Create Archive of Website

If you have a website that you're retiring but would like to copy it from your current registrar without having to sFTP/FTP the following command will copy it in Linux.

wget --recursive --domain [domain] --no-parent --page-requisites [location]

Nginx - Count of IP addresses in /var/log/nginx/access.log

Inspired by this post which supposedly works for Apache and also works for my Nginx installation. It seems however, some of the comments didn't work quite right on our Debian Lenny equipment.

Did away with grep (instead using cat) and tweaked to fix sorting not working correctly.

cat access.log | cut -d' ' -f1 | sort | uniq -c | sort -rg

If your log is big enough to scroll multiple pages when logged in remotely.

cat access.log | cut -d' ' -f1 | sort | uniq -c | sort -rg | less

Bulk File Rename

As usual there are command I use just often enough to forget them when the time cose. In our case we had a group of files that has the file extension in caps which was causing a problem with some 3rd party javascript so instead of modifying the .js files we change the extension to lower case (.JPG to .jpg).

for file in *.JPG ; do mv $file `echo $file | sed 's/\(.*\.\)JPG/\1jpg/'` ; done

Find Text Within Files

If you're a Linux Geek who likes to spend time figuring out a problem and writing grep commands this post is not for you. If you're using Linux, specifically a Debian distro such as Ubuntu, and need a quick solution and do not have a mastery of the command line nor time to write scripts for Gnome Nautilus this is your post!

Background:
Started using Ubuntu again due to finally having decent support in Google Talk for Drag/Drop file transfer and Google Voice Chat in Empathy. Since I use this quite a bit in my business it was imperative this work correctly.

Problem:
The ability to search inside of a variety of files (such as: pdfs, office 2K3 and 2K7 files, text files, etc.) for specific text. Tried apps such as sagasu, regexxer and even tried a few Nautilus scripts but they didn't perform what I needed.

Solution:
Google Desktop
If you've used it before you know how it works, if you haven't download it and give it a try. Here is a short post about how to install, setup and use Google Desktop on Ubuntu.

Problem with Debian "apt-get upgrade"

Recently when attempting to upgrade a server that was being neglected we ran across an issue with Debian stating.

trying to overwrite `/usr/sbin/add-shell’, which is also in package passwd

The solution was simple and was required on a few similar errors where the overwrite and package names changed. Again we had really neglected this server and had not run the apt-get upgrade in a long time.

The fix?
dpkg -r --force all passwd
apt-get -f install

To explain briefly the "dpkg -r --force" forces the removal of the package "passwd". The "all" switch turns on(or off) all force options. An important note taken directly from the dpkg man page.
Warning: These options are mostly intended to be used by experts only. Using them without fully understanding their effects may break your whole system.

If you are worried and do not understand the possible risks DO NOT run the above dpkg command. The "apt-get -f install" is less risky but again, research before you apply.

Good Things Come To Those Who Wait

...and wait....and wait.

Finally a Google Talk for Linux that has not only text messaging but voice chat as well. Thank you James for the quick post about which IM client to use. Thank you to the team who built the IM for making our life on Linux just a tad bit more enjoyable.

Increase Readability In Firefox on Ubuntu

If you are familiar with ClearType in XP you know what a difference it can make. For the most part we have been satisfied with the look and readability of fonts in Ubuntu Intrepid however the default Firefox fonts were rough. This simple fix went a long way in increasing readability and strain on my eyes.


Go to Edit > Preferences > Content > Advanced [Under Fonts & Colors section]. Change to the following settings for the Serif:, Sans-serif:, and Monospace: settings.

source: firefox to look good, configure fonts

Install Flashplayer in Firefox on Ubuntu Intrepid 64-bit

Before starting the install make absolutely sure that you have run the following command which should uninstall nspluginwrapper and flashplugin-nonfree.

user@machine~> sudo apt-get remove nspluginwrapper

Now you can use the following short installation guide
How to Install Adobe Flash Player 64-bit on Ubuntu 8.10

Cross Platform Password Programs

If you are considering a password program or looking to migrate from your current password program I'd highly recommend you consider the following two applications. They are cross platform so you can keep the databases and Windows portable versions on a USB key and reference the databases from your Linux version or you can copy them from your Windows installation to your Linux install or the other way around if you switch operating systems with any consistency.


KeePass
Windows Version
Install on Debian based distro by using apt-get install keepassx


PasswordSafe
Windows Version
Can install on Debian based distro by using apt-get install password-gorilla

rdesktop Minimize From Full Screen

After spending the time to get a Xubuntu workstation to VPN into a Windows environment I realized that I've become accostomed to using Windows Remote Desktop in full screen however when running rdesktop in full screen the small bar at the top that allow you to minimize didn't appear. The solution is easy however, CTRL+ALT+ENTER and it reduced size so you can see your XFCE panel.

If you're not sure how to launch rdesktop in full screen here is an example of the command that is launched from an icon on my panel.

rdesktop -T "AD Main" -d ADDomain -u ADAdmin -f -a 15 -P 192.168.0.1


021 no connection named [OpenSWAN]

Since it had been over a year since I had last used OpenSWAN I couldn't remember why this error occurred. Searching Google for 021 no connection named "blah" was not producing a fix. Finally, and I admit this a little embarrassed I read the manual and found that I had forgot to do after configured the /etc/ipsec.conf file.

user@machine ~> sudo ipsec auto --add myconnection


So if you see the following in the future don't forget to add the connection.

user@machine ~> sudo ipsec whack --name myconnection --initiate
021 no connection named "myconnection"


Restarting Pluto [OpenSWAN]

If you've suffered the same fate as I have...the inability to locate solutions on restarting pluto aside from the infamous RTM this post if for you.

First we'll make sure pluto is running and get the PID

user@machine ~> sudo ipsec setup --status
IPsec running - pluto pid: 8794
pluto pid 8794
No tunnels up


Now we restart everything related to ipsec

user@machine ~> sudo /etc/init.d/ipsec restart
ipsec_setup: Stopping Openswan IPsec...
ipsec_setup: Starting Openswan IPsec 2.4.6...
ipsec_setup: insmod /lib/modules/2.6.22-14-generic/kernel/net/key/af_key.ko
ipsec_setup: insmod /lib/modules/2.6.22-14-generic/kernel/net/ipv4/xfrm4_tunnel.ko
ipsec_setup: insmod /lib/modules/2.6.22-14-generic/kernel/net/xfrm/xfrm_user.ko


Last but not lease lets make sure pluto restarted and has a new PID

user@machine ~> ipsec setup --status
IPsec running - pluto pid: 9193
pluto pid 9193
No tunnels up

There are probably other ways of accomplishing this as well however so if you know please add a comment letting everyone know.

Adding SSL Site To Nginx Quick Start

If now is one of those times you just want to Get 'Er Done and not have a long drawn out process for doing something, welcome.

Step 1: Install OpenSSL
user@machine ~> sudo apt-get install openssl

Step 2: Create Self Signed Certificate [Answer questions in a way that makes you feel good ;)]
user@machine ~> sudo openssl req -new -x509 -days 365 -nodes -out /root/nginx.cert -keyout /root/nginx.key

Step 3: Add SSL options to nginx.conf [/etc/nginx/nginx.conf maybe?]

server {
listen 443;
server_name smallbiztechguy.blogspot.com;
error_page 403 404 500 502 503 504 /nginx-errors/error.htm;

ssl on;
ssl_certificate /root/nginx.cert;
ssl_certificate_key /root/nginx.key;

location / {
charset utf-8;
root /var/www;
index index.htm;
}

}


Step 4: Restart nginx

user@machine ~> ps aux | grep -i nginx
user@machine ~> kill -HUP [id from ps command above]

Internet Explorer on Linux

If you've ever had a need for running Internet Explorer, in this case IE6, on Linux there is a simple solution. First question might be why in the world would you want to? In some cases you need to test a website on IE to be sure it renders properly and in some cases you might be accessing Microsoft software that is a little picky about the browser that is accessing it.

The solution exists in IEs4Linux, a creation of Sérgio Luís Lopes Júnior. He has done a great job of allowing IE5, IE5.5, IE6 and now a beta of IE7 to be installed and run on Linux. The installation is fairly straightforward for all but IE7 and even that isn't too difficult by Linux standards. The performance isn't quite a snappy as I'd like to see but considering the price, $free, I can't complain too much.

Create an ISO from a DVD

If you need to create backups of DVDs there are a multitude of ways people recommend doing them on the web. The easiest I found can be accomplished in a single command and creates an ISO file that can be mounted later if necessary. Even tried it with an old, uncopyrighted DVD movie and it worked well.

user@machine ~> dd if=/dev/dvd0 of=~/dvdcopy.iso bs=2048

Some might ask: Why the bs=2048 parameter?

The reason is that often burnt cds/dvds have a number of null sectors
appended to the actual iso on the disc. If you simply use dd without a block
count the resulting iso will not be identical to the iso from which the dvd
was produced. The same remarks apply to the use of cat /dev/dvd > your.iso.
Not to say that the result won't work, it might. But if you do `md5sum
your.iso` the sum will not match that of the original iso if the there are
any blocks of padding copied up.
source

To remount:

user@machine ~> mkdir /media/dvdcopy
user@machine ~> mount -t iso9660 -o loop ~/dvdcopy.iso /media/dvdcopy

There are some posts on the web stating that if mounting a DVD that the type needs to be set to udp instead of iso9660 however the iso9660 value worked fine.

Encrypted Offsite Backup - Part I

For a couple of years now I've been performing offsite backups using the following process.

1. If not already loaded, boot into my Windows XP partition on my laptop
2. Copy all important files to a directory inside of the My Documents folder
3. Zip everything up using 7zip
4. Encrypt the zip file created in step 3 using AxCrypt
5. Copy the file to my backup server in another state using scp

The process didn't usually bother me too much because I'd kick off the zipping and copying processes before heading to bed so they'd just run at night. The problem with that process is obvious, it requires me manually initiating the process. As I've become painfully aware of most of us IT guys neglect our backups until it is too late. In my situation I've got a mixed home network running Debian, Ubuntu, Windows XP, uNSLUng and another distro or two when the urge strikes me. Well it came time to pony up and get with the program so I figured out a solution that once set up would give me secure offsite backups but also eliminate the time of doing the copying and zipping manually.

The solution involves using encfs to encrypt the data before copying it to the offsite machines using a mount created with sshfs. Below is the bash function I created that does the action for me, I'll be updating to use a key later but for now it just runs through cron and prompts me for the password.

function offsiteBackup
{

## set variables
DIRBASE=$HOME/Backup
DIRENC=$DIRBASE/encrypt
DIRDEC=$DIRBASE/decrypt

## remove encryption/decryption directories to make sure everything is gone
echo "Cleaning up from previous backups"
rm -rf $DIRBASE
mkdir -p "$DIRENC"
mkdir -p "$DIRDEC"

## mount the encryption and decryption directories
echo "Directories mounted"
encfs $DIRENC $DIRDEC

## copy all files that will be backed up
## NOTE: since the remote machines were mounted using sshfs only
##        the cp command is necessary.
echo "Copying files backup files from remote locations"
cp /media/remote1/settings.xml $DIRDEC
cp /media/remote1/mystuff $DIRDEC

echo "Files copied to the backup unencrypted directory"

## umount the decrypted directory
fusermount -u $DIRDEC

## create the zipped archive backup file
tar -czvf backup.tar.gz $DIRENC

## remove encryption/decryption directories to make sure everything is gone
echo "Removing backup directories"
rm -rf $DIRBASE

}

In Part II I'll be investigating if this same process can be followed with the Windows boxes using cygwin or some other alternative but for now this should get the Linux junkies started.

Special thanks for Tom Lowry at the University of Arizona Computer Science department for this post that got me started.

Mount Disk by UUID

Have you ever been in a situation where you are wanting to mount your disk drives (USB, ATA, SATA, Firewire, etc.) by some unique identifier due to the occasional weirdness where you get the joy of what used to be your USB thumb drive -> /mnt/disk is now /mnt/disk2? If not then move on and pretend like you never stumbled across this post, if you have been scratching your head or read a convoluted solution that was driving your crazy this is your post.

The most consistent way I've found is to mount a device by the device's UUID, also known as its Universally Unique Identifier. What this alpha numeric looks like can depend on the device but I've yet to discover two devices that have the same UUID.

In the example below we'll assume we want to mount /dev/sda1 as /media/windows

Step 1 - Get the UUID of your drive by executing the following:
user@machine ~> sudo vol_id -u /dev/sda1
208861208860Z1A6

Step 2 - Add the proper line to your /etc/fstab file to property mount the partition in the future, snippet below:
# /dev/sda1
UUID=208861208860Z1A6 /media/windows ntfs defaults,umask=007,gid=46 0 1

Another example which mounts a FAT32 partition:
# dev/sda#
UUID=A123-B456 /media/fat32 vfat gid=46,umask=000 0 1

If you wish to 'reload' the fstab file instead of rebooting:
user@machine ~> mount -a

Setting up a Linksys NSLU2 with Debian

In case you've been out of the loop for awhile there is a nice little network device called NSLU2 put out by Linksys. It is not the perfect device nor the end all solution to having a small machine that can run multiple USB drives as well as other USB accessories however it is cheap and has a large following which helps if you're not one of those types that likes to spend long weekends figured out things. Even if you do atleast you can easily get the operating system installed and get moving on to the business at hand.

As of right now there are a few issues with the default Debian installer so the manual method must be used however it is fairly simple. The first order of business is to download the Debian image to install on the NSLU2. Finally just head over to Martin's How-To and you should be up in running in no time.

Best wishes.

Scan for Wireless Networks on Ubuntu

Now that I'm beginning to travel more I run into situations where I need to search for wireless networks. Since I'm running Xubuntu some of the utilities that would exist in the more full featured desktops do not exist in Xubuntu but I found a few alternatives.

One solution is to install the wifi-radar package. Another that I found was running the following commands:

user@machine ~> sudo iwlist scan
user@machine ~> sudo iwconfig wlan0 mode Ad-Hoc {In this example wlan0 is the name of the wireless inteface}

The interface can also be edited with:
user@machine ~> sudo iwconfig

Remote Mount Directories using SSH

Now that I've moved over to using Linux as my primary client OS I decided it was time to remotely mount my NSLU2 (using Unslung) using SSH instead of messing with the Samba shares. It seems that there is a package for doing that on Ubuntu called sshfs, see this post on Debian Administration for more information.

One recommendation would be to try the sshfs package instead as I didn't require the module installation as discussed in the above post. In my case the build failed at which point just install sshfs worked 1st time, no other issues.