Category: Bash

Temporary Disabling Bash History

Say that you’ve got to pass some password as command line argument to something. It would probably be a bad idea to store it in your ~/.bash_history, but clearing the file isn’t desired either. So you need to temporary disable the command history for the current session. You can do it by unsetting the HISTFILE environment variable.

unset HISTFILE

The result is that while the session is active you can access the history as usual, but it won’t be saved to the disk. History for other sessions, will behave as usual.

Django Backup Script

This is a backup script for Django projects. It’s able to automate backups to a local folder and a remote FTP server of both the database and files. It somewhat old and has a few limitations, mainly supporting only MySQL and not supporting the new way for specifying databases introduced in Django 1.2.

It’s loosly based on my WordPress backup script and inspired the database settings auto-detection found in the newer Wordrpess backup script.

Usage is simple:

$ django_backup /path/to/my/proj
$ django_backup --db-only /path/to/my/proj

The latter command only backups the database.

The script uses a few configuration variables in the top of the script to set the folder which the local backups are kept in and the remote FTP server settings. The database settings are extracted directly from the settings.py of the backed-up project.
Continue reading Django Backup Script

Improved FTP Backup for WordPress

This script backups both the database and files of a WordPress blog into a remote FTP server (while keeping a local copy). It’s an update of my WordPress Backup to FTP script. The main changes are auto-detecting database settings and better support for caching plugins (specifically WP-Cache). The new version makes it easier to backup multiple WordPress blogs to the same FTP server.
Continue reading Improved FTP Backup for WordPress

Batch Renaming Using sed

I was reorganizing my music library and decided to change the naming convention I’ve used. This task is just asking to be automated. Since the filename change could be described using regular expression, I looked for a way to use sed for the renaming process.

The files I had, had the following pattern as filename ARTIST – SONG – TRACK – ALBUM

James Brown - I Got You (I Feel Good).ogg  - 01 - Classic James Brown

I wanted to rename it to ARTIST – ALBUM – TRACK – NAME

James Brown - Classic James Brown - 01 - I Got You (I Feel Good).ogg

Describing the change as a sed program is easy:

s/\(.*\) - \(.*\) - \(.*\) - \(.*\).ogg/\1 - \4 - \3 - \2.ogg/

Now all that has to be done is to pass each filename to mv and pass it again after it went through the sed script. This could be done like this:

for i in *; do
  mv "$i" "`echo $i | sed "s/\(.*\) - \(.*\) - \(.*\) - \(.*\).ogg/\1 - \4 - \3 - \2.ogg/"`";
done

The important part is the

`echo $i | sed "s/\(.*\) - \(.*\) - \(.*\) - \(.*\).ogg/\1 - \4 - \3 - \2.ogg/"`

which pipes the filename to sed and returns it as an argument for mv.

To see what renaming will be done one can alter a bit the above command, and get

for i in *; do
  echo "$i" "->" "`echo $i | sed "s/\(.*\) - \(.*\) - \(.*\) - \(.*\).ogg/\1 - \4 - \3 - \2.ogg/"`";
done

While will effectively print a list of lines of the form oldname -> newname.

Of course this technique isn’t limited to the renaming I’ve done. By changing the pattern given to sed, one can do any kind of renaming that can be described as a regular expression replacement. Also one can change the globbing (the *) in the for loop to operate only on specific files, that match a given pattern, in the directory instead of all of them.

WordPress Backup to FTP

Update: A newer version of the script is available.

This script allows you to easily backup your WordPress blog to an FTP server. It’s actually a modification of my WordPress Backup to Amazon S3 Script, but instead of saving the backup to Amazon S3 it uploads it to an FTP server. Another update is that now the SQL dump includes the database creation instructions so you don’t need to create it manually before restoring from the backup.

Although I’ve written it with WordPress in mind (to creates backups of my blog), it isn’t WordPress specific. It can be used to backup any website that consists of a MySQL database and files. I’ve successfully used it to backup MediaWiki installation.
Continue reading WordPress Backup to FTP

Backup a SourceForge hosted SVN repository – sf-svn-backup

SourceForge urges their users to backup the code repositories of their projects. As I have several projects hosted with SourceForge, I should do it too. Making the backups isn’t complicated at all, but because it isn’t automated properly, I’ve been lazy with it.

sf-svn-backup was written in order to simply automate the process. The script is pretty simple to use, just pass as the first argument the project name and the script will write down to stdout the dump file.

For example:

sf-svn-backup openyahtzee > openyahtzee.dump

The project name should be it’s UNIX name (e.g. openyahtzee and not Open Yahtzee). Because the script writes the dump file directly to stdout it’s easy to pipe the output first through a compression program such as gzip to create compressed SVN dump files.

tarsum – Calculate Checksum for Files inside Tar Archive

Update: I’ve released tarsum-0.2, a new version of tarsum.

Some time ago, I got back a hard disk back from data recovery. One of the annoying issues I encountered with the recovered data was corrupted files. Some files looked like they were recovered successfully but their content was corrupted. The ones that were configuration files, where usually easy to detect, as it raised errors in programs that tried to use them. But when such error occurs in some general text file, (or inside the data of an SQL dump), the file may seem correctly fine unless closely inspected.

I have an habit of storing old backups on CDs (they are initially made to online storage), I do it in order to reduce backup costs. But the recovered/corrupted data issue raised some concerns about my ability to recover using this disks. Assuming that I have a disk failure, and I couldn’t recover from my online backups for reason, how can I check the integrity of my CD backups?

Only storing and comparing hash signature for the whole archive, is almost useless. It allows you to validate whether all the files are probably fine, but it can’t tell apart one corrupted file in the archive from a completed corrupted archive. My idea was to calculate checksum (hash) for each file in the data and store the signature in a way that would allow me to see which individual files are corrupted.

This is where tarsum comes to the rescue. As it’s name applies it calculate checksum for each file in the archive. You can download tarsum from here.

Using tarsum is pretty straight forward.

tarsum backup.tar > backup.tar.md5

Calculates the MD5 checksums of the files. You can specify other hashes as well, by passing a tool that calculates it (it must work like md5sum).

tarsum --checksum=sha256sum backup.tar > backup.tar.sha256

To verify the integrity of the files inside the archive we use the diff command:

tarsum backup.tar | diff backup.tar.md5 -

where backup.tar.md5 is the original signature file we created. This is possible because the signatures are sorted alphabetically by the file name inside the archive, so it the order of the files is always the same.

Note that if you use an updated version of GNU tar, tarsum can also operate directly on compressed archives (e.g. tar.bz2, tar.gz).

s3backup – Easy backups of Folders to Amazon S3

This is an updated version of my previous backups script – Backup Directories to Amazon S3 Script. The new script works much better and safer. Unlike the old script, the new one creates the tarballs in a temporary file under /tmp, and allows more control over the backup process.

Continue reading s3backup – Easy backups of Folders to Amazon S3

WordPress Backup to Amazon S3 Script

This is an updated version of my WordPress Backup Script. The new version basically does the same thing: backup up a wordpress blog (actually any site that consists of files and a MySQL database). The new thing about the script is that instead of only saving the backup locally, it also uploads it to Amazon S3.

Continue reading WordPress Backup to Amazon S3 Script

WordPress Backup Script

This is a small script I’ve written to automate my server-side backups of my blogs. It creates a backup of both the database and the actual WordPress files. 

#!/bin/bash

# (C) 2008 Guy Rutenberg - http://www.guyrutenberg.com
# This is a script that creates backups of blogs.

DB_NAME=
DB_USER=
DB_PASS=
DB_HOST=

#no trailing slash
BLOG_DIR=
BACKUP_DIR=


echo -n "dumping database... "
mysqldump --user=${DB_USER} --password=${DB_PASS} --host=${DB_HOST} ${DB_NAME} \
 | bzip2 -c > ${BACKUP_DIR}/${DB_NAME}-$(date +%Y%m%d).sql.bz2
if [ "$?" -ne "0" ]; then
	echo -e "\nmysqldump failed!"
	exit 1
fi
echo "done"


echo -n "Creating tarball... "
tar -cjf ${BACKUP_DIR}/${BLOG_DIR##*/}-$(date +%Y%m%d).tar.bz2 ${BLOG_DIR}
if [ "$?" -ne "0" ]; then
	echo -e "\ntarball creation failed!"
	exit 1
fi
echo "done"

Continue reading WordPress Backup Script