In more than one occasion I’ve been asked to copy or backup an entire drive from one remote system to a second remote remote system, my preferred method for accomplishing this task is by making use of three different tools available in UNIX-like operating systems. Thanks to DD, Gzip, and OpenSSH I am able to make an exact copy of a drive, compress the resulting drive image while sending the resulting data to another remote system. The case for making use of this method is for when you either lack the local space to store the image or want to keep a backup in a remote system. It doesn’t hurt to backup valuable data, accidents happen. Also, it would be wise to compress the resulting image, I rather have the system spend the extra effort compressing the image using Gzip than to thave the uncompressed image taking more space than it should. What is great about this combination is that you can backup and restore the image using the same tools.
Remember to un-mount the drive you wish to clone. Not doing so may result in possible data corruption.
You can use the lshw (Hardware Lister) command to discover all available drives.
[root@system7]# lshw -C disk
*-cdrom
description: DVD-RAM writer
product: CDDVDW SH-S203N
vendor: TSSTcorp
physical id: 0.0.0
bus info: scsi@1:0.0.0
logical name: /dev/cdrom
logical name: /dev/dvd
logical name: /dev/scd0
logical name: /dev/sr0
version: SB01
serial: NECVMWarVMware IDE CDR101.00
capabilities: removable audio cd-r cd-rw dvd dvd-r dvd-ram
configuration: ansiversion=5 status=nodisc
*-disk
description: SCSI Disk
physical id: 0.0.0
bus info: scsi@2:0.0.0
logical name: /dev/sda
size: 50GiB (53GB)
capabilities: partitioned partitioned:dos
configuration: signature=000e4a4e
This command will copy, compress and send the image to the remote server.
dd if=/dev/sdb | gzip -c –fast | ssh user@ip ‘dd of=/home/user/sdb.img.gz’
Explanation: DD has been instructed to copy the drive /dev/sdb. Gzip will be used for compression, - c means Write on standard output, keep original files unchanged, - -fast mean Compress faster at the expense of high compression ration. The image will then be transferred via OpenSSH using the provided user credentials to the user directory /home/user. Note: the backup image makes use of the .gz extension to indicate compression is being used.
Restoring the image is not that different from the command used to backup the image.
ssh user@ip ‘dd if=/home/user/sdb.img.gz’ | gunzip -1 - | dd of=/dev/sdb
Explanation: Using OpenSSH log in to the remote system where the image is stored and with DD pull the image. Gunzip will be used to decompress the image crated by Gzip. Once again DD will be in charged of writing the image to /dev/sdb.
Remember that using OpenSSH or any other encrypted protocol for the transfer will slow down the progress and specifically impact CPU load during the cloning. But, because I am moving data across the Internet the security provided by OpenSSH is worth it. In a not so scientific test done between a white box server(sender) and a HP Quad Core server(receiver) it took 40.6 minutes to compress 80GB of data in to a 21GB image and transfer the result at a rate of 8.4MB/s(over a Gigabit network). The white box server uses an Intel E3200 CPU which during the test stayed at around 87% utilization, a faster CPU should provide better throughput.
dd, gzip, openssh — Oct 6, 2010