Tag Archives: gzip

How to Gzip Files on Windows

7-Zip can be used to create more than 7z compressed files, it can also create GZip compressed files and directories. GZip can compressed a single file on its own, however if you need to compress a directory an extra step is required where the directory is first archived with the help of tar and then compressed with GZip. This tutorial will cover single file compression and directory compression.

First Things First

Download 7zip from http://www.7-zip.org/ and install the application.

Gzip Single File

Start the 7zip application and browse the location where the file you want to compress is located.

Right click on the file_name > 7-Zip > Add to archive…

Gizp Windows

For Archive format: select gzip and click on OK to start compressing.

Gzip Windows

Compression progress.

Gzip Windows

The file is now compressed.

Gzip Windows

Gzip Directory

Compressing a directory takes an extra step, first you need to create a tar of the directory you want to compress.

Gzip Windows

Select the source directory > 7-Zip > Add to archive…

Gzip Windows

For Archive format: select tar and click on OK.

Gzip Windows

Progress being displayed.

Gzip Windows

Once again go back to 7-Zip select the new tar file(filename.tar) > 7-Zip > Add to archive…

Gzip Windows

For Archive format select gzip and click on OK.

Gzip Windows

Progress being displayed.

Gzip Windows

The resulting compressed directory with the new extension ending in .tar.gz.

Gzip Windows

Conclusion

While GZip files may not be all that common in Windows they are commonly used in other operating systems like Linux and FreeBSD. This post is specially useful since many Linux installation only have support for GZip, including support for Zip and 7z is not even an option. If you have any questions feel free to leave a comment below.

Links

Homepage: http://www.7-zip.org/

Copy and Restore a Drive with Dc3dd, Gzip and a Network Share

In the past I’ve written about combining dd, Gzip and OpenSSH to image a drive over a network, while the above works just fine there are better tools available that can improve the results or at least make it easier to clone a drive. By better I mean:

  1. Unlike dd, Dc3dd comes with a progress indicator
  2. With Gigabit links OpenSSH appears to become the bottleneck, even when resources are available network throughput is low

Keep in mind

Before we start the tutorial I recommend using a one of the many available Live-CD distributions like Parted Magic or Deft Linux because they include all the tools I’ll be using in this post. This tutorial assumes a working network share is already available.

The process for imaging a drive over a network revolves around five commands ls, Dc3dd, Gzip, Gunzip and mount.cifs.

Prepare

Locate the drive you need to image, we can use the ls command to list all available drives and their respective partitions, decide whether you need to copy the partitions or the entire drive.

# ls /dev/sd*

/dev/sda   /dev/sda1  /dev/sda2  /dev/sdb   /dev/sdb1

* /dev/sda and /dev/sda represent the actual drive, /dev/sda1/2 and /dev/sdb1 represent a partition within the drive.

Create a mount point for the network share.

# mkdir /media/netshare

Mount the network share.

# mount.cifs //192.168.1.8/072012/ /media/netshare/ -o user=tempuser,password=tempuser

Change the following:

  1. mount.cifs //192.168.1.8/072012/ – This is the network share IP address and share name
  2. /media/netshare/ – This is the mount point we created with mkdir.
  3. -o user=tempuser,password=tempuser – This is the username and password for the network share

Verify that the network share mounted successfully.

# df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb1             2.0G  8.0K  2.0G   1% /media/usb
//192.168.1.8/072012/
                       80G   23G   58G  29% /media/netshare

Copy

Star imaging or cloning the drive, compress and direct the output to the network share mount point.

# dc3dd if=/dev/sda progress=on | gzip -1 > /media/netshare/sda.img.gz

warning: sector size not probed, assuming 512
dc3dd 6.12.3 started at 2012-07-21 07:11:13 +0000
command line: dc3dd if=/dev/sda progress=on
compiled options: DEFAULT_BLOCKSIZE=32768
sector size: 512 (assumed)
9830400+0 sectors in
9830400+0 sectors out
5033164800 bytes (4.7 G) copied (??%), 444.728 s, 11 M/s
dc3dd completed at 2012-07-21 07:18:39 +0000
  1. dc3dd if=/dev/sda – Dc3dd will work with drive /dev/sda
  2. progress=on – Dc3dd will display the progress
  3. gzip -1 > – Gzip will be used for compression, -1 means fast compression
  4. /media/netshare/sda.img.gz – This is the network share path

My network share is in a Windows Server 2003 and as you can see the image was successfully created and compressed.

Restore

To restore the image we use Gunzip to decompress and Dc3dd to write the decompressed data over to the target disk. I recommend using the progress=on option when working with large images, its nice to have an idea of the progress being made.

# gunzip -c /media/netshare/sda.img.gz | dc3dd of=/dev/sda progress=on

warning: sector size not probed, assuming 512
dc3dd 6.12.3 started at 2012-07-21 22:55:04 +0000
command line: dc3dd of=/dev/sda
compiled options: DEFAULT_BLOCKSIZE=32768
sector size: 512 (assumed)
9830400+0 sectors in
9830400+0 sectors out
5033164800 bytes (4.7 G) copied (??%), 192.915 s, 25 M/s
dc3dd completed at 2012-07-21 22:58:19 +0000

Conclusion

This is my preferred method for imaging drives over the network because the tools required to copy and restore are easily found in most distributions. If you have any questions feel free comment below.

Clone a Hard Drive over the Network with DD

In more than one occasion I’ve been asked to copy or backup an entire drive from one remote system to a second remote remote system, my preferred method for accomplishing this task is by making use of three different tools available in UNIX-like operating systems. Thanks to DD, Gzip, and OpenSSH I am able to make an exact copy of a drive, compress the resulting drive image while sending the resulting data to another remote system.

The case for making use of this method is for when you either lack the local space to store the image or want to keep a backup in a remote system. It doesn’t hurt to backup valuable data, accidents happen. Also, it would be wise to compress the resulting image, I rather have the system spend the extra effort compressing the image using Gzip than to thave the uncompressed image taking more space than it should.

What is great about this combination is that you can backup and restore the image using the same tools.

Remember to un-mount the drive you wish to clone. Not doing so may result in possible data corruption.

Locate The Drive You Wish To Backup

You can use the lshw (Hardware Lister) command to discover all available drives.

[root@system7]# lshw -C disk

*-cdrom
       description: DVD-RAM writer
       product: CDDVDW SH-S203N
       vendor: TSSTcorp
       physical id: 0.0.0
       bus info: scsi@1:0.0.0
       logical name: /dev/cdrom
       logical name: /dev/dvd
       logical name: /dev/scd0
       logical name: /dev/sr0
       version: SB01
       serial: NECVMWarVMware IDE CDR101.00
       capabilities: removable audio cd-r cd-rw dvd dvd-r dvd-ram
       configuration: ansiversion=5 status=nodisc
  *-disk
       description: SCSI Disk
       physical id: 0.0.0
       bus info: scsi@2:0.0.0
       logical name: /dev/sda
       size: 50GiB (53GB)
       capabilities: partitioned partitioned:dos
       configuration: signature=000e4a4e

Backup To Remote Server

This command will copy, compress and send the image to the remote server.

dd if=/dev/sdb | gzip -c --fast | ssh user@ip 'dd of=/home/user/sdb.img.gz'

Explanation: DD has been instructed to copy the drive /dev/sdb.
Gzip will be used for compression, – c means Write on standard output, keep original files unchanged, – -fast mean Compress faster at the expense of high compression ration.

The image will then be transferred via OpenSSH using the provided user credentials to the user directory /home/user. Note: the backup image makes use of the .gz extension to indicate compression is being used.

Restore From Remote Server

Restoring the image is not that different from the command used to backup the image.

ssh user@ip 'dd if=/home/user/sdb.img.gz' | gunzip -1 - | dd of=/dev/sdb

Explanation: Using OpenSSH log in to the remote system where the image is stored and with DD pull the image. Gunzip will be used to decompress the image crated by Gzip. Once again DD will be in charged of writing the image to /dev/sdb.

Keep In Mind

Remember that using OpenSSH or any other encrypted protocol for the transfer will slow down the progress and specifically impact CPU load during the cloning. But, because I am moving data across the Internet the security provided by OpenSSH is worth it.

In a not so scientific test done between a white box server(sender) and a HP Quad Core server(receiver) it took 40.6 minutes to compress 80GB of data in to a 21GB image and transfer the result at a rate of 8.4MB/s(over a Gigabit network). The white box server uses an Intel E3200 CPU which during the test stayed at around 87% utilization, a faster CPU should provide better throughput.