wget --help

Basic use

The simplest command Wget can offer. This will download the the file movie.mp4 from domain.com, notice how the the protocol HTTP had to specified can be changed for FTP.

wget http://domain.com/movie.mp4

Resume interrupted download

Let’s say you are downloading a 2 GB file then Internet connection dies for some reason Wget only managed to download 1.5 GB out of the 2 GB. The -c option can be used to tell Wget to resume the partially downloaded file, no need to download again.

wget -c http://domain.com/bigfile.iso

Mirror a site

Mirror or make a copy of a web site with the -m option. Self explanatory.

wget -m http://domain.com

Bandwidth throttling

Not everybody has FIOS (not me) for those who for example share a DSL or ISDN line are some what limited in the amount of bandwidth they can spare. Wget provides a way to limit the amount of bandwidth used, basically you can limit how much bandwidth it can use.

wget --limit-rate=20k http://domain.com/movie.mp4

Increase re-tries

When dealing with an Internet connection that suffers from high latency resulting in long download times the downloads will likely fail to complete. Instead of letting the unknown happen raise the default amount of retries (20) to say 30. Hopefully this will give you a higher margin to complete the download.

wget --tries=45 http://domain.com/movie.mp4

Disguised as a web browser

Some sites will not allow download managers alone to retrieve the files stored on their server, instead they expect to see a web browser like Mozilla or IE. Well this minor obstacle can be overcome by telling Wget to disguised it self as Mozilla to retrieve the file.

wget -c -U Mozilla http://domain.com/movie.mp4

Download files from FTP server

To download the same file from a FTP server just change the protocol when issuing the Wget command.

wget ftp://domain.com/movie.mp4

Let’s say the site happens to be password protected, Wget allows the user to login with credentials.

wget ftp://username:password@domain.com/path/to/file

Feel free to mix and match the options above nobody got any where by refusing to experiment.

Interesting Links
http://en.wikipedia.org/wiki/Wget

Accidents happen for example a Flash drive that might have been accidentally formatted or certain files erased from a hard drive. The good news is that as long as the data is not overwritten there is a high margin for a successful recovery. After all the recycle bin is not a secure way of destroying data by any standards.

For this article I will be using PhotoRec by CG Security, Photorec has many advantages for example it’s open source, cross platform, configurable and is relatively light while operating. The entire operation will take some time depending in size of the drive to be inspected. Do not expect the files to recovered with the original file name, instead Photorec will assign them a number. The output data should be stored on a second drive, don’t even think about about using the same drive.

The recovery technique I use is called Data Carving, I’ll let the collective knowledge explain what it means.

Data Carving is a data recovery technique that allows for data with no file system allocation information to be extracted by identifying sectors and clusters belonging to the file. Data Carving usually searches through raw sectors looking for specific desired file signatures. The fact that there is no allocation information means that the investigator must specify a block size of data to carve out upon finding a matching file signature. This presents the challenge that the beginning of the file is still present and that there is (depending on how common the file signature is) a risk of many false hits. Also, data carving requires that the files recovered be located in sequential sectors (rather than fragmented) as there is no allocation information to point to fragmented file portions. This method can be time and resource intensive.

Excerpt from Wikipedia

Download PhotoRec from CGSecurtity

PhotoRec is part of the TestDisk Suite.
http://www.cgsecurity.org/wiki/PhotoRec

Step 1

After PhotoRec starts you will be presented with all the drives PhotoRec was able to detect. For demonstration purposes I will be using a 2 GB flash drive.

Disk /dev/sdb - 2097 MB / 1999 MiB (RO) - OCZ ET1208AD

And press enter to proceed.

Step 2

Select the partition table in my case it’s.

[ Intel ] intel/PC partition

Step 3

In this case I want to inspect the whole disk for data.

D No partition 0 0 1 254 245 54 4095999 [Whole disk]

Step 4

If you noticed in the bottom of the terminal you are also presented with the following options.

[Options] [File Opt]

You can select [Options] or [File Opt] to see the available file extensions that PhotoRec can recover, you can also unchecked the file extensions you don’t want to recover. Or configure how persistent PhotoRec should be.

Step 5

Now specify the file system on which PhotoRec will attempt to recover the data from. My USB drive was formatted to FAT32.

[ Other ] FAT/NTFS/HFS+/ReiserFS/…

Step 6

PhotoRec is now presenting you with the option of choosing the default output directory /home/user or your own directory. You might want to create a directory just for the recovery which is usually spread across several directories.

Step 7

After selecting the output directory the recovery process will start, PhotoRec will let you know of the remaining time and number of files found.

The recovery process is done. All recovered files are enumerated.

Output

Example of the recovery, all data is spread across multiple directories. The recovered files are renamed.

Not all is lost sometimes most of the data can be recovered. Keep in mind some media files may be beyond recovery. Always back up your data and there will be no need for a recovery, otherwise I hope this article helps.

http://en.wikipedia.org/wiki/Data_recovery

Testdisk

Sometimes accidents happen and partitions are deleted or the system may become un-bootable for some reason. Useful for those nasty Windows viruses which may damage the partition table.

PhotoRec

PhotoRec was design to recover data from storage media like hard disk and Flash drives alike that may have been accidentally formated or deleted. PhotoRec aims to recover common formats like PDF, HTML MP3 to name a few. When doing a recovery you might only want to recover the actual files that contain the data not the software it used to run on.

DBAN

By far the best tool for those who want to re-use hard drives which may or may not contain sensitive data. Instead of just erasing data DBAN rewrites random data generated by Mersenne twister or ISAAC on the hard drive multiple times. How secure it is?, well it holds several official certifications from government agencies.

Ophcrack

Useful for when the user forgets his or her password and has no other way of obtaining the password to the system. It beats other commercial tools I have used previously.

GParted

For those who boot into multiple operating systems and need to re-size the primary partition or have to re-size the partition where Windows resides. Some PC manufacturers do not provide an actual CD/DVD instead the OS is install in to a partition.

UNetbootin

Some tools are offered in the form of an ISO for a reason meaning they have to burned in to a CD/DVD. UNetbootin allows a USB drive to become bootable reducing the amount of CD/DVD’s I have to burn (waste). UNetbootin removes the hassle of having to make the ISO bootable, just point to the ISO and UNetbootin does the rest.

Download the virtual machine

mencoder filename.mov -ovc xvid -oac mp3lame -xvidencopts fixed_quant=4 -o output.mp4

Basically Mencoder was told to obtain the best quaility with fixed_quant=4. The quaility level ranges from 1-31. Remember because we changed the container and then told it to keep the highest quality avaiable the file will result in a larger output.

Example :

• Original file promo.mov 63.3 MB
• Encoded file promo.mp4 51.6 MB

It should not be that hard to make changes the line above to fit your needs.

Sources :

http://en.linuxreviews.org/HOWTO_Convert_video_files
http://www.linux.com/feature/121385
http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-selecting-codec.html

I wish you all a happy 2009, it’s a new year to start new projects and take on new adventures. I wish the best to all of you and thank you for reading this blog. Let’s improve on 2009 and make the best out of it. Again happy new year and take care.

I really mean it.

Recently I noticed that for the first time CentOS was available in the form of a network installation. Which means instead of downloading the whole operating system only a small ISO (less than 8 MB) has to be downloaded which is enough to boot from and connect to the remote CentOS servers. No need to download 4 GB’s of data. It’s also the perfect solution for multiple internal deployments. The whole process may take around 30 to 40 minutes.

Remember the available architectures are i386 and x86_64.

CentOS mirror : http://mirror.centos.org/centos/5/isos/

After you burn the ISO in to a CD/DVD we can start the network installation.

Network installation

For the Installation Method choose HTTP to fetch the remaining packages from the CentOS servers.

Configure TCP/IP

We will start the installation by choosing DHCP in order to grab an IP from the local area network.

HTTP setup

Now we need to indicate where the CentOS files are located on the CentOS servers, provide the following information when asked.

Web site name : mirror.centos.org
CentOS directory : centos/5.2/os/i386

Press OK to start fetching the files from the internet.

Now let the magic begin.

The hard work is done, proceed to customise the CentOS installation like usual by selecting the packages you want install. This the best and most efficient way of installing CentOS.

Yeah, you heard me no CD/DVD burning in here. VMWare player lacks the ability to create new virtual machines by it self, this problem can be overcomed by using third party solutions or a blank virtual machine. However one small feature found on VMWare Workstation still missing which is the ability to install the operating system from ISO without having to burn in it into a CD/DVD first. Mostly it’s about reducing recordable media waste.

You can download a blank virtual machine ready for installation from this link.

You might also find this article interesting : How to Create a virtual machine with VMWare Player

Install from a ISO

In the case of a blank virtual machine you need to add the following values and indicate the location of the ISO.

Add the following line.

ide1:0.fileName = “locationofiso.iso”

Locate and change the following line.

ide1:0.deviceType = “cdrom-raw”

To.

ide1:0.deviceType = “cdrom-image”

Close and save the configuration file. Now start the virtual machine and proceed with the regular installation.

After the installation is done change the values back to.

ide1:0.fileName = “auto detect”
ide1:0.deviceType = “cdrom-raw”

Save and enjoy your virtual machine.

It happens to all of us, sometimes segments of the network may become unusable or slow. The documentation created by the previous administrator may say Gigabit ethernet is supported or enabled across the network, yet only a fraction of the bandwidth is available.

Iperf can measure the throughput of a network, Iperf operates in a client server mode. Best of all it’s open source and cross platform. Iperf is the perfect tool to measure wired and wireless networks, bandwidth is measured through TCP tests.

Iperf installation

For the network test I will install Iperf on two systems, one will act as the server and the other as the client. Iperf is available on the Ubuntu repositories.

sudo apt-get install iperf

Iperf server

Now turn one of the systems in to the Iperf server.

iperf -s

Iperf server output.

————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————

Iperf client

On the Iperf client issue the following command to connect with the Iperf server and initiate the test.

iperf -c 192.168.1.103

Client side output.

————————————————————
Client connecting to 192.168.1.103, TCP port 5001
TCP window size: 16.0 KByte (default)
————————————————————
[ 3] local 192.168.1.101 port 59291 connected with 192.168.1.103 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 113 MBytes 94.5 Mbits/sec

You can see that on this test Iperf successfully transfered 113 MB of data at a network speed of 94.2 Mbits/sec.

You can also tell Iperf to wait before giving you the network results, by adding -t 30 at the end of the command. This will tell Iperf to wait and in the mean time transferer more data across the network.

Interesting links
http://sourceforge.net/projects/iperf

Essentially all I had to do was install lm-sensors and the GNOME sensor applet. Keep in mind for this article I used a DG31GL Intel Motherboard and a Q6600 Intel processor, all sensors were detected without a problem. Results may vary depending on the hardware.

Installation

First we need to install the vital lm-sensors and the GNOME applets that will later present the information on the desktop.

Next we do a sensor detection (results will vary based on hardware support).

After the sensor detection was done you will have to answer a series of questions, accepting the default works just fine except the last question. Don’t forget to answer Yes on the last question.

The detection was successful now we have to answer Yes in order to add the new sensors to /etc/modules, later on it will allow the GNOME applet to fetch the required data for display.

Add the GNOME applet

To add the applet responsible for displaying the data go to.

Right click on the top panel > click on Add to Panel.

Search for Hardware Sensors Monitor > click on Add.

The applet will display the temperature data on the system, processor, hard drive and AUX Temp. Not all of the data is necessary or useful, if you like to change what’s displayed.

Right click on the applet and select Preferences. Move on to the Sensors tab and check or unchecked what you don’t like.

Links of interest
http://sensors-applet.sourceforge.net
http://www.lm-sensors.org/