Tag Archives: md5

Using Hashdeep To Ensure Data Integrity

Posted on by

On a previous post I discussed the value of md5deep, now I am going to show you another tool by the name of hashdeep. As expected with hashdeep you can: recurse entire directories, perform matching, audit known hashes. Hashdeep grants auditing capabilities to the administrator.

Official Hashdeep description

Computes multiple hashes, or message digests, for any number of files while optionally recursively digging through the directory structure. By default the program computes MD5 and SHA-256 hashes, equivalent to -c md5,sha256. Can also take a list of known hashes and display the filenames of input files whose hashes either do or do not match any of the known hashes. Can also use a list of known hashes to audit a set of FILES. Errors are reported to standard error. If no FILES are specified, reads from standard input.

Hashdeep is the indicated tool if you need to recurse entire directories, I have personally hashed directories as large as 300GB without any problems.

Recursive directory hashing

The most common use, hashdeep can recurse a directory and output the results to a text file(you can change the extension). The -e parameter is optional.

hashdeep -e -r directory_name/ > output.txt
  • -r Recursive mode
  • -e Estimate time

Recursive drive hashing

Or if you wish you can hash an entire drive.

hashdeep -r G:\
  • -r Recursive mode
  • G:\ Name of the drive

Perform an audit using a list of known hashes (simple form)

Once you have a list of known hashes you can perform an audit of a directory to see if any changes where made.

hashdeep -r -a -k output.txt dir
  • -r Recursive mode
  • -a Audit mode
  • -k Load list of known hashes
  • output.txt File containing hashes
  • dir Name of the directory in question

For this example some files where changed, which resulted in a failed audit.

hashdeep: Audit failed

Perform an audit using a list of known hashes (advanced form)

This option will give detailed information about the audit.

hashdeep -v -r -a -k output.txt dir
  • -v Verobose mode
  • -r Recursive mode
  • -a Audit mode
  • -k Load listof known hashes
  • output.txt File containing hashes
  • dir Name of the directory in question
hashdeep: Audit failed
          Files matched: 41
Files paritally matched: 0
            Files moved: 0
        New files found: 1
  Known files not found: 1

Audit a list of known hashes and display the hash and location of those that failed to math against the list

This option will give you the name name, location, and hash of those files that failed to pass the audit.

hashdeep -r -X -v -k output.txt dir
  • -r Recursive mode
  • -X Display each failed hash that does not match the list of known hashes
  • -v Verbose mode
  • -k Load list of known hashes
  • output.txt File containing hashes
  • dir Name of the directory in question
%%%% HASHDEEP-1.0
%%%% size,md5,sha256,filename
## Invoked from: C:\Users\Luis\Desktop
## C:\> hashdeep -r -X -v -k output.txt dir
## 
6,dcd989387b401ac29bf44755f31c0952,5a3edf2142ffde0b2d9803d845c795c24bfdd610d2b9d68408f5207d47e11b4a,C:\Users\Luis\Desktop\dir\New Text Document - Copy (10).txt

Hash a directory with md5deep – Part 1

Posted on by

File integrity is no joke and system administrators know the severity of dealing corrupt files. The problem administrators face is that unlike users, administrator have to manage servers with hundreds of thousands of files in each directory that have to be hashed. This lends it self to a few problems: hashing file by file is out of the question and a waste of time, which means we are left to find a solution that can work on a recursive manner.

md5deep checksums

Recursive operation – md5deep is able to recursively examine an entire directory tree. That is, compute the MD5 for every file in a directory and for every file in every subdirectory.

For the Windows platform you can use md5deep which is a cross platform solution, works perfectly and best of all can handle a large number of files without breaking a sweat. It will literally create a checksum for all the files on the directory where md5deep is being used, the results can then exported to a single file where they can be used for verification at a later time.

md5deep is a software package used in the computer security, system administration and computer forensics communities for purposes of running large numbers of files through any of several different cryptographic digests.
Source: Wikipedia http://en.wikipedia.org/wiki/Md5deep

md5deep is managed from the command line, however it’s very easy to use considering it will be in charge of creating checksums for a large number of individual files.

How do we use it ?

md5deep example

First of all you can download md5deep from SourceForge.net.

After you download md5deep I recommend you move the executable over to the C:\Windows directory for easy access from the command prompt.

For this tutorial I will be hashing a single directory. Start by opening a command prompt :

Start > Accessories > Command Prompt

On the Command Prompt type :

md5deep -rel "test_directory" > results_file.md5

Explanation of the command

  • r = recursive operation
  • e = compute estimated time remaining for file name
  • l = print relative paths for file name
  • “test_directory” = this points md5deep to the directory you wish to checksum
  • > results_file.md5 = is the file where all the results will be written to, you can name it whatever you want.

After md5deep is done you can open results_file.md5 to view and analyze the results.

Conclusion

md5deep is a simple tool that should be part of your arsenal after all who can complain about file integrity.

Part 2 of this tutorial: Compare Hashes With md5deep

md5deep Home page
Md5deep manual

How to generate and verify an MD5 hash on Linux

Posted on by

An MD5 hash is commonly used to compare and verify the integrity of a file. To create the MD5 hash we can use the tool MD5SUM which is included in Linux and Unix like operating systems. MD5SUM can calculate, generate, and verify MD5 hashes.

Remember if the file becomes damaged or corrupted the hash will no longer match.

Generate the hash

We want to generate an MD5 hash for the file.iso and save the result to a text file named file.iso.md5.

md5sum file.iso > file.iso.md5

Verify hash

Because we have the hash in a text file we can place the hash file and target file in the same folder then issue the following command, if the hash matches the output will be OK.

md5sum -c file.iso.md5

Or if you want to view the string of characters and do a visual comparison then use.

md5sum file.iso

And compare it with the hash file you already have.

Despite suffering from weaknesses MD5 still remains the best option when it comes to integrity verification.