Wednesday, June 27, 2012

Installing hashdog


Yesterday I gave a presentation on the SANS DFIR 2012 conference regarding cryptographic checksums and how those can be used in computer forensic. In my talk I demonstrated a tool called ‘hashdog’ which can be used to generate databases with hashes of files. Files to either look for or discard in a forensic investigation. This blog post will take you to the steps needed to install ‘hashdog’ with its dependencies.

The first thing you need to do is to download ‘hashdog’ from its github repository which can be found here; https://github.com/parosterbergmedina/hashdog. Being written in perl, ‘hashdog’ has a few perl-modules that it depends on and before continuing we have to make sure that these modules are installed. My system is running Debian stable so all examples are from that distribution – your packages might be named differently. Verify that you have the following perl-modules installed on your system by running the ‘apt-get’ command listed below.

pmedina@forensic:~/hashdog$ sudo apt-get install perl-base perl-modules libdigest-md5-file-perl libdigest-sha-perl libstring-crc32-perl
Reading package lists... Done
Building dependency tree
Reading state information... Done
libdigest-md5-file-perl is already the newest version.
libdigest-sha-perl is already the newest version.
libstring-crc32-perl is already the newest version.
perl-base is already the newest version.
perl-modules is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Digest::MD5::File

There is a bug in the perl module ‘Digest::MD5::File’ and the way it handles files that ends with a trailing space (0x20). Bug 76174 has been addressed with the release of version 0.08 but Debian stable still uses the old 0.07 version. If you are running Debian stable and you do not want to have any issues with these types of files, you have two options. You can either manually install version ‘0.08’ from testing or unstable, or just simply edit the ‘File.pm’ yourself and fix the problem by following the steps outlined below.

The first step is to verify that this bug affects you.

pmedina@forensic:~/hashdog$ echo "this is a test" > "/tmp/testfile "
pmedina@forensic:~/hashdog$ ls /tmp/ | grep testfile | xxd
0000000: 7465 7374 6669 6c65 200a                 testfile .
pmedina@forensic:~/hashdog$ md5sum "/tmp/testfile "
e19c1283c925b3206685ff522acfe3e6  /tmp/testfile
pmedina@forensic:~/hashdog$ perl -MDigest::MD5::File=file_md5_hex -e 'print file_md5_hex("/tmp/testfile ") ."\n"';

pmedina@forensic:~/hashdog$

If the perl command did not produce a MD5 hash, we will patch the ‘File.pm’ file and verify that our fix worked.

pmedina@forensic:~/hashdog$ cat /tmp/patch
53c53
<         open my ($fh), $file or return;
---
>       open (my $fh,"<", $file) or return;
pmedina@forensic:~/hashdog$ patch /usr/share/perl5/Digest/MD5/File.pm -i /tmp/patch -o /tmp/updated
patching file /usr/share/perl5/Digest/MD5/File.pm
pmedina@forensic:~/hashdog$ sudo mv /tmp/updated /usr/share/perl5/Digest/MD5/File.pm
pmedina@forensic:~/hashdog$ chmod +r /usr/share/perl5/Digest/MD5/File.pm
pmedina@forensic:~/hashdog$ perl -MDigest::MD5::File=file_md5_hex -e 'print file_md5_hex("/tmp/testfile ") ."\n"';
e19c1283c925b3206685ff522acfe3e6

7-Zip

To test if a file is an archive and can be unpacked, ‘hashdog’ uses the program 7-Zip. 7-Zip, which has been ported to Linux as p7zip, is currently at version 9.20 while Debian stable still uses version 9.04. Even though most of the functionality in ‘hashdog’ will still work using that old version of pkzip, support for more archives has been added in later versions so upgrading might be a good idea. To use the versions included in Debian stable, simply install the packages ‘p7zip-full’ and ‘p7zip-rar’ using ‘apt-get’. This will automatically solve any dependencies as well.

pmedina@forensic:~/hashdog$ sudo apt-get install p7zip-full p7zip-rar
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  p7zip-full p7zip-rar
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/1,454 kB of archives.
After this operation, 3,867 kB of additional disk space will be used.
Selecting previously deselected package p7zip-full.
(Reading database ... 43741 files and directories currently installed.)
Unpacking p7zip-full (from .../p7zip-full_9.04~dfsg.1-1_amd64.deb) ...
Selecting previously deselected package p7zip-rar.
Unpacking p7zip-rar (from .../p7zip-rar_9.04~ds.1-1_amd64.deb) ...
Processing triggers for man-db ...
Setting up p7zip-full (9.04~dfsg.1-1) ...
Setting up p7zip-rar (9.04~ds.1-1) ...
pmedina@forensic:~/hashdog$ 7z | grep -i zip
7-Zip 9.04 beta  Copyright (c) 1999-2009 Igor Pavlov  2009-05-30
p7zip Version 9.04 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,1 CPU)
  -so: write data to stdout (eg: 7z a dummy -tgzip -so Doc.txt > archive.gz)

If you want to use version 9.20 of p7zip, the packages needs to be downloaded and installed manually. My forensic workstation is running Debian for the AMD64 platform so the packages I will be downloading and installing are compiled for that specific platform. Do a search for ‘p7zip’ and ‘p7zip-rar’ on http://packages.debian.org if you want to find packages for a different version.

pmedina@forensic:~/hashdog$ wget --no-verbose http://http.us.debian.org/debian/pool/non-free/p/p7zip-rar/p7zip-rar_9.20.1~ds.1-3_amd64.deb
2012-06-27 20:33:21 URL:http://http.us.debian.org/debian/pool/non-free/p/p7zip-rar/p7zip-rar_9.20.1~ds.1-3_amd64.deb [57870/57870] -> "p7zip-rar_9.20.1~ds.1-3_amd64.deb" [1]
pmedina@forensic:~/hashdog$ md5sum *.deb
deb71878cd75b0e7d70f1b62e08a77c7  p7zip-full_9.20.1~dfsg.1-4_amd64.deb
85ad60c37ebfc2bf8d7dc4cc7d26e6cc  p7zip-rar_9.20.1~ds.1-3_amd64.deb
pmedina@forensic:~/hashdog$ sudo dpkg --install p7zip-full_9.20.1~dfsg.1-4_amd64.deb p7zip-rar_9.20.1~ds.1-3_amd64.deb
(Reading database ... 43811 files and directories currently installed.)
Preparing to replace p7zip-full 9.04~dfsg.1-1 (using p7zip-full_9.20.1~dfsg.1-4_amd64.deb) ...
Unpacking replacement p7zip-full ...
Preparing to replace p7zip-rar 9.04~ds.1-1 (using p7zip-rar_9.20.1~ds.1-3_amd64.deb) ...
Unpacking replacement p7zip-rar ...
Setting up p7zip-full (9.20.1~dfsg.1-4) ...
Processing triggers for man-db ...
Setting up p7zip-rar (9.20.1~ds.1-3) ...
pmedina@forensic:~/hashdog$ 7z | grep -i zip
7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,1 CPU)

Hashdog

Now all the dependencies of ‘hashdog’ should be taken care of and you should be able to start using the program without any issues. To verify this, simply execute the program.

pmedina@forensic:~/hashdog$ ./hashdog.pl
[*] hashdog.pl version: 0.71 written by Par Osterberg Medina
specify a file or directory to process with '--input'
usage: hashdog.pl [--input] [--md5sum-file|--sha1sum-file|--rds-file] {options}

        --input|-i {file/dir}    file or directory to process
        --md5sum-file {file}     generate a file with md5 checksums
        --md5sum-fullpath        use full file paths in the file with md5 checksums
        --sha1sum-file {file}    generate a file with sha-1 checksums
        --sha1sum-fullpath       use full file paths in the file with sha-1 checksums
        --rds-file {file}        generate a file with checksums using the RDS format
        --rds-fullpath           use full file paths in the RDS file
        --archive-bin {file}     path to the 7-Zip binary
        --archive-skip {list}    comma sperated list of archive types to not expand
        --min-filesize {number}  minimun filesize in bytes to process
        --tmp                    specify the the tmp folder to use
        --verbose|-v             verbose output
        --debug|-d               turn on debug output
pmedina@forensic:~/hashdog$



No comments:

Post a Comment