Tuesday, September 28, 2010
One of Those Uh-Oh Moments - Recovering a Digital Needle From a Digital Haystack
Well, what to do. The easy way out would be to swallow my pride, rebuild the virtual machine, create a new secret key, and email the new public key to my correspondent and ask him to reencrypt the original message and resend it. The harder, but more educating, way out of the problem would be to recover the file. The data was likely still on the drive, the pointers to that data were likely gone or corrupted at best. I had nothing to lose by trying and I needed to hone my data recovery/ digital forensics skills. So, using dd and ssh via a script I wrote about six years ago, I created a bit by bit image of the virtual hard drive on another system. I loaded the image file into autopsy and the results weren't good. Autopsy would only be able to perform keyword searches on the image. The data layer really couldn't be seen or analyzed. I tried a few keyword searches, but it was futile. I called it a night at that point and went to bed.
The next morning, I decided the best approach would be to go with scalpel or some other data carver. Data carvers search for files using the file's header and footer information. Once the file is recovered, you just rename it to what you called it previously and you are good to go. Searching for "secring.gpg magic bytes, header, scalpel.conf" on Google led me nowhere. So, I generated two new public and private keys on my laptop and an Ubuntu virtual machine and compared their headers and footers. Each file had a slightly different header and footer.
I could tell you what I tried, but I'll give you the results. I created four new gpg entries in scalpel.conf, and two of them worked perfectly.
gpg with header "\x95\x03\xbe" and footer "\xb0\x02\x00\x00" --> 1 files
gpg with header "\x99\x01\x0d" and footer "\xb0\x02\x00\x03" --> 3 files
As you can see above, I could have used a longer header sequence by appending \x04\x4c or \x04\x4c\xa2 to each header sequence. The longer the sequence, the fewer results one gets. This cuts down on false positives, but it may result in no files being recovered at all. One just has to do some experimental runs with scalpel to tune the scalpel.conf configuration file. In this case, I only wanted gpg files, so I commented out every entry except the newly created gpg entries. After making a copy of the image, I split the image into 10GB files with split, i.e.
split -b 10GB livecd
which results in three 9.3GB files called xaa, xab, xac, and a 2.1GB file called xad. Running scalpel on the 30GB file resulted in an estimated 28 hour second pass scan. Narrowing the search to just gpg files and scanning 9.3GB files resulted in a complete file carving (2 passes) within 20 minutes per 9.3GB file. The gpg files were recovered from the xaa subfile.
I rebuilt the corrupted virtual machine once I had the recovered files. I created a .gnupg directory within the user's home directory and copied the pubring.gpg and secring.gpg files into that directory. Using apt-get, you can install thunderbird and the enigmail xpi plugin seamlessly:
apt-get install thunderbird enigmail gnupg (if gnupg isn't installed).
You can test gnupg like so:
gpg --list-keys (for querying keys stored in the public keyring), and
gpg --list-secret-keys (for keys stored in the secret keyring).
Then you start thunderbird and give it your name, email address, and password. It's smart enough to set up your gmail server settings for you. You just give enigmail your secret key passphrase and you can decrypt any messages encrypted using your keys.
I could have searched for the pubring.gpg using my email address since it is embedded in the public key using grep:
grep -r "email@example.com" /scalpelresults/
will search all of the carved files for that text expression. To eliminate carved emails, pipe the output to grep with the v option like so:
grep -r "firstname.lastname@example.org" /scalpelresults/ | grep -v "To: John Moore" | grep -v "From:
John Moore" | grep -v "Return-Path"
but fortunately, just browsing for the 4 files listed in the audit.txt file worked, since scalpel placed them in their own subdirectories.
Links to this post: