List of binary (TC) hashes / John format for TC?

[Progatiko]

I am trying to find truecrypt header in a dumped, partially damaged backup partition. Password is known.

I could create ~100M chunks of data 512 bytes each. Now I am looking for a possibility to try them all in the fastest possible way. I wrote a multi-threaded script that uses native truecrypt, but it'll take years...

Hashcat seems to be way faster with my test headers, but starting a separate process for each chunk is inefficient (and will again take years). Hashcat doesn't accept directories of files as a source, nor could I find a way to combine multiple binary hashes in single source file.

John the Ripper has a small script that hexifies headers. I could use it to store my TC header as a hash string, but it seems hashcat doesn't accept it for the mode 6233 which I need. And John the Ripper itself doesn't seem to support cascaded TC algorithms, one of which was used in my case (mode 6233 successfully decodes test header of similar container, but john the ripper fails).

I'd be glad for any ideas how can I proceed...

[Progatiko]

I'd really appreciate if somebody can point me in right direction how can I achieve the required behavior...

[Banaanhangwagen]

I do not understand what you are trying to do, since you have knowledge of the password.
You want to test if Hashcat finds the known pwd? If yes, follow these steps: https://hashcat.net/wiki/doku.php?id=fre...pt_volumes

You do not need the JtR script. Hashcat accepts a file/container as input. (Out of convenience, one cuts out the correct 512 bytes from the file/container/partition; that is enough for Hashcat to work on)

[Progatiko]

I do not understand what you are trying to do, since you have knowledge of the password.
You want to test if Hashcat finds the known pwd? If yes, follow these steps: https://hashcat.net/wiki/doku.php?id=fre...pt_volumes

You do not need the JtR script. Hashcat accepts a file/container as input. (Out of convenience, one cuts out the correct 512 bytes from the file/container/partition; that is enough for Hashcat to work on)

As I mentioned above I have a damaged backup dump where the filesystem is in chunks. Inside that dump there  is a 200GB truecrypt file container. Since there are just a few files besides it in the dump, I could roughly identify the beginning and the end of the file (corresponding chunks), but not exactly the needed 512 bytes (either of primary or backup TC header). My intention is to check a few millions of 512 chunks with the password I know to find the correct position of the file start (or, well end in case of backup header).

[Banaanhangwagen]

What is your ultimate goal? Opening the TC-container and recovering the files inside, I suppose.

Without being an expert on this domain, I suggest you should consider the following:
- If the filesystem is broken (correct ?), one can only recover files in it by searching for their headers; since a TC-container does not have a header, that is going to be a difficult one
- you could look at entropy, to make a more educated guess
- If I understand it correctly, you want to try each sector as if it was a TC-header;
- Since you are talking about 200GB, are we sure that the file is on a continuous way saved by the file system? Or is it fragmented? If it is fragmented, I won't even bother to try to recreate the container

[Progatiko]

What is your ultimate goal? Opening the TC-container and recovering the files inside, I suppose.

Without being an expert on this domain, I suggest you should consider the following:
- If the filesystem is broken (correct ?), one can only recover files in it by searching for their headers; since a TC-container does not have a header, that is going to be a difficult one
- you could look at entropy, to make a more educated guess
- If I understand it correctly, you want to try each sector as if it was a TC-header;
- Since you are talking about 200GB, are we sure that the file is on a continuous way saved by the file system? Or is it fragmented? If it is fragmented, I won't even bother to try to recreate the container

Well... Without calling myself an expert, I'd say I have some experience in forensics...

In this particular case I am working with a partial dump of borgbackup program. I could decrypt and dump it in sequential order of chunks... The backup itself should contain four TC containers, it's their pure data, no filesystem involved at all (at that layer).

I could successfully decrypt the first TC container (also 200Gb), since it started right at the beginning of the dump. It also proved that once I'll find the header, I have good chances of getting the content (at least partially with continuous sequence of chunks).

My problem is, that according to brogbackup docs (and I spoke with maintainers too) there is some metadata in between and there also can be another small TC container which size is known roughly and password not at all.

All together it results in the problem I described: I know very roughly where my target TC container starts (ultimate goal) and at maximum need to test ~100M binary TC hashes of 512b each. And I need some instrument that will allow me to feed them to hashcat (or other fast tool).

[philsmd]

I agree with @Karamba that an entropy check could be useful (btw: hashcat already does this internally https://github.com/hashcat/hashcat/blob/...#L222-L224 , but if you think it would reduce your list of chunks a lot, you could do it externally to avoid looking at too many "files").

hashcat unfortunately only allows to load 1 TrueCrypt "hash" at any time... that's normally not a problem, but a little bit of a bummer in your very specific situation (when you need to test a lot of different TrueCrypt containers "automatically").

I think it's a good strategy to pre-filter them (for instance only use 512 byte blocks that have enough entropy and maybe even rank/priorize them depending on their location on the disk, on the likelihood of a good candidate block etc etc) and then just run a bash/shell/batch script over it, something like this:

Code:

for f in chunks/*; do for t in 6213 6223 6233; do ./hashcat -m $t $f dict.txt; done; done | tee always_track_progress.log

you could even exit the loops when hashcat returns a success, e.g.

Code:

ret=$?
if [ "$ret" -eq 0 ]; then break; fi

all together:

Code:

for f in chunks/*; do for t in 6213 6223 6233; do ./hashcat -m $t $f dict.txt; ret=$?; if [ "$ret" -eq 0 ]; then break; fi; done; if [ "$ret" -eq 0 ]; then break; fi; done | tee always_track_progress.log

Also the loop over all hash types 6213, 6223 and 6233 is NOT needed if you know exactly which hashing algorithm was used when creating the TrueCrypt container (RIPEMD160 vs SHA512 vs Whirlpool).

This also assumes that no boot-mode was used (otherwise you would need to only use -m 6243 for hashcat).

Of course, this might not be the fastest strategy, because it needs to initialize/start hashcat again and again and again.... the time needed for cracking might be much smaller than the speed of starting and initializing the devices. To optimize this you could also play around with other options in hashcat, like -D 1 (only use CPU) or select only one device with -d 1, or use --backend-ignore-cuda etc etc

In this very specific situation it could also be faster to just develop and test a very special perl/python/php etc script that does everything for you (testing the entropy and also trying to decrypt the data similar to what hashcat does internally, but this might involve much more work for you).

I think the ranked/priorized testing of the blocks within a shell loop could be quite fast/feasible (especially because hashcat does an entropy check and therefore rejects a lot of garbage blocks etc).

[Progatiko]

@philsmd, thank you very much for such a detailed answer! Especially for your effort writing the shell script examples and further hashcat launch optimization ideas. It would be great if hashcat will have required possibility in later releases, but at the moment it seems to be a useful workaround.

I doubt entropy check will reduce the required amount significantly. My past experience with truecrypt shown that the chunks inside the container body have enough entropy to be a theoretical header most of the time. Anyways I gonna script it and see what reduction I can achieve. Do you have any particular recommendation for external tool that you would use in my case? I am thinking about going the python way