hashcat Forum
On-the-fly loading of gz wordlists - Printable Version

+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Support (https://hashcat.net/forum/forum-3.html)
+--- Forum: hashcat (https://hashcat.net/forum/forum-45.html)
+--- Thread: On-the-fly loading of gz wordlists (/thread-9248.html)



On-the-fly loading of gz wordlists - Mem5 - 05-23-2020

Quote:* changes v5.1.0 -> v6.0.0
(...)
- Support on-the-fly loading of compressed wordlists in zip and gzip format
Seen in hashcat beta 1803.

Just tried with a gz file:

Code:
$ wc -l  dic*
314'265 dic.gz
19'487'556 dic

Hashcat sees:
Code:
Dictionary cache built:
* Filename..: dic.gz
* Passwords.: 314266
* Keyspace..: 205218

Progress.........: 205218/205218 (100.00%)

Edit:
Code:
$ file dic.gz
dic.gz: gzip compressed data, last modified: Sun Apr 12 05:01:34 2020, from Unix


Looks like the file has not been unzipped?

Thanks.


RE: On-the-fly loading of gz wordlists - royce - 05-23-2020

WFM:

Code:
$ hashcat --version
v5.1.0-1789-gc7da6357

$ file rockyou.txt.gz
rockyou.txt.gz: gzip compressed data, was "rockyou.txt", last modified: Thu Jul  7 14:27:39 2016, from Unix

$ wc -l rockyou.txt.gz
213432 rockyou.txt.gz
$ gunzip -cd rockyou.txt.gz | wc -l
14344391

$ cat brandon.hash
fc275ac3498d6ab0f0b4389f8e94422c

$ hashcat -m 0 -w 4 -O -D 2 brandon.hash rockyou.txt.gz --potfile-disable
[...]
Dictionary cache built:
* Filename..: rockyou.txt.gz
* Passwords.: 14344391
* Bytes.....: 139921497
* Keyspace..: 14344384
* Runtime...: 3 secs
[...]
fc275ac3498d6ab0f0b4389f8e94422c:brandon



RE: On-the-fly loading of gz wordlists - Mem5 - 05-23-2020

Weird.. I may have failed somewhere in the compress step..


RE: On-the-fly loading of gz wordlists - Mem5 - 05-24-2020

From a performance perspective I cannot see any loss of speed; how does hashcat handle gigas of compressed wordlist without spending minutes/hours on decompression?


RE: On-the-fly loading of gz wordlists - royce - 05-24-2020

I'm not sure, but I think that decompressed data can be used "on the fly" - as soon as it's decompressed, without having to wait for the entire file to decompress.


RE: On-the-fly loading of gz wordlists - Mem5 - 05-25-2020

Ok.. but what about "Dictionary cache building", hashcat have to decompress all the file first to know the statistics, number of passwords etc?


RE: On-the-fly loading of gz wordlists - royce - 05-25-2020

Yes, that seems right. It would have to decompress the whole thing first, enough to analyze the statistics and then cache them. so I assume there would be some duplicated work, just like dictionary cache building without compression.