09-23-2019, 04:15 AM
The newest version of the compiled PCFG is finally generating password guesses! There's a ton of features/improvements I still want to make to it, (the top one is I'm pretty sure I'm horribly handling non-ASCII UTF-8 characters), but it's at the point where end to end testing can be conducted.
To generate guesses with the code in the repo, generate/copy a ruleset/grammar using the Python PCFG toolset, and then copy the entire "Rules" folder into the compiled_pcfg directory. Yes, better figuring out how to set up the PCFG trainer to support multiple projects is also on my to-do list.
I was running into challenges using it to pipe guesses into hashcat using the following command:
<Path omitted>/pcfg_guesser | ./hashcat64.exe -a 0 -m 100 ../../research/password_lists/hashcat_fmt_test_list.hsh
But that may be because I was running it under windows subsystem for Linux under "Ubuntu", so weirdness can pop up. The error was that it seemed like the pcfg_guesser died/stopped generating guesses while Hashcat was still starting up. If I cat a dictionary file in instead, hashcat will crack passwords. This probably points to an area I need to dig into on my end to handle blocking output gracefully. Or it could be something entirely else ¯\_(ツ)_/¯. What I really need to do is simply test it out on a real Linux system.
I was able to do some initial testing using John the Ripper though.
./pcfg_guesser | ../JohnTheRipper/run/john --stdin -format=Raw-MD5 ../../research/password_lists/test_list.txt
Side note: You can invoke JtR's status output when piping in guesses via stdin using the following command:
kill -SIGUSR1 <PID of JTR>
Doing this, I was getting about 4 to 5 million guesses a second. As an example:
0g 0:00:03:17 0g/s 5253Kp/s 5253Kc/s 183066MC/s jessykira3..liberty@2009
It does slow down the longer you run a cracking session, but so far I haven't run one using the compiled PCFG for more than an hour. That being said, this is about 20 times faster than the Python PCFG guesser so that's a huge improvement!
Next on my to-do list:
1) Run the compiled PCFG on a different computer. If issues still arise with piping guesses into Hashcat, dig into that.
2) Start abstracting the functions listed earlier that will be used to integrate this into other programs. Aka sc_pcfg_next, sc_pcfg_init, etc.
3) Testing, and more testing. For example I want to make sure it is generating the same guesses as the Python PCFG.
To generate guesses with the code in the repo, generate/copy a ruleset/grammar using the Python PCFG toolset, and then copy the entire "Rules" folder into the compiled_pcfg directory. Yes, better figuring out how to set up the PCFG trainer to support multiple projects is also on my to-do list.
I was running into challenges using it to pipe guesses into hashcat using the following command:
<Path omitted>/pcfg_guesser | ./hashcat64.exe -a 0 -m 100 ../../research/password_lists/hashcat_fmt_test_list.hsh
But that may be because I was running it under windows subsystem for Linux under "Ubuntu", so weirdness can pop up. The error was that it seemed like the pcfg_guesser died/stopped generating guesses while Hashcat was still starting up. If I cat a dictionary file in instead, hashcat will crack passwords. This probably points to an area I need to dig into on my end to handle blocking output gracefully. Or it could be something entirely else ¯\_(ツ)_/¯. What I really need to do is simply test it out on a real Linux system.
I was able to do some initial testing using John the Ripper though.
./pcfg_guesser | ../JohnTheRipper/run/john --stdin -format=Raw-MD5 ../../research/password_lists/test_list.txt
Side note: You can invoke JtR's status output when piping in guesses via stdin using the following command:
kill -SIGUSR1 <PID of JTR>
Doing this, I was getting about 4 to 5 million guesses a second. As an example:
0g 0:00:03:17 0g/s 5253Kp/s 5253Kc/s 183066MC/s jessykira3..liberty@2009
It does slow down the longer you run a cracking session, but so far I haven't run one using the compiled PCFG for more than an hour. That being said, this is about 20 times faster than the Python PCFG guesser so that's a huge improvement!
Next on my to-do list:
1) Run the compiled PCFG on a different computer. If issues still arise with piping guesses into Hashcat, dig into that.
2) Start abstracting the functions listed earlier that will be used to integrate this into other programs. Aka sc_pcfg_next, sc_pcfg_init, etc.
3) Testing, and more testing. For example I want to make sure it is generating the same guesses as the Python PCFG.