Adding PCFGs to Hashcat's Brain

[lakiw]

The five functions in general are the following (there's no template yet so I'm more like thinking loud here):

• sc_pcfg_init - A function which resets all internal structures of the generator as it would be started freshly from the commandline. It will also provide the mandatory and optional parameters a user can specify in a struct. It will return a context to work with. The context enables multi threading functionality.
  
• sc_pcfg_keyspace - A function which simply returns the total number of candidates which the generator will create based on the parameter configuration. If the total number is unknown this has some disadvantages. For instance, the ETA can not be computed or it may not be possible to distribute it via hashtopolis. In this case return (u64) -1 and hashcat will assume the generator will give a negative returncode in the seek/next function (explained next).
  
• sc_pcfg_seek - Seek to a specific candidate position. This is mandatory, the parameter will be just a number. Will also have a returncode if there's no such position
  
• sc_pcfg_next - Output the next password candidate (based on the context)
  
• sc_pcfg_shutdown - A cleanup function
  

If you can agree to this, I'll formalize the structure in a C header and you can extend it with PCFG parameters you need.

- atom

Those five functions sound very reasonable and I understand your need for them. "seek" and "keyspace" will require a dramatically different way of generating guesses from a grammar that will mean that guesses are no longer generated in probability order. That's not a showstopper but I just want to highlight there will be trade-offs. 

Long story short, I'll need some time to think about how to create an efficient index function. Providing a generic template using the above functions would probably help with integrating other guess generators though so designing a repeatable manner to provide them will likely help for adding other non-pcfg attack modes in the future. 

As dumb as it sounds, one hesitation I have is for researchers thinking the "index compatible" version of PCFGs is the same as a probability order guess generator. By definition, if we don't use probability order for generating guesses, the resulting guess generator will be less precise/accurate. Maybe we could come up with a new name for the algorithm we use in Hashcat. I'm open to suggestions!

There's a general misunderstanding here. There's no need to add code to the brain if you're adding a new generator to hashcat.

As crazy as it sounds, I'm mulling adding support for using a hashcat brain server into the python pcfg cracker. One thing I've struggled with in the past is once you start a cracking session with the current PCFG, you are kind of locked in to running it to completion. Having the ability to not re-hash previously made guesses or attacks would be really useful.

One step at a time though. I'll start looking into different ways to provide those five functions to integrate PCFGs into Hashcat's slow guess generator. As Matlink mentioned there may also be use in adding support for guess generators to Hashcat that don't have an index function. As he pointed out, if you want to generate guesses in probability order with PCFGs, there's no way I can think of to provide a practical index function.