Can shannon entropy be used to optimize password carcking?

[Complexoctopus]

I am pretty sure 99% of passwords are not random so I'd imagine if hashcat had a mode in which it would not wast time on hashing random passwords cracking speed would increase.

Or perhaps an AI could be developed which would be trained on existing wordlists and by prioritizing more likely password candidates over random passwords make brute forcing much more efficient.

[philsmd]

what attack type ?

hashcat has built-in support for markov chains see --help

it's probably not a good idea to have/generate password candidates that are gibberish ... and store large dictionaries that are just random data... so the "filtering" is actually often done way before the cracking starts (e.g. using leaked password lists etc)

[Complexoctopus]

what attack type ?

hashcat has built-in support for markov chains see --help

it's probably not a good idea to have/generate password candidates that are gibberish ... and store large dictionaries that are just random data... so the "filtering" is actually often done way before the cracking starts (e.g. using leaked password lists etc)

Thanks I will check it out!

I was not talking about the AI which can generate new passwords, I was talking about the AI that can filter passwords hashcat comes up with (during mask attack) before these passwords are hashed. But I guess this wouldn't be practical for fast algorithms.

I have no idea how wordlists can be helpful, none of my passwords are there and I did not even try to make complex passwords that are hard to memorize.

[blacktraffic]

I am pretty sure 99% of passwords are not random so I'd imagine if hashcat had a mode in which it would not wast time on hashing random passwords cracking speed would increase.

Or perhaps an AI could be developed which would be trained on existing wordlists and by prioritizing more likely password candidates over random passwords make brute forcing much more efficient.

People have been working on various statistical techniques - e.g. https://github.com/lakiw/pcfg_cracker , OMEN https://github.com/RUB-SysSec/OMEN and the PACK analysis tools - https://github.com/iphelix/pack 

I think it's cleaner and easier to use a candidate generator rather than doing the filtering in hashcat itself. 

In terms of Shannon entropy itself, the password Qwertyuiop{ has a reasonable entropy score, but isn't a very good password - so the measure is not ideal for the sort of thing we're working with here. (I did consider it myself, but turns out not to be terribly helpful)