Can shannon entropy be used to optimize password carcking?

[blacktraffic]

I am pretty sure 99% of passwords are not random so I'd imagine if hashcat had a mode in which it would not wast time on hashing random passwords cracking speed would increase.

Or perhaps an AI could be developed which would be trained on existing wordlists and by prioritizing more likely password candidates over random passwords make brute forcing much more efficient.

People have been working on various statistical techniques - e.g. https://github.com/lakiw/pcfg_cracker , OMEN https://github.com/RUB-SysSec/OMEN and the PACK analysis tools - https://github.com/iphelix/pack 

I think it's cleaner and easier to use a candidate generator rather than doing the filtering in hashcat itself. 

In terms of Shannon entropy itself, the password Qwertyuiop{ has a reasonable entropy score, but isn't a very good password - so the measure is not ideal for the sort of thing we're working with here. (I did consider it myself, but turns out not to be terribly helpful)