09-10-2011, 01:06 AM
I don't have exact details, but you can find them in the JtR source (charset.c, charset.h).
Intuitively, it seems pretty straightforward.
People have tendencies about which characters they use to construct passwords.
Perhaps they're 100x more likely to use an "s" than a "Z".
So instead of systematically marching through the char set range (i.e. A-Z,a-z), a much more efficient search can be performed using high likelihood characters first, moving to lower likelihood characters later.
JtR has the capability to create such a charset from a dictionary - rockyou, for example. (google)
Note that the rockyou charset is almost 2MB in size, so it must have more than just frequency information - perhaps positional statistics as well.
When using a charset such as this, you get a large number of successful decrypts up front, tapering to lower numbers with elapsed time.
Intuitively, it seems pretty straightforward.
People have tendencies about which characters they use to construct passwords.
Perhaps they're 100x more likely to use an "s" than a "Z".
So instead of systematically marching through the char set range (i.e. A-Z,a-z), a much more efficient search can be performed using high likelihood characters first, moving to lower likelihood characters later.
JtR has the capability to create such a charset from a dictionary - rockyou, for example. (google)
Note that the rockyou charset is almost 2MB in size, so it must have more than just frequency information - perhaps positional statistics as well.
When using a charset such as this, you get a large number of successful decrypts up front, tapering to lower numbers with elapsed time.