The PCFG attack (-a 4) guesses passwords in order of how likely they are. PCFG stands for Probabilistic Context-Free Grammar, which is a long name for a fairly simple idea.
Real passwords have shapes people keep reusing. monkey12, nicole2010, soccer! and Daniel99 are all the same idea with different parts filled in. A PCFG splits a password into two questions:
monkey12 is a six-letter word followed by two digits.monkey and the digits are 12.
A trained ruleset assigns probabilities to both parts. It records that “six letters then two digits” is a common shape, that monkey is a common six-letter word, and that 12 is a common pair of digits. Multiply the three numbers together to get the probability of monkey12.
A wordlist supplies candidates in file order without indicating which are most likely. A PCFG assigns probabilities, so the most likely guesses come first.
You can see it straight away. This is the real start of a default -a 4 run:
$ hashcat -a 4 --stdout 123456 1234567 123123 12345 123456789 123321 password 12345678 5201314 7777777 1234 1234567890 112233 159753 111111 654321 000000 123654 666666 qwerty
The grammar produces that order automatically.
hashcat ships default-passwords, trained on passwords, and uses it when you name none, so the short form is all you need to get going:
$ hashcat -m 0 -a 4 example0.hash
To use a ruleset of your own, name it after the hash file:
$ hashcat -m 0 -a 4 example0.hash /path/to/ruleset
Rules work too, and apply to the candidates the grammar produces:
$ hashcat -m 0 -a 4 example0.hash -r rules/best66.rule
A ruleset is just a directory of plain text files. Each line is a value, a tab, and a probability.
$ ls ruleset Alpha Capitalization Context Digits Grammar Keyboard Omen Other Years
Grammar/grammar.txt holds the shapes, most likely first:
M 0.40004227816694393 D6 0.051105353916786035 A6D2 0.037839499289255306 A6 0.037156138225371546 A7 0.025911933402529241
D6 means six digits. A6 means six letters. A6D2 means six letters then two digits, which is the shape of monkey12. The letter names which directory the parts come from, and the number is both a length and a file name, so A6 means Alpha/6.txt:
| letter | directory | what it holds |
|---|---|---|
A | Alpha | words, always stored lowercase |
C | Capitalization | which letters of the word before it are uppercase |
D | Digits | runs of digits |
O | Other | runs of symbols |
K | Keyboard | keyboard walks like qwerty or 1qaz2wsx |
X | Context | things that do not fit the others |
Y | Years | four digit years |
Case is handled separately. Words in Alpha are always lowercase, and every A token carries a matching C token that defines which letters to uppercase afterwards. LLLLLL leaves the word unchanged. ULLLLL capitalises the first letter. That is why monkey and Monkey are one entry rather than two.
Token M is the other half of the model, a Markov chain rather than a grammar shape.
Name several rulesets and hashcat merges them, splitting the run evenly between them by default:
$ hashcat -m 0 -a 4 example0.hash default-passwords my-russian
An even split will not always suit your target. weights= changes the share each one gets:
$ hashcat -m 0 -a 4 example0.hash default-passwords my-russian weights=2:1
Merging is worth doing because two rulesets trained on different material can reach candidates that neither reaches alone. It also costs you if you get the weights wrong, because half the run goes on candidates that were never likely.
Applies to master. A keyboard walk is a password typed as a path across the keys rather than spelled out, 1qaz or 1q2w3e4r. What the path produces depends on the layout under the fingers. The path that gives 1qaz on a US keyboard gives 1qay on a German one and a Cyrillic string on a Russian one. default-passwords was trained on English leaks, so the walks in it are the ones a US keyboard produces, and against somebody who types on another layout those are the wrong strings.
After 8.0.0 hashcat ships 10 rulesets that carry the same walks on other layouts, built from kwprocessor's keymaps: keyboard-walks-english, -russian, -arabic, -french, -czech, -greek, -hebrew, -persian, -thai and -vietnamese. Each holds walks and no words at all, so merging one in adds that layout's walks and replaces none of what it is merged with:
$ hashcat -m 0 -a 4 example0.hash default-passwords keyboard-walks-russian weights=9:1
On its own a walk ruleset shows what it carries:
$ hashcat -a 4 --stdout keyboard-walks-russian | head -8 ф1234 ф12345 1йфя ф123456 2цыч "1234 1й2ц3у4к "12345
Every walk that default-passwords knows is read on the target layout and keeps the probability the training measured. A path the layout cannot produce is left out, and a tail of further generated walks ranks below the trained ones. keyboard-walks-english repeats the US layout that default-passwords already covers, so it is there for a run that merges the walks with a ruleset of your own rather than with the one hashcat ships.
When you know the shape of the password but not the word, mask= takes a mask in -a 3 syntax and keeps only the candidates the grammar spells that fit it:
$ hashcat -m 0 -a 4 example0.hash mask=?u?l?l?l?l?d?d?d
This is usually a far better attack than the same mask under -a 3, because the grammar rules most of the shapes out before the run starts and orders what is left. The mask above is 40 times smaller than the brute force, and a looser one saves more. The mask attack page has the figures and the two caveats, of which the important one is that a masked run has no OMEN escape.
This is the part that surprises people. The mask is applied to the grammar before the run rather than to the candidates after it. hashcat walks the terminal lists byte position by byte position, keeps only the entries the mask admits where that position falls, and drops any shape whose layout cannot satisfy the mask at all. What survives is then ordered by probability over the survivors alone.
So a literal in the middle of a mask does not select passwords that have anything to do with the literal. It selects the likeliest passwords that happen to carry those bytes in that place. Pin red at positions 3 to 5 and the color plays no part in the result:
$ hashcat -a 4 --stdout mask=?a?ared?a?a | head paredes heredia firedog boredom jared12 jared11 Paredes firedup zgredek coreduo
Look at where the letters fall rather than at the words. paredes, heredia, firedog and boredom are seven letter words carrying r, e and d in those three places. jared12 is a different shape entirely, a five letter word and two digits, where the same three bytes end the word instead of sitting inside it. Paredes is the capitalized form of a word already further up the list. One mask reached all three, because each was filtered inside its own shape rather than tested against a finished string.
A known fragment is useful even if you do not know the surrounding characters. Fixing three bytes constrains the possible shapes, which in turn constrains the words around them. The grammar orders the remaining candidates by the probability of the whole password given the known fragment. Under -a 3, the same mask gives 81,450,625 candidates in Markov order. Here it gives 17,876,251, most likely first.
Applies to master. Sometimes what you know is a fragment and not its position: a year of birth, four digits of a PIN you saw typed, a name. contains= arrived after 8.0.0 and holds the grammar to the candidates that carry the fragment somewhere:
$ hashcat -a 4 --stdout contains=1987 | head 19871987 19871987 198700 121987 111987 321987 231987 1987123 1987 198765
The run is the one without the setting, minus every candidate that lacks the fragment, at the same cost per candidate. No probability is renormalised. What it buys is the size of the run: the bundled ruleset goes from 34,935,457,766,817,424 candidates to 8,288,273,198,779, which --total-candidates will print for you.
The fragment is read as a mask by the same parser mask= uses, and each place it could stand is filtered as described above, with every other position left open. So everything a mask means applies to the fragment. contains=19?d?d asks for any year of the 1900s, -1 through -8 define your own charsets, and a literal matches byte for byte, so contains=tom will not admit Tom while -1 tT contains=?1om admits both.
Several fragments, comma separated, must all be present, in any order, anywhere, and each on bytes of its own:
$ hashcat -a 4 --stdout contains=1987,tom | head tommy1987 tomas1987 tom1987 tomi1987 tomek1987 tomtom1987 tomato1987 tomcat1987 tomtom1987 phantom1987
That run holds 13,655,775,190 candidates, and adding a third fragment with contains=1987,tom,123 brings it to 135,010,767. Each fragment filters what the ones before it left, at the places they left free, so contains=93,31 admits 9331 and 3193 but not 931, where the two would have to share the 3.
Every place a fragment can stand costs the filtered grammar a terminal list, and short fragments on a large ruleset can stand almost anywhere. There is a cap, and hashcat refuses the run rather than exhausting memory:
$ hashcat -a 4 --stdout contains=a,b contains a,b: filtering this grammar needs more than the 4096 terminal lists a ruleset can hold
A single fragment of one character is fine on the bundled ruleset. Two of them are not, so give each fragment the characters you actually know. A comma inside a fragment is written with a charset, as in -1 , contains=a?1b, and naming the same fragment twice is refused.
A candidate that holds a fragment twice comes out twice. Both lists above show it: 19871987 is there two times under contains=1987, and tomtom1987 two times under contains=1987,tom. Each place is filtered on its own, so a candidate is produced once for every way it can hold the fragments apart, four times for 19871987 under contains=19,87. Within one cost level the candidates of a shape come out place by place, so their order can differ from the same run without the setting, while the order across cost levels is unchanged.
Like a mask, contains= drops the OMEN escape, because the escape has no terminals to filter. It cannot be combined with mask=, which already fixes every position, so write the characters into the mask where they belong. It does work with pwmin, pwmax, hint words and lookup=, and like a mask it cannot go with hintaccount.
hashcat does not train rulesets. That is a separate job, done with pcfg_cracker, and hashcat reads its ruleset format directly.
hashcat also does not ship a ruleset trained on ordinary language, because the text you would train it on carries a licence of its own. If you want one, train it yourself from material you are allowed to use.
The ruleset hashcat ships is trained on passwords, and those are mostly English. If the people whose passwords you are testing write in another language, a grammar trained on that language can include words absent from the default one.
hashcat does not ship such a ruleset, because the text you would train it on carries a licence of its own. So you build it yourself, and it is less work than it sounds.
To automate the process, use wiki-pcfg-pipeline for the first three steps below. It uses Wikipedia rather than the corpus in this walkthrough. It streams a language's dump, writes both a plain wordlist and a count-weighted one, and trains the grammar with pcfg-go's pcfg_trainer -p. The resulting ruleset works with -a 4. It provides two features absent from this walkthrough. First, segmentation is chosen per script. A plain \w+ split breaks Indic words at their vowel signs and separates Hebrew points. Second, it segments words in Chinese, Japanese and Thai, which are written without spaces. It also includes kbd_translit.py to remap a wordlist or ruleset by physical key position across 13 layouts. This helps with passwords typed under the wrong keyboard layout. The project is new and small, so read it before running it.
The walkthrough below stays worth reading if you want a different corpus, or want to see what each step does.
The Leipzig Corpora Collection publishes word lists for hundreds of languages, free to download. Pick a language, a source such as news or web, a year, and a size. The archives are named after those choices:
$ wget https://downloads.wortschatz-leipzig.de/corpora/deu_news_2023_10K.tar.gz $ tar xzf deu_news_2023_10K.tar.gz
Start small. A 10K corpus trains in well under a second and is enough to see whether the idea helps at all. The bigger sizes cost correspondingly more.
The file you want inside is the one ending in -words.txt. Each line holds an id, the word, and how often it appeared, separated by tabs:
1 ! 132 2 " 883 167 wieder 231 168 hatte 216
The low ids are punctuation rather than words, as you can see.
A trainer reads one entry per line and can also take a count before each entry. This is useful here because the corpus includes the frequency of every word.
This keeps the lines whose word holds a letter, drops the punctuation, and writes them as count:word:
$ awk -F'\t' '$2 ~ /[[:alpha:]]/ && $2 !~ /[[:space:]]/ {print $3":"$2}' \
deu_news_2023_10K-words.txt > deu_train.txt
$ head -3 deu_train.txt
4465:der
4170:die
3335:und
Pick one of the trainers from the list below. Any of them will do this, and all of them are far quicker than the original Python one.
$ git clone https://github.com/matrix/hpcfg $ cd hpcfg $ make $ ./hpcfg -t deu_train.txt -g deu_ruleset -w hpcfg: 32.764 lines, 149.549 passwords, 28.712 terminals, 12.117 OMEN contexts
-w indicates that the lines carry a count before each entry. The separator defaults to :, and -s changes it if your data uses something else.
It works the encoding out for itself, so a corpus full of umlauts, accents or Cyrillic needs no extra flag.
-i shows you what came out, which is worth a look the first time:
$ ./hpcfg -i deu_ruleset Program : hpcfg 1.0 Trained : 149.549 passwords (deu_train.txt) Encoding : utf-8 Structures : 864, escape takes 40.002% of the mass Terminals : 28.712 in 78 lists the structures reach
$ hashcat -m 0 -a 4 hash.txt /path/to/deu_ruleset --stdout die der und in das im zu es
That is a German grammar guessing German, most likely first.
A language ruleset has the words but not the decoration people add to them. The default one has the decoration but not the words. Name both and hashcat merges them at run time, which reaches candidates that neither reaches alone:
$ hashcat -m 0 -a 4 hash.txt default-passwords /path/to/deu_ruleset
An even split is the default and will not always suit the target. weights= sets the share each one gets, so this leans three to one towards the password side:
$ hashcat -m 0 -a 4 hash.txt default-passwords /path/to/deu_ruleset weights=3:1
Getting the weights wrong costs real time, because a share of the run goes on candidates that were never likely. It is worth trying a few values against a hash list you have already partly cracked.
-a 4 is based on. It reads the same ruleset layout. If you use it, pass -e utf-8: it tries to autodetect the encoding, and without the chardet package it offers to fall back to ascii, which throws away the very characters you built the ruleset for.pcfg_trainer writes a ruleset that -a 4 loads directly, and it builds to one static binary with no interpreter to install, which makes it the least trouble of these to hand to hashcat.--ruleset-format pcfg-cracker.The C and Go trainers are both far quicker than the Python one, and both are actively worked on, so read the figures in their own repositories rather than trusting a number quoted here.
Why Cpcfg is not in that list. Cynosure Prime's Cpcfg trains the same model in C and is quick at it, so people do ask. hashcat cannot load what it writes. It is file format compatible with pcfg-go rather than with the canonical layout: probabilities are stored as an integer count/total ratio instead of a float, and it adds directories of its own. Train with it and generate with pcfg-go's guesser, or use one of the trainers above when the ruleset has to go into -a 4.
The full reference is docs/hashcat-pcfg.md in the hashcat source. It covers the settings, how hashcat's version differs from the original PCFG tool, and what merging does in detail.