Combination mode doesn't generate all combinations

[Kgx Pnqvhm]

Hashcat's Combination mode attack seems to not generate all the combinations.

With the goal of using hashcat's words Combination attack-mode=1, my first step is to convert more of the KoreLogic john rules to hashcat, and to fix up the already converted that don't do the capitalizing, and use a smaller number of special characters.

But, by using hashcat's combination mode to help generate the rules, I've found that past a certain number of words, the combination mode doesn't give all the combinations.

(Running hashcat-cli32.exe v0.36 on an XP3 Celeron system with 1 GB of RAM.)

Starting with a rule to append one special character gives, before the capitalization rules, the 33 lines of:

Code:

$!
$
$"
$#
$$
$%
$&
$'
$(
$)
$*
$+
$,
$-
$.
$/
$:
$;
$<
$=
$>
$?
$@
$[
$\
$]
$^
$_
$`
${
$|
$}
$~

Then, running that through hashcat with attack-mode=1 with a rule of just "^c" to prepend the "c" to the above rules to get the capitalized rules, using --debug-mode=3 --debug-file=debug.txt to get the resultant list, I get the expected 33 additional rules. Added together, the final rule is 66 lines.

Using that 66 line append one special rule as the basis to generate a rule that appends two specials, by running that file though hashcat's combination mode with the noop.rule, the expected output would be 66 * 66 = 4,356 lines, but only 4,355 lines come out, meaning that one got left out.

Backtracking, and using just the 33 main rules, without the capitalized versions, gets the expected 33 * 33 = 1,089 lines.

To test further, sending that 1,089 line file though the combination mode, which should get 1,089*1,089 = 1,185,921 lines results in only 1,184,515 lines, which is 1,406 short.

What is going wrong?

[Kgx Pnqvhm]

Trying words, rather than symbols, got these results:

50*50 = 2,500 expected vs 2,499 output; 1 short
60*60 = 3,600 expected vs 3,600 output; OK
70*70 = 4,900 expected vs 4,894 output; 6 short
80*80 = 6,400 expected vs 6,399 output; 1 short
100*100 = 10,000 expected vs 99,992; 8 short
200*200 = 40,000 expected vs 39,964; 36 short

Has any extensive testing of the (Word) Combination attack mode been done?

[atom]

i can just guess now, but maybe it related to multiple threads writing to the same output file. can you try to reproduce this with -n 1? if the bug is gone, it seems i have to add some mutexing mechanism into debug file..

[Kgx Pnqvhm]

i can just guess now, but maybe it related to multiple threads writing to the same output file. can you try to reproduce this with -n 1? if the bug is gone, it seems i have to add some mutexing mechanism into debug file..

Re-trying with just one thread fixed the problem.

Does this mean that hashcat would be generating all the combinations in an actual attack, and the problem is just what gets into the debug file?

[atom]

thats good news. but yes, i am still just guessing. if i am right this should produce invalid lines in your debug output file. i mean this is some kind of race condition. the result file will look always diffrent and with no repeating pattern but there should be lines that are invalid. maybe with 1 or 2 char more than expected or 1 or 2 char less. could you please verify that too?

and as a temp workaround you can use -n 1.

[Kgx Pnqvhm]

After finding that using just one thread solved the problem, I erased the test files.

Making a new quick test file of 70 items, all the same length, and running four times in a row (with the default 8 threads) got 4 different outcomes, with only one correct: 4,895; 4,900; 4,898; 4,899. But all the resultant lines were the same length.

[Kgx Pnqvhm]

Hashcat in straight words attack-mode=0 has the same problem.

I ran the simple prepend rule on the 1,185,921 4 specials text body to get the capitalized version of the rules, with the default 8 threads, and even though the ending display had that number, the debug output file had only 1,185,737 lines.

(This was a second test of that operation. The first test I ran oon the file with comments, and it came out funny so I re-tested on the rules alone.)