+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Misc (https://hashcat.net/forum/forum-15.html)
+--- Forum: Organisation and Events (https://hashcat.net/forum/forum-24.html)
+--- Thread: Official Best64 Challenge Thread (/thread-1002.html)
RE: Official Best64 Challenge Thread - Hash-IT - 03-29-2012
Congratulations arex1337, well done.
I enjoyed reading your write up and thank you for putting so much work into making hashcat's rules even better.
It was also nice to see that you not only shared your new rules but also shared the method and code you used to make your best 64. You are very generous and I hope you enjoyed doing it.
Thank you for your contribution and I hope this is only the start of your "ideal rule" search !!
RE: Official Best64 Challenge Thread - forumhero - 03-29-2012
i enjoyed arex1337's write up as well. very nicely done and congrats
RE: Official Best64 Challenge Thread - M@LIK - 03-29-2012
yea me too arex1337.. me too : )
THANKS!!
RE: Official Best64 Challenge Thread - arex1337 - 03-29-2012
I updated my write-up with some post-challenge details, plus a chart that shows everybody's progression! http://thepasswordproject.com/2012-03-28_my_participation_in_the_best64_challenge
RE: Official Best64 Challenge Thread - Hash-IT - 03-29-2012
Good grief !!!
What are you arex1337 ?? A scientist or a Jedi ?
Great stuff, even though I don't pretend for a minute to understand it all !
Thanks for your contribution.
RE: Official Best64 Challenge Thread - atom - 03-30-2012
This is the write-up how I rebuild the new best64.rule out of all the submissions but for real-life use.
This new best64.rule will go into all hashcat distributions.
I took arex best 80's results posted here: http://thepasswordproject.com/2012-03-28_my_participation_in_the_best64_challenge#post-competition
1. Check how useful it really is
To do this i made a MD5 hashlist of rockyou.txt which is the ultimate real-life password resource we have.
Code:
$ cat /root/dict/untouched/rockyou.txt | perl /root/dict2hash.pl > rockyou.md5The resulting hashlist has 14 million hashes. This is to much.
Code:
$ wc -l rockyou.md5Quote:14344391 rockyou.md5
I reduced them a bit by randomly fetching out 1 million hashes. This is the hashlist we will work with.
Code:
$ sort -R rockyou.md5 | head -1000000 > rockyou.md5.1000000
$ wc -l rockyou.md5.1000000Quote:1000000 rockyou.md5.1000000
Now run the top 10k dictionary and the best 80 rules from the contest against it.
Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r best80.ruleQuote:...
Recovered.: 17097
Here is the original best64.rule. We will use it later for comparison with the final result:
Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r rules/best64.ruleQuote:...
Recovered.: 12345
Funny number.. Well, but thats it for real
2. Clean up arex's rules
There is a lot of useless stuff inside. Will do:
- Deobfuscate rules, stuff like ^7 { which is actually just $7
- Remove dupes that come out after deobfuscation.
- Remove as much as possible hashcat specific functions to make resulting hashlist jtr/pwp compatible.
- Remove useless functions to increase performance while rules are applied at runtime in rule-engine: Making it faster!
Code:
$2$4]] :
$4]]] ] ]
T0 T0
$1$2i42@2 $1
i42@2T0u u
}}D1{ ]
$si42@2 $s
]]]d ] ] ] d
$3$2]]]]] ] ] ]
krKz3x13 r
i42@2$2 $2
}}}}'4 } } } } '4
:so0 so0
$0$1i42@2 $0 $1
}]]{ } ] ] {
$1$2$3*22 $1 $2 $3
i42@2$9Z1 $9 $9
lx31 D3
$b]$7 $7
dx28Y4'4d d x28 Y4 '4 d
}}}}}'5 } } } } } '5
z3x13$3 $3
^5{ $5
$6 $9 $6 $9
f*A5'8x14 f *A5 '8 x14
si1 si1
z3x13$4 $4
$0$0$7}[ $0 $0
]]$a ] ] $a
]]]$m$a$n ] ] ] $m $a $n
+5]}}}}'4 +5 ] } } } } '4
*05x03d'3p1 *05 x03 d '3 p1
]]]$1$2$3 ] ] ] $1 $2 $3
i42@2$1$1 $1 $1
$9li42@2 $9
$1 $2 $1 $2
]]]$y ] ] ] $y
^b^b^p^h^p]]]] ^b ^b ^p ^h ^p ] ] ] ]
$2$2'5$1 '5 $1
Z4'8x42 Z4 '8 x42
$0 $0
$6 $6
i42@2$2$3 $2 $3
$2 $2 $2 $2
^&x03{{{{{{ x02 { { { { { {
$1]]]$e$r ] ] $e $r
se3 se3
] ] $i $e ] ] $i $e
]]l]$o ] ] ] $o
$8z3x13 $8
i01i42@2 ^1
}}}}}}Y4'4d } } } } } } Y4 '4 d
$2 $1 $2 $1
^t@tD2D2 D2 D2
{{{{{{{{x31'4 '5 D3
p2'7p1x58 p2 '7 p1 x58
]]D1] ] ] D1 ]
*44D4 D4
+0+0+0+0+0+0+0+0 +0 +0 +0 +0 +0 +0 +0 +0
*04+0'4 *04 +0 '4
$8$8 $8 $8
x14dp2'6 x14 d p2 '6
]]]l$p$h$p$b$b ] ] ] $p $h $p $b $b
Z5*75'5{x02 Z5 *75 '5 { x02
$7$7 $7 $7
x21i42@2 D2
^e^h^ti42@2 ^e ^h ^t
] ] ] $d $o $g ] ] ] $d $o $g
}}-0x12 } } -0 x12
o0to0b o0b
l[^d^t@t [ ^d
$e $e
l]$sl]$a ] $a
$.]]l]$s ] ] $s
+0+0+0x12 +0 +0 +0 x12
:}}} } } }
$1$3 $1 $3
o1ao0ml ^m ^a
Z5'6x31]p1 Z5 '6 x31 ] p1
$0$2 $0 $2Some rules could not get deobfuscated/optimized.
Some of the changes might have an influence on the cracked results since I removed lots of stuff, too.
To make sure the changes are not so bad, here is a backup run to see its not so far away from the 17097 from the original set.
Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r best80-de.ruleQuote:...
Recovered.: 17050
OK, thats fine
2. Remove the both rules that are only efficient cracking the phpbb hashlist
Code:
^b ^b ^p ^h ^p ] ] ] ]
] ] ] $p $h $p $b $bNow we need to run again to see what we lost. In theory, should be zero:
Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r best80-de.ruleQuote:...
Recovered.: 17050
Zero! Toda!
3. Regroup them
By doing this I mean I am doing no changes at all to the rules itself. But you will see, it gives a great new view on everything.
Code:
## nothing, reverse, case... base stuff
:
r
u
T0
## simple number append
$0
$1
$2
$3
$4
$5
$6
$7
$8
$9
## special number append
$0 $0
$0 $1
$0 $2
$1 $1
$1 $2
$1 $3
$2 $1
$2 $2
$2 $3
$6 $9
$7 $7
$8 $8
$9 $9
$1 $2 $3
## high frequency append
$e
$s
## high frequency overwrite at end
] $a
] ] $s
] ] $a
] ] $e $r
] ] $i $e
] ] ] $o
] ] ] $y
] ] ] $1 $2 $3
] ] ] $m $a $n
] ] ] $d $o $g
## high frequency prepend
^1
^e ^h ^t
## high frequency overwrite at start
o0b
o0d
o0m o1a
## leetify
so0
si1
se3
## simple extracts
D2
D2 D2
D3
D4
## undouble word
'5 D3
'5 $1
## removes suffixes from 'strongified' passwords in dict
]
] ]
] ] ]
] ] ] d
] ] D1 ]
## rotates
+5 ] } } } } '4
x02 { { { { { {
} ] ] {
} } -0 x12
} } }
} } } } '4
} } } } } '5
} } } } } } Y4 '4 d
## unknown
*04 +0 '4
*05 x03 d '3 p1
+0 +0 +0 +0 +0 +0 +0 +0
+0 +0 +0 x12
Z4 '8 x42
Z5 '6 x31 ] p1
Z5 *75 '5 { x02
d x28 Y4 '4 d
f *A5 '8 x14
p2 '7 p1 x58
x14 d p2 '6Verify results:
Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r best80-de.ruleQuote:...
Recovered.: 17050
Perfect!
4. Reduce the 78 rules down to 64 by removing the rules that are less efficient
It is possible that two or more different rules can crack the same hash just because of the many words in our dictionary.
So what we focus in are all these hashes that have been cracked by only one single rule.
This will happend more or less often for each rule. We add a counter for each rule and with each hit we increase it.
When we are finished we simply sort out these rules that have the lowest number in their counter.
To do this, you can read arex way of doing it, see his writeup: http://thepasswordproject.com/2012-03-28_my_participation_in_the_best64_challenge#pick_64_rules_that_combined_will_match_the_largest_possible_number_of_hashes
5. Result
Here is our final new best64.rule:
Code:
## nothing, reverse, case... base stuff
:
r
u
T0
## simple number append
$0
$1
$2
$3
$4
$5
$6
$7
$8
$9
## special number append
$0 $0
$0 $1
$1 $1
$1 $2
$2 $1
$2 $2
$2 $3
$6 $9
$7 $7
$8 $8
$9 $9
$1 $2 $3
## high frequency append
$s
## high frequency overwrite at end
] ] $a
] ] $e $r
] ] $i $e
] ] ] $o
] ] ] $y
] ] ] $1 $2 $3
] ] ] $m $a $n
## high frequency prepend
^1
^e ^h ^t
## leetify
so0
si1
se3
## simple extracts
D2 D2
D3
D4
## undouble word
'5 $1
## removes suffixes from 'strongified' passwords in dict
]
] ]
] ] ]
] ] ] d
] ] D1 ]
## rotates
+5 ] } } } } '4
x02 { { { { { {
} ] ] {
} } -0 x12
} } } } '4
} } } } } '5
} } } } } } Y4 '4 d
## unknown
*04 +0 '4
*05 x03 d '3 p1
+0 +0 +0 +0 +0 +0 +0 +0
Z4 '8 x42
Z5 *75 '5 { x02
d x28 Y4 '4 d
f *A5 '8 x14
p2 '7 p1 x58
x14 d p2 '6And our final value:
Quote:...
Recovered.: 15357
The original best64.rule was:
Quote:...
Recovered.: 12345
The original best64.rule was already very efficient on the random hashlist.
The new one is 24.39% more efficient.
Again, thanks to all participating in the contest.
RE: Official Best64 Challenge Thread - Hash-IT - 03-30-2012
Fantastic work there atom. you have clearly put a lot of effort into that, it is much appreciated.
I have just deleted my humble attempt to do this as it was junk in comparison.
Thank you.
RE: Official Best64 Challenge Thread - frekvent - 03-31-2012
I have experienced the same issue as described phish (but the original post seems to be deleted). I tried to simplify rules by doing some simple substitutions:
- sXY@Y -> @X@Y
- iNX@X -> @X
- ^X@X -> @X
- $X@X -> @X
Oh and thanks to atom for compiling the new best64.rule! I have tested it against some random hashes with some random wordlists and got some very good results! I did spot a minor simplification you could do. It turns out that "}}D1{" can be simplified to a single "]".
Anyway thanks a lot! It is truly amazing how much the original rules could be improved upon!
RE: Official Best64 Challenge Thread - atom - 03-31-2012
the post was not deleted, just splitted into a seperate thread, since this thread is about the challange, not about strange rule engine behaiviors. please read the http://hashcat.net/forum/thread-1027.html then you hopefully understand why it works how it works...
RE: Official Best64 Challenge Thread - atom - 03-31-2012
(03-31-2012, 12:10 AM)frekvent Wrote: It turns out that "}}D1{" can be simplified to a single "]".
This is cool. I will update the post and the best64.rule. Thanks!