statsprocessor v0.03 - Printable Version

statsprocessor v0.03 - Printable Version

+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Support (https://hashcat.net/forum/forum-3.html)
+--- Forum: hashcat-utils, maskprocessor, statsprocessor, md5stress, wikistrip (https://hashcat.net/forum/forum-28.html)
+--- Thread: statsprocessor v0.03 (/thread-1265.html)

Pages: 1 2

statsprocessor v0.03 - atom - 06-10-2012

THIS THREAD IS OUTDATED, PLS CONTINUE HERE: https://hashcat.net/forum/thread-1285.html

I'm very proud to present you a new toy: statsprocessor

Download here: http://hashcat.net/files/statsprocessor-0.04.7z

This is a standalone word-generator like the maskprocessor, but based on markov-attack!

The markov-attack is a statistically based brute-force like attack, but instead of specifying a charset or a mask, we specify a file.

This file is once generated by ourself in a previous step. It contains statistical informations which are made out an automated analysis of a given dictionary.

To do the analysis we use another new tool called "hcstatgen" which is part of the new hashcat-utils package, starting with version 0.6.

Download 32 bit here: http://hashcat.net/files/hashcat-utils-0.7-32.7z
Download 64 bit here: http://hashcat.net/files/hashcat-utils-0.7-64.7z

The second program, the new tool "statprocessor", generates the words based on the statistical order of the .hcstat file.

NOTE: In Brute-Force Attack or in Mask Attack we can limit the keyspace by setting a smaller charset in order to reduce the attack-time.

In Markov Attack we have something similar, the "threshold". All you do is to specify a number. The higher the number, the higher the threshold to add a new link between two characters on the two-level table on which the markov-attack bases on.

This background is not so important, just remember that the higher the value, the smaller the keyspace and thus the faster the attack.

If you set the threshold to 0 its like a real Brute-Force attack but with markov ordering.

Here is a example usage:

Generate the .hcstat out of the rockyou.txt dictionary

Code:
root@sf:~/statsprocessor-0.01# /root/hashcat-utils-0.6/hcstatgen.bin rockyou.hcstat < /root/dict/untouched/rockyou.txt 

Reading input...

Sorting tables...

Writing stats...

NOTE: I will add rockyou.hcstat as an example to the distribution so that new users have something to instantly play with.

Now lets run the statprocessor just to see what comes out. I will set an totally oversized threshold which is not for concrete usage but this way the output is so small that you will get an idea of how it looks like

Code:
root@sf:~/statsprocessor-0.01# ./sp64.bin --pw-min 5 --pw-max 5 --threshold 400000 rockyou.hcstat | head -20

anana

anane

anani

anara

anare

anari

anala

anali

anale

analo

aname

anama

anami

anamo

anaml

anasa

anat

anena

anene

anera

I will explain a bit in detail next. This if for those who want to know how markov-attack works internally. Otherwise you can skip to the next command.

As you can see, it starts with the char "a". That is because "a" is the most used char in rockyou.txt.

The first character is what we call the root-character. All the following char are looked up from the markov-table.

The table is an 1-to-n table. The key can be any char. For this char, the table holds an ORDERED array structure which char follows next.

From what we see here is that the char "n" is the most followed char after the char "a". Because of some statistical accident the char "a" is followed most by the char "n", so it ends up in a loop.

In the next iteration, the 2nd most followed char after "n" is the "e", and so on.

As you can see after the word "anasa" there is a four char word "anat" following, even though we specified --pw-min 5.

This happend because of the high threshold. There is simply no char following the "t" char more than 400000 times in rockyou.txt.

Since we always focus on performance, see how fast this beast is

Code:
root@sf:~/statsprocessor-0.01# ./sp64.bin --pw-min 5 --pw-max 5 rockyou.hcstat | wc -l    

1634805578

root@sf:~/statsprocessor-0.01# time ./sp64.bin -o /dev/null --pw-min 5 --pw-max 5 rockyou.hcstat        

real    0m18.234s

user    0m17.981s

sys     0m0.244s

This makes it 89.6 Mplains/s fast - on a single cpu core!

This 89.6 Mplains/s is fast enough to feed oclHashcat-plus for slow algorithms like DCC2 or WPA/WPA2. Here is an example run

Code:
root@sf:~/oclHashcat-plus-0.09# /root/statsprocessor-0.01/sp64.bin --pw-min 8 /root/statsprocessor-0.01/rockyou.hcstat | ./oclHashcat-plus64.bin -m 2500 test.hccap -n 80

...

Starting attack in wordlist stdin mode...    

Status.......: Running

Input.Mode...: Pipe

Hash.Target..: hashcat (00:aa:aa:aa:aa:aa <-> aa:aa:aa:aa:aa:aa)

Hash.Type....: WPA/WPA2

Time.Running.: 10 secs

Time.Util....: 10001.1ms/241.1ms Real/CPU, 2.5% idle

Speed........:   172.0k c/s Real,   181.3k c/s GPU

Recovered....: 0/1 Digests, 0/1 Salts

Progress.....: 1474560

Rejected.....: 0

HWMon.GPU.#1.: 98% Util, 60c Temp, 84% Fan

HWMon.GPU.#2.: 98% Util, 60c Temp, 30% Fan

As you can see, this is full speed for my hd6990.

But what if we want to use this with fast algorithms like MD5/SHA1? Well that would require a special new attack-mode for oclHashcat-lite or oclHashcat-plus. I will defenitly add this to a later version. Till that you can workaround this just by adding some workload to each generated word

Code:
$ /root/statsprocessor-0.01/sp64.bin --pw-min 6 /root/statsprocessor-0.01/rockyou.hcstat | ./oclHashcat-plus64.bin -n 80  example0.hash -r rules/hybrid/prepend_ld.rule -r rules/hybrid/prepend_ld.rule

...

Starting attack in wordlist stdin mode...    

007f8558448eba6c4f1d3fe33a0e52f7:jaanammi    

d77a07d8643df8b91975c5f3f8a582ee:brant124    

f8686f2dde3442b22166eac8bd0fa08b:c2anthea    

ec0962c15a328520b65d6172b8746774:alan1043    

63af54f143bd08246a78d371fc78eb25:deansail    

fc3d0a9d96d9a64722e05694033a8e05:vianchen    

3819f84ff69eab96e390816dba69481c:beans830

Well you got the idea...

NOTE

This is a very strong attack-mode. I think you will have a lot of fun with it.
Increment mode is already added. Like always, you can specify --pw-min and --pw-max.
The default threshold is set to 48.

RE: statsprocessor v0.01 - M@LIK - 06-10-2012

I didn't understand everything xD
But this looks very interesting!
Good job atom!

RE: statsprocessor v0.01 - Hash-IT - 06-10-2012

Wow..

M@LIK might not have understood everything but I didn't understand anything ! Big Grin

However it looks great !! Thank you very much atom !

RE: statsprocessor v0.01 - Pixel - 06-10-2012

Great! more tools to play with thanks atom

RE: statsprocessor v0.01 - rurapenthe - 06-10-2012

Very nice Atom! :-)

I'm playing with it already... gonna be fun.

RE: statsprocessor v0.01 - .::Rizwan::. - 06-10-2012

All thing flies by over my head : lol
Very Nice Atom,A very handy tool indeed

RE: statsprocessor v0.02 - atom - 06-10-2012

In the meanwhile I've finished v0.02. This one accepts an optional typical hashcat mask.

Download it here: http://hashcat.net/files/statsprocessor-0.02.7z

This can be used as a filter. It is best explained with two examples:

Without filter-mask (as in v0.01)

Code:
root@sf:~/statsprocessor-0.02# ./sp64.bin --pw-min 3 rockyou.hcstat | head -10

ana

ane

ani

ang

and

ann

ano

ant

an1

ank

With new filter-mask

Code:
root@sf:~/statsprocessor-0.02# ./sp64.bin --pw-min 3 rockyou.hcstat ?l?l?d | head -10

an1

an2

an0

an3

an4

an9

an8

an7

an6

an5

RE: statsprocessor v0.02 - Hash-IT - 06-11-2012

Steady on atom !!!

I've not learned how to use the first one properly yet !!! Big Grin

RE: statsprocessor v0.02 - ntk - 06-11-2012

awsome. thanks

RE: statsprocessor v0.03 - atom - 06-13-2012

Changes:

Added -t shortcut for --threshold
Added --reverse parameter which lets the markov generator generate the words from right to left, not from left to right.

Code:
root@sf:~/statsprocessor-0.03# ./sp64.bin --pw-min 4 -t 4 rockyou.hcstat | head -10 anan anar anal anam anen aner anel anem anir anil root@sf:~/statsprocessor-0.03# ./sp64.bin --pw-min 4 -t 4 rockyou.hcstat --reverse | head -10 nana rana lana mana nena rena lena mena rina lina
Changed how that most important parameter "--threshold" is interpreted by statsprocessor to a bit more easy-to-understand usage. You just define a maximum. This is the maximum number of chars which are added to the markov-table. One exception: In case of "0" it will allow all chars and you will end up in an ordered brute-force.

Code:
root@sf:~/statsprocessor-0.03# ./sp64.bin --threshold 1 rockyou.hcstat | head -10 a an ana anan anana ananan ananana anananan anananana ananananan root@sf:~/statsprocessor-0.03# ./sp64.bin --threshold 2 rockyou.hcstat | head -10 a e an ar en er ana ane ara are root@sf:~/statsprocessor-0.03# ./sp64.bin --threshold 0 rockyou.hcstat | head -10 a e 1 0 i 2 o n r l [*]
Added parameter -s and -l like in maskprocessor. Well you can also call it support for restore and distribution

Code:
root@sf:~/xy/statsprocessor-0.03# ./sp64.bin rockyou.hcstat ?a?a?a -s ctu -l ctk ctu cts ct1 ctc ctl ctm ct2 ctz ctb ct0 ctw ctn ctk [*]