Bottleneck, and my 31 character limit. Ideas?

[atom]

OK, here's the solution:

##
## generate base
##

$ cat perm-input.txt
0123456789

$ /root/hashcat-utils-1.2/permute.bin < perm-input.txt | more
0123456789
1023456789
2013456789
0213456789
1203456789

$ /root/hashcat-utils-1.2/permute.bin < perm-input.txt | perl -ne 'chomp; my @x = split ""; printf "%s-\n", join ("-", @x);' > wordlist.txt

$ more wordlist.txt
0-1-2-3-4-5-6-7-8-9-
1-0-2-3-4-5-6-7-8-9-
2-0-1-3-4-5-6-7-8-9-
0-2-1-3-4-5-6-7-8-9-
1-2-0-3-4-5-6-7-8-9-

$ wc -l wordlist.txt
3628800 wordlist.txt

$ md5sum wordlist.txt
18adc4eacda246044fe166016f30f422 wordlist.txt

##
## generate amplifier
##

Now a little patch for permute.c from hashcat-utils to generate the amplifier rules for A, B and C (10, 11, 12):

Code:

$ diff permute.c permutepatch.c
38a39,128
> void conv (char *line_buf, int line_len, int *map)
> {
>   static const char grp_pos[] = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z' };
>
>   int A_sav = 0;
>   int B_sav = 0;
>   int C_sav = 0;
>
>   int i;
>   int p;
>
>   for (i = 0, p = 0; i < line_len; i++, p += 2)
>   {
>     if (line_buf[i] == 'A')
>     {
>       A_sav = p;
>
>       p++;
>     }
>     else if (line_buf[i] == 'B')
>     {
>       B_sav = p;
>
>       p++;
>     }
>     else if (line_buf[i] == 'C')
>     {
>       C_sav = p;
>
>       p++;
>     }
>   }
>
>   int key = (A_sav <<  0)
>           + (B_sav <<  8)
>           + (C_sav << 16);
>
>   if (map[key] == 1) return;
>
>   map[key] = 1;
>
>   if (A_sav < B_sav && A_sav < C_sav)
>   {
>     printf ("i%c1 i%c0 i%c- ", grp_pos[A_sav + 0], grp_pos[A_sav + 1], grp_pos[A_sav + 2]);
>
>     if (B_sav < C_sav)
>     {
>       printf ("i%c1 i%c1 i%c- ", grp_pos[B_sav + 0], grp_pos[B_sav + 1], grp_pos[B_sav + 2]);
>       printf ("i%c1 i%c2 i%c- ", grp_pos[C_sav + 0], grp_pos[C_sav + 1], grp_pos[C_sav + 2]);
>     }
>     else
>     {
>       printf ("i%c1 i%c2 i%c- ", grp_pos[C_sav + 0], grp_pos[C_sav + 1], grp_pos[C_sav + 2]);
>       printf ("i%c1 i%c1 i%c- ", grp_pos[B_sav + 0], grp_pos[B_sav + 1], grp_pos[B_sav + 2]);
>     }
>   }
>   else if (B_sav < A_sav && B_sav < C_sav)
>   {
>     printf ("i%c1 i%c1 i%c- ", grp_pos[B_sav + 0], grp_pos[B_sav + 1], grp_pos[B_sav + 2]);
>
>     if (A_sav < C_sav)
>     {
>       printf ("i%c1 i%c0 i%c- ", grp_pos[A_sav + 0], grp_pos[A_sav + 1], grp_pos[A_sav + 2]);
>       printf ("i%c1 i%c2 i%c- ", grp_pos[C_sav + 0], grp_pos[C_sav + 1], grp_pos[C_sav + 2]);
>     }
>     else
>     {
>       printf ("i%c1 i%c2 i%c- ", grp_pos[C_sav + 0], grp_pos[C_sav + 1], grp_pos[C_sav + 2]);
>       printf ("i%c1 i%c0 i%c- ", grp_pos[A_sav + 0], grp_pos[A_sav + 1], grp_pos[A_sav + 2]);
>     }
>   }
>   else if (C_sav < A_sav && C_sav < B_sav)
>   {
>     printf ("i%c1 i%c2 i%c- ", grp_pos[C_sav + 0], grp_pos[C_sav + 1], grp_pos[C_sav + 2]);
>
>     if (A_sav < B_sav)
>     {
>       printf ("i%c1 i%c0 i%c- ", grp_pos[A_sav + 0], grp_pos[A_sav + 1], grp_pos[A_sav + 2]);
>       printf ("i%c1 i%c1 i%c- ", grp_pos[B_sav + 0], grp_pos[B_sav + 1], grp_pos[B_sav + 2]);
>     }
>     else
>     {
>       printf ("i%c1 i%c1 i%c- ", grp_pos[B_sav + 0], grp_pos[B_sav + 1], grp_pos[B_sav + 2]);
>       printf ("i%c1 i%c0 i%c- ", grp_pos[A_sav + 0], grp_pos[A_sav + 1], grp_pos[A_sav + 2]);
>     }
>   }
>
>   printf (" ] ^[ $]\n");
> }
>
51a142,143
>   int *map = (int *) calloc (256 * 256 * 256, sizeof (int));
>
72c164
<     puts (line_buf);
---
>     conv (line_buf, line_len, map);
74c166
<     while ((k = next_permutation (line_buf, p, k)) != line_len) puts (line_buf);
---
>     while ((k = next_permutation (line_buf, p, k)) != line_len) conv (line_buf, line_len, map);
76c168
<     puts (line_buf);
---
>     conv (line_buf, line_len, map);

$ cat input
0123456789ABC

$ ./permutepatch < input > amplifier.rule

$ wc -l amplifier.rule
1716 amplifier.rule

$ md5sum amplifier.rule
77dac3e8ac31eec1318d60ee60ef341c amplifier.rule

##
## this should work against a hash (and even a hashlist) with full gpu acceleration
## cracking should be nearly instant
##

$ ./oclHashcat64.bin 3206c0ed6f2770d72a470c839b0550ae wordlist.txt -r amplifier.rule -w 3
oclHashcat v1.36 starting...

Device #1: Tahiti, 3022MB, 1000Mhz, 32MCU
Device #2: Tahiti, 3022MB, 1000Mhz, 32MCU
Device #3: Tahiti, 3022MB, 1000Mhz, 32MCU

Hashes: 1 hashes; 1 unique digests, 1 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates
Rules: 1716
Applicable Optimizers:
* Zero-Byte
* Precompute-Init
* Precompute-Merkle-Demgard
* Meet-In-The-Middle
* Early-Skip
* Not-Salted
* Not-Iterated
* Single-Hash
* Single-Salt
* Scalar-Mode
* Raw-Hash
Watchdog: Temperature abort trigger set to 90c
Watchdog: Temperature retain trigger set to 80c
Device #1: Kernel ./kernels/4098/m00000_a0.Tahiti_1573.4_1573.4 (VM)_1428240294.kernel (522804 bytes)
Device #2: Kernel ./kernels/4098/m00000_a0.Tahiti_1573.4_1573.4 (VM)_1428240294.kernel (522804 bytes)
Device #3: Kernel ./kernels/4098/m00000_a0.Tahiti_1573.4_1573.4 (VM)_1428240294.kernel (522804 bytes)

Cache-hit dictionary stats wordlist.txt: 76204800 bytes, 3628800 words, 6227020800 keyspace


INFO: approaching final keyspace, workload adjusted

3206c0ed6f2770d72a470c839b0550ae:[5-10-3-0-2-7-11-9-6-8-1-12-4]

Session.Name...: oclHashcat
Status.........: Cracked
Rules.Type.....: File (amplifier.rule)
Input.Mode.....: File (wordlist.txt)
Hash.Target....: 3206c0ed6f2770d72a470c839b0550ae
Hash.Type......: MD5
Time.Started...: Mon Apr 6 03:11:38 2015 (1 sec)
Speed.GPU.#1...: 1807.6 MH/s
Speed.GPU.#2...: 1771.0 MH/s
Speed.GPU.#3...: 1769.9 MH/s
Speed.GPU.#*...: 5348.5 MH/s
Recovered......: 1/1 (100.00%) Digests, 1/1 (100.00%) Salts
Progress.......: 3453911324/6227020800 (55.47%)
Skipped........: 0/3453911324 (0.00%)
Rejected.......: 0/3453911324 (0.00%)
Restore.Point..: 1048576/3628800 (28.90%)
HWMon.GPU.#1...: 64% Util, 33c Temp, 25% Fan
HWMon.GPU.#2...: 62% Util, 33c Temp, 25% Fan
HWMon.GPU.#3...: 64% Util, 33c Temp, 25% Fan

Started: Mon Apr 6 03:11:38 2015
Stopped: Mon Apr 6 03:11:40 2015

PS: Check the progress value

[Rolf]

D C D

[epixoip]

odd restriction
programming limitation
To top it off - closed source - I can't just change the code

If you don't understand the restriction nor why it exists, what makes you think you'd be any better off having the source code?

[InfDoleo]

odd restriction
programming limitation
To top it off - closed source - I can't just change the code

If you don't understand the restriction nor why it exists, what makes you think you'd be any better off having the source code?

So that I CAN understand why it exists, and modify it if necessary.

If a combination or mask attack CAN be longer than 31, it's not that its completely impossible. If it has to do with 32 bytes of space in the GPU itself, and it's fed the data differently because of the way combination and mask attacks work, there doesnt seem to be any good reason why a straight dictionary attack wouldnt either. Split the string in 2 parts before feeding it in, and simulate a combination attack.

If it's not that simple because the combining is done IN the GPU, then you find a workaround. And if thats not possible, then you LEARN why. Learning why is always preferable to "No. Just because."

And, since it's closed source, it therefore IS impossible, instead of maybe possible.

That being said there was a reply in my other thread that seems to show a way to do it using rule-based attacks, something I hadn't yet read up on.

Doesn't change the fact that it sucks it's closed source. There are other things I'd like to do with GPU hashing that it seems like oclhashcat does MUCH better than the open source solutions like cgminer and the like. I have an app I want to research, and having a solid starting point in a programming area I am not familiar with, GPU interfacing, would have been nice.

Not sure why the heck I'm being jumped upon here.

[InfDoleo]

Wow, Atom, thats awesome. I hadn't yet looked into rule based attacks. By reading the patch, it's limited to A B and C - can you, if you don't mind (you've already gone above and beyond writing that up), explain exactly what it's doing? I'm going to run it here soon and inspect the output so I can try and see for myself, but a human explanation always speeds things up.

I see you're an admin - are you the developer? Excuse my ignorance, I've worked with oclHashcat for less than 24 hours now, and only learned of it's existence 12 hours before that.

[unix-ninja]

Not sure why the heck I'm being jumped upon here.

If you want open source, get John the Ripper or Cryptohaze.

If you want to learn about the design limitations of oclHashcat, you are better off reading ocl docs and trying to write your own SHA1 cracking. That will teach your more than reading hashcat source.

[atom]

Well yeah from what you need to know about GPU's that you need an amplfier for fast hashes:

https://hashcat.net/wiki/doku.php?id=fre...king_speed

Rules are a good way to archieve such an amplification. A good number of rules is 1000. The more the better:

https://hashcat.net/wiki/doku.php?id=fre...full_speed

You just need to find a way to create such a scenerio. Means we have to divide a part of the workload from the problem and applying it back again later. From a math standpoint it's like doing this:

N!(13) = 6,227,020,800
N!(10) = 3,628,800

So the quotient is 1,716 which is the first number > 1,000 so that's why I took it so that's the reason why the rules use ABC, they could also use 6789 if i wanted to do that.

Technically we just generate N!(10) now for the base-word input and then we use rules to mutate those in a way that it does exactly what's needed to the same as we would use N!(13). To get to all we need to find is the offset points for the remaining A, B, C values. Those are not so easy to find because they themself modify the base word with each insert rule applied that's why i cheated and used the permute.bin itself to print them out.

[InfDoleo]

So, lets see if I understand this correctly:

Finding a way to amplify the wordlist speeds up performance by keeping the same section of wordlist in memory, and modifying it and going again, and again, on the same, now changed, wordlist. Correct?

And what you posted above takes a smaller subset of my wordlist, and then used ruleset to change it over and over again inside the GPU and to keep working on it, without slowing down.

Got that part.

One thing I'm having a hard time understanding, and I've read the wiki on it, I guess I need a better explanation... I ran this today, and ^C'ed out of the rule making line after there were a few hundred lines to check out. And for the life of me I cannot make sense of what the rules do. Some I get... the ] and the ^[ and $] at the end, I found the reference in the rules wiki page. But the 'i' on each number? I dont see that anywhere.. and some of the other letters/codes in there as well.

Is there a better page so I can learn how to write rules myself for future use?

[atom]

Sure it's the insert rule. Rules are described here in detail:

https://hashcat.net/wiki/doku.php?id=rule_based_attack

The first parameter is the offset, the 2nd the actual char to insert. In our case we need multiple inserts:

1, 0, -
1, 1, -
1, 2, -

[InfDoleo]

Sure it's the insert rule. Rules are described here in detail:

https://hashcat.net/wiki/doku.php?id=rule_based_attack

The first parameter is the offset, the 2nd the actual char to insert. In our case we need multiple inserts:

1, 0, -
1, 1, -
1, 2, -

Aah! I totally missed that. I was looking for a single character 'i' so totally missed the iNX. That leads to more confusion then....

I was looking at the output'ed rule file and the first few lines look like this:

iK1 iL0 iM- iN1 iO1 iP- iQ1 iR2 iS- ] ^[ $]
i01 i10 i2- iN1 iO1 iP- iQ1 iR2 iS- ] ^[ $]
i21 i30 i4- iN1 iO1 iP- iQ1 iR2 iS- ] ^[ $]
i41 i50 i6- iN1 iO1 iP- iQ1 iR2 iS- ] ^[ $]
i61 i70 i8- iN1 iO1 iP- iQ1 iR2 iS- ] ^[ $]

insert character 'K' at postition 1? Inser 'L' at position 0? Is that right? Or are the positions 0-9A-Z to handle 36 positions? The page says "For character positions other than 0-9 use A-Z (A=11)" - is that basically saying "For positions over 9, use A-Z, for a total of 36 possible positions"?

So then next question - the whole reason I'm doing this rule/mask based is that this application I'm working on WILL grow to larger numbers... 0-13, 14, 15, etc - and when that happens the password will be longer than 31 characters, the maximum. Is it not possible to reference a character in a position greater than 36 then?

Thanks a ton for all your help.