Rule Piping Request
#1
Atom would you please consider this feature in hashcatplus ?

Currently if a user wishes to modify their wordlists, on the fly, in hashcatplus they can use rules.

This is a very powerful and useful feature but it has one slight draw back. The rule files can get very long and complex even for very simple and common password modifications.

For example, say the user wants a simple modification to their lists.

Rule File

l
u
c

In the above rule we see that the user wishes to lower case all, upper case all and then capitalise the first letter of each line of their wordlist. Fortunately this only requires 3 lines of rule code. The problem arises when a user decides to make another seemingly simple modification to their word list, appending numbers to the password.

Number suffixing is very common so it is not unreasonable to want to do this, say 0 – 100. The problem is by simply wanting to append a small number range to a password whilst also retaining the previous (very common) modifications l,u,c this rule list is already about 300 lines long ! Another, simple and common password padding is appending years, “bob1960” for example. So to catch these words with the rule file l,u,c we are already running into 3000 lines, 4000 if we include an original pass “:”.

So can I make a feature request that allows “Rule Piping” ? This will allow users to have a primary and secondary rule file.

Example code would look like…

oclHashcat-plus32.exe --hash-type 2500 examplewpa.hccap wordlist.txt --rules-file example.rule --rules-file2 example.rule -o Found.txt

(If no “--rules-file2 example.rule” exists then hashcatplus would work as it does now and assume a single rule file).

Rulefile1 > password Rulefile2 > output . Rules are taken from lists 1 and run down the entire list 2, then the second rule is taken from list 1 and run down list 2 and so on.

Rule file 1 contains l,u,c while rule file 2 contains $0 - $1$0$0

The combined first rule sent to the engine would be l so Password would be all lower case, then rule 2 would append a number 0 result = password0.

The above would allow for considerably smaller rule lists (3 lines in list 1 and 100 lines in list 2) which offer other possibilities which will also be very helpful when using the toggle rule.

I was going to suggest a rule that appended numbers that only took one line to write something like $num0-100 or similar but I think piping one list into the other offers more flexibility.

Better rules and rule management mean more rules for the GPU thus (hopefully) optimising it. !!

Thank you.
#2
how about using oclHashcat with -j option?
#3
Well if oclHashcat can be piped to oclhashcatplus then that will probably do it !! Smile

Thank you for the tip, I actually looked in hashcat to see if there was that option but never thought of oclHashcat. There are too many versions with different features !!! Smile

I only concentrate on WPA so I don't really look at the others much, although I actually did on this occasion but still missed it !

Does the -j option do what I was asking for ? I would still appreciate this feature in hashcatplus though.
Thank you.

I have just come back to edit this, I noticed the following on the wiki.
Quote:This project will be deprecated soon. All features getting migrated to oclHashcat-plus.

There are no “rule-files” possible since the rules are applied in loading-phase. Only one rule per side.

Rules can not apply on masks.

So can my request still be considered please ?




#4
wait, we have a standalone solution for this: http://hashcat.net/wiki/rules_with_maskprocessor
#5
the problem is not generating the rules at all but storing them.
#6
That’s right undeath, also I do wonder just how large these files can be before it hits performance ?

I am always looking to reduce code, lists or rules in an effort to be lean and mean as it were, ha ha ! Smile

Atom, can you please tell us if there are any limitations to rules, either file size or number of lines within which may affect performance ?

Thanks.
#7
yes there are limitations. in hashcat it tried to sort out dupliated rules and therefore has to manage an lookup-tree which requires memory. this means the more rules you add, the more startup-time is requires because the bigger the table gets, the longer the lookup takes. in oclHashcat-plus you have no such check. if the user add duplicate rules, its his problem. but therefore you have more memory overhead per rule. i would say the maximum number of rules for hashcat is around 3 million, for oclHashcat-plus around 1 million. sure, if the user has special hardware these number can increase.

just one thing to mention. the cracking time will not change by lowering the number of rules in comparision to an built-in special attack like you suggested. in other words, the special attack also have to generate the plaintext value and also has to run it through the algorithm. the require time is the same. so from my view large ruleset are of the same efficiency "rate".
#8
Thanks again for your reply atom.

Wow !! oclHashcat-plus around 1 million rules is hardly a limitation !! Smile

I am grateful you took the time to explain that as I was concerned about performance when reading through large or very large rule lists.

When I wrote this

Quote:Better rules and rule management mean more rules for the GPU thus (hopefully) optimising it. !!

I was referring to the fact you have taught me that running rules in the GPU is faster and more efficient than reading directly from a word list or mask-processor. As the GPU is so much faster this technique is used to utilise the GPU to its maximum. See I do pay attention ! Wink

I understand the responsibility is now on the user to check for duplicate rules in their lists. I think this is a better idea as the user should check their lists first. This only has to be done once and this prevents hashcat having to do it every time. Good idea !

I understand that having some rules “built in” would not improve performance in any way. My post or request was to help tidy things up a little for the user. The ability to insert one small user defined list with “:,l,c,u” in it as the first primer list and a secondary list with $1 - $1$0$0” etc would be so much easier to manage from a users point of view, also offering the possibility of interesting combinations.

Your answer explained the limitations (or rather, lack of limitation ) of a large single list from a speed / performance point of view but as a list management technique it would have positive benefits to an end user. So would it cause problems to have a list 1 and 2 in hashcatplus ?

The default supplied rule lists in the hashcatplus directory could be much smaller and cover more ground. As I have mentioned before there are problems with those lists which I have fixed. I am about to make new ones to share for new users and I thought it would be good to find out if I could interest you in this feature before I start making them as it would affect their size greatly.

Thanks.
#9
I have just thought of another benefit to this idea, a user could make their own filter lists.

say rule 1 contained...
@0@1@2@3@4@5@6'@7@8@9

A user could remove all numbers from the list before applying the second rule of their choice.

Perhaps also rule list 1 could have some other type of filtering, filter for non printable ascii ?
#10
rules for filtering non-printable ascii. this sounds somehow a bad idea. this have to be done with each execution of hashcat. if its that important, why not clean them in the source, in other words, in the dictionary. a simple perl script can do this.