Limiting the consecutive occurrence

Limiting the consecutive occurrence - Printable Version

+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Deprecated; Ancient Versions (https://hashcat.net/forum/forum-46.html)
+--- Forum: Very old oclHashcat-plus Support (https://hashcat.net/forum/forum-23.html)
+--- Thread: Limiting the consecutive occurrence (/thread-1201.html)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13

RE: Limiting the consecutive occurrence - Pixel - 06-01-2012

(06-01-2012, 01:55 AM)M@LIK Wrote:
Pixel Wrote: OK I'll explain more, what I meant was two repeat instances total any where in the whole line, so in QVHHRIRA, the two Hs is one instance and the two Rs would be the second instance.

Oh I get you, that wasn't very easy to do.
No more than 2 instances of a duplicate character:

Code:
/$.$.*\1.*$.$.*\2.*$.$.*\3/d /$.$.*\1.*$.$.*$.$.*\2.*\3/d /$.$.*\1.*$.$.*$.$.*\3.*\2/d /$.$.*$.$.*\1.*\2.*$.$.*\3/d /$.$.*$.$.*\1.*$.$.*\2.*\3/d /$.$.*$.$.*\1.*$.$.*\3.*\2/d /$.$.*$.$.*\2.*\1.*$.$.*\3/d /$.$.*$.$.*\2.*$.$.*\1.*\3/d /$.$.*$.$.*\2.*$.$.*\3.*\1/d /$.$.*$.$.*$.$.*\1.*\2.*\3/d /$.$.*$.$.*$.$.*\1.*\3.*\2/d /$.$.*$.$.*$.$.*\2.*\1.*\3/d /$.$.*$.$.*$.$.*\2.*\3.*\1/d /$.$.*$.$.*$.$.*\3.*\1.*\2/d /$.$.*$.$.*$.$.*\3.*\2.*\1/d

Yup all these, don't forget to remove the previous rule:

Code:
/$.$.*$.$.*\1.*\2/d /$.$.*$.$.*\2.*\1/d

Thanks M@LIK your the best. Oh WOW, lot of commands for this one. Do you know and good sed tutorials that are good? I have had a look but, just wanted to see if you had any that you would recommend having a you at?

M@LIK I think I found a bug:

QVHHABFY
QVHHABFZ
QVHHABGA
QVHHABGB
QVHHABGC
QVHHABGD
...

QVHHBJDA
QVHHBJDB
QVHHBJDC
QVHHBJDD
QVHHBJDE
QVHHBJDF

The red ones are missing .

Code:
mp64.exe QVHH?u?u?u?u --stop-at QVHHRIZZ|sed -f  awesome_script.txt>Q.txt

In the text file EDIT BELOW

Code:
/\(.\)\1\1/d

/\(.\).*\1.*\1/d

/\(.\).*\1.*\(.\).*\2/d

/\(.\).*\1.*\(.\).*\2.*\(.\).*\3/d

/\(.\).*\1.*\(.\).*\(.\).*\2.*\3/d

/\(.\).*\1.*\(.\).*\(.\).*\3.*\2/d

/\(.\).*\(.\).*\1.*\2.*\(.\).*\3/d

/\(.\).*\(.\).*\1.*\(.\).*\2.*\3/d

/\(.\).*\(.\).*\1.*\(.\).*\3.*\2/d

/\(.\).*\(.\).*\2.*\1.*\(.\).*\3/d

/\(.\).*\(.\).*\2.*\(.\).*\1.*\3/d

/\(.\).*\(.\).*\2.*\(.\).*\3.*\1/d

/\(.\).*\(.\).*\(.\).*\1.*\2.*\3/d

/\(.\).*\(.\).*\(.\).*\1.*\3.*\2/d

/\(.\).*\(.\).*\(.\).*\2.*\1.*\3/d

/\(.\).*\(.\).*\(.\).*\2.*\3.*\1/d

/\(.\).*\(.\).*\(.\).*\3.*\1.*\2/d

/\(.\).*\(.\).*\(.\).*\3.*\2.*\1/d

EDIT:

It looks like its this line

Code:
/\(.\).*\1.*\(.\).*\2/d

it seems to work without this line. what did this line do?

RE: Limiting the consecutive occurrence - Hash-IT - 06-01-2012

Excellent work M@LIK.

I am just going to summarise the regular expressions part of this before we all get lost on what it all means.

So this is the regular expression code only not SED. Can you help me finish the explanation of the code M@LIK ?

Code:
.)\1\1  =  No more than 2 consecutive characters.  Add \1 for more.

(.).*\1.*\1  =  No more than 2 characters in a given line.  Add .*\1 for more.

(.).*\1.*(.).*\2  =  

(.).*(.).*\1.*\2  =  

(.).*(.).*\2.*\1  =  

(ABCD|BCDE|CDEF|DEFG|EFGH|FGHI|GHIJ|HIJK|IJKL|JKLM|KLMN|LMNO|MNOP|NOPQ|OPQR|PQRS|QRST|RSTU|STUV|TUVW|UVWX|VWXY|WXYZ)  =  No characters in alphabetical sequence of more than 3

M@LIK, can I interest you in writing a wiki page for the hashcat wiki on this subject ? I think many people are interested in it but are just not able to help. Just take a look at the number of times this thread has been read !!! Big Grin

RE: Limiting the consecutive occurrence - M@LIK - 06-01-2012

Pixel Wrote: M@LIK I think I found a bug:
...
The red ones are missing .

Opos! My bad, we should have removed all these three lines:

Code:
/\(.\).*\1.*\(.\).*\2/d

/\(.\).*\(.\).*\1.*\2/d

/\(.\).*\(.\).*\2.*\1/d

Pixel Wrote: Thanks M@LIK your the best. Oh WOW, lot of commands for this one. Do you know and good sed tutorials that are good? I have had a look but, just wanted to see if you had any that you would recommend having a you at?

I often look here: http://www.grymoire.com/Unix/Sed.html
But that only covers about 30% of sed, It's really an unlimited tool.

Hash-IT Wrote: I am just going to summarise the regular expressions part of this before we all get lost on what it all means.

So this is the regular expression code only not SED. Can you help me finish the explanation of the code M@LIK ?

Code:
.)\1\1 = No more than 2 consecutive characters. Add \1 for more. (.).*\1.*\1 = No more than 2 characters in a given line. Add .*\1 for more. (.).*\1.*(.).*\2 = (.).*(.).*\1.*\2 = (.).*(.).*\2.*\1 = (ABCD|BCDE|CDEF|DEFG|EFGH|FGHI|GHIJ|HIJK|IJKL|JKLM|KLMN|LMNO|MNOP|NOPQ|OPQR|PQRS|QRST|RSTU|STUV|TUVW|UVWX|VWXY|WXYZ) = No characters in alphabetical sequence of more than 3

We removed the last 4 rules, so no need to explain them.
However, it's all in:

Code:
(.) = Any character.

\1 or \2 or \3 ... \9 = Any occurrence of that same character.

Note that the \[1-9] are considered in sequence:

Code:
(.)\1 = Any two same-characters next to each other. E.g: AA or BB or 33. BUT NOT: AB, 12.

(.)\1(.)\2 = This will match: AABB, NN55, KK.., ^^$$. BUT NOT: ABAB, MNMN, 1313.

.* = Means there is a possibility that the characters are not next to each other:

     E.g: (.).*\1 Will match: 1B1, 11, 1SDH1, 1MBKDOEF1, OIU11TRW. BUT NOT: ABCD, 1234

Hash-IT Wrote: M@LIK, can I interest you in writing a wiki page for the hashcat wiki on this subject ? I think many people are interested in it but are just not able to help. Just take a look at the number of times this thread has been read !!!

Sounds good! My exams are still going, I'll finish by the end of next week, let's wait til then.

RE: Limiting the consecutive occurrence - Hash-IT - 06-01-2012

Thanks for the clarification M@LIK !

(06-01-2012, 03:44 PM)M@LIK Wrote: Sounds good! My exams are still going, I'll finish by the end of next week, let's wait til then.

Oh, sorry you did mention you had exams and I completely forgot about that. I won't ask you anything directly until the end of next week as your exams MUST come first. The results can affect the rest of your life so don't waste you time on forums !!!

Good luck with them and I look forward to you having some spare time (after your exams) to talk more about this subject.

RE: Limiting the consecutive occurrence - ntk - 06-01-2012

thanks Pixel and Hash-IT for the default password information.

Good luck to your exams, M@LIK.

RE: Limiting the consecutive occurrence - Hash-IT - 06-01-2012

(06-01-2012, 11:30 PM)ntk Wrote: thanks Pixel and Hash-IT for the default password information.

You're welcome. Big Grin

RE: Limiting the consecutive occurrence - ntk - 06-02-2012

not wanting to confuse anyone. Post this contribution just in case someone wants to experiment it ONLY.
-------------------------------------------------------------------------------------
from http://forums.gentoo.org/viewtopic-t-686732-postdays-0-postorder-asc-start-25.html

Akkara wrote:

$ time egrep '^have|^want' file.txt >/dev/null
real 1m5.023s
user 1m4.886s
sys 0m0.093s

$ time egrep '^(have|want)' file.txt >/dev/null
real 1m5.469s

========================================================================

SOLUTION
alias c_grep='LC_ALL=C egrep' then use c_grep something

user 1m5.346s
sys 0m0.070s

Using sed is fast:
Code:
$ time sed -n -e '/^have/p' -e '/^want/p' file.txt >/dev/null
real 0m0.264s
user 0m0.263s
sys 0m0.000s
Why is there such a large disparity?
--------------------------------------------------------------------------------------
Genone wrote:

grep is apparently heavily affected by locale settings (e.g. bug 283149), so try running it with LC_ALL=C to see if that changes anything.

--------------------------------------------------------------------------------------
Akkara wrote:

That's the problem!
Code:
$ time LC_ALL=C egrep '^have|^want' file.txt >/dev/null
real 0m0.143s
user 0m0.143s
sys 0m0.000s

RE: Limiting the consecutive occurrence - Hash-IT - 06-02-2012

I have just generated "A" in full then filtered with regular expression to remove more than 2 adjacent same characters only.

Changed the full .txt file from ...... 74.8GB to ....... 74.1GB Tah dah !!!

Hmmm... not much of a saving really. Sad

RE: Limiting the consecutive occurrence - Hash-IT - 06-02-2012

@Pixel

Bad news Pixel, I have a SKY key which doesn't duplicate a character. I think it was just pure chance the ones we knew had that.

Sorry to break your theory, but well spotted anyway. However you may still wish to continue it as there may be a more likely chance that it occurs than not. So it's not all bad news !

RE: Limiting the consecutive occurrence - undeath - 06-02-2012

you guys seriously need some math courses.