+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Misc (https://hashcat.net/forum/forum-15.html)
+--- Forum: User Contributions (https://hashcat.net/forum/forum-25.html)
+--- Thread: bash script for email rules (/thread-2581.html)
bash script for email rules - liquiz - 09-02-2013
Hello Everyone,
I have been slogging through the southwest md5 hash set that they released a few years ago. It's really my first try at this and for those of you have never seen the southwest hash set its over 110 million. Suffice to say in a file that size there are some odd things you stumble across. Email addresses may not be the oddest thing but some of the domains are which has got to generate hashes that your are likely to never figure out.
The bash script itself is probably nothing special but it will save someone some time so you don't have to do it yourself.
All you need is a input file, you can modify the script if you like to change names or add/remove some some of the rules being applied, named domainlist-master.input. The script will generate 4 files, 2 rules and 2 dictionary.
master-lemail.dict - lower case email dictionary
master-uemail.dict - upper case email dictionary
master-lemail.rule - lower case email rule
master-uemail.rule - upper case email rule
CAVEAT EMPTOR - I cobble together code all the time, it doesnt always end up being pretty nor do I guarantee it will work for you
bash script that is normally run on Ubuntu and Scientific Linux
Code:
# log/dictionary for things in lower case
lrulefile="master-lemail.rule"
ldictfile="master-lemail.dict"
# log/dictionary for things in upper case
urulefile="master-uemail.rule"
udictfile="master-uemail.dict"
function printdict () {
mystring=$1
for pr in "@" "#" "~"
do
echo "\$${pr}${mystring}" | tee -a $udictfile
echo "\$${pr}${mystring}" | tee -a $ldictfile
done
}
function printrule () {
mystring=$1
myustring=$(echo "${mystring}" | tr '[a-z]' '[A-Z]')
for pr in "@" "#" "~"
do
for x in ":" "d" "l" "u" "c" "d" "f"
do
echo "${x}\$${pr}${mystring}" | tee -a $lrulefile
echo "${x}\$${pr}${myustring}" | tee -a $urulefile
done
done
}
for foo in $(cat domainlist-master.input)
do
mystring2="${foo}"
mystring=""
for (( i=0; i<${#foo}; i++ )); do
myc=$(echo ${foo:$i:1})
mystring="${mystring}\$${myc}"
done
printdict $mystring
printrule $mystring
doneA brief overview of what I have observed, a majority of what you find will be the username@domain.com followed by username#domain.com then username~domain.com and so on. If you follow standard analysis you will find that lower case will get a higher percentage # of hits than uppercase.
The real secret sauce in this will be your input file, some of whats in mine come from Hashit/T0XIC/Blandy UK but by far and away most of my work was in .edu and rr.com. I also spent some time scraping leaks in pastebin for various domains. The scraping of pastebin type sites has probably led to more of the unusual entries which turned up as hits in the southwest md5 set.
Based on the response I will see about posting either the rule files themselves or the input file or even both.
Snapshot of compressed information since Im not sure what the limit will end up being for attaching a files.
-rw-r--r--. 1 root root 24K Sep 2 12:34 domainlist-master.7z
-rw-r--r--. 1 root root 308K Sep 2 12:35 emailrules-dictionary.7z
Peace
RE: bash script for email rules - epixoip - 09-02-2013
i'm having kind of a difficult time reading your script, you seem to be adverse to both indentation and POSIX.
anyway, if this does what i think it does, you don't need to use the rule engine for this, nor do you need an external script or program to generate these candidates. this functionality is all built into plus.
examples,
-a 1 usernames domains -k '^@'
-a 1 usernames domains -j u -k '^@ u'
-a 1 usernames domains -k '^#'
-a 1 usernames domains -j u -k '^# u'
-a 1 usernames domains -k '^~'
-a 1 usernames domains -j u -k '^~ u'
RE: bash script for email rules - liquiz - 09-02-2013
(09-02-2013, 07:15 PM)epixoip Wrote: i'm having kind of a difficult time reading your script, you seem to be adverse to both indentation and POSIX.
anyway, if this does what i think it does, you don't need to use the rule engine for this, nor do you need an external script or program to generate these candidates. this functionality is all built into plus.
examples,
-a 1 usernames domains -k '^@'
-a 1 usernames domains -j u -k '^@ u'
-a 1 usernames domains -k '^#'
-a 1 usernames domains -j u -k '^# u'
-a 1 usernames domains -k '^~'
-a 1 usernames domains -j u -k '^~ u'
Apologies on that, I didnt catch the formatting until after the post. I will edit that and post the after formatted version. I use a script called beautify-bash.py (http://arachnoid.com/python/beautify_bash_program.html) and the rule script started on one system that had it but finished somewhere completely different.
If I get the chance today I'll try and decipher the example, rules and some of the flags still give me problems.
The script and its uses where done prior to the 15 version of plus which didnt handle long names etc .. so Im using it mostly with hashcat itself. The runs take a bit to finish so I have just started looking at how 15 handles them.
Peace
RE: bash script for email rules - liquiz - 09-02-2013
Just to follow up to put things in context.
Starting with oclHashcatplus v0.15 you have two methods to go at email addresses.
What epixoip suggests is using a combination attack and the specifying a rule for one side or the other. I didn't have access to 15 so I hadn't even considered this approach.
-a 1 username domain -k '^@'
That syntax translates to
-a 1 = combination attack
username = left dictionary
domain = right dictionary
-k '^@' = pre-pend an @ symbol to each word in the domain dictionary
Many thanks to epixoip for taking the time to go over that with me.
The other approach is a rules based approach which is the one I took.
Both approaches would appear to be viable with the only differences probably being speed.
Peace
RE: bash script for email rules - liquiz - 09-02-2013
Attached is the input file that I use for my script, just remove the .txt from the end.
You should be able to use this for both the combination and the rules based email list approach.
If for some reason the forum link is giving you problems try http://www.int21h.org/research/domainlist-master.7z
Thats a short term alternative until I find a better place to store files.
Peace
RE: bash script for email rules - liquiz - 09-07-2013
I was getting ready to test the speed of the two approaches and noticed something in the rule based approach.
Even though v15 of plus will handle long passwords it still struggles with long rules.
I still plan on timing them but I also need take into account/verify the amount of coverage each approach takes.
Peace