Interessting converation on IRC

[atom]

Tonight there was an interessting conversation on #hashcat IRC. A User "jnpplf" who's new to hashcat joined and asked couple of questions that we often see from people who have some background on hashcracking but either come from other tools or were not active in the hashcracking world for a few years and have outdated information.

I think it's an interessting read so I asked for epixoip's and jnpplf's permission to post it here and they agreed.

Code:

jnpplf (~jnpplf@unaffiliated/jnpplf) has joined #hashcat jnpplf If my password dictionary is a) numeric only, and b) huge, is there any feasible way of storing it in something that supports integer storage to save space, but also be able to use it with hashcat? jnpplf Rough math, I believe that would make my wordlist 186GB as text (though I have others that would require much, much more space), or 78GB as ints epixoip doesn't make any sense to have a large numeric-only dictionary epixoip straight wordlist attacks are slower than brute force, and brute forcing digits is fast. epixoip single 7970 can do len 1 - 12 in just over 3m jnpplf So, use a mask to 'generate' the numbers? epixoip yes jnpplf That makes sense epixoip for example, -a 3 -i ?d?d?d?d?d?d?d?d?d?d?d?d jnpplf How much benefit would I see from precomputing the hashes? epixoip zero jnpplf Really? epixoip there's a reason we don't use rainbow tables anymore. jnpplf Is this a disk speed thing? epixoip no, it's a "tmto is impractical and slow" thing epixoip gpus have by and large deprecated rainbow tables jnpplf Interesting. How about if the hashes are generated with some hardcore key stretching / bcrypt with high rounds? jnpplf I'm off of my original question and into theory now, apparently my info is way out of date epixoip you can't have a rainbow table for bcrypt anyway epixoip or any other salted algorithm epixoip so that question does not apply jnpplf Oh, you can't run bcrypt without a salt? epixoip nope epixoip nor would you want to jnpplf Also news to me. jnpplf catches up to 3 years ago jnpplf Rainbow tables are /really/ redundant? epixoip yes jnpplf Dang. epixoip no one uses them anymore jnpplf Well that's an item crossed off of my todo list :) epixoip haha jnpplf Don't suppose you have a link to the tmto for rainbow tables being redundant so I can read up on it? epixoip it's common knowledge, don't know if anyone has bothered to write about it jnpplf It's not THAT common epixoip it is among password crackers. epixoip it's not really something you need to read much about. just do the math jnpplf If I were trivially capable of doing that math, I probably wouldn't be here epixoip rainbow tables are large, inflexbile, and you have to have one set per length, and per algorithm. they also don't scale well at all with multiple hashes. epixoip gpus are fast, flexbile, and scale linearly jnpplf Interesting. Maybe I'll just spend my time getting better with hashcat's mask syntax then epixoip good idea jnpplf So if I wanted to, say, crack (dog|cat)1234, would I have to run two separate attempts? I can't figure out a way of writing a mask to match that. jnpplf That's supposed to be a logical OR rather than a literal epixoip you wouldn't use a mask attack for that epixoip you'd use a hybrid attack jnpplf dictionary + mask jnpplf I seeeeeeeee epixoip right jnpplf Good stuff, thanks jnpplf : dont read on it, do a benchmark, digit wordlist vs gpu bruteforcing epixoip no worries jnpplf oo, had another question before I shut up jnpplf What if the second half of the password is derived from the first half? I could pipe in valid values somehow, but that means I don't apply either the mask or dictionary? Just generate something from a shell script and get hashcat to accept external input? epixoip oclhashcat supports reading from stdin epixoip cpu hashcat can read from a named pipe jnpplf so generate_passwords.sh > oclhashcat [...] will work? epixoip |, not > epixoip but yes jnpplf Would | be for cpu hashcat? jnpplf wouldn't* epixoip no epixoip for cpu hashcat you'd have to use mkfifo to create a named pipe epixoip then specify the name of the pipe as a wordlist jnpplf Ah. Funky, and beyond my understanding. epixoip generate_passwords.sh > oclhashcat would replace the contents of the oclhashcat binary with the output of generate_passwords.sh jnpplf so totally_optimised_awesome_script | oclhashcat -awesome_args is my best bet jnpplf epixoip, good point epixoip yep jnpplf Any idea what the fastest language might be for generating those? I feel like bash isn't the best candidate. jnpplf Or, alternatively, is that not going to be the bottleneck? epixoip doesn't matter epixoip it's going to be slow regardless jnpplf slow? jnpplf That's exactly what I don't want. epixoip it's unavoidable in this instance, as you're generating plains on the cpu and then transferring them to the gpu epixoip you will not be able to fully utilize the gpu in this instance jnpplf Unless I could pregenerate the list ;) jnpplf Or is the CPU still faster? epixoip straight wordlist attacks are also slow on gpu epixoip becuase you're not giving them enough work epixoip in order to fully utilize gpus with wordlists you have to add rules epixoip for straight wordlist attacks, gpu is either same speed or slower than cpu jnpplf Hmmm, doesn't look like any of the rules would help in my scenario epixoip you could always write your own rules, if needed. jnpplf that's the maskprocessor stuff? epixoip external mask processor epixoip standalone implementation of the hashcat mask engine jnpplf Hmmmmmmmm. So the rules still rely on hashcat's mask formatting, which rules out appending the derived part of the password. Might be able to speed up the prefix generation though? jnpplf Not sure it would help versus using a dictionary for prefixes as they're known epixoip no epixoip the rules are used with wordlists, and have their own syntax. it's a small programming language, really. epixoip see http://hashcat.net/wiki/doku.php?id=rule_based_attack jnpplf My problem is that, while the rules look suitably wonderful, the help for maskprocessor makes it look extremely limited epixoip the rules have nothing to do with maskprocessor epixoip mask processor is just for processing masks. epixoip the two have nothing to do with eachother jnpplf Oh epixoip maskprocessor is mostly so that you can add hashcat's mask functionality to other programs, like pyrit or john the ripper. jnpplf "Using maskprocessor to generate rules" jnpplf That's confusing :p epixoip oh epixoip yeah i suppose that is misleading epixoip english is atom's second language so you will have to forgive him. epixoip oh no no epixoip i see what you're referring to epixoip in that instance, that's actually using mask processor to help you generate repetitive rules epixoip which is most certainly valid epixoip like if you wanted to generate a rule that appended two lower alpha to each word in a word list epixoip you wouldn't want to type that shit by hand jnpplf Wouldn't that just be a hybrid attack though? jnpplf dictionary + 2 alpha as a mask? epixoip so you can just use mask processor to generate the rule file for you: ./mp '$?l$?l' epixoip yeah, but in some instances you can't use a hybrid attack epixoip then you can use multi-rules epixoip in some cases it's also more efficient to use rules than hybrid attack epixoip such as when using mask + dict with a very small mask epixoip in that case it would be much more efficient to use rules jnpplf But if the mask is long, it's more efficient to use the hybrid? epixoip yes epixoip if possible jnpplf Brain is about to crawl out of my ear :p epixoip what if you want to use a hybird attack + rules? epixoip can't do it epixoip but you can use rule chaining epixoip anyway that's not really maskprocessor's raison d'etre, that was really just mentioned as an aside epixoip just something neat that you could do with mp jnpplf The examples on the rule attack page make it look like it's most effective if you're trying to fuzz the wordlist. If the format is fairly predictable, does that make it less useful? epixoip you could also accomplish the same thing with a shell one-liner, but it would be slower. epixoip if the format is predictable then you can simply write predicatble rules jnpplf OK, this is going to be much easier if I just explain what I'm doing :p epixoip for example, if the algorithm is md5(strtoupper(pass)), then you can use the single rule 'u' for all of your attacks jnpplf dict + known number of digits + single character checksum epixoip ok, in that case just use -a 6 epixoip what charset is the checksum comprised of? jnpplf numeric epixoip ok so then it's really just dict + known number of digits epixoip -a 6 dict.txt ?d?d?d?d?d epixoip or whateer jnpplf Would that be faster than calculating the final digit? epixoip yes, much faster jnpplf So I'm solving the wrong problem :p epixoip with hybrid attacks, the canddiates are generated on the gpu, enabling full acceleration jnpplf So the GPU would have to be over 10 times faster than the CPU to make that worthwhile epixoip and the extra digit on the end only adds 1^10 complexity epixoip yes epixoip and the GPU is a lot more than 10 times faster than the CPU jnpplf Interesting. epixoip in the case of MD5, GPU is probably 100x faster than CPU epixoip maybe more jnpplf This is more likely to be SHA-1 jnpplf Just because 2013 :) epixoip sha-1 should be about 60x faster than cpu jnpplf And that assumes a nice saucy AMD card, rather than my midrange NVIDIA? jnpplf ATI*, I guess epixoip that doesn't mean much. raw md5 is still the most widely used algorithm for password storage, even in 2013. epixoip amd is correct. ati doesn't exist anymore. epixoip and yes, those figures are for a 7970 jnpplf I know, but I rarely see them called AMD cards epixoip we call them amd cards here jnpplf Check you guys out :p epixoip nvidia sucks for password cracking, so it definitely won't be 60x faster epixoip but it will probably be at least 10x faster than cpu jnpplf But then the AMD benchmarks seem to be about 6 times faster than CUDA vn but it rocks for gaming jnpplf Also that :p epixoip who plays games? vn not you, obviously. epixoip obviously. jnpplf Worst case, as long as using the GPU isn't obviously horrible compared to the CPU, even with cuda, I'd write for AMD and then offload the processing to someone with an AMD card jnpplf Or 4, as this case may be :) jnpplf That said, EC2 offers some GPU nodes now, right? Are they AMD based? epixoip no epixoip they're super old teslas epixoip and telsas are worse than their GT* counterparts for crypto epixoip ec2 is a massive waste of money if you intend to use it for password cracking epixoip if this is something you plan to do often, go pick up a cheap AMD gpu, like a 7950 epixoip costs less than an ec2 gpu instance. epixoip 10x the performance jnpplf I am learning all the things this evening epixoip good, good jnpplf Is this related to your day job? epixoip yes. jnpplf infosec, or specificially hash cracking? jnpplf Stricture? :) epixoip yes, stricture jnpplf Oh cool. I've sent some business your way :) epixoip orly? jnpplf Well, in truth I've referred people to you jnpplf It may not have become business epixoip right on, we appreciate it jnpplf You're the only company I know of that does it as a core offering epixoip yup epixoip i don't know of any others, either epixoip although we're trying to get into hardware sales as well jnpplf ASICs, or just selling people 7950s? :p epixoip no, not asics. just dedicated clusters epixoip we have a few customers we're building clusters for now, using 7970s and 7990s jnpplf Why would someone buy a cluster of commodity hardware? epixoip it's not commodity hardware epixoip we use enterprise grade hardware with commodity gpus, and a warranty epixoip we use commodity gpus because there's no better option for crypto epixoip some would say firepro would be more "appropriate," but our customers don't want to spend 4x as much for half the performance. epixoip and i don't blame them, i certainly wouldn't advise them to do it jnpplf Had never heard of them until now. epixoip firepro? jnpplf Yeah jnpplf Just googled them up epixoip that's amd's professional workstation and server product line epixoip but they still use the same GPUs as the radeon line epixoip just underclocked jnpplf So how many people are at Stricture? 1 - 10 on LinkedIn, but only one with a profile epixoip 5 jnpplf Interesting jnpplf Time to go fumble my way through some masks then, I guess epixoip i already gave you the mask :P jnpplf Yeah but I'm a spaz ;) epixoip -a 6 dict ?d?d?d?d?d epixoip plug n play epixoip (that's assuming it's word + 4 digits + 1 digit checksum) jnpplf If I need to go through a few different masks/dicts, is there an obvious efficiency to trying to get them all going in the same execution? epixoip alter mask appropriately. epixoip yeah. the more work you give it, the more of the gpu you will utilize epixoip you have to keep it busy to make efficient use of it jnpplf So... if my dict were (it isn't) one digit long jnpplf In fact, ignore that jnpplf If I know the first n characters of a password, at what point does it become more efficient to use a dictionary versus a mask? jnpplf Can my mask contain a constant? epixoip yes, your mask can contain a constant jnpplf So all passwords begin with 'j', use j?d?d epixoip yup epixoip and if that's the case, then it would be more efficient to use a mask jnpplf How about ja..jc? epixoip -1 abc j?1?d?d/d... epixoip er, ? not / jnpplf And still faster than a dictionary? epixoip yes epixoip the mask engine generates candidates on the gpu epixoip which makes it very fast jnpplf So this comes back to the 10x / 60x / Nx speed increase of the GPU over the CPU? epixoip right. jnpplf So I'd need the keyspace of the additional mask to be 60 or more values if my GPU were 60x faster than my CPU epixoip i'm not sure what you mean jnpplf Oh, erm... jnpplf So if I'm cracking dict + mask, it becomes more efficient to use mask+mask as long as the first mask isn't generating more than 60 candidate values jnpplf assuming a 60x speed increase on the GPU epixoip no jnpplf balls epixoip that only applies if you're doing additional work epixoip like in the case of just brute forcing the checksum epixoip we'd have to 1^10 additional work epixoip if you're doing a straight mask like in the example above, then it would still only be 1^10 additional complexity over the script that calculates the checksum. jnpplf Sure jnpplf So.... you're saying the dictionary is FASTER than the mask as a prefix? epixoip the mask attack will always be much faster than a dictionary or hybrid attack because all of the canddiates are generated on the GPU, there's no memory or host-to-device transfering happening. epixoip no. jnpplf So why would you ever use a dictionary? epixoip because most people don't crack plaintexts that follow some defined pattern like this epixoip i'm speaking to your specific case, where you have this one specific pattern that you're exploiting jnpplf OK, so say I'm cracking 8 digit numeric only jnpplf Oh wait jnpplf COnfusing myself, gimme a sec :p epixoip 8 digit numeric you would absolutely want to do a mask attack, no question jnpplf What if part of it were predictable though? jnpplf Yep, did it again epixoip then you have a choice jnpplf That's when you use the static mask :p epixoip right epixoip you can use constants in the mask, or you can do hybrid jnpplf I think my brain's hitting its new info limit epixoip the former would likely be faster epixoip let me give you some advice: stop talking/thinking about it, and just go try it :P jnpplf Yeah epixoip once you start doing it you will understand jnpplf I've done it before, I was looking for efficiency improvements :) epixoip there are several different ways to skin this cat epixoip if you know that the password is exactly 8 characters long, starts with a j, and ends in 5 digits, then mask attack is the obvious choice epixoip you can just blow through it with j?l?l?d?d?d?d?d jnpplf If it were 8-starting-with-j or 9-starting-with-x, would I run 2 separate jobs or try to force that into a mask? epixoip two separate masks epixoip the attacks will run very quickly jnpplf Does hashcat accept multiple masks in one run? epixoip no epixoip but you can use a for loop epixoip for m in j?l?l?d?d?d?d?d x?l?l?l?d?d?d?d?d; do ./hashcat -a 3 $m ; done jnpplf Gotcha jnpplf OK, that's all I'm going to ask :) jnpplf Going to go melt my lovely graphics card jnpplf Thanks a lot for the help epixoip :) epixoip no worries epixoip have fun! jnpplf rages into the wilderness jnpplf -a 6 -1 abc hashes.txt j?1?d?d?d... doesn't run jnpplf Just gives me a syntax error jnpplf Same if I shift -1 abc to after the hashfile name jnpplf headdesks jnpplf Wait. I'm missing the hash type. jnpplf hnnnngh, it always takes me ages to figure out how many args I'm supposed to be passing to this thing epixoip using sha1 right? jnpplf Yeah jnpplf I added the hash type jnpplf It was acting like it didn't have enough args, I split my mask into two and it tries to open one half as a file... epixoip ./hashcat -m 120 -n 80 -u 1024 -o jnpplf.pot hash.txt -a 6 -1 abc hashes.txt j?1?d?d?d epixoip oh epixoip i just copy/pasted your line epixoip and i see your problem epixoip er nm, no i don't. jnpplf hash.txt, hashes.txt? epixoip er yes i do epixoip sorry, i'm tired epixoip you're using -a 6 instead of -a 3 epixoip and hash.txt, hashes.txt, whatever your file is called jnpplf Well you have both, is my point jnpplf You have an output, an input, and something else epixoip that's because i just copy/pasted your line and tacked it on mine without reading it :) jnpplf Ah jnpplf Seems to be running now :) epixoip ./hashcat -m 120 -n 80 -u 1024 -o jnpplf.pot hash.txt -a 3 -1 abc j?1?d?d?d jnpplf That's the ticket, though without the performance args at the moment jnpplf ooo, this is 550MH/s, versus the 300 or so I was getting with my original approach. Fun :) epixoip add the performance args, should be able to get a bit more. jnpplf GPU is 99% util epixoip sure, but -u for example optimizes the number of loops per iteration. epixoip so you will still likely see better performance jnpplf Those args are 100% indecipherable to me ;) I'll try with though jnpplf Seems a tiny bit faster jnpplf 555 vs 553 at first glance jnpplf Computer running like crap ;) epixoip what gpu is this? jnpplf 660Ti epixoip ah jnpplf That's also on a single hash epixoip oh you're doing a single ahsh? jnpplf Will generate a whole crapload to test against later epixoip you don't want to use -plus then jnpplf Just trying to get stuff stood up for now epixoip you want to use -lite jnpplf I don't even have lite. Separately download? epixoip yes epixoip lite is optimized for brute forcing a single hash jnpplf I'm surprised they're separate binaries though jnpplf Surely that would be easy to switch in code jnpplf After all, it knows I'm doing a single hash because I had to use --force epixoip completely different code base jnpplf Oh really? r4d1x not it isnt epixoip no? jnpplf grabs popcorn epixoip atom told me they were quite different r4d1x sry my reply was to "Surely that would be easy to switch in code" epixoip oh oh okay :) jnpplf Ah epixoip you had me very confused :P r4d1x epixoip: have you tried running amd and nvidia in the same box at all? epixoip not with hashcat, but with multiforcer we have epixoip or were you asking at an os level, like installing drivers for both and having xorg see both r4d1x just wondering if you had tried with hashcat epixoip ah. no i've not tried with hashcat r4d1x might have to throw a 580 in and see what happens jnpplf Hmm. If I write a mask of j?1?d?d?d..., where -1 is abc, that will only try ja, jb, jc, right? Not jab, jac, jabc... r4d1x so, im listening to JFK tower, and the controller asks this guy to turn left into a storm, pilot comes back, I'm not turning into that storm, controller repeats turn left, pilot declares emergency, controller says he cant, pilots repeats declaring an emergency, controller clears him to land jnpplf I've got 1 constant, second character is a choice of 2, 3rd onwards should all be ?d, but my Plain.Text starts with **** r4d1x moral of the story, if you dont want to comply with the controller, declare and emergency r4d1x s/and/an epixoip jnpplf: yes, that's correct. it would only try ja, jb, jc, and not jab, jac, jabc, etc. jnpplf In fact, why would Plain.Text not show me the full value being tested anyway? epixoip r4d1x: that's awesome r4d1x I know right? r4d1x turn left r4d1x no r4d1x turn left r4d1x FUCK YOU r4d1x cleared to land epixoip jnpplf: because it's trying 550 million combinations per second; you want it should print all 550 million of them? :) jnpplf Then what jnpplf 's Plain.Text even used for? epixoip to give you a rough indication of where you are. jnpplf Starting from the back? :-/ jnpplf Surely the first 4 digits are the most important jnpplf Hmmm, something is up. jnpplf Why, if I gave it a 10 digit mask, did it try 8 digits? jnpplf Exhausted that, moved onto 9 r4d1x becuse you didnt set --pw-min=10 r4d1x lite assumes you want to increment jnpplf I assumed the length was defined by the number of characters I put in the mask jnpplf Ah epixoip are you doing cpu or gpu? epixoip oh you'r eusing lite jnpplf I am now, yeah epixoip right, lite increments jnpplf Gotcha. jnpplf But the mask will work fine on its own in plus? r4d1x unless you specify --increment jnpplf Freaky :p epixoip yeah, plus requires an explicit switch (-i) to enable incrementing r4d1x well, when that increment stuff was added I raised hell r4d1x but I dont care about lite epixoip yeah i don't like autoincrement jnpplf Neither do I, normally jnpplf This would explain why the progress values were so fast ;)