hcxtools - solution for capturing wlan traffic and conversion to hashcat formats

[ZerBea]

Nice to hear that. Thanks.
Updated hcxpcaptool - new option:
-o <file> : output hccapx file
Less overhead than wlancap2hcx but more crackable handshakes.
Only the best handshake (combination of mac_sta, mac_ap, ESSID) is converted.
Now we run several stages to detect a handshake. That makes us a little bit slower on big caps
(usually merged caps) which contains many, many different networks.
The identitiy list is no longer limited to 258 characters per identity.
And some other improvements.
hcxpcaptool and hcxdumptool are still under heavy construction. That means not all wlandump-ng and wlancap2hcx options are implemented, yet. But they represent the third generation of hcxtools an will replace older wlanhcx.... tools step by step.

[ZerBea]

Updated hcxpcaptool - new (long) options (now we went into direct sync with hashcat and JtR!):

--time-error-corrections  : maximum allowed time gap (default: 10000s)
--nonce-error-corrections : maximum allowed nonce gap (default: 8) [should be the same value as in hashcat]

Use the same values as in hashcat or JtR.
That means, if you convert the cap to hccapx using --nonce-error-corrections=128
you must(!) use the same value in hashcat or in JtR!

The --time-error-corrections is the maximum allowed timegap between 2 messages
within the authentication.

Result: only one (the best !) handshake per combination mac_sta, mac_ap, ESSID.
No longer need to remove duplicates from your hashfile!

If you have bad reception, use higher values for both options!

That makes us more flexible and brings us more crackable handshakes (~20% more).

And keep in mind:
The quality of a hccapx file is based solely on on the skills of the attack tool and the conversion tool.
If the attacker or the dumper fails - the hccapx will be crappy (even on a messagepair M1M2M3M4)!

But nevertheless, I added an option to convert to raw handshakes (all handshakes):
-O <file> : output raw hccapx file

[hulley]

Thank you ZerBea, wonderful work!

[ZerBea]

Hi hulley.
I'm not shure if it's a wondeful work. It's a hard work for the tool to reduce overhead from a hccapx file.
And its much, much more than comparing only mac_sta, mac_ap and ESSID or messagepair.
I failed epically to reduce this overhead (on a hccapx), so I decided to implement this in hcxpcaptool (and that needed a complete rewrite).

Some words about --nonce-error-corrections
hcxpcaptool doesn't correct the nonce (like hashcat it does - hashcat can do this much better).
Using --nonce-error-corrections reduces the overhead, because only handshakes up to this value
are converted. If you decide to convert up to a value of 256 you will get more handshakes (on crappy reception),
but you need also the same value in hashcat to recover the password from them.
In this case the price tag is  speed drop!

[ZerBea]

And this is a nice test that explains why I gave it up to clean a hccapx:

Convert a cap to hccapx using both options -o and -O
$ hcxpcaptool -O testO.hccapx -o testo.hccapx 201801161903.cap
start reading from 201801161903.cap
summary:                                        
--------
file name..............: 201801161903.cap
file type..............: pcap 2.4
network type...........: DLT_IEEE802_11 (105)
endianess..............: little endian
read errors............: flawless
packets inside.........: 3153
skippedpackets.........: 0
probe requests.........: 125
probe responses........: 254
association requests...: 128
reassociation requests.: 21
EAPOL packets..........: 2612
raw handshakes.........: 179
usable handshakes......: 36

as you can see:
raw handshakes.........: 179
usable handshakes......: 36

Run hashcat using the following options on the "cleaned hccapx":
$ hashcat -m 2500 --nonce-error-corrections=8 --potfile-disable --logfile-disable --outfile-format=2 -o foundlisto testo.hccapx wordlist:
Speed.Dev.#1.....:   438.8 kH/s (2.66ms)

4 passwords recovered:
$ sort foundlisto | uniq | wc
     4       4      57

followed by a test on the "raw hccapx"
$ hashcat -m 2500 --nonce-error-corrections=0 --potfile-disable --logfile-disable --outfile-format=2 -o foundlistO testO.hccapx wordlist:
Speed.Dev.#1.....:   481.2 kH/s (2.66ms)

4 passwords recovered:
$ sort foundlistO | uniq | wc
     4       4      57

Now make up your mind:
- clean or not clean,
- nonce-error-corrections or not nonce-error-corrections,
- low value nonce-error-corrections or high value nonce-error-corrections
or any combination of this

BTW:
Do not wonder about both low speed values compared to a synthetic benchmark of a 1080TI (my wordlist is very small and that drops speed).

[ee10]

Latest version does not compile on macOS

Code:

wlancap2hcx.c:24:10: fatal error: 'openssl/sha.h' file not found
#include <openssl/sha.h>
        ^~~~~~~~~~~~~~~
1 error generated.

[ee10]

Fixed it by installing openssl using Homebrew

[ZerBea]

Well, the dependencies are a real problem and I'll get rid of this.
The latest update has only included minor changes (faster detection of driver crash on 4.4 kernel).
Please do a git clone again. I removed an ugly debug message (hallo - implemented to test Frame Check Sequence FCS).

Major update is done on hcxpcaptool:
-o <file> : output hccapx file
-O <file> : output raw hccapx file
-x <file> : output hccap file
-X <file> : output raw hccap file
-j <file> : output john WPAPSK-PMK file
-J <file> : output raw john WPAPSK-PMK file
-E <file> : output wordlist (autohex enabled) to use as input wordlist for cracker
-I <file> : output identity list
         : needs to be sorted unique
-P <file> : output possible WPA/WPA2 plainmasterkey list
-T <file> : output management traffic information list
         : european date : timestamp : mac_sta : mac_ap : essid
-A <file> : output nonce information list
         : european date : timestamp : mac_sta : mac_ap : message : replaycount : nonce
-S <file> : output EAPOL information list
         : european date : timestamp : mac_sta : mac_ap : message : replaycount : eapol_len : eapol
-H <file> : output dump raw packets in hex
-V        : verbose (but slow) status output
-h        : show this help
-v        : show version

--time-error-corrections  : maximum allowed time gap (default: 600s)
--nonce-error-corrections : maximum allowed nonce gap (default: 8)
                         : should be the same value as in hashcat

Prime goal was the detection of handshakes. You can test the new functions with the demo cap from this post:
https://hashcat.net/forum/thread-7207.html
Download the repository. Inside is a demo cap file: wpa2psk-linksys.cap (i think original from aircrack-ng trunk)

$ hcxpcaptool -o test.hccapx wpa2psk-linksys.cap
start reading from wpa2psk-linksys.cap
                                             
summary:                                        
--------
file name..............: wpa2psk-linksys.cap
file type..............: pcap 2.4
network type...........: DLT_IEEE802_11 (105)
endianess..............: little endian
read errors............: flawless
packets inside.........: 499
skippedpackets.........: 0
packets with FCS.......: 0
beacons................: 85
probe requests.........: 11
probe responses........: 6
association requests...: 4
association responses..: 4
authentications........: 8
deauthentications......: 3
EAPOL packets..........: 9
best handshakes........: 1
1 handshake(s) written to test.hccapx

$ hashcat -m 2500 test.hccapx wpa-sec.txt
hashcat (v4.0.1-84-g7c1fb014) starting...
Session..........: hashcat
Status...........: Cracked
Hash.Type........: WPA/WPA2
Hash.Target......: linksys (AP:00:0b:86:c2:a4:85 STA:00:13:ce:55:98:ef)
Time.Started.....: Fri Jan 19 19:01:33 2018 (1 sec)
Time.Estimated...: Fri Jan 19 19:01:34 2018 (0 secs)
Guess.Base.......: File (wpa-sec.txt)
Guess.Queue......: 1/1 (100.00%)
Speed.Dev.#1.....:   421.8 kH/s (2.74ms)
Recovered........: 1/1 (100.00%) Digests, 1/1 (100.00%) Salts
Progress.........: 40493/40493 (100.00%)
Rejected.........: 0/40493 (0.00%)
Restore.Point....: 0/40493 (0.00%)
Candidates.#1....:          -> ١٢٣٤٥٦٧٨٩
HWMon.Dev.#1.....: Temp: 34c Fan: 29% Util: 86% Core:1835MHz Mem:5005MHz Bus:16

This cap is a really nice playground (thanks to JohnDN90). You can play around with the raw options and the error-corrections values. Depending on this options you will retrieve the best handshake or all 9 handshakes (3 are crackable):

$ hcxpcaptool -O test.hccapx *.cap
start reading from wpa2psk-linksys.cap
                                             
summary:                                        
--------
file name..............: wpa2psk-linksys.cap
file type..............: pcap 2.4
network type...........: DLT_IEEE802_11 (105)
endianess..............: little endian
read errors............: flawless
packets inside.........: 499
skippedpackets.........: 0
packets with FCS.......: 0
beacons................: 85
probe requests.........: 11
probe responses........: 6
association requests...: 4
association responses..: 4
authentications........: 8
deauthentications......: 3
EAPOL packets..........: 9
raw handshakes.........: 9
best handshakes........: 1
9 handshake(s) written to test.hccapx

$ hashcat -m 2500 test.hccapx wpa-sec.txt
hashcat (v4.0.1-84-g7c1fb014) starting...
Session..........: hashcat
Status...........: Exhausted
Hash.Type........: WPA/WPA2
Hash.Target......: test.hccapx
Time.Started.....: Fri Jan 19 19:12:55 2018 (1 sec)
Time.Estimated...: Fri Jan 19 19:12:56 2018 (0 secs)
Guess.Base.......: File (wpa-sec.txt)
Guess.Queue......: 1/1 (100.00%)
Speed.Dev.#1.....:   264.5 kH/s (2.77ms)
Recovered........: 3/9 (33.33%) Digests, 0/1 (0.00%) Salts
Progress.........: 40493/40493 (100.00%)
Rejected.........: 0/40493 (0.00%)
Restore.Point....: 40493/40493 (100.00%)
Candidates.#1....:          -> ١٢٣٤٥٦٧٨٩
HWMon.Dev.#1.....: Temp: 39c Fan: 29% Util: 84% Core:1835MHz Mem:5005MHz Bus:16

[hulley]

Hi ZerBea

is there a file size limit in hcxpcaptool?
I have a 52.3mb and 54.1mb caps, hang at

start reading from pentest.cap
300000 packets processed - be patient!
[hours]
the two files hang at the same amount of packets.

[ZerBea]

Hi hulley.
No, the only limit is the size of your RAM (hcxpcaptool will tell you if we are running out of RAM). And that is the problem.
Handshake detection is time intensive and RAW handshake detection is very time intensive!
Time increases exponentially. That means for every M1, M2, M3, M4 and ESSID
we run threw the whole file! And caps with zeroed timestamps (typically by wpaclean converted) makes it again slower.
(EAPOL packets) * (EAPOL packets) * (unique(beacons + associationrequests + reassociationrequests + proberequestsproberesonses) * nonce-error-corections * time-error-corrections
Or for the example above: 9 * 9 * 1 * 8 * 600
If you have a merged cap or a cap containing many different ESSIDs and many EAPOL frames you should be patient.

BTW: Do you use the latest pushed version (14 hour ago - improved calculation speed by 1000%)

1. let's see how much packets are inside - we use a ryzen 1700 and the size of this cap is 302.338454 MB (the cap is a merged cap - own captures and different files from www, including deadly cleaned files):
$ hcxpcaptool 20170228.cap
start reading from 20170228.cap
                                             
summary:                                        
--------
file name..............: 20170228.cap
file type..............: pcap 2.4
network type...........: DLT_IEEE802_11 (105)
endianess..............: little endian
read errors............: flawless
packets inside.........: 1409389
skippedpackets.........: 0
packets with FCS.......: 0

1409389 = packet count
you will see it here: xxxxx packets processed - be patient!

2. now get the best hashes:
$ time hcxpcaptool -o test.hccapx 20170228.cap
start reading from 20170228.cap
                                             
summary:                                        
--------
file name..............: 20170228.cap
file type..............: pcap 2.4
network type...........: DLT_IEEE802_11 (105)
endianess..............: little endian
read errors............: flawless
packets inside.........: 1409389
skippedpackets.........: 0
packets with FCS.......: 0
warning................: zero value timestamps detected
WDS packets............: 14
beacons................: 398623
probe requests.........: 38459
probe responses........: 102777
association requests...: 61752
reassociation requests.: 12143
EAPOL packets..........: 789871
best handshakes........: 1735

1735 handshake(s) written to test.hccapx

real 0m29,577s
user 0m28,797s
sys 0m0,780s

3. or the raw hashes:
$ time hcxpcaptool -O test.hccapx 20170228.cap
start reading from 20170228.cap
                                             
summary:                                        
--------
file name..............: 20170228.cap
file type..............: pcap 2.4
network type...........: DLT_IEEE802_11 (105)
endianess..............: little endian
read errors............: flawless
packets inside.........: 1409389
skippedpackets.........: 0
packets with FCS.......: 0
warning................: zero value timestamps detected
WDS packets............: 14
beacons................: 398623
probe requests.........: 38459
probe responses........: 102777
association requests...: 61752
reassociation requests.: 12143
EAPOL packets..........: 789871
raw handshakes.........: 20414
best handshakes........: 1735

20414 handshake(s) written to test.hccapx

real 0m31,066s
user 0m30,221s
sys 0m0,844s


http://wpa-sec.stanev.org/  switched to hashcat and hcxtools.
You can upload your big caps there to see how many passwords can be recovered (by the improved system) using common wordlists. Service is complete free.


Cheers
Mike