hcxtools - solution for capturing wlan traffic and conversion to hashcat formats

[ZerBea]

This is a really good reason to hexify ESSIDs within the new hash line (22000):

http://support.sjcam.com/support/solutio...obile-wifi-

SSID: SJ4000WIFIdf\r\n\r\n9c4633d808
or in hex:
534a343030305749464964660d0a0d0a39633436333364383038

where SJ4000WIFI5e (hex: 534a34303030574946496466) is the name of the cam
and 9c4633d808 something VENDOR (e.g.: part of MAC) specific.

802.11 allow (ctrl characters 0x0d 0x0a) inside the ESSID information element (IE tag) field:

534a34303030574946496466
0d0a
0d0a
39633436333364383038

Take a look at the video. The ESSID goes over at least 2 lines in the menu
https://www.youtube.com/watch?v=rBxVGoXJzlA

[WPA_Catcher]

Hi ZerBea

I notice today that you have chosen to remove the ability for users to define mac_ap and mac_client as it makes hcxdumptool's stealth capability ineffective.

Some of us have no need for stealth and the ability to easily identify hcxdumptools mac_ap and mac_client is useful when viewing captures with wireshark.

I assume those who require stealth capabilities are unlikely to define mac_ap or mac_client anyway so could the option please be retained?

As you know I am learning and just playing with your wonderful tools as a hobby. I wondered if you might consider a feature request for people like me who are interested in how all this works.

When the new hcxpcapngtool outputs the new format 22000 could there be something like the hashcat debug option with rules where the user can see which rule was effective? It would be very useful to people learning if hcxpcapngtool could have a switch whereby if enabled hcxpcapngtool would output a .pcapng which contains only the items it used to produce each hash line and which it believed were the best to create the output hash line?

I am learning to make my own beacon, m1,m2,m3,m4 selections with wireshark and it would be interesting to see how my selections differ from the ones hcxpcapngtool makes. Also people who are experts at this sort of thing may also be able to help improve hcxpcapngtool if they notice that a better choice could have been made. I am not suggesting there is any likelihood of that by the way

[ZerBea]

Adding the option to run a user defined MAC_AP and/or MAC_CLIENT was only a test to demonstrate that it is useless.
Shortly after the start hcxdumptool adapts to the WiFi traffic and uses more and more received MAC addresses and ESSIDs.

Wireshark is a good tool to analyze traffic. But take care:
In monitor mode the adapter does not check to see if the cyclic redundancy check (CRC) values are correct for packets captured, so some captured packets may be corrupted (malformed).
http://www.ict-optimix.eu/images/a/ad/WiFiBitError.pdf
https://en.wikipedia.org/wiki/Monitor_mode
By latest commits, I added tons of code to detect this!

There is no need for an additional .pcapng output, because Atom added a similar feature to hashcat:
Now outfile contain the PMKID/MIC for the recovered hash. Just compare this field with the hashline field and take the option which was in use to successfully convert it.
Start with --all --ignore-ie --max-essids=5 --nonce-error-corrections=256 --eapoltimeout=43200000
and go more and more restrict.
You can use simple bash commands to do this comparison.
Read more here:
https://hashcat.net/forum/thread-8891-po...l#pid47266

Improving hcxpcapngtool isn't easy, because every code change due to an analysis of a single capfile has a huge impact on other capfiles (based on analysis of wpa-sec.stanev.org).

[WPA_Catcher]

Ah it being a demonstration explains a lot, I was getting a little lost with my experiments.

You are always one step ahead, it sounds like you are checking everything. I hope you enjoy polishing your code to ensure there are no bad captures/hashes as there is nothing worse for a WPA network tester than burning electricity on an unbreakable capture. I hope you get super nerdy about these checks as I watch your progress on GitHub!

Is there a way for a user to use your checking filters in hcxpcapngtool to purify a pcapng file without outputting the hashes? Basically use the quality checks of hcxpcapngtool to produce only a quality capture file and remove any bad packets? Perhaps you intend to add the filters to hcxdumptool so nothing bad even gets written to disk!

Thank you for your time as always.

[ZerBea]

You can do this running hcxpcapngtool without options to save files:

Code:

$ hcxpcapngtool test.cap
reading from test.cap...

summary capture file
file name................................: test.cap
version (pcap/cap).......................: 2.4 (very basic format without any additional information)
timestamp minimum (GMT)..................: 08.05.2018 03:28:56
timestamp maximum (GMT)..................: 08.05.2018 03:51:36
link layer header type...................: DLT_IEEE802_11_RADIO (127)
endianess (capture system)...............: little endian
packets inside...........................: 9305
frames with correct FCS..................: 1874
BEACON (total)...........................: 4055
PROBEREQUEST.............................: 326
PROBEREQUEST (directed)..................: 113
PROBERESONSE.............................: 1347
ASSOCIATIONREQUEST (total)...............: 54
ASSOCIATIONREQUEST (PSK).................: 32
REASSOCIATIONREQUEST (total).............: 15
REASSOCIATIONREQUEST (PSK)...............: 7
EAPOL messages (total)...................: 2952
EAPOL RSN messages.......................: 2954
EAPOL WPA messages.......................: 2
ESSID (total unique).....................: 1500
ESSID changes (mesured maximum)..........: 6 (warning)
EAPOLTIME gap (measured maximum usec)....: 1826255
REPLAYCOUNT gap (measured maximum).......: 8
EAPOL M1 messages........................: 2398
EAPOL M2 messages........................: 445
EAPOL M3 messages........................: 93
EAPOL M4 messages........................: 16
EAPOL pairs (total)......................: 9011
EAPOL pairs (best).......................: 27
EAPOL M12E2..............................: 24
EAPOL M32E2..............................: 2
EAPOL M34E4..............................: 1
PMKID (total)............................: 64
PMKID (useless)..........................: 2
PMKID (best).............................: 16
malformed packets (total)................: 1512
BROADCAST MAC error (malformed packets)..: 1014
IE TAG length error (malformed packets)..: 342
ESSID error (malformed packets...........: 145
EAPOL messages (malformed packets).......: 60

versus:
$ hcxpcapngtool test.cap --ignore-ie
reading from test.cap...

summary capture file
file name................................: test.cap
version (pcap/cap).......................: 2.4 (very basic format without any additional information)
timestamp minimum (GMT)..................: 08.05.2018 03:28:56
timestamp maximum (GMT)..................: 08.05.2018 03:51:36
link layer header type...................: DLT_IEEE802_11_RADIO (127)
endianess (capture system)...............: little endian
packets inside...........................: 9305
frames with correct FCS..................: 1874
BEACON (total)...........................: 4055
PROBEREQUEST.............................: 326
PROBEREQUEST (directed)..................: 113
PROBERESONSE.............................: 1347
ASSOCIATIONREQUEST (total)...............: 54
ASSOCIATIONREQUEST (PSK).................: 32
REASSOCIATIONREQUEST (total).............: 15
REASSOCIATIONREQUEST (PSK)...............: 7
EAPOL messages (total)...................: 2952
EAPOL RSN messages.......................: 2954
EAPOL WPA messages.......................: 2
ESSID (total unique).....................: 1500
ESSID changes (mesured maximum)..........: 6 (warning)
EAPOLTIME gap (measured maximum usec)....: 1826255
REPLAYCOUNT gap (measured maximum).......: 8
EAPOL M1 messages........................: 2398
EAPOL M2 messages........................: 445
EAPOL M3 messages........................: 93
EAPOL M4 messages........................: 16
EAPOL pairs (total)......................: 9051
EAPOL pairs (best).......................: 29
EAPOL M12E2..............................: 26
EAPOL M32E2..............................: 2
EAPOL M34E4..............................: 1
PMKID (total)............................: 64
PMKID (useless)..........................: 2
PMKID (best).............................: 19
malformed packets (total)................: 1512
BROADCAST MAC error (malformed packets)..: 1014
IE TAG length error (malformed packets)..: 342
ESSID error (malformed packets...........: 145
EAPOL messages (malformed packets).......: 60

Malformed packets detected!
In monitor mode the adapter does not check to see if the cyclic redundancy check (CRC)
values are correct for packets captured. The device is able to detect the Physical Layer
Convergence Procedure (PLCP) preamble and is able to synchronize to it, but if there is
a bit error in the payload. This can lead to unexpected results.
Please analyze the dump file with Wireshark.

Than compare the results:
EAPOL pairs (best).......................: 27
vs
EAPOL pairs (best).......................: 29

I can't recommend to edit a hcxdumptool pcapng file, because this will remove important informations!
$ hcxdumptool --help:
Do not edit, merge or convert this pcapng files, because it will remove optional comment fields!

$ hcxpcapngtool --help
Do not edit, merge or convert pcapng files! This will remove optional comment fields!
Detection of bit errors does not work on cleaned dump files!

[ZerBea]

Latest git head is able to detect EAP-TLS (tunneled TLSv1.2)- WAP3/SAE - SAE SHA384 SUITE B frames with 24 byte MIC (WPA3 ENTERPRISE).

example pcapng in Wireshark default capture format (gzip compressed):
https://github.com/wireshark/wireshark/b....pcapng.gz

Code:

$ hcxpcapngtool wpa3-suiteb-192.pcapng.gz
decompressing wpa3-suiteb-192.pcapng.gz to /tmp/wpa3-suiteb-192.pcapng.gz.tmp
reading from wpa3-suiteb-192.pcapng.gz.tmp...

summary capture file
file name.................................: wpa3-suiteb-192.pcapng.gz
version (pcapng).........................: 1.0
operating system.........................: N/A
application..............................: N/A
interface name...........................: N/A
interface vendor.........................: 000000
weak candidate...........................: N/A
MAC ACCESS POINT.........................: 000000000000 (incremented on every new client)
MAC CLIENT...............................: 000000000000
REPLAYCOUNT..............................: 0
ANONCE...................................: 0000000000000000000000000000000000000000000000000000000000000000
SNONCE...................................: 0000000000000000000000000000000000000000000000000000000000000000
timestamp minimum (GMT)..................: 29.10.2019 08:35:04
timestamp maximum (GMT)..................: 29.10.2019 08:35:04
link layer header type...................: DLT_IEEE802_11_RADIO (127)
endianess (capture system)...............: little endian
packets inside...........................: 97
BEACON (total)...........................: 2
PROBEREQUEST.............................: 2
PROBERESONSE.............................: 1
AUTHENTICATION (OPEN SYSTEM).............: 6
ASSOCIATIONREQUEST (total)...............: 3
ASSOCIATIONREQUEST (SAE SHA384 SUITE B)..: 3
IDENTITIES...............................: 1
EAP (total)..............................: 15
EAP CODE REQUEST.........................: 7
EAP CODE RESPONSE........................: 7
EAP ID...................................: 2
EAP-TLS messages.........................: 10
EAPOL messages (total)...................: 12
EAPOL RSN messages.......................: 12
ESSID (total unique).....................: 1
EAPOL M1 messages........................: 3
EAPOL M2 messages........................: 3
EAPOL M3 messages........................: 3
EAPOL M4 messages........................: 3

[WPA_Catcher]

Hi ZerBea and sorry for not replying yesterday I started reading those links you provided and as usual started following other links etc... LOL

As you know I have no option but to use hccap files (not even hccapx) as hashcat no longer supports my old GPU.

I am trying to use only your tools instead of doing everything the aircrack way. The following might be helpful for others stuck in my position and who need to still use hccap.

Once I have my pcapng file from hcxdumptool I use hcxpcapngtool to output to hccap and importantly I also output 22000.

Unfortunately hcxpcapngtool is unable to split the hccap output into individual network files.

So to get a more useful result I use "split -b 392 (pathtopcapngfile)" to make individual hccap files but they have random names. I then have to look through each file to find the ESSID I need. This can be tedious when there are sometimes 50 or more networks and I am not smart enough to know how to automate this search.

I often find many .hccap's of the same network I am interested in and I once assumed any output from hcx-anything produced a valid handshake but I have discovered some are not crackable. However thanks to your posts here I can now check for this.

Once I have found all the individual .hccap files outputted by hcxpcapngtool I open the .22000 file and look for the Key MIC that matches one of the hccap files and I have found this to be the one that is usually crackable.

I understand you wish to dump old formats but if there is anything you could advise or even alter within hcx-anything to help users stuck in my situation I would be extremely grateful.

I suppose the most useful or important issue would be for hcxpcapngtool not to output any hccap files which are not crackable. Is it possible for you to please add the Key MIC check during conversion?

Thanks again for your time.

[ZerBea]

Good idea, and this will help old hccap users a lot.
Your hashcat version doesn't support nonce-error-corrections (NC). That exactly is the reason why some of your hashes are not recoverable. NC is an amazing feature (used by hashcat and hcxtools) to compensate a packet loss during reception. Usually it works on M1 frames, transmitted by an access point (AP) during the same authentication sequence. It works as long as the AP doesn't renew his random values and his EAPOL counters.
Take a look at this example to explore the advantage of NC:
https://hashcat.net/forum/thread-8910-po...l#pid47400

"I suppose the most useful or important issue would be for hcxpcapngtool not to output any hccap files which are not crackable. Is it possible for you to please add the Key MIC check during conversion?"
-> If I were able to do this, we do not need hashcat any longer and hashcat will become obsolete.
Validating MICs and/or PMKIDs is hashcat domain. hcxtools/hcxdumptool only have the power to check a single weak candidate, a single PMK or a zeroed PMK. Apart from this, I promised Atom not to do this (coding a GPU based hash cracker). Also I don't want to blame me, because hashcat can do this much faster and better than I ever could do it.

BTW:
hccap isn't removed:
$ hcxhashtool -h
--hccap=<file> : output to ancient hccap file

convert pcapng to 22000 hashline, get the the hashes you would like to test and save them to hccap

Once you split your hccap files just do a search for the ESSID:
$ grep -r your_essid
and it will show you the hccap file which matches.

BTW 2:
Grouping to old hccap format doesn't make sense, because your hashcat version doesn't handle reuse of PBKDF2. Running an old version of hashcat, it is much better to convert only one single hash and feed hashcat with it. Choose this hash carefully, otherwise you will waste your time and the time of your old GPU.

[WPA_Catcher]

Thank you for your reply I am going through the things you have said (slowly) but I just wanted to respond to this.

BTW 2:
Grouping to old hccap format doesn't make sense, because your hashcat version doesn't handle reuse of PBKDF2. Running an old version of hashcat, it is much better to convert only one single hash and feed hashcat with it. Choose this hash carefully, otherwise you will waste your time and the time of your old GPU.

I don't want to "group" old hccap's but I don't know how else to output multiple networks to separate hccap files.

hcxhashtool is obviously the best way forward for me and as usual you are one step ahead. As far as I know all hashes in the new .22000 format are 100% crackable, is that right? If so then my request is for hcxhashtool to be able to output all hashes contained within a .22000 file to individual hccaps.

I guess if hcxhashtool only uses the hashes in .22000 to convert then the user would not need to worry about chosing a bad hash?

If you can do this for us you will save many peoples old GPU's from becoming obsolete! I am a huge fan of your work and it is great to be able to talk directly with you.

[ZerBea]

Done by latest commit:
https://github.com/ZerBea/hcxtools/commi...08fbde51ee

$ hcxhastool -h
--hccap-single : output to ancient hccap single files (MAC + count)

But be warned:
This way is much more complicated than to choose the best 22000 hashline and convert it to a hccap file containing this single hash.
Doing this by new hccap-single option you have to use a hex viewer!