hcxtools - solution for capturing wlan traffic and conversion to hashcat formats

[ZerBea]

All characters lower 0x20 and greater than 0x7e are converted to $HEX[xxxxxxx].
There is no plan to support other charactersets, because hashcat is able to handle the output from hcxtools.

[ZerBea]

New chipset reported to work with hcxdumptool:
Bus 001 Device 002: ID 148f:2573 Ralink Technology, Corp. RT2501/RT2573 Wireless Adapter

[recombinant]

Alright; I am pretty new to hxctools (I was introduced to this suite of utilities by the recent PMKID attack blogpost), so please forgive my ignorance (and my haste, because I am just too curious to be satisfied by doing the research, I guess).
I'm used to the interface provided by the aircrack suite--especially besside-ng for automated WPA attacks.

Besside-ng has the ability to output a list of networks that it has found the handshake for, including the ESSID in plaintext, the BSSID, and the fact that the handshake has been found for the network.

Is there any possible way for me to derive a similar list of networks and which information (PMK/Handshake/PMKID) is available for retrieving the PSK from hcxdumptool's pcapng output? (EDIT: Found a partial workaround for the handshakes, at least, in the edited section 1)

I'm also used to using wpaclean to slim down the file to the absolute minimum available. Is there a way to do that with hcxtools?

Additionally, I usually upload my caps to stanev's wpa-sec. Does stanev's wpa-sec site support the PMKID derived from the recent PMKID attack, or is it only going to show networks that have the handshake captured? (Pretty sure I know the answer somewhat, see edited section 2)

I tried reading the documentation to understand this, but I either missed it (likely; sorry!) or the information is not present.

Finally, does hcxdumptool truly need to scan through channels other than 1, 6, and 11? I thought every other channel had overlap with those three.

Thanks a ton for your work, ZerBea.

EDIT:

After doing some research and playing around a bit, I think I have some questions of my own answered.
1) hcxpcaptool can output to an hccapx file. From there, you can use wlanhcx2cap to output to a format that can be read by aircrack-ng to list the ESSIDS, BSSIDS, and the handshakes available.
2) The PMKID retrieval process is fundamentally different from the handshake retrieval process. Also, when converting to hccapx, you can see how many PMKIDs are in the file(s) and how many Handshakes are in the file(s). Unfortunately, this does not tell you if those are discrete, individual networks, or anything else. hcxdumptool seems to not discriminate. With this in mind, though, I still have no 'easy' way to see what ESSIDs the PMKID-pwned networks have, and also can't see them when I upload to wpa-sec, presumably. Perhaps wpa-sec can change in the future to incorporate PMKID attacks? Also, unfortunately, the WPA upload feature for multiple pcaps will put in duplicate entries to wpa-sec instead of consolidating them all into the minimum necessary information. Perhaps that can be changed in a future version to minimize data transfer to the internet?

[ZerBea]

Hi recombinant.

Does stanev's wpa-sec site support the PMKID derived from the recent PMKID attack, or is it only going to show networks that have the handshake captured?
-> wpa-sec is working on that feature (PMKID). But you can do a feature request here:
https://github.com/RealEnder/dwpa/issues

Is there any possible way for me to derive a similar list of networks and which information (PMK/Handshake/PMKID) is available for retrieving the PSK from hcxdumptool's pcapng output?
-> run hcxpcaptool -o hashlist.hccapx -z hashlist.16800 test.pcapng
-> take a look into the hashlist.16800. MAC_AP, MAC_STA and ESSID are inside.
-> run wlanhcxinfo -i hashlist.hccapx -a -e

For better understanding:
hcxdumptool is the dumper. For further going analysis use hcxtools (for example wlanhcxinfo to get informations about the handshakes inside a hccapx file or wlanhcx2ssid to stip handshakes you like to work on).


Finally, does hcxdumptool truly need to scan through channels other than 1, 6, and 11? I thought every other channel had overlap with those three.
-> use the -c option (-c 1,6,11) - overlapping only works if you are close to the access point.

I'm also used to using wpaclean to slim down the file to the absolute minimum available.
-> do not clean your cap files! There is absolutely no need to clean hcxdumptool pcapng files.
and from wpa-sec (https://wpa-sec.stanev.org/?):
"Note: please do not use any additional tools to strip or modify the capture files, since they can mangle handshakes and lead to uncrackable results."

Is there a way to do that with hcxtools?
-> yes, but it isn't recommended!
$ hcxpcaptool -o hashes.hccapx test.pcapng
$ wlanhcx2cap -i hashes.hccapx -o cleaned

hcxdumptool seems to not discriminate.
-> it is not the task of hcxdumptool (but you can use a filter list) - use hcxtools for that purpose

Also, unfortunately, the WPA upload feature for multiple pcaps will put in duplicate entries to wpa-sec instead of consolidating them all into the minimum necessary information.
-> no,  ‎that is so wanted by wpa-sec (reuse PBKDF2 and PMK cracking is activated internal there, to speed up cracking process). wpa-sec will make sure that the best handshake will be used.

Perhaps that can be changed in a future version to minimize data transfer to the internet?
-> Why? hcxtools are able to handle gz compressed pcapng files. wpa-sec accepts this, because hcxtools running inside. Use gzip to compress the cap. So there is absolutely no need to clean the files.

[SIMBA_1983]

Hello, ZerBea
First of all - thanks for your work!

My question is: if the crack speed is the same for both 2500 and 16800 modes then hashcat does the same mathematical calculations for both formats, right? If so, is there any way to convert from 2500/hccapx to 16800/PMKID file formats?

Advantages:
-no need to run two passes on the same wordlist - one for each mode. (Running multiple hccapx/PMKIDs in single run is much much quicker then running single hccapx/PMKID each time)
-running hcxdumptool I can unintentionally capture both PMKID and handshake for the same AP. The password is be the same, so why run crack twice?
-text 16800 format is way easier for storing, sorting and managing captured data then the binary 2500.

[recombinant]

Awesome response, ZerBea. Thank you for the clearcut answers--it is nice to see that everyone is already on top of most of the things I already desire.

Is there any way that wlanhcxinfo can get a feature update equivalent to the hccapx -a -e option? Something that could enumerate which networks had been pwned, and in what way, would be nice. Perhaps that would be better suited for hcxpcaptool, though? Obviously, with the current method, we would have to write a script to compare the .hccapx and the .16800 file to find exactly which networks were pwned specifically, but that information is probably already in the pcapng, right?

I hope what I am saying makes sense: Wlanhcxinfo provides us the ability to see SSIDs in plaintext ASCII, while viewing the .16800 file directly only allows us to see the bytes themselves.

Thanks again for the work--are you accepting pull requests?

[ZerBea]

Hi SIMBA_1983.
My question is: if the crack speed is the same for both 2500 and 16800 modes then hashcat does the same mathematical calculations for both formats, right?
-> 16800 is a little bit faster, because we do not calculate a PTK. Only PBKDF2 is calculated by both hashmodes. The rest of the calculation is different.

If so, is there any way to convert from 2500/hccapx to 16800/PMKID file formats?
-> No, hccapx doesn't store this informations. And if you use a dumper which saves only M2 and M3, PMKID get lost, too.

Some words about hccapx:
With the introduction of WPA3, hccapx will die!

[ZerBea]

Hi recombinant

are you accepting pull requests?
-> yes, but under the restrictions of README.md:
- Multiple stand-alone binaries - designed to run on  Arch Linux.
- All of these utils are designed to execute only one specific function.
hcxdumptool = attack and dump - nothing else
hcxpcaptool = conversion and basic informations about content of a cap, pcap, pcapng file to determine if it's damaged) - nothing else
wlanhcxinfo = info about content of hccapx file - nothing else
wlanhcx2ssid = select special records to work on hashcat - nothing else
...
So:
- I will not add things that makes the code slower or more complex.
- I will not add things that can be done by scripts (comparing lists or potfiles)
- I will not add scripts which can do this.
- I will not add special code for other distributions than Arch Linux
- Form follows function and not: function follows form (no beautiful lists, status output and no correlation of data)

we would have to write a script ...
-> Yes. Such scripts are running on wpa-sec and I have some similar ones in my environment. If you need a scipt to correlate the data, you have to code it!

find exactly which networks were pwned specifically, but that information is probably already in the pcapng, right?
-> Yes, but you can find it in 16800.hash, 16801.hash, hccapx, too!

If you like to work on ESSIDs (ASCII/UTF-8) - be warned and make sure your terminal will display this!
We did some analysis on submitted caps to wpa-sec and we found zeroed ESSIDs, damaged ESSIDs with CTRL, UTF-8, ASCII and non ASCII characters inside. We also found zeroed PSKs and PSKs with CTRL, UTF-8, ASCII and non ASCII characters inside.
... and we found tons of deadly cleaned cap files with zeroed timestamps.

Right now, I only need the ESSID to calculate a PMK (by PBKDF2) and that's all. Than I prefer hashmodes 2501 and 16801. (wpa-sec use -m 2501 an every incomming cap file, first)
With the introduction of WPA3, PBKDF2 will die:
PMK = KDF-512(keyseed, "SAE KCK and PMK", *(commit-scalar + peer-commit-scalar) modulo r)
and the PMKID:
PMKID = L((commit-scalar + peer-commit-scalar) modulo r, 0, 128)
...and PBKDF2 allready died on WPA ENTERPRISE:
https://hashcat.net/forum/thread-7717-po...l#pid41571


BTW:
hcxdumptoo/hcxtools/hcxkeys are  n o t  aircrack-ng, kismet or tcpdump
The tools are designed as analysis tools to develop new procedures (like the PMKID attack). So you should know what you are doing!
Take a look at a hcxdumptool pcapng file and compare it with an aircrack-ng, kismet or tcpdump cap file. I'm sure you see the difference:
hcxdumptool provides three different types of pcapng files (wep traffic, unencrypted IPv4/IPv6 traffic and EAP/EAPOL traffic).
No useless management, control or data frames inside the EAP/EAPOL traffic pcapng. One beacon proberequest/proberesponse and all authentications (authentication, associationrequest/response, rassociationrequest/response, EAP/EAPOL) are stored. We store all authentication frames, to determine if some frames changed during the attacks (remember: designed as analysis tools)

I recommend to use different hashcat potfiles instead of one single potfile:
2500.pot
2501.pot
16800.pot
16801.pot
You can run simple bash commands (cat, cut, awk) on them to get all informations you need.
The rest of the information can be retrieved form the hashfiles (hccapx, 16800/16801).
Now you can correlate the data in an easy way by bash scripts.


Some internal infos about hcxdumptool:
- designed to run on a raspberry pi
- handle 512 access points simultaneously
- handle 512 clients simultaneously
- 32 filter list entries

[recombinant]

Your thoroughness and helpfulness is incredible. Thank you for the all of the information; I learn more useful things every time you reply.
I understand your diligence when it comes to making each tool do one thing; that makes a lot of sense to me. I see now how you intended each of those utilities to be used.

One quick followup question: How did you learn so much about wireless authentication and the associated attacks?

[ZerBea]

How did you learn so much about wireless authentication and the associated attacks?
-> 35 years cryptanalysis and traffic analysis.