Missing Frames and hcxpcapngtool

[null]

I am using ESP Hash Monster on a M5Stack Core2 to capture wlan packeks. I can easily capture lots of handshakes (all four messages) and occasionaly a PMKID as well. When I attempt to convert these captures to a Hashcat accepted format using hcxpcapngtool, I always get the message that frames are missing.

What exact frames do I need in order to crack a WPA2 PSK? More than the 4-way handshake and/or PMKID?
What exactly is meant by the "total/useless/best" output, and how can the PMKID be both useless and best?

Yes, these questions are not specifically Hashcat-related and they are newb for sure, so I appreciate a nudge in the right direction, or someone to point out what it is I am obviously missing. I've tried to find answers in the documentation but have come up empty so far.

Here is the output from the tool, which includes a four-way handshake, and a PMKID (I think):

summary capture file
--------------------
file name................................: 0001.pcap
version (pcap/cap).......................: 2.4 (very basic format without any additional information)
timestamp minimum (GMT)..................: 31.12.1969 19:29:03
timestamp maximum (GMT)..................: 01.01.1970 09:53:18
used capture interfaces..................: 1
link layer header type...................: DLT_IEEE802_11 (105)
endianess (capture system)...............: little endian
packets inside...........................: 1357
BEACON (total)...........................: 55
WPA encrypted............................: 27
EAPOL messages (total)...................: 1274
EAPOL RSN messages.......................: 1274
ESSID (total unique).....................: 28
EAPOLTIME gap (measured maximum usec)....: 666344089
EAPOL ANONCE error corrections (NC)......: working
REPLAYCOUNT gap (suggested NC)...........: 23
EAPOL M1 messages........................: 1061
EAPOL M2 messages........................: 57
EAPOL M3 messages........................: 116
EAPOL M4 messages........................: 40
EAPOL pairs (total)......................: 83
PMKID (total)............................: 1
PMKID (useless)..........................: 1
PMKID (best).............................: 1

Warning: missing frames!
This dump file contains no important frames like
authentication, association or reassociation.
That makes it hard to recover the PSK.

Warning: missing frames!
This dump file contains no undirected proberequest frames.
An undirected proberequest may contain information about the PSK.
That makes it hard to recover the PSK.


-----

I tried to attach the pcap but the forum doesn't allow them I guess.

[ZerBea]

The basics:
To recover the PSK of a WPA1, WPA2 or WPA2 key version 3 network you need either a 4way handshake (which contain M2 or not zeroed M4 and either a M1 or M3) or a PMKID and the network name ESSID.

But there are much more useful frames and information that can help to recover a PSK.
hcxpcapngtool is an analysis tool that will take additional information from the dumpfile and parse it to hashcat. That will make it easier to recover the PSK.
Unfortunately the tools that you use to attack the target and to dump the traffic to a cap/pcap file do not take care about this! hcxpcapngtool detect this missing frames as well as the missing radio tap information and give you a warning.
Please notice: It is only a warning and not an ERROR

To get rid of this, I suggest to use hcxdumptool (attack and dump):

Code:

$ hcxpcapngtool -o test.22000 -E wordlist -I wordlist -U wordlist hcxdumptool.pcapng
hcxpcapngtool 6.2.4-83-g48d77b7 reading from hcxdumptool.pcapng...

summary capture file
--------------------
file name.................................: hcxdumptool.pcapng
version (pcapng).........................: 1.0
operating system.........................: Linux 5.10.32-2-ARCH
application..............................: hcxdumptool 6.2.4-14-g69872e0
interface name...........................: wlan0
interface vendor.........................: 00e061
openSSL version..........................: 1.1
weak candidate...........................: 12345678
MAC ACCESS POINT.........................: 00bb3a0d66ce (incremented on every new client)
MAC CLIENT...............................: f0a225ad2c04
REPLAYCOUNT..............................: 61865
ANONCE...................................: 6b27634f5e19df3b62aea9255ecc1ccbf0c22028b0d28427389080355947207c
SNONCE...................................: 523b2d541ed34297a92a2a52a3e79d07f2d391fe41100e66d5e1b55e02f49211
timestamp minimum (GMT)..................: 29.04.2021 11:29:42
timestamp maximum (GMT)..................: 29.04.2021 21:55:51
used capture interfaces..................: 1
link layer header type...................: DLT_IEEE802_11_RADIO (127)
endianess (capture system)...............: little endian
packets inside...........................: 15573
packets received on 2.4 GHz..............: 14529
ESSID (total unique).....................: 1270
BEACON (total)...........................: 1968
BEACON (detected on 2.4 GHz channel).....: 1 3 4 5 6 7 8 9 10 11 13
BEACON (SSID unset)......................: 117
BEACON (SSID zeroed).....................: 20
PROBEREQUEST.............................: 245
PROBEREQUEST (directed)..................: 16
PROBERESPONSE (total)....................: 759
AUTHENTICATION (total)...................: 347
AUTHENTICATION (OPEN SYSTEM).............: 347
ASSOCIATIONREQUEST (total)...............: 45
ASSOCIATIONREQUEST (PSK).................: 45
REASSOCIATIONREQUEST (total).............: 24
REASSOCIATIONREQUEST (PSK)...............: 24
EAP (total)..............................: 6
EAP CODE request.........................: 4
EAP-TLS messages.........................: 4
EAPOL messages (total)...................: 11685
EAPOL RSN messages.......................: 11664
EAPOL WPA messages.......................: 21
EAPOLTIME gap (measured maximum usec)....: 60667444
EAPOL ANONCE error corrections (NC)......: working
REPLAYCOUNT gap (suggested NC)...........: 12
EAPOL M1 messages (total)................: 4378
EAPOL M2 messages (total)................: 6833
EAPOL M3 messages (total)................: 222
EAPOL M4 messages (total)................: 252
EAPOL pairs (total)......................: 38361
EAPOL pairs (best).......................: 50
EAPOL ROGUE pairs........................: 30
EAPOL pairs written to combi hash file....: 50 (RC checked)
EAPOL M12E2 (challenge)..................: 35
EAPOL M32E2 (authorized).................: 13
EAPOL M34E4 (authorized).................: 2
PMKID (useless)..........................: 32
PMKID (total)............................: 349
PMKID (best).............................: 108
PMKID ROGUE..............................: 95
PMKID written to combi hash file.........: 108


session summary
---------------
processed pcapng files................: 1

You may have noticed that this dumpfile in pcapng format, taken by hcxdumptool, contain a full radio tap header and much more information than your dumpfile.

To make it clear to you what is missing in your dump file, please read this:
https://github.com/evilsocket/pwnagotchi/issues/835
and get the example mentioned there:
https://github.com/evilsocket/pwnagotchi...nctest.zip
now try it:

Code:

$ hcxpcapngtool -o eapol.22000 -E wordlist test.pcap
$ hashcat -m 22000 --nonce-error-corrections=8 eapol.22000 wordlist

and you'll know what I mean.

Do not wonder about the pcap file format. The example was converted from pcapng to pcap so that old school tools (e.g. based on libpcap) are able to handle it. Unfortunately neither cap nor pcap file format is able to store additional comment fields that will help to recover the PSK.

[null]

Thank you for sending this, I really appreciate the detail.

So to make sure I understand: The issue is that the Hash Monster is filtering out certain packets (namely, undirected probe requests) before writing to pcap, and the hxcpcapngtool is looking for those, and other useful frames that aren't being properly captured. Also, the additional info that the tool is looking for can only be stored in pcapng format.

Since you rightly pointed out that the output of the Hash Monster is pcap (not pcapng), I ran the capture through the old hcxpcaptool as well:

reading from 0001.pcap

summary capture file:
---------------------
file name........................: 0001.pcap
file type........................: pcap 2.4
file hardware information........: unknown
capture device vendor information: 000000
file os information..............: unknown
file application information.....: unknown (no custom options)
network type.....................: DLT_IEEE802_11 (105)
endianness.......................: little endian
read errors......................: flawless
minimum time stamp...............: 01.01.1970 00:29:03 (GMT)
maximum time stamp...............: 01.01.1970 14:53:18 (GMT)
packets inside...................: 1357
skipped damaged packets..........: 0
packets with GPS NMEA data.......: 0
packets with GPS data (JSON old).: 0
packets with FCS.................: 0
beacons (total)..................: 55
beacons (WPS info inside)........: 4
beacons (MESH-ID inside).........: 2
EAPOL packets (total)............: 1274
EAPOL packets (WPA2).............: 1274
PMKIDs (zeroed and useless)......: 1
PMKIDs (not zeroed - total)......: 1
PMKIDs (WPA2)....................: 2
PMKIDs from access points........: 1
best handshakes (total)..........: 5 (ap-less: 0)
best PMKIDs (total)..............: 1

summary output file(s):
-----------------------
0 handshake(s) written to test.pcap


-----

I'm still fuzzy on this point:

>Unfortunately neither cap nor pcap file format is able to store additional information that will help to recover the PSK.

Does this mean that no handshakes or combination of frames captured in pcap format can be used to crack the PSK? Or only that these particular tools want more information to make the process "easier"?

Is there a toolchain for taking pcap files, without the additional frames that hxcpcapngtool is looking for, and pass them to Hashcat for cracking?

[ZerBea]

For sure, hcxpcaptool/hcxpcapngtool take all basic information (BEACON, 4way handshake and/or PMKID) from a cap and pcap file and convert it to a hash file accepted by hashcat. That is old school basics.

This new tool chain:
hcxdumptool (attack the CLIENT and store additional information in pcapng comment field) -> hcxpcapngtool (evaluate and parse this information using hc22000 file format) -> hashcat (recover the PSK)
Additional information that are stored in pcapng comment fields, e.g.:
- replay count used for the attack
- MACs used for the attack
- NONCEs used for the attack
- weak candidate PSK

Is there a toolchain for taking pcap files, without the additional frames that hxcpcapngtool is looking for, and pass them to Hashcat for cracking?
That depend on the tool that you use (which tool do you use?) to attack the target and to dump the traffic to a cap/pcap/pcapng file. It also depend on filtering options that you use. Usually all 802.11 frames can/should be stored in cap, pcap and pcapng file format.


Please try the example mentioned above and ask yourself this question:
How much time will take hashcat to recover the PSK from the example by brute force method?

[null]

The Hash monster is simply the tool I am using to capture the handshakes, and output them in the pcaps. From there I am planning to use Hashcat to actually do the brute-forcing/cracking.

The project is here: https://github.com/G4lile0/ESP32-WiFi-Hash-Monster

[ZerBea]

Looks like the WiFi hash monster is filtering some useful 802.11 frames out.
How about an issue report on their github?

Some examples of useful 802.11 frames (there are much more - take a look at them by tshark or Wireshark):

Code:

PROBEREQUEST frame
may contain a PSK in clear

AUTHENTICATION frame
useful to detect the beginning of an AUTHENTICATION sequence

ASSOCIATION/REASSOCIATION frame
useful to detect the beginning of an ASSOCIATION sequence

EAP frame
may contain an identity

It is definitely not a good idea to filter this frames out.

BTW1:
pwnagotchi and bettercap received already a fix and store this 802.11 frames, now.

The same applies to the radio tap header (DLT_IEEE802_11_RADIO). It contain useful information about the quality of the received frame.

Code:

Frame 1: 255 bytes on wire (2040 bits), 255 bytes captured (2040 bits) on interface wlan0, id 0
Radiotap Header v0, Length 24
    Header revision: 0
    Header pad: 0
    Header length: 24
    Present flags
        Present flags word: 0xa000402e
            .... .... .... .... .... .... .... ...0 = TSFT: Absent
            .... .... .... .... .... .... .... ..1. = Flags: Present
            .... .... .... .... .... .... .... .1.. = Rate: Present
            .... .... .... .... .... .... .... 1... = Channel: Present
            .... .... .... .... .... .... ...0 .... = FHSS: Absent
            .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: Present
            .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: Absent
            .... .... .... .... .... .... 0... .... = Lock Quality: Absent
            .... .... .... .... .... ...0 .... .... = TX Attenuation: Absent
            .... .... .... .... .... ..0. .... .... = dB TX Attenuation: Absent
            .... .... .... .... .... .0.. .... .... = dBm TX Power: Absent
            .... .... .... .... .... 0... .... .... = Antenna: Absent
            .... .... .... .... ...0 .... .... .... = dB Antenna Signal: Absent
            .... .... .... .... ..0. .... .... .... = dB Antenna Noise: Absent
            .... .... .... .... .1.. .... .... .... = RX flags: Present
            .... .... .... .... 0... .... .... .... = TX flags: Absent
            .... .... .... .0.. .... .... .... .... = Channel+: Absent
            .... .... .... 0... .... .... .... .... = MCS information: Absent
            .... .... ...0 .... .... .... .... .... = A-MPDU Status: Absent
            .... .... ..0. .... .... .... .... .... = VHT information: Absent
            .... .... .0.. .... .... .... .... .... = frame timestamp: Absent
            .... .... 0... .... .... .... .... .... = HE information: Absent
            .... ...0 .... .... .... .... .... .... = HE-MU information: Absent
            .... .0.. .... .... .... .... .... .... = 0 Length PSDU: Absent
            .... 0... .... .... .... .... .... .... = L-SIG: Absent
            ...0 .... .... .... .... .... .... .... = TLVs: Absent
            ..1. .... .... .... .... .... .... .... = Radiotap NS next: True
            .0.. .... .... .... .... .... .... .... = Vendor NS next: False
            1... .... .... .... .... .... .... .... = Ext: Present
        Present flags word: 0x00000820
            .... .... .... .... .... .... .... ...0 = TSFT: Absent
            .... .... .... .... .... .... .... ..0. = Flags: Absent
            .... .... .... .... .... .... .... .0.. = Rate: Absent
            .... .... .... .... .... .... .... 0... = Channel: Absent
            .... .... .... .... .... .... ...0 .... = FHSS: Absent
            .... .... .... .... .... .... ..1. .... = dBm Antenna Signal: Present
            .... .... .... .... .... .... .0.. .... = dBm Antenna Noise: Absent
            .... .... .... .... .... .... 0... .... = Lock Quality: Absent
            .... .... .... .... .... ...0 .... .... = TX Attenuation: Absent
            .... .... .... .... .... ..0. .... .... = dB TX Attenuation: Absent
            .... .... .... .... .... .0.. .... .... = dBm TX Power: Absent
            .... .... .... .... .... 1... .... .... = Antenna: Present
            .... .... .... .... ...0 .... .... .... = dB Antenna Signal: Absent
            .... .... .... .... ..0. .... .... .... = dB Antenna Noise: Absent
            .... .... .... .... .0.. .... .... .... = RX flags: Absent
            .... .... .... .... 0... .... .... .... = TX flags: Absent
            .... .... .... .0.. .... .... .... .... = Channel+: Absent
            .... .... .... 0... .... .... .... .... = MCS information: Absent
            .... .... ...0 .... .... .... .... .... = A-MPDU Status: Absent
            .... .... ..0. .... .... .... .... .... = VHT information: Absent
            .... .... .0.. .... .... .... .... .... = frame timestamp: Absent
            .... .... 0... .... .... .... .... .... = HE information: Absent
            .... ...0 .... .... .... .... .... .... = HE-MU information: Absent
            .... .0.. .... .... .... .... .... .... = 0 Length PSDU: Absent
            .... 0... .... .... .... .... .... .... = L-SIG: Absent
            ...0 .... .... .... .... .... .... .... = TLVs: Absent
            ..0. .... .... .... .... .... .... .... = Radiotap NS next: False
            .0.. .... .... .... .... .... .... .... = Vendor NS next: False
            0... .... .... .... .... .... .... .... = Ext: Absent
    Flags: 0x00
        .... ...0 = CFP: False
        .... ..0. = Preamble: Long
        .... .0.. = WEP: False
        .... 0... = Fragmentation: False
        ...0 .... = FCS at end: False
        ..0. .... = Data Pad: False
        .0.. .... = Bad FCS: False
        0... .... = Short GI: False
    Data Rate: 1,0 Mb/s
    Channel frequency: 2412 [BG 1]
    Channel flags: 0x00a0, Complementary Code Keying (CCK), 2 GHz spectrum
        .... .... ...0 .... = Turbo: False
        .... .... ..1. .... = Complementary Code Keying (CCK): True
        .... .... .0.. .... = Orthogonal Frequency-Division Multiplexing (OFDM): False
        .... .... 1... .... = 2 GHz spectrum: True
        .... ...0 .... .... = 5 GHz spectrum: False
        .... ..0. .... .... = Passive: False
        .... .0.. .... .... = Dynamic CCK-OFDM: False
        .... 0... .... .... = Gaussian Frequency Shift Keying (GFSK): False
        ...0 .... .... .... = GSM (900MHz): False
        ..0. .... .... .... = Static Turbo: False
        .0.. .... .... .... = Half Rate Channel (10MHz Channel Width): False
        0... .... .... .... = Quarter Rate Channel (5MHz Channel Width): False
    Antenna signal: -89 dBm
    RX flags: 0x0000
        .... .... .... .... .... ..0. = Bad PLCP: False
    Antenna signal: -89 dBm
    Antenna: 0
802.11 radio information
    PHY type: 802.11b (HR/DSSS) (4)
    Short preamble: False
    Data rate: 1,0 Mb/s
    Channel: 1
    Frequency: 2412MHz
    Signal strength (dBm): -89 dBm
    [Duration: 2040µs]
        [Preamble: 192µs]

It is not a good idea to filter this header out.

BTW2:
The hash modes 2500 (hccapx - that include cap2hccapx) and 16800 are outdated:
https://hashcat.net/wiki/doku.php?id=cracking_wpawpa2

BTW3:
State of the art tools use pcapng format instead of cap/pcap format, e.g. Wireshark/tshark (5.3.2. Output File Formats)
https://www.wireshark.org/docs/wsug_html...ction.html

BTW4:
A good starting point to learn 802.11 is here:
https://mrncciew.com/2014/10/27/cwap-802...tresponse/
https://mrncciew.com/2014/10/10/802-11-m...ion-frame/
https://mrncciew.com/2014/10/28/802-11-m...qresponse/
https://mrncciew.com/2014/08/24/cwsp-eap-basics/

Please notice:
In contrast to many other hash modes, a successful 802.11 attack always(!) starts on the RF channel. Excessive injecting DEAUTHENTICATION frames is far away from a successful attack.
If the attack failed, you waste GPU time and/or hashcat will fail, too.

[CUwindows00]

I try to use hcxpcapngtool to convert 50,000 cap handshake 22000 format, the algorithm is very accurate, so far I have not found that the correct password will be missed, not to mention there is a function of NC technology, which can be used with confidence

[ZerBea]

Unfortunately it's not as good as I'd like it to be.

On "crappy" dump files or dump files captured by passive tools or cleaned dump files it may fail.

BTW:
There is absolutely no need to filter the reception branch or to clean a dump file!
What has been lost on reception (or filtered out) is gone for ever.
What has not been saved by the dump tool is gone for ever.
What has been cleaned by "wpa cleaning tools" is gone for ever.