hashcat Forum
New hccapx format explained - Printable Version

+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Misc (https://hashcat.net/forum/forum-15.html)
+--- Forum: User Contributions (https://hashcat.net/forum/forum-25.html)
+--- Thread: New hccapx format explained (/thread-6273.html)

Pages: 1 2 3 4 5

New hccapx format explained - atom - 02-06-2017

A few days ago a user came into the #hashcat IRC channel and reported to have problems cracking one of his WPA handshake captures. No worries, the user knew the password to the WPA handshake so this seemed to be a ligitimate request.

After some debugging it was clear what is the problem. It's the tools he used to convert the .cap file to .hccap!

We've tested the following:
  • aircrack-ng (latest version from GitHub)
  • cap2hcap (latest version from SVN)
After some debugging it was clear what they did wrong: They did not verify the replay counter attribute which is used to distinguish between different WPA handshakes of the same AP and Station. Both tools failed doing this correctly. The result is an .hccap file that looks valid from hashcats perspective but it will never crack even if you had the correct password in your wordlist.

Especially if you run a deauth attack to accelerate the handshake capture process, it's possible that there's multiple WPA handshakes running in parallel (and getting captured) from the same AP and Station. Each of them will have different nonce's (random data) which is part of the calculation that hashcat is doing to verify if a password candidate is correct or not. To do a successfull verification it's important to use the correct nonce's and to do this it's important to match the handshake packets correctly.

I've reported this to the Aircrack-NG Team, trac ticket is here: http://trac.aircrack-ng.org/ticket/651

As this ticket is open since 8 years, and the last submission for cap2hccap is also 3 years ago, we've decide to write our own converter. This new converter is now finished and is called "cap2hccapx". We've added it as a new tool to https://github.com/hashcat/hashcat-utils, whose newest version we've already released as hashcat-utils v1.4.

While we were implementing cap2hccapx we learned a bit better about the information exchanged in the so-called 4-way WPA handshake and from which packets we can obtain valuable information. There's basically two possible combinations of messages (also called exchanges) in the handshake process that can be used to crack the authentication, not just one. 

The most important message of the 4 messages is the second one. We always need that one. After that, in addition to the beacon requests/responses (which provide us with the network name, aka ESSID, and MAC of the access point, aka BSSID), we at least need one of these messages:
  • The 1st one: packet from the access point (AP) to the client (STA) which initiate the connection request
  • The 3rd one: packet from the access point (AP) to the client (STA) which is a response to the message number 2
If the 3rd exchange packet was captured it is a proof that the password was correct and therefore the handshake for us can be marked as verified (we call it "authenticated"). That's what we already used in the past as it seemed to be what people are looking for.

On the other hand, if we only are in possession of the 1st and 2nd one, we can't tell for sure if the connection was established or even if the password was correct. However, it has it's own usecase. For example whenever you see a Client probing for his AP this typically also shows you the ESSID. As an attacker, you can now simply setup an AP with exactly this ESSID. If the Client does not verify the MAC address (most clients don't do this), it will try to connect to your AP. Of course this will fail, because you AP doesn't use the same Password as the Client does. However, the Client will have used the correct Password. This so called "non-authenticated" Handshake can get cracked with hashcat now, too. If you succeed to crack it, you can re-setup your AP with the cracked password and the connection with the Client will be successful giving you access to the Client from a more "internal" network position or to do any man-in-the-middle attack. Alternatively, you could also connect to the real AP, in case you know where it is located.

We very quickly recognized that all of this new type of information somehow will force us to introduce a new hashcat capture format. We call it "hccapx". You can see the details about the new format here: https://hashcat.net/wiki/hccapx (and if you want you can compare it to the old hccap format https://hashcat.net/wiki/hccap).

All in all, we added several additional entries that help hashcat to know if the file is a valid one (because we added a file magic "HCPX" at the start of the file), we added a format version number and of course the information about whether the authentication was successful (see the "authenticated" entry) and therefore if message 3 was captured too. If your .hccapx file contains both types, hashcat will try to crack both. This will create no additional cost because all the verification is done after the slow PBKDF-HMAC-SHA1 part. IOW, you get this features for free.

The hccapx is a new format and we know that you might have some old .hccap files in your archives. But believe us, the changes were needed and we love them. Without these changes we can't ensure that the converted data is even crackable (because as you already know, output from other tools might not be crackable at all, because of the missing checks and faulty matching algorithm). Therefore it also doesn't make any sense to write a converter from hccap to hccapx.

In case you were using the old converter site, we've also updated that one: https://hashcat.net/cap2hccapx/

Finally, with these changes in hashcat-utils and hashcat, we expect to get much fewer problem reports and hopefully you will crack more WPA/WPA2 networks too.

RE: New hccapx format explained - martin.po21 - 02-06-2017

Very nice indeed, a big thanks to developers!
However, I tested this on a capture file I already cracked and therefor I know capture is indeed valid (ATT?d?d?d network with default pass of 10 digits)- after converzion to hccapx by your online converter and cracking attempt in hashcat-3.30+103, hashcat returned this error: "Hashfile 'C:\Users\xxxx\Desktop\4516_14x641x9xx.hccapx': Invalid hccapx eapol size"

Any idea of what I did wrong?

Edit: I just realized this is in dev. forum. Please remove my reply if troubleshooting does not belong here. I'm sorry for inconvenience

RE: New hccapx format explained - rico - 02-07-2017

Great stuff, handshake capturing always a tricky business. I just gave cap2hccapx a quick spin but didn't have much luck either I'm afraid. cap2hccapx seems to identify incomplete handshakes as good?

Example from a .cap file I found, filtering on EAPOL:
t1. Wireshark reports Message 2 of 4 (airodump-ng picks up no handshake)
t2. Wireshark reports Message 4 of 4 (airodump-ng picks up no handshake)
t3. Wireshark reports Message 3&4 of 4 (airodump-ng pick up no handshake)
t4. Wireshark reports Message 1,2,3&4 of 4 (airodump-ng reports WPA handshake found)

So three failures and one success. Then "aircrack-ng -J" picks up the singular complete handshake "WPA (1 handshake)" as expected.

However, cap2hccapx reports 4 handshakes found:

Networks detected: 1

[*]BSSID=d0:3f:0f:d5:03:25 ESSID=TST260FF (Length: 8)
 --> STA=60:02:b4:e2:5c:a6, Authenticated=1, Replay Counter=0
 --> STA=60:02:b4:e2:5c:a6, Authenticated=0, Replay Counter=0
 --> STA=60:02:b4:e2:5c:a6, Authenticated=0, Replay Counter=0
 --> STA=60:02:b4:e2:5c:a6, Authenticated=1, Replay Counter=0

Written 4 WPA Handshakes to: tst260ff


Latest beta fails to read hccapx file:
Code:
hashcat-3.30+103>hashcat64.exe -a 3 -m 2500 --session=tst260ff -o tst260ff_found.txt --outfile-format=3 tst260ff.hccapx --custom-charset1=35679 08?1?d?d?d?d?d?d?d
hashcat (v3.30-103-g65d5921) starting...

<SNIP old CUDA version warnings>
OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: GeForce GTX 760, 512/2048 MB allocatable, 6MCU

OpenCL Platform #2: Intel(R) Corporation
========================================
* Device #2: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz, skipped

Hashfile 'tst260ff.hccapx': Invalid hccapx eapol size
Hashfile 'tst260ff.hccapx': Invalid hccapx signature
Hashfile 'tst260ff.hccapx': Invalid hccapx signature
No hashes loaded

Started: Mon Feb 06 21:54:32 2017
Stopped: Mon Feb 06 21:54:33 2017


RE: New hccapx format explained - atom - 02-07-2017

(02-06-2017, 11:39 PM)martin.po21 Wrote: However, I tested this on a capture file I already cracked and therefor I know capture is indeed valid (ATT?d?d?d network with default pass of 10 digits)- after converzion to hccapx by your online converter and cracking attempt in hashcat-3.30+103, hashcat returned this error: "Hashfile 'C:\Users\xxxx\Desktop\4516_14x641x9xx.hccapx': Invalid hccapx eapol size"

Should be fixed with hashcat beta 104. If it does not work, please send me the .cap file so that I can reproduce locally.

(02-06-2017, 11:39 PM)martin.po21 Wrote: Edit: I just realized this is in dev. forum. Please remove my reply if troubleshooting does not belong here. I'm sorry for inconvenience

Moved to a better forum, thanks!

RE: New hccapx format explained - atom - 02-07-2017

(02-07-2017, 01:01 AM)rico Wrote: Great stuff, handshake capturing always a tricky business. I just gave cap2hccapx a quick spin but didn't have much luck either I'm afraid. cap2hccapx seems to identify incomplete handshakes as good?

Kind of yes, and it's good you notice it because this is another new feature I forgot to mention in the above introduction text. From your example it's clear that you are facing exactly the problem mentioned above. That is the replay counter collisions (all Replay Counter variables in your dump are 0 while AP and STA are the same). That means with aircrack-ng there's a 50% chance you will not be able to crack it (with the correct password candidate and when exported as .hccap) because of the wrongly mixing of the nonce's. If it works this time it's possible it will not work with the next .cap you're producing.

The problem remains also with hccapx because the damage is done already. However, cap2hccapx knows about hashcats ability to crack N number of handshakes of the same ESSID for the price of one. That's why it's exporting all possible matches (valid combinations of MAC-AP<->MAC-STA and valid replay counter). The time it takes to crack them with hashcat is the same as it would be only one handshake. If you want to verify just capture the runtime of hashcat cracking just one or all four, it's the same. However, the chances to crack at least one of them is 100% (if you have the correct password candidate) and that's the important difference.

One more thing to mentioning:

Quote:t2. Wireshark reports Message 4 of 4 (airodump-ng picks up no handshake)

Note that packet 4 is not needed for cracking with hashcat. This one is dropped.

Quote:Hashfile 'tst260ff.hccapx': Invalid hccapx eapol size
Hashfile 'tst260ff.hccapx': Invalid hccapx signature
Hashfile 'tst260ff.hccapx': Invalid hccapx signature

This one is odd and shouldn't occur. To reproduce locally I need the .cap. Please send it to me.

RE: New hccapx format explained - atom - 02-07-2017

After some testing, that error:

Quote:Invalid hccapx eapol size

and/or

Quote:Invalid hccapx signature

is somehow related to windows beta binaries. We'll try to fix.

RE: New hccapx format explained - rico - 02-07-2017

Thanks, atom. I won't be able to get back to my PC to send the .cap for another 10 hours (if you even still need it).

RE: New hccapx format explained - atom - 02-07-2017

I've just upload hashcat beta 104. With it, you should be able to load the hccapx on windows, too. If it works I don't need the .cap.

RE: New hccapx format explained - rico - 02-07-2017

Thanks again, works great with hashcat-3.30+119 on Windows 10 x64.

I would suggest adjusting the reported hashrate though. I got very excited to see a huge speed bump and thought some new optimisation had been added...but then saw the same old rate benchmarking "hashcat -b". Then I twigged that the reported hashrate had tripled but only because there were three handshakes in the file... Estimated completion time remains the same.

E.g.
Bench (don't laugh, this is just fun for me):

Code:
OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: GeForce GTX 760, 512/2048 MB allocatable, 6MCU

OpenCL Platform #2: Intel(R) Corporation
========================================
* Device #2: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz, 2047/16323 MB allocatable, 8MCU

Hashtype: WPA/WPA2

Speed.Dev.#1.....:    26158 H/s (58.41ms)
Speed.Dev.#2.....:    14226 H/s (71.28ms)
Speed.Dev.#*.....:    40384 H/s


Mask attack 8 UPPER:
Code:
Hash.Type........: WPA/WPA2
Hash.Target......: tst260ff.hccapx
Time.Started.....: Tue Feb 07 19:53:41 2017 (9 mins, 1 sec)
Time.Estimated...: Thu Apr 06 07:01:38 2017 (57 days, 9 hours)
Input.Mask.......: ?u?u?u?u?u?u?u?u [8]
Input.Queue......: 1/1 (100.00%)
Speed.Dev.#1.....:    79449 H/s (58.40ms)
Speed.Dev.#2.....:    46828 H/s (64.01ms)
Speed.Dev.#*.....:   126.3 kH/s
Recovered........: 0/3 (0.00%) Digests, 0/3 (0.00%) Salts


RE: New hccapx format explained - c4p0ne - 02-08-2017

When loading .cap files into recovery software like Elcomsoft's or Passcape's which were reported by those programs as "Invalid Handshakes", I always wondered why when converting those to hccap, and then re-importing them into that software, why they were suddenly reported as "valid".... This development is gonna prevent a lot of sweat from my brow as I no longer have to wonder if handshakes claimed as "valid" can ACTUALLY be trusted to be so. Incidentally, this is the program I used to use (under Windows) because of its easy drag-drop. Guess it's going in the garbage now. https://github.com/wpatoolkit/Cap-Converter