hashcat Forum

Full Version: Low utilization with multi GPU
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
As stated, I recently upgraded my cracking rig from 1-1080Ti to 2x1080TI + 3x1070Ti taking advantage of the cheap GPU's with the mining exodus. The issue is, I have the 2x1080Ti's on water, when I only had1 gpu performance was great in hashcat, gpu was 100% utilized and rates were 630Mh/s+ (wpa2) (As this was also my display gpu). After adding the rest of them, and in no matter what order I run them in ( using -d 1,2 for example) each GPU will only get to about 36% utilization and the performance therefore is terrible. Is there an easy fix for this I am missing?

I will add that I have tweaked the cards , power and clock rate etc via afterburner (currently in windows, will test results in linux today as well) and using the benchmark feature (-b) they all put out great numbers, but when using in practice its down to ~36% utilization .. see benchmark below

OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: GeForce GTX 1080 Ti, 2816/11264 MB allocatable, 28MCU
* Device #2: GeForce GTX 1080 Ti, 2816/11264 MB allocatable, 28MCU
* Device #3: GeForce GTX 1070 Ti, 2048/8192 MB allocatable, 19MCU
* Device #4: GeForce GTX 1070 Ti, 2048/8192 MB allocatable, 19MCU

Benchmark relevant options:
===========================
* --optimized-kernel-enable

Hashmode: 2500 - WPA-EAPOL-PBKDF2 (Iterations: 4096)

Speed.Dev.#1.....:   663.8 kH/s (84.77ms) @ Accel:128 Loops:64 Thr:1024 Vec:1
Speed.Dev.#2.....:   626.5 kH/s (44.77ms) @ Accel:128 Loops:32 Thr:1024 Vec:1
Speed.Dev.#3.....:   439.0 kH/s (87.31ms) @ Accel:128 Loops:64 Thr:1024 Vec:1
Speed.Dev.#4.....:   459.6 kH/s (83.42ms) @ Accel:128 Loops:64 Thr:1024 Vec:1
Speed.Dev.#*.....:  2188.9 kH/s
Benchmark numbers look fine, you probably do not feed your GPU with enough workload on the base. Try running in -a 3 mode.
Hmm .. same result:

Session..........: GL-brute-8-10
Status...........: Running
Hash.Type........: WPA-EAPOL-PBKDF2
Hash.Target......: d:\Training\wireless\captures\Greenlight.hccapx
Time.Started.....: Tue Aug 14 13:06:25 2018 (28 secs)
Time.Estimated...: Next Big Bang (> 10 years)
Guess.Mask.......: ?a?a?l?l?l?l?l?a?a?a [10]
Guess.Queue......: 1/1 (100.00%)
Speed.Dev.#1.....: 230.7 kH/s (5.72ms) @ Accel:32 Loops:16 Thr:1024 Vec:1
Speed.Dev.#2.....: 230.3 kH/s (5.73ms) @ Accel:32 Loops:16 Thr:1024 Vec:1
Speed.Dev.#3.....: 156.5 kH/s (5.78ms) @ Accel:32 Loops:16 Thr:1024 Vec:1
Speed.Dev.#4.....: 156.9 kH/s (5.72ms) @ Accel:32 Loops:16 Thr:1024 Vec:1
Speed.Dev.#*.....: 774.4 kH/s
Recovered........: 0/17 (0.00%) Digests, 0/1 (0.00%) Salts
Progress.........: 21561344/91935822600700000 (0.00%)
Rejected.........: 0/21561344 (0.00%)
Restore.Point....: 0/967745501060000 (0.00%)
Candidates.#1....: parieriner -> p:rdfanere
Candidates.#2....: p`jmnanere -> p]eginaner
Candidates.#3....: p?wmelller -> pNqeponere
Candidates.#4....: pMqeponere -> plaxvinere
HWMon.Dev.#1.....: Temp: 33c Fan: 0% Util: 37% Core:2113MHz Mem:4704MHz Bus:8
HWMon.Dev.#2.....: Temp: 35c Fan: 23% Util: 38% Core:2088MHz Mem:4908MHz Bus:8
HWMon.Dev.#3.....: Temp: 55c Fan: 7% Util: 35% Core:2025MHz Mem:3499MHz Bus:1
HWMon.Dev.#4.....: Temp: 52c Fan: 0% Util: 38% Core:2050MHz Mem:3499MHz Bus:1
You have 17 handshakes in your .hccapx file. Remove 16 of them.
I know this is not related to your problem but you said you have 2x1080Ti and 3x1070Ti and I only see 4 devices listed. You are missing a 1070Ti. Unless you made a typo.
soxrok2212 makes a good point.
I think I have seen in the past that number of handshakes doesn't have an effect on speed, I know I did not have a huge drop in hashrate on this same capture with one card, but I will try it with a single handshake just to be sure.

And yes I have 3 1070Ti's but during this run on only had 2 connected. I also tested with just the 2 1080Ti's and had the same result. Still pending testing on Linux. Hopefully be able to try that tomorrow.
Have not tweaked any settings in Linux so the performance is down but just quick testing and the issue does not appear to follow, hitting 97% utilization on each card.

Session..........: hashcat
Status...........: Running
Hash.Type........: WPA-EAPOL-PBKDF2
Hash.Target......: /media/cpu_pirate/shared/Training/wireless/captures/Greenlight.hccapx
Time.Started.....: Thu Aug 16 09:52:20 2018 (1 min, 59 secs)
Time.Estimated...: Fri Mar 24 08:34:47 2248 (229 years, 219 days)
Guess.Mask.......: ?a?a?a?a?a?a?a?a [8]
Guess.Queue......: 1/1 (100.00%)
Speed.Dev.#1.....: 467.7 kH/s (6.49ms) @ Accel:32 Loops:16 Thr:1024 Vec:1
Speed.Dev.#2.....: 447.9 kH/s (6.79ms) @ Accel:32 Loops:16 Thr:1024 Vec:1
Speed.Dev.#*.....: 915.6 kH/s
Recovered........: 0/17 (0.00%) Digests, 0/1 (0.00%) Salts
Progress.........: 108265472/6634204312890625 (0.00%)
Rejected.........: 0/108265472 (0.00%)
Restore.Point....: 0/69833729609375 (0.00%)
Candidates.#1....: Zarierin -> Z:[olerd
Candidates.#2....: Y`=monan -> Y]/merin
HWMon.Dev.#1.....: Temp: 46c Fan: 0% Util: 96% Core:1847MHz Mem:5005MHz Bus:8
HWMon.Dev.#2.....: Temp: 50c Fan: 29% Util: 97% Core:1771MHz Mem:5005MHz Bus:8