Only 3/8 GPU used

[ssjtoma]

Hi all,
I successfully got cudaHashcat running but it only uses 3 out of 8 nVidia GPU's in the system. I tried specifying -d 1,2,3,4,5,6,7,8 but still only 3 cards are crunching on a test MD5 hash. Is that normal for short masks (4 on the left and 4 on the right with increment)?

Also, I noticed that the speed is 100x faster with --gpu-async. The cards are brand new Tesla S2050's (not GTS 8800 or similar).

Status.......: Running
Hash.Type....: MD5
Mode.Left....: Mask '?1?1?1?1' (81450625)
Mode.Right...: Mask '?1?1?1' (857375)
Speed.GPU*...: 7800.3M/s
Recovered....: 0/1 Digests, 0/1 Salts
Progress.....: 11560905932800/69833729609375 (16.55%)
Running......: 25 mins, 43 secs
Estimated....: 2 hours, 4 mins
HW.Monitor.#1: 0% GPU, 0c Temp
HW.Monitor.#2: 0% GPU, 0c Temp
HW.Monitor.#3: 0% GPU, 0c Temp
HW.Monitor.#4: 99% GPU, 0c Temp
HW.Monitor.#5: 92% GPU, 0c Temp
HW.Monitor.#6: 99% GPU, 0c Temp
HW.Monitor.#7: 0% GPU, 0c Temp
HW.Monitor.#8: 0% GPU, 0c Temp

EDIT:
Using -n 800 and -gpu-loops=1024 I got one more card running, still not sure why the rest idle. Also, interestingly the speed is lower with the extra card crunching:
Status.......: Running
Hash.Type....: MD5
Mode.Left....: Mask '?1?1?1?1' (81450625)
Mode.Right...: Mask '?1?1?1' (857375)
Speed.GPU*...: 7259.7M/s
Recovered....: 0/1 Digests, 0/1 Salts
Progress.....: 652475136000/69833729609375 (0.93%)
Running......: 1 min, 31 secs
Estimated....: 2 hours, 38 mins
HW.Monitor.#1: 99% GPU, 0c Temp
HW.Monitor.#2: 0% GPU, 0c Temp
HW.Monitor.#3: 0% GPU, 0c Temp
HW.Monitor.#4: 99% GPU, 0c Temp
HW.Monitor.#5: 99% GPU, 0c Temp
HW.Monitor.#6: 99% GPU, 0c Temp
HW.Monitor.#7: 0% GPU, 0c Temp
HW.Monitor.#8: 0% GPU, 0c Temp

[Rolf]

When I had 3 GTX 480 gpus, some weren't always used.
Turns out it's about combinations on the left side.
Try ?1?1?1?1?1 ?1?1

[ssjtoma]

When I had 3 GTX 480 gpus, some weren't always used.
Turns out it's about combinations on the left side.
Try ?1?1?1?1?1 ?1?1

Thanks, that did fire up the rest of the GPU's as well, but the performance is no better then when it only used 3:

Status.......: Running
Hash.Type....: MD5
Mode.Left....: Mask '?1?1?1?1?1' (773
Mode.Right...: Mask '?1?1' (9025)
Speed.GPU*...: 7745.4M/s
Recovered....: 0/1 Digests, 0/1 Salts
Progress.....: 357493964800/698337296
Running......: 47 secs
Estimated....: 2 hours, 29 mins
HW.Monitor.#1: 99% GPU, 0c Temp
HW.Monitor.#2: 99% GPU, 0c Temp
HW.Monitor.#3: 99% GPU, 0c Temp
HW.Monitor.#4: 99% GPU, 0c Temp
HW.Monitor.#5: 99% GPU, 0c Temp
HW.Monitor.#6: 99% GPU, 0c Temp
HW.Monitor.#7: 99% GPU, 0c Temp
HW.Monitor.#8: 99% GPU, 0c Temp

EDIT:
cudaHashcat-lite seems to be using 6 of the GPU's and has much superior speed then cudaHashcat with all gpu's. Weird:

Status.......: Running
Hash.Type....: MD5
Time.Running.: 48 secs
Time.Left....: 15 secs
Plain.Text...: **#J1@
Plain.Length.: 6
Speed........: 11.4G/s
Progress.....: 553199861760/735091890625 (75.26%)
HW.Monitor.#1: 99% GPU, 0c Temp
HW.Monitor.#2: 99% GPU, 0c Temp
HW.Monitor.#3: 99% GPU, 0c Temp
HW.Monitor.#4: 99% GPU, 0c Temp
HW.Monitor.#5: 99% GPU, 0c Temp
HW.Monitor.#6: 99% GPU, 0c Temp
HW.Monitor.#7: 0% GPU, 0c Temp
HW.Monitor.#8: 0% GPU, 0c Temp

[atom]

you need to feed it with enough work. that is by increasing the combinations of the left side. try -1 ?l?d?s?u ?1?1?1?1 ?1?1?1?1 -n 80

[atom]

or try oclHashcat-lite. it manages everything for you

[ssjtoma]

or try oclHashcat-lite. it manages everything for you

Thanks, oclHashcat-lite is faster but it still only uses 6 of the GPU's with -1 ?l?u?s?d ?1?1?1?1?1?1?1 (that is 7 char).

[atom]

i think its still the same problem. its the sake of massive parallelism. give it enough work to do.

[ssjtoma]

i think its still the same problem. its the sake of massive parallelism. give it enough work to do.

Nope, it seems like the cards are crunching, just not reporting correctly their utilization. I tried splitting the work into two sets, which halved the performance and the second set reported 0% utilization with 5G/s performance. I have latest drivers installed so not sure where the error is (temperatures are not reported either for any of the cards)

Status.......: Running
Hash.Type....: MD5
Time.Running.: 41 secs
Time.Left....: 1 day, 17 hours
Plain.Text...: ***-DHfa
Plain.Length.: 8
Speed........: 5559.2M/s
Progress.....: 230183403520/838004755312500 (0.03%)
HW.Monitor.#1: 0% GPU, 0c Temp
HW.Monitor.#2: 0% GPU, 0c Temp
HW.Monitor.#3: 0% GPU, 0c Temp
HW.Monitor.#4: 0% GPU, 0c Temp

[atom]

pls check output of nvidia-smi if it behaves the same its maybe some driver problem

[ssjtoma]

pls check output of nvidia-smi if it behaves the same its maybe some driver problem

It's a driver issue. Had to switch from HP recommended drivers to nvidia's, that fixed the utilization display, not the temperature issue. Any way to have hashcat use freeipmi instead of nvidia-smi (that seems to be able to get the temps right)?