Dual GPU & OPenCL/Mesa Issues

[DrRobotnik]

Hi,

apologies if this is in the wrong subforum.

I recently posted here for some advice on my previous GPU (https://hashcat.net/forum/thread-6265.html) and after banging my head against a wall, decided to bite the bullet and invest in a better GPU that was compatible with the AMD-GPU-pro driver (why i didnt just go for NVidia then, i dont know, i hate myself)

Purchased a AMD R9 290x, and after again banging my head trying to get the AMDGPU-pro driver installed on The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali) with 4.9.6 kernel (its wouldnt install the module against that kernel version, only 4.4) i managed to get this GPU running on hashcat on Mint 18 with a 4.4 kernel.

Great! so i bought a second GPU, another R9 290x. Plug and play i think.

However this is where i encountered the problems. The PC recognizes both cards, and can benchmark both at around 190K each for WPA. But when actually cracking, i get the following errors:

Code:

hashcat (v3.30) starting...

OpenCL Platform #1: Advanced Micro Devices, Inc.
================================================
* Device #1: Hawaii, 2047/3865 MB allocatable, 14MCU
* Device #2: Hawaii, 2047/4043 MB allocatable, 14MCU
* Device #3: AMD FX(tm)-4100 Quad-Core Processor, skipped

OpenCL Platform #2: Mesa
========================
* Device #4: AMD HAWAII (DRM 3.2.0 / 4.4.0-53-generic, LLVM 3.9.1), 2047/4094 MB allocatable, 44MCU
* Device #5: AMD HAWAII (DRM 3.2.0 / 4.4.0-53-generic, LLVM 3.9.1), 2047/4094 MB allocatable, 44MCU

Hashes: 1 digests; 1 unique digests, 1 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates
Rules: 390625

Applicable Optimizers:
* Zero-Byte
* Single-Hash
* Single-Salt
* Slow-Hash-SIMD

Watchdog: Temperature abort trigger set to 90c
Watchdog: Temperature retain trigger set to 75c

* Device #1: build_opts '-I /usr/share/hashcat/OpenCL -D VENDOR_ID=1 -D CUDA_ARCH=0 -D VECT_SIZE=1 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=4 -D KERN_TYPE=2500 -D _unroll -cl-std=CL1.2'
* Device #1: Kernel /root/.hashcat/kernels/m02500.b9797364.kernel (1228600 bytes)
* Device #1: Kernel /root/.hashcat/kernels/amp_a0.b9797364.kernel (776312 bytes)
* Device #2: build_opts '-I /usr/share/hashcat/OpenCL -D VENDOR_ID=1 -D CUDA_ARCH=0 -D VECT_SIZE=1 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=4 -D KERN_TYPE=2500 -D _unroll -cl-std=CL1.2'
* Device #2: Kernel /root/.hashcat/kernels/m02500.b9797364.kernel (1228600 bytes)
* Device #2: Kernel /root/.hashcat/kernels/amp_a0.b9797364.kernel (776312 bytes)
* Device #4: build_opts '-I /usr/share/hashcat/OpenCL -D VENDOR_ID=16 -D CUDA_ARCH=0 -D VECT_SIZE=4 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=4 -D KERN_TYPE=2500 -D _unroll -cl-std=CL1.2'
* Device #4: Kernel m02500.a465c7df.kernel not found in cache! Building may take a while...
clBuildProgram(): CL_BUILD_PROGRAM_FAILURE

input.cl:20:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:118:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:169:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:203:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:331:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:384:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:420:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:548:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier

* Device #4: Kernel /usr/share/hashcat/OpenCL/m02500.cl build failure. Proceeding without this device.

* Device #5: build_opts '-I /usr/share/hashcat/OpenCL -D VENDOR_ID=16 -D CUDA_ARCH=0 -D VECT_SIZE=4 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=4 -D KERN_TYPE=2500 -D _unroll -cl-std=CL1.2'
* Device #5: Kernel m02500.a465c7df.kernel not found in cache! Building may take a while...
clBuildProgram(): CL_BUILD_PROGRAM_FAILURE

input.cl:20:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:118:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:169:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:203:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:331:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:384:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:420:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier
input.cl:548:1: error: OpenCL version 1.1 does not support the 'static' storage class specifier

* Device #5: Kernel /usr/share/hashcat/OpenCL/m02500.cl build failure. Proceeding without this device.

and the cracking will proceed with only one GPU. Sometimes its device #1, sometimes #2, but never both at the same time.

Looking at the messaging, it would appear that OpenCL version 1.1 is not compatible, but this is what was installed with the AMDGPU-pro driver. I dont think my hardware is at fault as both were able to benchmark successfully together and if both run when cracking (at times seperately) i dont think its the hardware.

Really stumped as to why these dont work together, and cannot seem to upgrade OpenCL to 1.2 or 2.0 if compatible (cards are compatible with 2.0 after reading the wiki) or if this would even fix the issue.

Any advice would be greatly appreciated.

Thanks

DrR

[epixoip]

Get rid of Mesa.

[DrRobotnik]

Get rid of Mesa.

How? Removing mesa-opencl-icd automatically removed hashcat. And reinstalling hashcat reinstalls mesa?

Is there a version of hashcat that runs without mesa, or a way to remove mesa without uninstalling hashcat?

[epixoip]

Get rid of The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali). Or stop installing hashcat via The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali)'s repository. Or run Hashcat with --opencl-platforms 1 to only use the AMD platform. Or do all three. Either way, Mesa is the issue here. And if The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali) installs Mesa along with hashcat, then The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali) is the issue as well.

[DrRobotnik]

Thanks for the advice. Wasnt aware that hashcat via the The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali) repo would be the cause.

Have reinstalled hashcat from source on Mint, and am no longer seeing the previous errors with OpenCL/Mesa.

However, still only one GPU is being utilized (the below is a test cap to confirm both GPUs are working together):

Code:

hashcat (v3.30) starting...

OpenCL Platform #1: Advanced Micro Devices, Inc.
================================================
* Device #1: Hawaii, 2047/3884 MB allocatable, 14MCU
* Device #2: Hawaii, 2047/4043 MB allocatable, 14MCU
* Device #3: AMD FX(tm)-4100 Quad-Core Processor, skipped

Hashes: 1 digests; 1 unique digests, 1 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates
Rules: 390625

Applicable Optimizers:
* Zero-Byte
* Single-Hash
* Single-Salt
* Slow-Hash-SIMD

Watchdog: Temperature abort trigger set to 90c
Watchdog: Temperature retain trigger set to 75c

Starting attack in stdin mode...

Session..........: hashcat
Status...........: Running
Hash.Type........: WPA/WPA2
Hash.Target......: ********
Time.Started.....: Sat Feb 18 10:11:07 2017 (6 secs)
Time.Estimated...: Sat Feb 18 10:11:13 2017 (0 secs)
Input.Base.......: Pipe
Input.Mod........: Rules (rule.rule)
Speed.Dev.#1.....:   199.0 kH/s (70.95ms)
Speed.Dev.#2.....:        0 H/s (0.00ms)
Speed.Dev.#*.....:   198.9 kH/s
Recovered........: 0/1 (0.00%) Digests, 0/1 (0.00%) Salts
Progress.........: 913952
Rejected.........: 0
Restore.Point....: 0
Candidates.#1....: AAAAAAAC -> ZZZZAAAC
Candidates.#2....: [Copying]
HWMon.Dev.#1.....: Temp: 59c Fan: 30% Core:1025Mhz Mem:1350Mhz Lanes:16
HWMon.Dev.#2.....: Temp: 38c Fan: 30% Core: 300Mhz Mem: 150Mhz Lanes:16

i know that both GPUs work independantly, and can benchmark both GPUs together, so dont think theres an issue with hardware, but have been wrong several times before with this process.

Any advice?

DrR

[Mangix]

Report the bug to Mesa. There's probably nothing that can be done.

[epixoip]

The pipe may be the culprit here. Can you test with other attack modes to see if it is?