I'm having trouble with hashcat 6 working with any of the NVidia 2000 series GPUs I've tried. I've tried on a laptop RTX 2060 (Linux Mint) and a hashing rigs with a mix of GTX 1080 Founders and GTX 2080 Supers (Ubuntu 18.04) with the exact same results on the 2080 GPUs only. The 1080's work just fine...
I have easier access to the laptop so I will use that for the examples below --
First - without nvidia-cuda-toolkit installed:
And the nvidia packages that are installed when it works:
As you can see, it works, but complains about falling back to OpenCL. In addition, hashcat 6 pegs the CPU at 100% when running in OpenCL fallback. This is not ideal. hashcat 5 can run for weeks (probably forever) without using more than 20% of the CPU, so we are currently reverted back to hashcat 5 because of this problem.
With Cuda Toolkit installed:
Here is the package list for all nvidia when the cuda toolkit is installed:
As you can see, the GTX 2080's just don't work with nvidia-cuda-toolkit and dependencies installed. I've tried using the Ubuntu default repositories and installing the NVidia drivers with apt, and also downloading the drivers directly from NVidia and installing them that way. No change.
I have used the precompiled hashcat binaries and built from source. I've also tried the same with the latest beta of hashcat. No change.
Additionally, hashcat 6.0 with only the 1080 GPU AND cuda toolkit still pegs the CPU at 100%, so we still can't use hashcat 6 with just those GPU without CPU issues.
Any ideas why this doesn't work?
I have easier access to the laptop so I will use that for the examples below --
First - without nvidia-cuda-toolkit installed:
Code:
$ sudo hashcat-6.0.0/hashcat.bin -m 1000 -b
hashcat (v6.0.0) starting in benchmark mode...
Benchmarking uses hand-optimized kernel code by default.
You can use it in your cracking session by setting the -O option.
Note: Using optimized kernel code limits the maximum supported password length.
To disable the optimized kernel code in benchmark mode, use the -w option.
* Device #1: CUDA SDK Toolkit installation NOT detected.
CUDA SDK Toolkit installation required for proper device support and utilization
Falling back to OpenCL Runtime
* Device #1: WARNING! Kernel exec timeout is not disabled.
This may cause "CL_OUT_OF_RESOURCES" or related errors.
To disable the timeout, see: https://hashcat.net/q/timeoutpatch
nvmlDeviceGetFanSpeed(): Not Supported
OpenCL API (OpenCL 1.2 CUDA 10.2.185) - Platform #1 [NVIDIA Corporation]
========================================================================
* Device #1: GeForce RTX 2060, 5632/5934 MB (1483 MB allocatable), 30MCU
Benchmark relevant options:
===========================
* --optimized-kernel-enable
Hashmode: 1000 - NTLM
Speed.#1.........: 37744.5 MH/s (53.21ms) @ Accel:64 Loops:1024 Thr:1024 Vec:8
Started: Thu Jul 9 17:51:19 2020
Stopped: Thu Jul 9 17:51:26 2020And the nvidia packages that are installed when it works:
Code:
$ sudo apt list --installed | grep -i nvidia
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
libnvidia-cfg1-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-common-435/bionic,bionic,bionic-updates,bionic-updates,now 435.21-0ubuntu0.18.04.2 all [installed]
libnvidia-common-440/bionic,bionic,bionic-updates,bionic-updates,bionic-security,bionic-security,now 440.100-0ubuntu0.18.04.1 all [installed,automatic]
libnvidia-compute-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-decode-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-encode-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-extra-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-fbc1-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-gl-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-ifr1-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-compute-utils-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-dkms-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-driver-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed]
nvidia-kernel-common-440/bionic,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-kernel-source-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-prime/bionic-updates,bionic-updates,now 0.8.8.2 all [installed]
nvidia-settings/bionic,now 440.64-0ubuntu0~0.18.04.1 amd64 [installed]
nvidia-utils-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
xserver-xorg-video-nvidia-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]As you can see, it works, but complains about falling back to OpenCL. In addition, hashcat 6 pegs the CPU at 100% when running in OpenCL fallback. This is not ideal. hashcat 5 can run for weeks (probably forever) without using more than 20% of the CPU, so we are currently reverted back to hashcat 5 because of this problem.
With Cuda Toolkit installed:
Code:
$ sudo hashcat-6.0.0/hashcat.bin -m 1000 -b
hashcat (v6.0.0) starting in benchmark mode...
Benchmarking uses hand-optimized kernel code by default.
You can use it in your cracking session by setting the -O option.
Note: Using optimized kernel code limits the maximum supported password length.
To disable the optimized kernel code in benchmark mode, use the -w option.
* Device #1: WARNING! Kernel exec timeout is not disabled.
This may cause "CL_OUT_OF_RESOURCES" or related errors.
To disable the timeout, see: https://hashcat.net/q/timeoutpatch
* Device #2: WARNING! Kernel exec timeout is not disabled.
This may cause "CL_OUT_OF_RESOURCES" or related errors.
To disable the timeout, see: https://hashcat.net/q/timeoutpatch
nvmlDeviceGetFanSpeed(): Not Supported
CUDA API (CUDA 10.2)
====================
* Device #1: GeForce RTX 2060, 5644/5934 MB, 30MCU
OpenCL API (OpenCL 1.2 CUDA 10.2.185) - Platform #1 [NVIDIA Corporation]
========================================================================
* Device #2: GeForce RTX 2060, skipped
Benchmark relevant options:
===========================
* --optimized-kernel-enable
Hashmode: 1000 - NTLM
nvrtcCompileProgram(): NVRTC_ERROR_INVALID_OPTION
nvrtc: error: invalid value for --gpu-architecture (-arch)
* Device #1: Kernel /home/jen42/hashcat-6.0.0/OpenCL/shared.cl build failed.
* Device #1: Kernel /home/jen42/hashcat-6.0.0/OpenCL/shared.cl build failed.
Started: Thu Jul 9 18:00:43 2020
Stopped: Thu Jul 9 18:00:44 2020Here is the package list for all nvidia when the cuda toolkit is installed:
Code:
$ sudo apt list --installed | grep -i nvidia
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
libnvidia-cfg1-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-common-435/bionic,bionic,bionic-updates,bionic-updates,now 435.21-0ubuntu0.18.04.2 all [installed]
libnvidia-common-440/bionic,bionic,bionic-updates,bionic-updates,bionic-security,bionic-security,now 440.100-0ubuntu0.18.04.1 all [installed,automatic]
libnvidia-compute-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-decode-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-encode-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-extra-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-fbc1-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-gl-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
libnvidia-ifr1-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-compute-utils-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-cuda-dev/bionic,now 9.1.85-3ubuntu1 amd64 [installed,automatic]
nvidia-cuda-toolkit/bionic,now 9.1.85-3ubuntu1 amd64 [installed]
nvidia-dkms-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-driver-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed]
nvidia-kernel-common-440/bionic,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-kernel-source-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
nvidia-opencl-dev/bionic,now 9.1.85-3ubuntu1 amd64 [installed,automatic]
nvidia-prime/bionic-updates,bionic-updates,now 0.8.8.2 all [installed]
nvidia-profiler/bionic,now 9.1.85-3ubuntu1 amd64 [installed,automatic]
nvidia-settings/bionic,now 440.64-0ubuntu0~0.18.04.1 amd64 [installed]
nvidia-utils-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]
xserver-xorg-video-nvidia-440/bionic,bionic-updates,bionic-security,now 440.100-0ubuntu0.18.04.1 amd64 [installed,automatic]As you can see, the GTX 2080's just don't work with nvidia-cuda-toolkit and dependencies installed. I've tried using the Ubuntu default repositories and installing the NVidia drivers with apt, and also downloading the drivers directly from NVidia and installing them that way. No change.
I have used the precompiled hashcat binaries and built from source. I've also tried the same with the latest beta of hashcat. No change.
Additionally, hashcat 6.0 with only the 1080 GPU AND cuda toolkit still pegs the CPU at 100%, so we still can't use hashcat 6 with just those GPU without CPU issues.
Any ideas why this doesn't work?