5x 1080 TI on linux
#1
Hello.

I bought 5 Nvidia 1080 TI Turbo with 11 GB of RAM. 
I have some problem to make it work Ubuntu 20.04.4 LTS

I followed this guide (https://www.unix-ninja.com/p/Building_a_...-_Part_III), the installation of NVIDIA driver seems good but when I start hashcat, I always have this error : 

Code:
No devices found/left.

When I run lspci, I correctly see my 1080 TI. 

thank you for your precious help
Reply
#2
(07-25-2022, 03:39 PM)fl0at0xff Wrote: Hello.

I bought 5 Nvidia 1080 TI Turbo with 11 GB of RAM. 
I have some problem to make it work Ubuntu 20.04.4 LTS

I followed this guide (https://www.unix-ninja.com/p/Building_a_...-_Part_III), the installation of NVIDIA driver seems good but when I start hashcat, I always have this error : 

Code:
No devices found/left.

When I run lspci, I correctly see my 1080 TI. 

thank you for your precious help

Those instructions are five years old, which counts as ancient lore in GPU land.  Install the latest NVIDIA drivers and CUDA toolkit that are available for the 1080.

from hashcat.net: "NVIDIA GPUs require "NVIDIA Driver" (440.64 or later) and "CUDA Toolkit" (9.0 or later)"
Reply
#3
(07-25-2022, 03:49 PM)pdo Wrote: Those instructions are five years old, which counts as ancient lore in GPU land.  Install the latest NVIDIA drivers and CUDA toolkit that are available for the 1080.

from hashcat.net: "NVIDIA GPUs require "NVIDIA Driver" (440.64 or later) and "CUDA Toolkit" (9.0 or later)"

Hum now seems better. I installed drivers version 515.57 with CUDA 11.7. 
I can run nvidia-smi and the output show me the result. 

It is important to not forget to blacklist Nouveau driver.
Now, how can I test if hashcat is working correctly with my GPU ?
Reply
#4
(07-25-2022, 04:41 PM)fl0at0xff Wrote:
(07-25-2022, 03:49 PM)pdo Wrote: Those instructions are five years old, which counts as ancient lore in GPU land.  Install the latest NVIDIA drivers and CUDA toolkit that are available for the 1080.

from hashcat.net: "NVIDIA GPUs require "NVIDIA Driver" (440.64 or later) and "CUDA Toolkit" (9.0 or later)"

Hum now seems better. I installed drivers version 515.57 with CUDA 11.7. 
I can run nvidia-smi and the output show me the result. 

It is important to not forget to blacklist Nouveau driver.
Now, how can I test if hashcat is working correctly with my GPU ?

just run the benchmark: hashcat -b
Reply
#5
(07-25-2022, 04:51 PM)pdo Wrote: just run the benchmark: hashcat -b

Ok. I always tried to crack a simple sha512crypt password "Password" using this command :

Code:
hashcat -m 1800 hash.txt ../wordlist/SecLists/Passwords/Common-Credentials/500-worst-passwords.txt

where hash.txt contains

Quote:$6$uwAT6nE2C3o0THWW$QsErk0YrwCmQdr0XLiJ34ZJlA0lGZ9cEvJxt9NvG4E9lVUwCvhj1h8rkGpA3q8vhTcSL4PkKm9Sf3JhG/WaNH.

Seems correct ? I Have this output but this seems not working. 

Code:
OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: NVIDIA GeForce GTX 1080 Ti, 2794/11176 MB allocatable, 28MCU
* Device #2: NVIDIA GeForce GTX 1080 Ti, 2794/11178 MB allocatable, 28MCU
* Device #3: NVIDIA GeForce GTX 1080 Ti, 2794/11178 MB allocatable, 28MCU

OpenCL Platform #2: The pocl project
====================================
* Device #4: pthread-Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz, skipped.

Hashes: 1 digests; 1 unique digests, 1 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates
Rules: 1

Applicable optimizers:
* Zero-Byte
* Single-Hash
* Single-Salt
* Uses-64-Bit

Minimum password length supported by kernel: 0
Maximum password length supported by kernel: 256

ATTENTION! Pure (unoptimized) OpenCL kernels selected.
This enables cracking passwords and salts > length 32 but for the price of drastically reduced performance.
If you want to switch to optimized OpenCL kernels, append -O to your commandline.

Watchdog: Temperature abort trigger set to 90c

* Device #1: build_opts '-cl-std=CL1.2 -I OpenCL -I /usr/share/hashcat/OpenCL -D LOCAL_MEM_TYPE=1 -D VENDOR_ID=32 -D CUDA_ARCH=601 -D AMD_ROCM=0 -D VECT_SIZE=1 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=16 -D KERN_TYPE=1800 -D _unroll'
* Device #2: build_opts '-cl-std=CL1.2 -I OpenCL -I /usr/share/hashcat/OpenCL -D LOCAL_MEM_TYPE=1 -D VENDOR_ID=32 -D CUDA_ARCH=601 -D AMD_ROCM=0 -D VECT_SIZE=1 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=16 -D KERN_TYPE=1800 -D _unroll'
* Device #3: build_opts '-cl-std=CL1.2 -I OpenCL -I /usr/share/hashcat/OpenCL -D LOCAL_MEM_TYPE=1 -D VENDOR_ID=32 -D CUDA_ARCH=601 -D AMD_ROCM=0 -D VECT_SIZE=1 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=16 -D KERN_TYPE=1800 -D _unroll'
Dictionary cache built:
* Filename..: ../wordlist/SecLists/Passwords/Common-Credentials/500-worst-passwords.txt
* Passwords.: 499
* Bytes.....: 3491
* Keyspace..: 499
* Runtime...: 0 secs
Reply
#6
(07-25-2022, 04:58 PM)fl0at0xff Wrote:
(07-25-2022, 04:51 PM)pdo Wrote: just run the benchmark: hashcat -b

Ok. I always tried to crack a simple sha512crypt password "Password" using this command :

Code:
hashcat -m 1800 hash.txt ../wordlist/SecLists/Passwords/Common-Credentials/500-worst-passwords.txt

where hash.txt contains

Quote:$6$uwAT6nE2C3o0THWW$QsErk0YrwCmQdr0XLiJ34ZJlA0lGZ9cEvJxt9NvG4E9lVUwCvhj1h8rkGpA3q8vhTcSL4PkKm9Sf3JhG/WaNH.

Seems correct ? I Have this output but this seems not working. 

Code:
OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: NVIDIA GeForce GTX 1080 Ti, 2794/11176 MB allocatable, 28MCU
* Device #2: NVIDIA GeForce GTX 1080 Ti, 2794/11178 MB allocatable, 28MCU
* Device #3: NVIDIA GeForce GTX 1080 Ti, 2794/11178 MB allocatable, 28MCU

OpenCL Platform #2: The pocl project
====================================
* Device #4: pthread-Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz, skipped.

Hashes: 1 digests; 1 unique digests, 1 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates
Rules: 1

Applicable optimizers:
* Zero-Byte
* Single-Hash
* Single-Salt
* Uses-64-Bit

Minimum password length supported by kernel: 0
Maximum password length supported by kernel: 256

ATTENTION! Pure (unoptimized) OpenCL kernels selected.
This enables cracking passwords and salts > length 32 but for the price of drastically reduced performance.
If you want to switch to optimized OpenCL kernels, append -O to your commandline.

Watchdog: Temperature abort trigger set to 90c

* Device #1: build_opts '-cl-std=CL1.2 -I OpenCL -I /usr/share/hashcat/OpenCL -D LOCAL_MEM_TYPE=1 -D VENDOR_ID=32 -D CUDA_ARCH=601 -D AMD_ROCM=0 -D VECT_SIZE=1 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=16 -D KERN_TYPE=1800 -D _unroll'
* Device #2: build_opts '-cl-std=CL1.2 -I OpenCL -I /usr/share/hashcat/OpenCL -D LOCAL_MEM_TYPE=1 -D VENDOR_ID=32 -D CUDA_ARCH=601 -D AMD_ROCM=0 -D VECT_SIZE=1 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=16 -D KERN_TYPE=1800 -D _unroll'
* Device #3: build_opts '-cl-std=CL1.2 -I OpenCL -I /usr/share/hashcat/OpenCL -D LOCAL_MEM_TYPE=1 -D VENDOR_ID=32 -D CUDA_ARCH=601 -D AMD_ROCM=0 -D VECT_SIZE=1 -D DEVICE_TYPE=4 -D DGST_R0=0 -D DGST_R1=1 -D DGST_R2=2 -D DGST_R3=3 -D DGST_ELEM=16 -D KERN_TYPE=1800 -D _unroll'
Dictionary cache built:
* Filename..: ../wordlist/SecLists/Passwords/Common-Credentials/500-worst-passwords.txt
* Passwords.: 499
* Bytes.....: 3491
* Keyspace..: 499
* Runtime...: 0 secs

Not sure what's going on there, but it doesn't look like you've got CUDA installed (correctly?)
Reply
#7
(07-25-2022, 05:02 PM)pdo Wrote: Not sure what's going on there, but it doesn't look like you've got CUDA installed (correctly?)

CUDA seems installed but I don't know how to check if I use it. 

Code:
god@crackfarm:~/test$ nvidia-smi
Mon Jul 25 15:07:30 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.226.00  Driver Version: 515.57      CUDA Version: 11.7    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|        Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|  0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 24%  36C    P0    54W / 250W |    87MiB / 11264MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|  1  NVIDIA GeForce ...  Off  | 00000000:03:00.0 Off |                  N/A |
| 23%  37C    P0    54W / 250W |    85MiB / 11264MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|  2  NVIDIA GeForce ...  Off  | 00000000:04:00.0 Off |                  N/A |
| 16%  44C    P0    56W / 250W |    85MiB / 11264MiB |      3%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                      GPU Memory |
|  GPU      PID  Type  Process name                            Usage      |
|=============================================================================|
|  No running processes found                                                |
Reply
#8
(07-25-2022, 05:08 PM)fl0at0xff Wrote:
(07-25-2022, 05:02 PM)pdo Wrote: Not sure what's going on there, but it doesn't look like you've got CUDA installed (correctly?)

CUDA seems installed but I don't know how to check if I use it. 

Code:
god@crackfarm:~/test$ nvidia-smi
Mon Jul 25 15:07:30 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.226.00  Driver Version: 515.57      CUDA Version: 11.7    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|        Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|  0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 24%  36C    P0    54W / 250W |    87MiB / 11264MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|  1  NVIDIA GeForce ...  Off  | 00000000:03:00.0 Off |                  N/A |
| 23%  37C    P0    54W / 250W |    85MiB / 11264MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|  2  NVIDIA GeForce ...  Off  | 00000000:04:00.0 Off |                  N/A |
| 16%  44C    P0    56W / 250W |    85MiB / 11264MiB |      3%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                      GPU Memory |
|  GPU      PID  Type  Process name                            Usage      |
|=============================================================================|
|  No running processes found                                                |

After installed the latest version of hc, it working better. 
When I launch benchmark, the GUI stucks after some minutes. do you have idea ? I still can press "S" for status but nothing is updated. All numbers and statistics are the same and the "q" key for quit does not quit the process.
Reply
#9
use
Code:
hashcat -I
to see which devices and how these are recognized (cuda or failback opencl), please post your output

given this last post:

OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: NVIDIA GeForce GTX 1080 Ti, 2794/11176 MB allocatable, 28MCU
* Device #2: NVIDIA GeForce GTX 1080 Ti, 2794/11178 MB allocatable, 28MCU
* Device #3: NVIDIA GeForce GTX 1080 Ti, 2794/11178 MB allocatable, 28MCU

OpenCL Platform #2: The pocl project
====================================
* Device #4: pthread-Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz, skipped.

it seems hahscat didnt use cuda for your cards, next, only 3 cards are recognized, you said 5?
Reply
#10
hello, below the output. Seems to have some bug with CUDA. I only connected 3 GPU for now. 

Code:
Successfully initialized NVIDIA CUDA library.

Failed to initialize NVIDIA RTC library.

* Device #1: CUDA SDK Toolkit not installed or incorrectly installed.
            CUDA SDK Toolkit required for proper device support and utilization.
            Falling back to OpenCL runtime.

Successfully initialized NVIDIA CUDA library.

Failed to initialize NVIDIA RTC library.

* Device #2: CUDA SDK Toolkit not installed or incorrectly installed.
            CUDA SDK Toolkit required for proper device support and utilization.
            Falling back to OpenCL runtime.

Successfully initialized NVIDIA CUDA library.

Failed to initialize NVIDIA RTC library.

* Device #3: CUDA SDK Toolkit not installed or incorrectly installed.
            CUDA SDK Toolkit required for proper device support and utilization.
            Falling back to OpenCL runtime.

* Device #4: Outdated POCL OpenCL driver detected!

OpenCL Info:
============

OpenCL Platform ID #1
  Vendor..: NVIDIA Corporation
  Name....: NVIDIA CUDA
  Version.: OpenCL 3.0 CUDA 11.7.99

  Backend Device ID #1
    Type...........: GPU
    Vendor.ID......: 32
    Vendor.........: NVIDIA Corporation
    Name...........: NVIDIA GeForce GTX 1080 Ti
    Version........: OpenCL 3.0 CUDA
    Processor(s)...: 28
    Clock..........: 1582
    Memory.Total...: 11176 MB (limited to 2794 MB allocatable in one block)
    Memory.Free....: 11008 MB
    OpenCL.Version.: OpenCL C 1.2
    Driver.Version.: 515.57
    PCI.Addr.BDF...: 01:00.0

  Backend Device ID #2
    Type...........: GPU
    Vendor.ID......: 32
    Vendor.........: NVIDIA Corporation
    Name...........: NVIDIA GeForce GTX 1080 Ti
    Version........: OpenCL 3.0 CUDA
    Processor(s)...: 28
    Clock..........: 1582
    Memory.Total...: 11178 MB (limited to 2794 MB allocatable in one block)
    Memory.Free....: 11008 MB
    OpenCL.Version.: OpenCL C 1.2
    Driver.Version.: 515.57
    PCI.Addr.BDF...: 03:00.0

  Backend Device ID #3
    Type...........: GPU
    Vendor.ID......: 32
    Vendor.........: NVIDIA Corporation
    Name...........: NVIDIA GeForce GTX 1080 Ti
    Version........: OpenCL 3.0 CUDA
    Processor(s)...: 28
    Clock..........: 1582
    Memory.Total...: 11178 MB (limited to 2794 MB allocatable in one block)
    Memory.Free....: 11008 MB
    OpenCL.Version.: OpenCL C 1.2
    Driver.Version.: 515.57
    PCI.Addr.BDF...: 04:00.0

OpenCL Platform ID #2
  Vendor..: The pocl project
  Name....: Portable Computing Language
  Version.: OpenCL 1.2 pocl 1.4, None+Asserts, LLVM 9.0.1, RELOC, SLEEF, DISTRO, POCL_DEBUG

  Backend Device ID #4
    Type...........: CPU
    Vendor.ID......: 128
    Vendor.........: GenuineIntel
    Name...........: pthread-Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz
    Version........: OpenCL 1.2 pocl HSTR: pthread-x86_64-pc-linux-gnu-skylake
    Processor(s)...: 4
    Clock..........: 3900
    Memory.Total...: 5873 MB (limited to 1024 MB allocatable in one block)
    Memory.Free....: 2904 MB
    OpenCL.Version.: OpenCL C 1.2 pocl
    Driver.Version.: 1.4

I think I need to install CUDA SDK too ?
Reply