Fedora 38 + AMD RX 7900 XT
#1
Hi,

I've got two devices;
GPU: AMD Radeon RX 7900 XT (Navi 31)
CPU+GPU: AMD Ryzen 9 7950X

On https://hashcat.net/hashcat/ it says "AMD GPUs on Linux require "AMDGPU" (21.50 or later) and "ROCm" (5.0 or later)". However, I'm unable to get it working with amdgpu 23.0.0 (xorg-x11-drv-amdgpu-23.0.0-1.fc38.x86_64) and rocm 5.5.0 (rocm-opencl-5.5.0-1.fc38.src.rpm) on Fedora 38.

If I go without the rocm-opencl package and with these packages installed:
Code:
$ rpm -qa | grep -iE '(amd|rocm|mesa|opencl)'
mesa-filesystem-23.0.3-3.fc38.x86_64
mesa-filesystem-23.0.3-3.fc38.i686
amd-gpu-firmware-20230404-149.fc38.noarch
opencl-headers-3.0-14.20230201git4c82e9c.fc38.noarch
opencl-filesystem-1.0-17.fc38.noarch
mesa-libglapi-23.0.3-3.fc38.x86_64
mesa-vulkan-drivers-23.0.3-3.fc38.x86_64
mesa-libOSMesa-23.0.3-3.fc38.x86_64
mesa-va-drivers-23.0.3-3.fc38.x86_64
mesa-dri-drivers-23.0.3-3.fc38.x86_64
mesa-libgbm-23.0.3-3.fc38.x86_64
mesa-libEGL-23.0.3-3.fc38.x86_64
mesa-libGL-23.0.3-3.fc38.x86_64
mesa-libGLU-9.0.1-8.fc38.x86_64
mesa-libxatracker-23.0.3-3.fc38.x86_64
mesa-libEGL-devel-23.0.3-3.fc38.x86_64
mesa-libGLU-devel-9.0.1-8.fc38.x86_64
mesa-libGL-devel-23.0.3-3.fc38.x86_64
rocm-clinfo-5.5.0-1.fc38.x86_64
rocm-comgr-16.1-2.fc38.x86_64
mesa-vulkan-drivers-23.0.3-3.fc38.i686
mesa-va-drivers-23.0.3-3.fc38.i686
mesa-libglapi-23.0.3-3.fc38.i686
mesa-dri-drivers-23.0.3-3.fc38.i686
mesa-libGL-23.0.3-3.fc38.i686
mesa-libgbm-23.0.3-3.fc38.i686
mesa-libEGL-23.0.3-3.fc38.i686
mesa-libOSMesa-23.0.3-3.fc38.i686
xorg-x11-drv-amdgpu-23.0.0-1.fc38.x86_64
rocm-comgr-devel-16.1-2.fc38.x86_64
rocm-device-libs-16.1-1.fc38.x86_64
rocm-smi-4.0.0-8.fc38.noarch
hsakmt-1.0.6-29.rocm5.5.0.fc38.x86_64
rocm-runtime-5.5.0-1.fc38.x86_64
rocminfo-5.5.0-1.fc38.x86_64
hsakmt-devel-1.0.6-29.rocm5.5.0.fc38.x86_64
rocm-runtime-devel-5.5.0-1.fc38.x86_64

Obviously only my CPU is detected:
Code:
$ hashcat -I
hashcat (v6.2.6-532-g939a97d14) starting in backend information mode
OpenCL Info:
============
OpenCL Platform ID #1
  Vendor..: The pocl project
  Name....: Portable Computing Language
  Version.: OpenCL 3.0 PoCL 3.1  Linux, Release, RELOC, SPIR, LLVM 16.0.0, SLEEF, FP16, DISTRO, POCL_DEBUG
  Backend Device ID #1
    Type...........: CPU
    Vendor.ID......: 1
    Vendor.........: AuthenticAMD
    Name...........: pthread-AMD Ryzen 9 7950X 16-Core Processor
    Version........: OpenCL 3.0 PoCL HSTR: pthread-x86_64-redhat-linux-gnu-znver4
    Processor(s)...: 32
    Clock..........: 5879
    Memory.Total...: 61372 MB (limited to 8192 MB allocatable in one block)
    Memory.Free....: 30654 MB
    Local.Memory...: 1024 KB
    OpenCL.Version.: OpenCL C 1.2 PoCL
    Driver.Version.: 3.1

When installing this rocm-opencl package:
Code:
$ dnf info rocm-opencl
Repository teams is listed more than once in the configuration


Installed Packages
Name        : rocm-opencl
Version      : 5.5.0
Release      : 1.fc38
Architecture : x86_64
Size        : 1.7 M
Source      : rocm-opencl-5.5.0-1.fc38.src.rpm
Repository  : @System
From repo    : updates
Summary      : ROCm OpenCL Runtime
URL          : https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime
License      : MIT
Description  : ROCm OpenCL language runtime.
            : Supports offline and in-process/in-memory compilation.

I get the following liste of devices (take note of "CL_OUT_OF_HOST_MEMORY"):
Code:
$ hashcat -I
hashcat (v6.2.6-532-g939a97d14) starting in backend information mode
clCreateCommandQueue(): CL_OUT_OF_HOST_MEMORY
clCreateCommandQueue(): CL_OUT_OF_HOST_MEMORY
OpenCL Info:
============
OpenCL Platform ID #1
  Vendor..: Advanced Micro Devices, Inc.
  Name....: AMD Accelerated Parallel Processing
  Version.: OpenCL 2.1 AMD-APP (3558.0)
  Backend Device ID #1
    Type...........: GPU
    Vendor.ID......: 1
    Vendor.........: Advanced Micro Devices, Inc.
    Name...........: AMD Radeon Graphics
    Version........: OpenCL 2.0
    Processor(s)...: 48
    Clock..........: 2482
    Memory.Total...: 24560 MB (limited to 20876 MB allocatable in one block)
    Memory.Free....: 0 MB
    Local.Memory...: 64 KB
    OpenCL.Version.: OpenCL C 2.0
    Driver.Version.: 3558.0 (HSA1.1,LC)
    PCI.Addr.BDF...: 03:00.0
  Backend Device ID #2
    Type...........: GPU
    Vendor.ID......: 1
    Vendor.........: Advanced Micro Devices, Inc.
    Name...........: AMD Radeon Graphics
    Version........: OpenCL 2.0
    Processor(s)...: 1
    Clock..........: 2200
    Memory.Total...: 512 MB (limited to 435 MB allocatable in one block)
    Memory.Free....: 0 MB
    Local.Memory...: 64 KB
    OpenCL.Version.: OpenCL C 2.0
    Driver.Version.: 3558.0 (HSA1.1,LC)
    PCI.Addr.BDF...: 12:00.0
OpenCL Platform ID #2
  Vendor..: The pocl project
  Name....: Portable Computing Language
  Version.: OpenCL 3.0 PoCL 3.1  Linux, Release, RELOC, SPIR, LLVM 16.0.0, SLEEF, FP16, DISTRO, POCL_DEBUG
  Backend Device ID #3
    Type...........: CPU
    Vendor.ID......: 1
    Vendor.........: AuthenticAMD
    Name...........: pthread-AMD Ryzen 9 7950X 16-Core Processor
    Version........: OpenCL 3.0 PoCL HSTR: pthread-x86_64-redhat-linux-gnu-znver4
    Processor(s)...: 32
    Clock..........: 5879
    Memory.Total...: 61372 MB (limited to 8192 MB allocatable in one block)
    Memory.Free....: 30654 MB
    Local.Memory...: 1024 KB
    OpenCL.Version.: OpenCL C 1.2 PoCL
    Driver.Version.: 3.1

When testing with "hashcat -b -m 0" it's failing with "clCreateCommandQueue(): CL_OUT_OF_HOST_MEMORY" and "No devices found/left."

If I then remove "rocm-opencl" and install mesa-libOpenCL instead, and run benchmark with --force I get this:
Code:
$ hashcat -b -m 0 --force
hashcat (v6.2.6-532-g939a97d14) starting in benchmark mode
Benchmarking uses hand-optimized kernel code by default.
You can use it in your cracking session by setting the -O option.
Note: Using optimized kernel code limits the maximum supported password length.
To disable the optimized kernel code in benchmark mode, use the -w option.
You have enabled --force to bypass dangerous warnings and errors!
This can hide serious problems and should only be done when debugging.
Do not report hashcat issues encountered when using --force.
OpenCL API (OpenCL 1.1 Mesa 23.0.3) - Platform #1 [Mesa]
========================================================
* Device #1: AMD Radeon Graphics (gfx1100, LLVM 16.0.1, DRM 3.49, 6.2.15-300.fc38.x86_64), 24448/24576 MB (6144 MB allocatable), 96MCU
* Device #2: AMD Radeon Graphics (gfx1036, LLVM 16.0.1, DRM 3.49, 6.2.15-300.fc38.x86_64), 31616/31710 MB (7927 MB allocatable), 2MCU
OpenCL API (OpenCL 3.0 PoCL 3.1  Linux, Release, RELOC, SPIR, LLVM 16.0.0, SLEEF, FP16, DISTRO, POCL_DEBUG) - Platform #2 [The pocl project]
============================================================================================================================================
* Device #3: pthread-AMD Ryzen 9 7950X 16-Core Processor, skipped
Benchmark relevant options:
===========================
* --force
* --backend-devices-virtual=1
* --optimized-kernel-enable
-------------------
* Hash-Mode 0 (MD5)
-------------------
clCompileProgram(): CL_COMPILE_PROGRAM_FAILURE
fatal error: cannot open file '/usr/lib64/clc/gfx1100-amdgcn-mesa-mesa3d.bc': No such file or directory
* Device #1: Kernel /usr/local/share/hashcat/OpenCL/shared.cl build failed.

Any ideas on what to try out next?
Reply
#2
I've stumbled across this post on github; https://github.com/RadeonOpenCompute/ROC...1544500199

Seems like RX 7900 XT (RDNA3) cards is not yet supported by rocm.

Has anyone been able to use a RDNA3 card with hashcat on linux?
Reply
#3
Smile 
This now works. I'm not sure what got fixed, so I'll post what updated packages I have.

Code:
$ rpm -qa | grep -iE '(amd|rocm|mesa|opencl)'
opencl-filesystem-1.0-17.fc38.noarch
rocm-comgr-16.1-2.fc38.x86_64
xorg-x11-drv-amdgpu-23.0.0-1.fc38.x86_64
rocm-comgr-devel-16.1-2.fc38.x86_64
hsakmt-1.0.6-29.rocm5.5.0.fc38.x86_64
rocminfo-5.5.0-1.fc38.x86_64
hsakmt-devel-1.0.6-29.rocm5.5.0.fc38.x86_64
mesa-libGLw-8.0.0-25.fc38.x86_64
mesa-demos-8.5.0-1.fc38.x86_64
mesa-libGLw-devel-8.0.0-25.fc38.x86_64
opencl-headers-3.0-15.20230509gite049b16.fc38.noarch
rocm-device-libs-16.2-1.fc38.x86_64
rocm-clinfo-5.5.1-8.fc38.x86_64
rocm-hip-5.5.1-8.fc38.x86_64
rocm-opencl-5.5.1-8.fc38.x86_64
rocm-runtime-5.5.0-2.fc38.x86_64
rocm-runtime-devel-5.5.0-2.fc38.x86_64
rocm-smi-5.5.1-3.fc38.x86_64
mesa-libGLU-9.0.3-1.fc38.x86_64
mesa-libGLU-devel-9.0.3-1.fc38.x86_64
amd-gpu-firmware-20230804-153.fc38.noarch
mesa-libxatracker-23.1.6-1.fc38.x86_64
mesa-libd3d-23.1.6-1.fc38.x86_64
mesa-libd3d-devel-23.1.6-1.fc38.x86_64
mesa-libxatracker-devel-23.1.6-1.fc38.x86_64
mesa-vulkan-drivers-23.1.6-1.fc38.x86_64
mesa-vulkan-drivers-23.1.6-1.fc38.i686
mesa-filesystem-23.1.6-1.fc38.x86_64
mesa-libgbm-23.1.6-1.fc38.x86_64
mesa-libEGL-23.1.6-1.fc38.x86_64
mesa-libglapi-23.1.6-1.fc38.x86_64
mesa-dri-drivers-23.1.6-1.fc38.x86_64
mesa-libOSMesa-23.1.6-1.fc38.x86_64
mesa-libGL-23.1.6-1.fc38.x86_64
mesa-libOpenCL-23.1.6-1.fc38.x86_64
mesa-filesystem-23.1.6-1.fc38.i686
mesa-libOpenCL-devel-23.1.6-1.fc38.x86_64
mesa-libGL-devel-23.1.6-1.fc38.x86_64
mesa-libOSMesa-devel-23.1.6-1.fc38.x86_64
mesa-libEGL-devel-23.1.6-1.fc38.x86_64
mesa-libgbm-devel-23.1.6-1.fc38.x86_64
mesa-omx-drivers-23.1.6-1.fc38.x86_64
mesa-vdpau-drivers-23.1.6-1.fc38.x86_64
mesa-va-drivers-freeworld-23.1.6-1.fc38.x86_64
mesa-libgbm-23.1.6-1.fc38.i686
mesa-libEGL-23.1.6-1.fc38.i686
mesa-libglapi-23.1.6-1.fc38.i686
mesa-dri-drivers-23.1.6-1.fc38.i686
mesa-libOSMesa-23.1.6-1.fc38.i686
mesa-libGL-23.1.6-1.fc38.i686

Code:
$ dnf info rocm-opencl
Repository teams is listed more than once in the configuration
keybase                                                                                                                                                                                                                                                                                      25 kB/s | 3.3 kB    00:00   
Installed Packages
Name        : rocm-opencl
Version      : 5.5.1
Release      : 8.fc38
Architecture : x86_64
Size        : 1.7 M
Source      : rocclr-5.5.1-8.fc38.src.rpm
Repository  : @System
From repo    : updates
Summary      : ROCm OpenCL platform and device tool
URL          : https://github.com/ROCm-Developer-Tools/clr
License      : MIT
Description  : ROCm OpenCL language runtime.
            : Supports offline and in-process/in-memory compilation.

I also got the latest hashcat and recompiled it against new libraries and packages;

Code:
$ hashcat -I
hashcat (v6.2.6-716-gdb4d8e23e) starting in backend information mode

* Device #2: Unstable OpenCL driver detected!

OpenCL Info:
============

OpenCL Platform ID #1
  Vendor..: Advanced Micro Devices, Inc.
  Name....: AMD Accelerated Parallel Processing
  Version.: OpenCL 2.1 AMD-APP (3558.0)

  Backend Device ID #1
    Type...........: GPU
    Vendor.ID......: 1
    Vendor.........: Advanced Micro Devices, Inc.
    Name...........: AMD Radeon Graphics
    Version........: OpenCL 2.0
    Processor(s)...: 48
    Clock..........: 2482
    Memory.Total...: 24560 MB (limited to 20876 MB allocatable in one block)
    Memory.Free....: 24448 MB
    Local.Memory...: 64 KB
    OpenCL.Version.: OpenCL C 2.0
    Driver.Version.: 3558.0 (HSA1.1,LC)
    PCI.Addr.BDF...: 03:00.0

OpenCL Platform ID #2
  Vendor..: Mesa
  Name....: Clover
  Version.: OpenCL 1.1 Mesa 23.1.6

  Backend Device ID #2
    Type...........: GPU
    Vendor.ID......: 1
    Vendor.........: AMD
    Name...........: AMD Radeon Graphics (gfx1100, LLVM 16.0.6, DRM 3.52, 6.4.11-200.fc38.x86_64)
    Version........: OpenCL 1.1 Mesa 23.1.6
    Processor(s)...: 96
    Clock..........: 2482
    Memory.Total...: 24576 MB (limited to 6144 MB allocatable in one block)
    Memory.Free....: 24448 MB
    Local.Memory...: 64 KB
    OpenCL.Version.: OpenCL C 1.1
    Driver.Version.: 23.1.6

OpenCL Platform ID #3
  Vendor..: The pocl project
  Name....: Portable Computing Language
  Version.: OpenCL 3.0 PoCL 4.0  Linux, Release, RELOC, SPIR, LLVM 16.0.5, SLEEF, DISTRO, POCL_DEBUG

  Backend Device ID #3
    Type...........: CPU
    Vendor.ID......: 1
    Vendor.........: AuthenticAMD
    Name...........: cpu-AMD Ryzen 9 7950X 16-Core Processor
    Version........: OpenCL 3.0 PoCL HSTR: cpu-x86_64-redhat-linux-gnu-znver4
    Processor(s)...: 32
    Clock..........: 5879
    Memory.Total...: 61961 MB (limited to 8192 MB allocatable in one block)
    Memory.Free....: 30948 MB
    Local.Memory...: 1024 KB
    OpenCL.Version.: OpenCL C 1.2 PoCL
    Driver.Version.: 4.0

OpenCL Platform ID #4
  Vendor..: Mesa/X.org
  Name....: rusticl
  Version.: OpenCL 3.0

68563.8 MH/s on md5 Smile

Code:
$ hashcat -b -m 0
hashcat (v6.2.6-716-gdb4d8e23e) starting in benchmark mode

Benchmarking uses hand-optimized kernel code by default.
You can use it in your cracking session by setting the -O option.
Note: Using optimized kernel code limits the maximum supported password length.
To disable the optimized kernel code in benchmark mode, use the -w option.

* Device #2: Unstable OpenCL driver detected!

This OpenCL driver may fail kernel compilation or produce false negatives.
You can use --force to override, but do not report related errors.

OpenCL API (OpenCL 2.1 AMD-APP (3558.0)) - Platform #1 [Advanced Micro Devices, Inc.]
=====================================================================================
* Device #1: AMD Radeon Graphics, 24448/24560 MB (20876 MB allocatable), 48MCU

OpenCL API (OpenCL 1.1 Mesa 23.1.6) - Platform #2 [Mesa]
========================================================
* Device #2: AMD Radeon Graphics (gfx1100, LLVM 16.0.6, DRM 3.52, 6.4.11-200.fc38.x86_64), skipped

OpenCL API (OpenCL 3.0 PoCL 4.0  Linux, Release, RELOC, SPIR, LLVM 16.0.5, SLEEF, DISTRO, POCL_DEBUG) - Platform #3 [The pocl project]
======================================================================================================================================
* Device #3: cpu-AMD Ryzen 9 7950X 16-Core Processor, skipped

Benchmark relevant options:
===========================
* --backend-devices-virtual=1
* --optimized-kernel-enable

-------------------
* Hash-Mode 0 (MD5)
-------------------

Speed.#1.........: 68563.8 MH/s (11.38ms) @ Accel:512 Loops:1024 Thr:32 Vec:1

Started: Thu Aug 24 12:10:55 2023
Stopped: Thu Aug 24 12:11:01 2023
Reply