Fedora 38 + AMD RX 7900 XT
#1
Hi,

I've got two devices;
GPU: AMD Radeon RX 7900 XT (Navi 31)
CPU+GPU: AMD Ryzen 9 7950X

On https://hashcat.net/hashcat/ it says "AMD GPUs on Linux require "AMDGPU" (21.50 or later) and "ROCm" (5.0 or later)". However, I'm unable to get it working with amdgpu 23.0.0 (xorg-x11-drv-amdgpu-23.0.0-1.fc38.x86_64) and rocm 5.5.0 (rocm-opencl-5.5.0-1.fc38.src.rpm) on Fedora 38.

If I go without the rocm-opencl package and with these packages installed:
Code:
$ rpm -qa | grep -iE '(amd|rocm|mesa|opencl)' mesa-filesystem-23.0.3-3.fc38.x86_64 mesa-filesystem-23.0.3-3.fc38.i686 amd-gpu-firmware-20230404-149.fc38.noarch opencl-headers-3.0-14.20230201git4c82e9c.fc38.noarch opencl-filesystem-1.0-17.fc38.noarch mesa-libglapi-23.0.3-3.fc38.x86_64 mesa-vulkan-drivers-23.0.3-3.fc38.x86_64 mesa-libOSMesa-23.0.3-3.fc38.x86_64 mesa-va-drivers-23.0.3-3.fc38.x86_64 mesa-dri-drivers-23.0.3-3.fc38.x86_64 mesa-libgbm-23.0.3-3.fc38.x86_64 mesa-libEGL-23.0.3-3.fc38.x86_64 mesa-libGL-23.0.3-3.fc38.x86_64 mesa-libGLU-9.0.1-8.fc38.x86_64 mesa-libxatracker-23.0.3-3.fc38.x86_64 mesa-libEGL-devel-23.0.3-3.fc38.x86_64 mesa-libGLU-devel-9.0.1-8.fc38.x86_64 mesa-libGL-devel-23.0.3-3.fc38.x86_64 rocm-clinfo-5.5.0-1.fc38.x86_64 rocm-comgr-16.1-2.fc38.x86_64 mesa-vulkan-drivers-23.0.3-3.fc38.i686 mesa-va-drivers-23.0.3-3.fc38.i686 mesa-libglapi-23.0.3-3.fc38.i686 mesa-dri-drivers-23.0.3-3.fc38.i686 mesa-libGL-23.0.3-3.fc38.i686 mesa-libgbm-23.0.3-3.fc38.i686 mesa-libEGL-23.0.3-3.fc38.i686 mesa-libOSMesa-23.0.3-3.fc38.i686 xorg-x11-drv-amdgpu-23.0.0-1.fc38.x86_64 rocm-comgr-devel-16.1-2.fc38.x86_64 rocm-device-libs-16.1-1.fc38.x86_64 rocm-smi-4.0.0-8.fc38.noarch hsakmt-1.0.6-29.rocm5.5.0.fc38.x86_64 rocm-runtime-5.5.0-1.fc38.x86_64 rocminfo-5.5.0-1.fc38.x86_64 hsakmt-devel-1.0.6-29.rocm5.5.0.fc38.x86_64 rocm-runtime-devel-5.5.0-1.fc38.x86_64

Obviously only my CPU is detected:
Code:
$ hashcat -I hashcat (v6.2.6-532-g939a97d14) starting in backend information mode OpenCL Info: ============ OpenCL Platform ID #1   Vendor..: The pocl project   Name....: Portable Computing Language   Version.: OpenCL 3.0 PoCL 3.1  Linux, Release, RELOC, SPIR, LLVM 16.0.0, SLEEF, FP16, DISTRO, POCL_DEBUG   Backend Device ID #1     Type...........: CPU     Vendor.ID......: 1     Vendor.........: AuthenticAMD     Name...........: pthread-AMD Ryzen 9 7950X 16-Core Processor     Version........: OpenCL 3.0 PoCL HSTR: pthread-x86_64-redhat-linux-gnu-znver4     Processor(s)...: 32     Clock..........: 5879     Memory.Total...: 61372 MB (limited to 8192 MB allocatable in one block)     Memory.Free....: 30654 MB     Local.Memory...: 1024 KB     OpenCL.Version.: OpenCL C 1.2 PoCL     Driver.Version.: 3.1

When installing this rocm-opencl package:
Code:
$ dnf info rocm-opencl Repository teams is listed more than once in the configuration Installed Packages Name        : rocm-opencl Version      : 5.5.0 Release      : 1.fc38 Architecture : x86_64 Size        : 1.7 M Source      : rocm-opencl-5.5.0-1.fc38.src.rpm Repository  : @System From repo    : updates Summary      : ROCm OpenCL Runtime URL          : https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime License      : MIT Description  : ROCm OpenCL language runtime.             : Supports offline and in-process/in-memory compilation.

I get the following liste of devices (take note of "CL_OUT_OF_HOST_MEMORY"):
Code:
$ hashcat -I hashcat (v6.2.6-532-g939a97d14) starting in backend information mode clCreateCommandQueue(): CL_OUT_OF_HOST_MEMORY clCreateCommandQueue(): CL_OUT_OF_HOST_MEMORY OpenCL Info: ============ OpenCL Platform ID #1   Vendor..: Advanced Micro Devices, Inc.   Name....: AMD Accelerated Parallel Processing   Version.: OpenCL 2.1 AMD-APP (3558.0)   Backend Device ID #1     Type...........: GPU     Vendor.ID......: 1     Vendor.........: Advanced Micro Devices, Inc.     Name...........: AMD Radeon Graphics     Version........: OpenCL 2.0     Processor(s)...: 48     Clock..........: 2482     Memory.Total...: 24560 MB (limited to 20876 MB allocatable in one block)     Memory.Free....: 0 MB     Local.Memory...: 64 KB     OpenCL.Version.: OpenCL C 2.0     Driver.Version.: 3558.0 (HSA1.1,LC)     PCI.Addr.BDF...: 03:00.0   Backend Device ID #2     Type...........: GPU     Vendor.ID......: 1     Vendor.........: Advanced Micro Devices, Inc.     Name...........: AMD Radeon Graphics     Version........: OpenCL 2.0     Processor(s)...: 1     Clock..........: 2200     Memory.Total...: 512 MB (limited to 435 MB allocatable in one block)     Memory.Free....: 0 MB     Local.Memory...: 64 KB     OpenCL.Version.: OpenCL C 2.0     Driver.Version.: 3558.0 (HSA1.1,LC)     PCI.Addr.BDF...: 12:00.0 OpenCL Platform ID #2   Vendor..: The pocl project   Name....: Portable Computing Language   Version.: OpenCL 3.0 PoCL 3.1  Linux, Release, RELOC, SPIR, LLVM 16.0.0, SLEEF, FP16, DISTRO, POCL_DEBUG   Backend Device ID #3     Type...........: CPU     Vendor.ID......: 1     Vendor.........: AuthenticAMD     Name...........: pthread-AMD Ryzen 9 7950X 16-Core Processor     Version........: OpenCL 3.0 PoCL HSTR: pthread-x86_64-redhat-linux-gnu-znver4     Processor(s)...: 32     Clock..........: 5879     Memory.Total...: 61372 MB (limited to 8192 MB allocatable in one block)     Memory.Free....: 30654 MB     Local.Memory...: 1024 KB     OpenCL.Version.: OpenCL C 1.2 PoCL     Driver.Version.: 3.1

When testing with "hashcat -b -m 0" it's failing with "clCreateCommandQueue(): CL_OUT_OF_HOST_MEMORY" and "No devices found/left."

If I then remove "rocm-opencl" and install mesa-libOpenCL instead, and run benchmark with --force I get this:
Code:
$ hashcat -b -m 0 --force hashcat (v6.2.6-532-g939a97d14) starting in benchmark mode Benchmarking uses hand-optimized kernel code by default. You can use it in your cracking session by setting the -O option. Note: Using optimized kernel code limits the maximum supported password length. To disable the optimized kernel code in benchmark mode, use the -w option. You have enabled --force to bypass dangerous warnings and errors! This can hide serious problems and should only be done when debugging. Do not report hashcat issues encountered when using --force. OpenCL API (OpenCL 1.1 Mesa 23.0.3) - Platform #1 [Mesa] ======================================================== * Device #1: AMD Radeon Graphics (gfx1100, LLVM 16.0.1, DRM 3.49, 6.2.15-300.fc38.x86_64), 24448/24576 MB (6144 MB allocatable), 96MCU * Device #2: AMD Radeon Graphics (gfx1036, LLVM 16.0.1, DRM 3.49, 6.2.15-300.fc38.x86_64), 31616/31710 MB (7927 MB allocatable), 2MCU OpenCL API (OpenCL 3.0 PoCL 3.1  Linux, Release, RELOC, SPIR, LLVM 16.0.0, SLEEF, FP16, DISTRO, POCL_DEBUG) - Platform #2 [The pocl project] ============================================================================================================================================ * Device #3: pthread-AMD Ryzen 9 7950X 16-Core Processor, skipped Benchmark relevant options: =========================== * --force * --backend-devices-virtual=1 * --optimized-kernel-enable ------------------- * Hash-Mode 0 (MD5) ------------------- clCompileProgram(): CL_COMPILE_PROGRAM_FAILURE fatal error: cannot open file '/usr/lib64/clc/gfx1100-amdgcn-mesa-mesa3d.bc': No such file or directory * Device #1: Kernel /usr/local/share/hashcat/OpenCL/shared.cl build failed.

Any ideas on what to try out next?
Reply
#2
I've stumbled across this post on github; https://github.com/RadeonOpenCompute/ROC...1544500199

Seems like RX 7900 XT (RDNA3) cards is not yet supported by rocm.

Has anyone been able to use a RDNA3 card with hashcat on linux?
Reply
#3
Smile 
This now works. I'm not sure what got fixed, so I'll post what updated packages I have.

Code:
$ rpm -qa | grep -iE '(amd|rocm|mesa|opencl)' opencl-filesystem-1.0-17.fc38.noarch rocm-comgr-16.1-2.fc38.x86_64 xorg-x11-drv-amdgpu-23.0.0-1.fc38.x86_64 rocm-comgr-devel-16.1-2.fc38.x86_64 hsakmt-1.0.6-29.rocm5.5.0.fc38.x86_64 rocminfo-5.5.0-1.fc38.x86_64 hsakmt-devel-1.0.6-29.rocm5.5.0.fc38.x86_64 mesa-libGLw-8.0.0-25.fc38.x86_64 mesa-demos-8.5.0-1.fc38.x86_64 mesa-libGLw-devel-8.0.0-25.fc38.x86_64 opencl-headers-3.0-15.20230509gite049b16.fc38.noarch rocm-device-libs-16.2-1.fc38.x86_64 rocm-clinfo-5.5.1-8.fc38.x86_64 rocm-hip-5.5.1-8.fc38.x86_64 rocm-opencl-5.5.1-8.fc38.x86_64 rocm-runtime-5.5.0-2.fc38.x86_64 rocm-runtime-devel-5.5.0-2.fc38.x86_64 rocm-smi-5.5.1-3.fc38.x86_64 mesa-libGLU-9.0.3-1.fc38.x86_64 mesa-libGLU-devel-9.0.3-1.fc38.x86_64 amd-gpu-firmware-20230804-153.fc38.noarch mesa-libxatracker-23.1.6-1.fc38.x86_64 mesa-libd3d-23.1.6-1.fc38.x86_64 mesa-libd3d-devel-23.1.6-1.fc38.x86_64 mesa-libxatracker-devel-23.1.6-1.fc38.x86_64 mesa-vulkan-drivers-23.1.6-1.fc38.x86_64 mesa-vulkan-drivers-23.1.6-1.fc38.i686 mesa-filesystem-23.1.6-1.fc38.x86_64 mesa-libgbm-23.1.6-1.fc38.x86_64 mesa-libEGL-23.1.6-1.fc38.x86_64 mesa-libglapi-23.1.6-1.fc38.x86_64 mesa-dri-drivers-23.1.6-1.fc38.x86_64 mesa-libOSMesa-23.1.6-1.fc38.x86_64 mesa-libGL-23.1.6-1.fc38.x86_64 mesa-libOpenCL-23.1.6-1.fc38.x86_64 mesa-filesystem-23.1.6-1.fc38.i686 mesa-libOpenCL-devel-23.1.6-1.fc38.x86_64 mesa-libGL-devel-23.1.6-1.fc38.x86_64 mesa-libOSMesa-devel-23.1.6-1.fc38.x86_64 mesa-libEGL-devel-23.1.6-1.fc38.x86_64 mesa-libgbm-devel-23.1.6-1.fc38.x86_64 mesa-omx-drivers-23.1.6-1.fc38.x86_64 mesa-vdpau-drivers-23.1.6-1.fc38.x86_64 mesa-va-drivers-freeworld-23.1.6-1.fc38.x86_64 mesa-libgbm-23.1.6-1.fc38.i686 mesa-libEGL-23.1.6-1.fc38.i686 mesa-libglapi-23.1.6-1.fc38.i686 mesa-dri-drivers-23.1.6-1.fc38.i686 mesa-libOSMesa-23.1.6-1.fc38.i686 mesa-libGL-23.1.6-1.fc38.i686

Code:
$ dnf info rocm-opencl Repository teams is listed more than once in the configuration keybase                                                                                                                                                                                                                                                                                      25 kB/s | 3.3 kB    00:00    Installed Packages Name        : rocm-opencl Version      : 5.5.1 Release      : 8.fc38 Architecture : x86_64 Size        : 1.7 M Source      : rocclr-5.5.1-8.fc38.src.rpm Repository  : @System From repo    : updates Summary      : ROCm OpenCL platform and device tool URL          : https://github.com/ROCm-Developer-Tools/clr License      : MIT Description  : ROCm OpenCL language runtime.             : Supports offline and in-process/in-memory compilation.

I also got the latest hashcat and recompiled it against new libraries and packages;

Code:
$ hashcat -I hashcat (v6.2.6-716-gdb4d8e23e) starting in backend information mode * Device #2: Unstable OpenCL driver detected! OpenCL Info: ============ OpenCL Platform ID #1   Vendor..: Advanced Micro Devices, Inc.   Name....: AMD Accelerated Parallel Processing   Version.: OpenCL 2.1 AMD-APP (3558.0)   Backend Device ID #1     Type...........: GPU     Vendor.ID......: 1     Vendor.........: Advanced Micro Devices, Inc.     Name...........: AMD Radeon Graphics     Version........: OpenCL 2.0     Processor(s)...: 48     Clock..........: 2482     Memory.Total...: 24560 MB (limited to 20876 MB allocatable in one block)     Memory.Free....: 24448 MB     Local.Memory...: 64 KB     OpenCL.Version.: OpenCL C 2.0     Driver.Version.: 3558.0 (HSA1.1,LC)     PCI.Addr.BDF...: 03:00.0 OpenCL Platform ID #2   Vendor..: Mesa   Name....: Clover   Version.: OpenCL 1.1 Mesa 23.1.6   Backend Device ID #2     Type...........: GPU     Vendor.ID......: 1     Vendor.........: AMD     Name...........: AMD Radeon Graphics (gfx1100, LLVM 16.0.6, DRM 3.52, 6.4.11-200.fc38.x86_64)     Version........: OpenCL 1.1 Mesa 23.1.6     Processor(s)...: 96     Clock..........: 2482     Memory.Total...: 24576 MB (limited to 6144 MB allocatable in one block)     Memory.Free....: 24448 MB     Local.Memory...: 64 KB     OpenCL.Version.: OpenCL C 1.1     Driver.Version.: 23.1.6 OpenCL Platform ID #3   Vendor..: The pocl project   Name....: Portable Computing Language   Version.: OpenCL 3.0 PoCL 4.0  Linux, Release, RELOC, SPIR, LLVM 16.0.5, SLEEF, DISTRO, POCL_DEBUG   Backend Device ID #3     Type...........: CPU     Vendor.ID......: 1     Vendor.........: AuthenticAMD     Name...........: cpu-AMD Ryzen 9 7950X 16-Core Processor     Version........: OpenCL 3.0 PoCL HSTR: cpu-x86_64-redhat-linux-gnu-znver4     Processor(s)...: 32     Clock..........: 5879     Memory.Total...: 61961 MB (limited to 8192 MB allocatable in one block)     Memory.Free....: 30948 MB     Local.Memory...: 1024 KB     OpenCL.Version.: OpenCL C 1.2 PoCL     Driver.Version.: 4.0 OpenCL Platform ID #4   Vendor..: Mesa/X.org   Name....: rusticl   Version.: OpenCL 3.0

68563.8 MH/s on md5 Smile

Code:
$ hashcat -b -m 0 hashcat (v6.2.6-716-gdb4d8e23e) starting in benchmark mode Benchmarking uses hand-optimized kernel code by default. You can use it in your cracking session by setting the -O option. Note: Using optimized kernel code limits the maximum supported password length. To disable the optimized kernel code in benchmark mode, use the -w option. * Device #2: Unstable OpenCL driver detected! This OpenCL driver may fail kernel compilation or produce false negatives. You can use --force to override, but do not report related errors. OpenCL API (OpenCL 2.1 AMD-APP (3558.0)) - Platform #1 [Advanced Micro Devices, Inc.] ===================================================================================== * Device #1: AMD Radeon Graphics, 24448/24560 MB (20876 MB allocatable), 48MCU OpenCL API (OpenCL 1.1 Mesa 23.1.6) - Platform #2 [Mesa] ======================================================== * Device #2: AMD Radeon Graphics (gfx1100, LLVM 16.0.6, DRM 3.52, 6.4.11-200.fc38.x86_64), skipped OpenCL API (OpenCL 3.0 PoCL 4.0  Linux, Release, RELOC, SPIR, LLVM 16.0.5, SLEEF, DISTRO, POCL_DEBUG) - Platform #3 [The pocl project] ====================================================================================================================================== * Device #3: cpu-AMD Ryzen 9 7950X 16-Core Processor, skipped Benchmark relevant options: =========================== * --backend-devices-virtual=1 * --optimized-kernel-enable ------------------- * Hash-Mode 0 (MD5) ------------------- Speed.#1.........: 68563.8 MH/s (11.38ms) @ Accel:512 Loops:1024 Thr:32 Vec:1 Started: Thu Aug 24 12:10:55 2023 Stopped: Thu Aug 24 12:11:01 2023
Reply