+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Support (https://hashcat.net/forum/forum-3.html)
+--- Forum: hashcat (https://hashcat.net/forum/forum-45.html)
+--- Thread: NVAPI_ERROR (/thread-10654.html)
NVAPI_ERROR - vitiura - 03-11-2022
Hello everyone!
I try to start hashcat (wordlist brutforce) on my rig (win server 2019 with 10 of RTX 3080ti) and it crashes after about 10-20 minutes with unknown error.
I tried:
1. Upgrade/Downgrade nvidia drivers;
2. Downgrade hashcat version (all down to 6.2.0);
2. Expand swap file up to 120 GB;
3. Use different mods (-w option) doesn't impact;
4. Use different wordlists (100 and 400 GB);
5. Use different hashtypes;
Any other conditions:
- Windows server 2019
- RAM 64GB
- Intel core i7 7700k
- NVIDIA EVGA RTX 3080ti (10 devices)
- ETH miner works stable
- Any of 7 cards works stable. The problem start with 8th device
- Temp max 60 C
- Don't use any boost. Just default settings.
Thank you for any help!
hashcat output below:
[s]tatus [p]ause [b]ypass [c]heckpoint [f]inish [q]uit => NvAPI_GPU_GetPerfPoliciesStatus(): NVAPI_ERROR
nuLaunchKernel(): unknown error UcuLaunchKernel(): unknown errorcuMemcpyHtoDAsync(): unknown error known Error
cvmlDeviceGetUtilizationRates(): UnvmlDeviceGetUtilizationRates(): Unknown Error
nvmlDeviceGetUtilizationRates(): Unknown Error
nvmlDeviceGetUtilizationRates(): Unknown ErrornvmlDeviceGetUtilizationRates(): U
nvmlDeviceGetUtilizationRates(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
cuMemcpyDtoHAsync(): unknown error
cuMemcpyDtoHAsync(): unknown error
nvmlDeviceGetFanSpeed(): Unknown Error
Session..........: hashcat
Status...........: Error
Hash.Mode........: 20 (md5($salt.$pass))
Hash.Target......: C:\hashcatgui\hashcat\hashes\test.txt
Time.Started.....: Wed Mar 09 11:46:21 2022 (13 mins, 15 secs)
Time.Estimated...: Wed Mar 09 12:24:39 2022 (25 mins, 3 secs)
Kernel.Feature...: Optimized Kernel
Guess.Base.......: File (D:\files\list.txt)
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........: 5354.7 kH/s (0.36ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#2.........: 5396.1 kH/s (0.32ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#3.........: 5472.6 kH/s (0.29ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#4.........: 5436.7 kH/s (0.38ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#5.........: 5002.4 kH/s (0.60ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#6.........: 5283.0 kH/s (0.66ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#7.........: 5344.5 kH/s (0.56ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#8.........: 5120.5 kH/s (0.38ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#9.........: 5220.3 kH/s (0.48ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#10.........: 4995.7 kH/s (0.57ms) @ Accel:1024 Loops:1 Thr:32 Vec:1
Speed.#*.........: 52626.5 kH/s
Recovered........: 0/3 (0.00%) Digests, 0/3 (0.00%) Salts
Progress.........: 41625845760/120741743394 (34.48%)
Rejected.........: 0/41625845760 (0.00%)
Restore.Point....: 13738967040/40247247798 (34.14%)
Restore.Sub.#1...: Salt:2 Amplifier:0-1 Iteration:0-1
Restore.Sub.#2...: Salt:2 Amplifier:0-1 Iteration:0-1
Restore.Sub.#3...: Salt:2 Amplifier:0-1 Iteration:0-1
Restore.Sub.#4...: Salt:2 Amplifier:0-1 Iteration:0-1
Restore.Sub.#5...: Salt:2 Amplifier:0-1 Iteration:0-1
Restore.Sub.#6...: Salt:2 Amplifier:0-1 Iteration:0-1
Restore.Sub.#7...: Salt:2 Amplifier:0-1 Iteration:0-1
Restore.Sub.#8...: Salt:2 Amplifier:0-1 Iteration:0-1
Restore.Sub.#9...: Salt:2 Amplifier:0-1 Iteration:0-1
Restore.Sub.#10...: Salt:2 Amplifier:0-1 Iteration:0-1
Candidate.Engine.: Device Generator
Candidates.#1....: ->
Candidates.#2....: ->
Candidates.#3....: ->
Candidates.#4....: ->
Candidates.#5....: ->
Candidates.#6....: ->
Candidates.#7....: ->
Candidates.#8....: ->
Candidates.#9....: ->
Candidates.#10....: ->
Hardware.Mon.#1..: Temp: 52c Core:1950MHz Mem:9242MHz Bus:1
Hardware.Mon.#2..: Temp: 51c Core:1859MHz Mem:9242MHz Bus:1
Hardware.Mon.#3..: Temp: 51c Core:1948MHz Mem:9242MHz Bus:1
Hardware.Mon.#4..: Temp: 54c Core:1979MHz Mem:9242MHz Bus:1
Hardware.Mon.#5..: Temp: 51c Core:1933MHz Mem:9242MHz Bus:1
Hardware.Mon.#6..: Temp: 46c Core:1933MHz Mem:9242MHz Bus:1
Hardware.Mon.#7..: Temp: 50c Core:1964MHz Mem:9242MHz Bus:1
Hardware.Mon.#8..: Temp: 51c Core:1933MHz Mem:9242MHz Bus:1
Hardware.Mon.#9..: Temp: 49c Core:1799MHz Mem:9492MHz Bus:1
Hardware.Mon.#10..: Temp: 54c Core:1999MHz Mem:9242MHz Bus:1
cuMemFree(): unknown error
cuMemFree(): unknown error
cuMemFree(): unknown error
cuMemFree(): unknown error
cuMemFree(): unknown error
cuMemFree(): unknown error
cuMemFree(): unknown error
RE: NVAPI_ERROR - slyexe - 03-13-2022
I would suspect it has an issue with the amount of memory being allocated based on the error appearing "cuMemFree(): unknown error".
The typical setup for such a large system is to have VRAM = RAM so in this scenario you would need a total of 120GB of RAM. I don't suspect a swap file would work as intended in this situation so either increase the physical RAM or disable cards.
RE: NVAPI_ERROR - vitiura - 03-14-2022
(03-13-2022, 04:37 AM)slyexe Wrote: I would suspect it has an issue with the amount of memory being allocated based on the error appearing "cuMemFree(): unknown error".
The typical setup for such a large system is to have VRAM = RAM so in this scenario you would need a total of 120GB of RAM. I don't suspect a swap file would work as intended in this situation so either increase the physical RAM or disable cards.
Thank you for support!
I was also inclined to this opinion. But when Hashcat works, according to the task manager, the RAM load of the entire system is no more than 30%. Do you think that this is not a true indicator?
RE: NVAPI_ERROR - Snoopy - 03-14-2022
can you post the output of hashcat -I
EDIT: was missing this part - Any of 7 cards works stable. The problem start with 8th device
-> did you used -D 2 -d 1,2,3,4,5,6,7 to use only 7 cards? if not see comment below
yeah like slyexe was mentioning, seems to be a problem with your system ram, you could use options -D 2 -d 1,2,3,4 ... to tell hashcat just to use the first x cards and see if this failure disappears
never the less, plz show output of hashcat -I
RE: NVAPI_ERROR - vitiura - 03-14-2022
(03-14-2022, 07:39 PM)Snoopy Wrote: EDIT: was missing this part - Any of 7 cards works stable. The problem start with 8th device
-> did you used -D 2 -d 1,2,3,4,5,6,7 to use only 7 cards?
Yes, I tried to explain that any combination with 7 CUDA devices works good and the problem is not with one of my devices.
-D 2 -d 1,2,3,4,5,6,7 and -D 2 -d 4,5,6,7,8,9,10 lead to the same result (it works). But even if I add another one CUDA device it will crash in 10-15 minutes of work.
Hashcat -I output:
CUDA Info:
==========
CUDA.Version.: 11.5
Backend Device ID #1 (Alias: #11)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:01:00.0
Backend Device ID #2 (Alias: #12)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:02:00.0
Backend Device ID #3 (Alias: #13)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:03:00.0
Backend Device ID #4 (Alias: #14)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:04:00.0
Backend Device ID #5 (Alias: #15)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:08:00.0
Backend Device ID #6 (Alias: #16)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:09:00.0
Backend Device ID #7 (Alias: #17)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:0b:00.0
Backend Device ID #8 (Alias: #18)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:0c:00.0
Backend Device ID #9 (Alias: #19)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:0d:00.0
Backend Device ID #10 (Alias: #20)
Name...........: NVIDIA GeForce RTX 3080 Ti
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB
Memory.Free....: 10054 MB
Local.Memory...: 99 KB
PCI.Addr.BDFe..: 0000:0e:00.0
OpenCL Info:
============
OpenCL Platform ID #1
Vendor..: NVIDIA Corporation
Name....: NVIDIA CUDA
Version.: OpenCL 3.0 CUDA 11.5.56
Backend Device ID #11 (Alias: #1)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 01:00.0
Backend Device ID #12 (Alias: #2)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 02:00.0
Backend Device ID #13 (Alias: #3)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 03:00.0
Backend Device ID #14 (Alias: #4)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 04:00.0
Backend Device ID #15 (Alias: #5)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 08:00.0
Backend Device ID #16 (Alias: #6)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 09:00.0
Backend Device ID #17 (Alias: #7)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 0b:00.0
Backend Device ID #18 (Alias: #8)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 0c:00.0
Backend Device ID #19 (Alias: #9)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 0d:00.0
Backend Device ID #20 (Alias: #10)
Type...........: GPU
Vendor.ID......: 32
Vendor.........: NVIDIA Corporation
Name...........: NVIDIA GeForce RTX 3080 Ti
Version........: OpenCL 3.0 CUDA
Processor(s)...: 80
Clock..........: 1800
Memory.Total...: 12287 MB (limited to 3071 MB allocatable in one block)
Memory.Free....: 10624 MB
Local.Memory...: 48 KB
OpenCL.Version.: OpenCL C 1.2
Driver.Version.: 496.13
PCI.Addr.BDF...: 0e:00.0
OpenCL Platform ID #2
Vendor..: Intel(R) Corporation
Name....: Intel(R) OpenCL
Version.: OpenCL 2.1
Backend Device ID #21
Type...........: GPU
Vendor.ID......: 8
Vendor.........: Intel(R) Corporation
Name...........: Intel(R) HD Graphics 630
Version........: OpenCL 2.1
Processor(s)...: 24
Clock..........: 1150
Memory.Total...: 26082 MB (limited to 1023 MB allocatable in one block)
Memory.Free....: 12992 MB
Local.Memory...: 64 KB
OpenCL.Version.: OpenCL C 2.0
Driver.Version.: 21.20.16.4534
Backend Device ID #22
Type...........: CPU
Vendor.ID......: 8
Vendor.........: Intel(R) Corporation
Name...........: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
Version........: OpenCL 2.1 (Build 359)
Processor(s)...: 8
Clock..........: 4200
Memory.Total...: 65227 MB (limited to 8153 MB allocatable in one block)
Memory.Free....: 32581 MB
Local.Memory...: 32 KB
OpenCL.Version.: OpenCL C 2.0
Driver.Version.: 6.6.0.359
RE: NVAPI_ERROR - Snoopy - 03-15-2022
well like slyexe mentioned, you dont have enough ram to use all cards, so upgrade your ram or stick to 7 cards (i would assume that even 7 cards will/could throw an error, depending on the attack
RE: NVAPI_ERROR - vitiura - 03-16-2022
(03-15-2022, 07:26 PM)Snoopy Wrote: well like slyexe mentioned, you dont have enough ram to use all cards, so upgrade your ram or stick to 7 cards (i would assume that even 7 cards will/could throw an error, depending on the attack
Thank you for your answer.
I almost completely agree with you, but I have some doubts. On another 4x1080ti rig (11 GB VRAM each) Hashcat works great with any type of attack in nightmare mode. At the same time, the motherboard has only 8 GB of RAM. VRAM/RAM ratio is 4!!!
UPD: I've solved problem! The error gone when I removed the -O flag from arguments (Enable optimized kernels). Honestly, I do not know if there will be any limits with this, but it works! Hope this post can help somebody.
RE: NVAPI_ERROR - slyexe - 03-17-2022
Glad you were able to solve the issue. I was going to reply back but was out of town for work. I was going to suggest trying to run 2 attacks simultaneously dividing the cards into 2 separate instances. But if you have resolved the problem I would say this would be irrelevant at this point in time.
RE: NVAPI_ERROR - MidnightHunter - 03-25-2022
(03-16-2022, 11:06 AM)vitiura Wrote:(03-15-2022, 07:26 PM)Snoopy Wrote: well like slyexe mentioned, you dont have enough ram to use all cards, so upgrade your ram or stick to 7 cards (i would assume that even 7 cards will/could throw an error, depending on the attack
Thank you for your answer.
I almost completely agree with you, but I have some doubts. On another 4x1080ti rig (11 GB VRAM each) Hashcat works great with any type of attack in nightmare mode. At the same time, the motherboard has only 8 GB of RAM. VRAM/RAM ratio is 4!!!
UPD: I've solved problem! The error gone when I removed the -O flag from arguments (Enable optimized kernels). Honestly, I do not know if there will be any limits with this, but it works! Hope this post can help somebody.
Hello my friend!
I have the same problem with my set (8 RTX3090 The-Distribution-Which-Does-Not-Handle-OpenCL-Well (Kali)) Tried to solve the problem in the same ways, but have no result
If I understand you correctly, you solve it, by deleting argument -O? But it turns off work with optimized Kernel, and speed drops by three times..
RE: NVAPI_ERROR - vitiura - 04-01-2022
[/quote]
If I understand you correctly, you solve it, by deleting argument -O? But it turns off work with optimized Kernel, and speed drops by three times..
[/quote]
Hey! I think not always by three times, but I agree that it's not the most graceful solution, but it works. If you have any easy ways to fix problem, I'll be glad to watch it
