TITAN RTX - performance issues?

[jason_tx]

Hi All, 
I have (2) TITAN RTX installed but...
I am currently trying to figure out why the benchmark numbers are so high compared to the actual numbers when running the latest hashcat. 
Any and all ideas/solutions would be greatly appreciated. 
I have updated to the latest NVIDI drivers, see the "--opencl-info" below:
Below is the "--opencl-info", the Benchmark, and Actual stats.

OpenCL Info:

Code:

OpenCL Info:

Platform ID #1
  Vendor  : NVIDIA Corporation
  Name    : NVIDIA CUDA
  Version : OpenCL 1.2 CUDA 10.2.87

  Device ID #1
    Type          : GPU
    Vendor ID      : 32
    Vendor        : NVIDIA Corporation
    Name          : TITAN RTX
    Version        : OpenCL 1.2 CUDA
    Processor(s)  : 72
    Clock          : 1770
    Memory        : 6144/24576 MB allocatable
    OpenCL Version : OpenCL C 1.2
    Driver Version : 440.97

  Device ID #2
    Type          : GPU
    Vendor ID      : 32
    Vendor        : NVIDIA Corporation
    Name          : TITAN RTX
    Version        : OpenCL 1.2 CUDA
    Processor(s)  : 72
    Clock          : 1770
    Memory        : 6144/24576 MB allocatable
    OpenCL Version : OpenCL C 1.2
    Driver Version : 440.97

Benchmark Results:

Code:

hashcat (v5.1.0) starting in benchmark mode...

Benchmarking uses hand-optimized kernel code by default.
You can use it in your cracking session by setting the -O option.
Note: Using optimized kernel code limits the maximum supported password length.
To disable the optimized kernel code in benchmark mode, use the -w option.

* Device #1: WARNING! Kernel exec timeout is not disabled.
            This may cause "CL_OUT_OF_RESOURCES" or related errors.
            To disable the timeout, see: https://hashcat.net/q/timeoutpatch
* Device #2: WARNING! Kernel exec timeout is not disabled.
            This may cause "CL_OUT_OF_RESOURCES" or related errors.
            To disable the timeout, see: https://hashcat.net/q/timeoutpatch
OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: TITAN RTX, 6144/24576 MB allocatable, 72MCU
* Device #2: TITAN RTX, 6144/24576 MB allocatable, 72MCU

Benchmark relevant options:
===========================
* --optimized-kernel-enable

Hashmode: 0 - MD5

Speed.#1.........: 54021.6 MH/s (44.26ms) @ Accel:128 Loops:1024 Thr:256 Vec:1
Speed.#2.........: 54290.1 MH/s (44.02ms) @ Accel:128 Loops:1024 Thr:256 Vec:1
Speed.#*.........:  108.3 GH/s

Hashmode: 100 - SHA1

Speed.#1.........: 19480.4 MH/s (61.53ms) @ Accel:128 Loops:512 Thr:256 Vec:1
Speed.#2.........: 19571.1 MH/s (61.23ms) @ Accel:128 Loops:512 Thr:256 Vec:1
Speed.#*.........: 39051.5 MH/s

Hashmode: 1400 - SHA2-256

Speed.#1.........:  8075.2 MH/s (74.28ms) @ Accel:128 Loops:256 Thr:256 Vec:1
Speed.#2.........:  8128.5 MH/s (73.76ms) @ Accel:128 Loops:256 Thr:256 Vec:1
Speed.#*.........: 16203.7 MH/s

Hashmode: 1700 - SHA2-512

Speed.#1.........:  2599.7 MH/s (57.58ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Speed.#2.........:  2622.5 MH/s (57.08ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Speed.#*.........:  5222.2 MH/s

Hashmode: 2500 - WPA-EAPOL-PBKDF2 (Iterations: 4096)

Speed.#1.........:  899.7 kH/s (40.60ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Speed.#2.........:  901.9 kH/s (40.48ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Speed.#*.........:  1801.6 kH/s

Hashmode: 1000 - NTLM

Speed.#1.........: 94411.2 MH/s (25.12ms) @ Accel:128 Loops:1024 Thr:256 Vec:1
Speed.#2.........: 94588.6 MH/s (24.99ms) @ Accel:128 Loops:1024 Thr:256 Vec:1
Speed.#*.........:  189.0 GH/s

Hashmode: 3000 - LM

Speed.#1.........: 50350.9 MH/s (47.34ms) @ Accel:128 Loops:1024 Thr:256 Vec:1
Speed.#2.........: 50626.8 MH/s (47.09ms) @ Accel:128 Loops:1024 Thr:256 Vec:1
Speed.#*.........:  101.0 GH/s

Hashmode: 5500 - NetNTLMv1 / NetNTLMv1+ESS

Speed.#1.........: 54596.3 MH/s (43.72ms) @ Accel:128 Loops:1024 Thr:256 Vec:1
Speed.#2.........: 55000.5 MH/s (43.40ms) @ Accel:128 Loops:1024 Thr:256 Vec:1
Speed.#*.........:  109.6 GH/s

Hashmode: 5600 - NetNTLMv2

Speed.#1.........:  4048.2 MH/s (74.08ms) @ Accel:128 Loops:128 Thr:256 Vec:1
Speed.#2.........:  4056.3 MH/s (73.92ms) @ Accel:128 Loops:128 Thr:256 Vec:1
Speed.#*.........:  8104.4 MH/s

Hashmode: 1500 - descrypt, DES (Unix), Traditional DES

Speed.#1.........:  1968.0 MH/s (76.11ms) @ Accel:8 Loops:1024 Thr:256 Vec:1
Speed.#2.........:  1944.0 MH/s (77.07ms) @ Accel:8 Loops:1024 Thr:256 Vec:1
Speed.#*.........:  3912.0 MH/s

Hashmode: 500 - md5crypt, MD5 (Unix), Cisco-IOS $1$ (MD5) (Iterations: 1000)

Speed.#1.........: 24096.0 kH/s (43.57ms) @ Accel:1024 Loops:500 Thr:32 Vec:1
Speed.#2.........: 24571.3 kH/s (42.53ms) @ Accel:1024 Loops:500 Thr:32 Vec:1
Speed.#*.........: 48667.3 kH/s

Hashmode: 3200 - bcrypt $2*$, Blowfish (Unix) (Iterations: 32)

Speed.#1.........:    29266 H/s (38.28ms) @ Accel:16 Loops:4 Thr:8 Vec:1
Speed.#2.........:    29355 H/s (38.05ms) @ Accel:16 Loops:4 Thr:8 Vec:1
Speed.#*.........:    58621 H/s

Hashmode: 1800 - sha512crypt $6$, SHA512 (Unix) (Iterations: 5000)

Speed.#1.........:  359.5 kH/s (81.02ms) @ Accel:512 Loops:128 Thr:32 Vec:1
Speed.#2.........:  359.5 kH/s (81.04ms) @ Accel:512 Loops:128 Thr:32 Vec:1
Speed.#*.........:  719.0 kH/s

Hashmode: 7500 - Kerberos 5 AS-REQ Pre-Auth etype 23

Speed.#1.........:  717.5 MH/s (52.19ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#2.........:  721.6 MH/s (51.87ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#*.........:  1439.1 MH/s

Hashmode: 13100 - Kerberos 5 TGS-REP etype 23

Speed.#1.........:  712.5 MH/s (52.55ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#2.........:  714.5 MH/s (52.40ms) @ Accel:128 Loops:64 Thr:64 Vec:1
Speed.#*.........:  1427.0 MH/s

Hashmode: 15300 - DPAPI masterkey file v1 (Iterations: 23999)

Speed.#1.........:  157.0 kH/s (79.63ms) @ Accel:128 Loops:128 Thr:256 Vec:1
Speed.#2.........:  156.7 kH/s (79.75ms) @ Accel:128 Loops:128 Thr:256 Vec:1
Speed.#*.........:  313.7 kH/s

Hashmode: 15900 - DPAPI masterkey file v2 (Iterations: 7999)

Speed.#1.........:  132.7 kH/s (70.81ms) @ Accel:256 Loops:128 Thr:32 Vec:1
Speed.#2.........:  133.9 kH/s (70.13ms) @ Accel:256 Loops:128 Thr:32 Vec:1
Speed.#*.........:  266.6 kH/s

Hashmode: 7100 - macOS v10.8+ (PBKDF2-SHA512) (Iterations: 35000)

Speed.#1.........:    31727 H/s (67.37ms) @ Accel:128 Loops:32 Thr:256 Vec:1
Speed.#2.........:    31960 H/s (66.88ms) @ Accel:128 Loops:32 Thr:256 Vec:1
Speed.#*.........:    63687 H/s

Hashmode: 11600 - 7-Zip (Iterations: 524288)

Speed.#1.........:    21950 H/s (52.06ms) @ Accel:128 Loops:256 Thr:256 Vec:1
Speed.#2.........:    22147 H/s (51.58ms) @ Accel:128 Loops:256 Thr:256 Vec:1
Speed.#*.........:    44097 H/s

Hashmode: 12500 - RAR3-hp (Iterations: 262144)

Speed.#1.........:    68469 H/s (66.99ms) @ Accel:4 Loops:16384 Thr:256 Vec:1
Speed.#2.........:    67441 H/s (68.04ms) @ Accel:4 Loops:16384 Thr:256 Vec:1
Speed.#*.........:  135.9 kH/s

Hashmode: 13000 - RAR5 (Iterations: 32767)

Speed.#1.........:    96142 H/s (47.51ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Speed.#2.........:    97547 H/s (46.80ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Speed.#*.........:  193.7 kH/s

Hashmode: 6211 - TrueCrypt PBKDF2-HMAC-RIPEMD160 + XTS 512 bit (Iterations: 2000)

Speed.#1.........:  599.2 kH/s (51.71ms) @ Accel:128 Loops:32 Thr:256 Vec:1
Speed.#2.........:  606.8 kH/s (50.95ms) @ Accel:128 Loops:32 Thr:256 Vec:1
Speed.#*.........:  1206.1 kH/s

Hashmode: 13400 - KeePass 1 (AES/Twofish) and KeePass 2 (AES) (Iterations: 6000)

Speed.#1.........:  237.2 kH/s (105.74ms) @ Accel:512 Loops:128 Thr:32 Vec:1
Speed.#2.........:  232.3 kH/s (107.97ms) @ Accel:512 Loops:128 Thr:32 Vec:1
Speed.#*.........:  469.5 kH/s

Hashmode: 6800 - LastPass + LastPass sniffed (Iterations: 500)

Speed.#1.........:  6014.0 kH/s (91.37ms) @ Accel:128 Loops:125 Thr:256 Vec:1
Speed.#2.........:  5493.7 kH/s (99.66ms) @ Accel:128 Loops:125 Thr:256 Vec:1
Speed.#*.........: 11507.7 kH/s

Hashmode: 11300 - Bitcoin/Litecoin wallet.dat (Iterations: 199999)

Speed.#1.........:    11698 H/s (64.08ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Speed.#2.........:    11672 H/s (64.23ms) @ Accel:128 Loops:64 Thr:256 Vec:1
Speed.#*.........:    23370 H/s

Started: Thu Oct 24 12:48:55 2019
Stopped: Thu Oct 24 12:57:47 2019

Actual:

Code:

hashcat64.exe -a 0 --session=2019-10-24 -m 0 -w 3 --status --status-timer=60 -p : -O --hwmon-disable -o "C:\Users\user\Desktop\results.txt" --outfile-format=3 "C:\Users\user\Desktop\hashes.txt" "C:\Users\user\Desktop\wordlists\hashes.org-2018.txt" "C:\Users\user\Desktop\wordlists\hashes.org-2019.txt"
hashcat (v5.1.0) starting...

* Device #1: WARNING! Kernel exec timeout is not disabled.
            This may cause "CL_OUT_OF_RESOURCES" or related errors.
            To disable the timeout, see: https://hashcat.net/q/timeoutpatch
* Device #2: WARNING! Kernel exec timeout is not disabled.
            This may cause "CL_OUT_OF_RESOURCES" or related errors.
            To disable the timeout, see: https://hashcat.net/q/timeoutpatch
OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: TITAN RTX, 6144/24576 MB allocatable, 72MCU
* Device #2: TITAN RTX, 6144/24576 MB allocatable, 72MCU

Hashes: 8 digests; 8 unique digests, 1 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates
Rules: 1

Applicable optimizers:
* Optimized-Kernel
* Zero-Byte
* Precompute-Init
* Precompute-Merkle-Demgard
* Meet-In-The-Middle
* Early-Skip
* Not-Salted
* Not-Iterated
* Single-Salt
* Raw-Hash

Minimum password length supported by kernel: 0
Maximum password length supported by kernel: 31

Watchdog: Hardware monitoring interface not found on your system.
Watchdog: Temperature abort trigger disabled.

INFO: Removed 2 hashes found in potfile.

Dictionary cache hit:
* Filename..: C:\Users\user\Desktop\wordlists\hashes.org-2018.txt
* Passwords.: 613377895
* Bytes.....: 6429547050
* Keyspace..: 613377895

[s]tatus [p]ause [b]ypass [c]heckpoint [q]uit =>

Session..........: 2019-10-24
Status...........: Running
Hash.Type........: MD5
Hash.Target......: C:\Users\user\Desktop\hashes.txt
Time.Started.....: Thu Oct 24 13:07:19 2019 (3 secs)
Time.Estimated...: Thu Oct 24 13:07:51 2019 (29 secs)
Guess.Base.......: File (C:\Users\user\Desktop\wordlists\hashes.org-2018.txt)
Guess.Queue......: 1/2 (50.00%)
Speed.#1.........:  9548.5 kH/s (0.39ms) @ Accel:128 Loops:1 Thr:256 Vec:1
Speed.#2.........:  9476.0 kH/s (0.39ms) @ Accel:128 Loops:1 Thr:256 Vec:1
Speed.#*.........: 19024.5 kH/s
Recovered........: 2/8 (25.00%) Digests, 0/1 (0.00%) Salts
Progress.........: 47191472/613377895 (7.69%)
Rejected.........: 5552/47191472 (0.01%)
Restore.Point....: 47191160/613377895 (7.69%)
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1
Restore.Sub.#2...: Salt:0 Amplifier:0-1 Iteration:0-1
Candidates.#1....: 16_amor_yas -> 1849633311sinaweibo
Candidates.#2....: 157220798927881 -> 184959591255691

[s]tatus [p]ause [b]ypass [c]heckpoint [q]uit =>

Session..........: 2019-10-24
Status...........: Quit
Hash.Type........: MD5
Hash.Target......: C:\Users\user\Desktop\hashes.txt
Time.Started.....: Thu Oct 24 13:07:19 2019 (6 secs)
Time.Estimated...: Thu Oct 24 13:07:53 2019 (28 secs)
Guess.Base.......: File (C:\Users\user\Desktop\wordlists\hashes.org-2018.txt)
Guess.Queue......: 1/2 (50.00%)
Speed.#1.........:  8991.3 kH/s (0.39ms) @ Accel:128 Loops:1 Thr:256 Vec:1
Speed.#2.........:  8997.2 kH/s (0.39ms) @ Accel:128 Loops:1 Thr:256 Vec:1
Speed.#*.........: 17988.5 kH/s
Recovered........: 2/8 (25.00%) Digests, 0/1 (0.00%) Salts
Progress.........: 103818998/613377895 (16.93%)
Rejected.........: 9974/103818998 (0.01%)
Restore.Point....: 103818948/613377895 (16.93%)
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1
Restore.Sub.#2...: Salt:0 Amplifier:0-1 Iteration:0-1
Candidates.#1....: 57600950 -> 5elmvale
Candidates.#2....: 5488934558 -> 5elmerhick
Started: Thu Oct 24 13:07:09 2019
Stopped: Thu Oct 24 13:07:28 2019

[undeath]

https://hashcat.net/wiki/doku.php?id=fre...full_speed

[Chick3nman]

I run hashcat on Titan RTX pretty much daily, this seems like reasonable enough speed given its just a straight wordlist attack and its dumped into the hashcat GUI(which we don't support officially). I would listen to the warnings and apply the timeout patch. Then I would stop using the GUI so you can control the flags that are being thrown better, no reason at all to disable the hardware monitoring system or set a status timer or -p unless are things you specifically need. From there, drop the -w 3 as setting that manually sometimes gives funny tuning values. A simple command is going to be easier for you to read and spot errors, as well less likely to cause issues with mixing of flags and different configurations when they arent needed.

[jason_tx]

https://hashcat.net/wiki/doku.php?id=fre...full_speed

Thanks for the link. I've obviously only dipped my toe in the hashcat possibility pool, so I appreciate the guidance.

After reading I tried a new command to see if I could generate enough 'work', yet I'm curious if anyone here could suggest better ones?! Any command suggestions for a decent number of bcrypt hashes?  
Below is the command I used, which did get me running up to around 20,000 MH/s total (2 cards combined). To be clear I'm satisfied with the speed now (knowing there isn't any actual issues), just don't know if there are better methods to generate more 'work'.  

Code:

pp64.exe "\hashcat-5.1.0\WORDLISTS\rockyou.txt\rockyou.txt" | hashcat64.exe -a 0 --session=2019-10-24 -m 0 -w 3 --status --status-timer=60 -p : -O --hwmon-disable -o "C:\Users\user\Desktop\results.txt" --outfile-format=3 -r "\hashcat-5.1.0\rules\OneRuleToRuleThemAll.rule" "C:\Users\user\Desktop\MD5_hashes.txt"

Status Result snapshot:

Code:

Session..........: 2019-10-24
Status...........: Running
Hash.Type........: MD5
Hash.Target......: C:\Users\user\Desktop\MD5_hashes.txt
Time.Started.....: Thu Oct 24 16:53:30 2019 (8 hours, 31 mins)
Time.Estimated...: Fri Oct 25 01:25:27 2019 (0 secs)
Guess.Base.......: Pipe
Guess.Mod........: Rules (\hashcat-5.1.0\rules\OneRuleToRuleThemAll.rule)
Speed.#1.........:  9579.7 MH/s (34.33ms) @ Accel:128 Loops:256 Thr:256 Vec:1
Speed.#2.........: 10059.8 MH/s (33.63ms) @ Accel:128 Loops:256 Thr:256 Vec:1
Speed.#*.........: 19642.6 MH/s
Recovered........: 2/8 (25.00%) Digests, 0/1 (0.00%) Salts
Progress.........: 597677088964608
Rejected.........: 0
Restore.Point....: 0
Restore.Sub.#1...: Salt:0 Amplifier:11008-11264 Iteration:0-256
Restore.Sub.#2...: Salt:0 Amplifier:49920-50176 Iteration:0-256
Candidates.#1....: Qbejtvc -> e15flf
Candidates.#2....: mkogulomo01 -> fbejoac077

Also I'm concerned that I'm doing something wrong or skipping some easier password possibilities considering I let this run for 16 hours and I did not resolve any new hashes ('2' were from potfile). Would it be better to run a bruteforce (..and does that generate enough 'work' to get the GPUs running)?

sorry for the barrage for questions, 
Thanks in advance for any insight. 

[jason_tx]

I run hashcat on Titan RTX pretty much daily, this seems like reasonable enough speed given its just a straight wordlist attack and its dumped into the hashcat GUI(which we don't support officially). I would listen to the warnings and apply the timeout patch. Then I would stop using the GUI so you can control the flags that are being thrown better, no reason at all to disable the hardware monitoring system or set a status timer or -p unless are things you specifically need. From there, drop the -w 3 as setting that manually sometimes gives funny tuning values. A simple command is going to be easier for you to read and spot errors, as well less likely to cause issues with mixing of flags and different configurations when they arent needed.

Noted! No more hashcat GUI, thanks for the advice.

[royce]

It's OK to use hashcat GUI if it works for what you're trying to do. But many of the things you'll eventually want to do are better done with the command line directly.