TITAN RTX - performance issues?

[jason_tx]

Hi All, 
I have (2) TITAN RTX installed but...
I am currently trying to figure out why the benchmark numbers are so high compared to the actual numbers when running the latest hashcat. 
Any and all ideas/solutions would be greatly appreciated. 
I have updated to the latest NVIDI drivers, see the "--opencl-info" below:
Below is the "--opencl-info", the Benchmark, and Actual stats.

OpenCL Info:

Code:

OpenCL Info: Platform ID #1   Vendor  : NVIDIA Corporation   Name    : NVIDIA CUDA   Version : OpenCL 1.2 CUDA 10.2.87   Device ID #1     Type          : GPU     Vendor ID      : 32     Vendor        : NVIDIA Corporation     Name          : TITAN RTX     Version        : OpenCL 1.2 CUDA     Processor(s)  : 72     Clock          : 1770     Memory        : 6144/24576 MB allocatable     OpenCL Version : OpenCL C 1.2     Driver Version : 440.97   Device ID #2     Type          : GPU     Vendor ID      : 32     Vendor        : NVIDIA Corporation     Name          : TITAN RTX     Version        : OpenCL 1.2 CUDA     Processor(s)  : 72     Clock          : 1770     Memory        : 6144/24576 MB allocatable     OpenCL Version : OpenCL C 1.2     Driver Version : 440.97

Benchmark Results:

Code:

hashcat (v5.1.0) starting in benchmark mode... Benchmarking uses hand-optimized kernel code by default. You can use it in your cracking session by setting the -O option. Note: Using optimized kernel code limits the maximum supported password length. To disable the optimized kernel code in benchmark mode, use the -w option. * Device #1: WARNING! Kernel exec timeout is not disabled.             This may cause "CL_OUT_OF_RESOURCES" or related errors.             To disable the timeout, see: https://hashcat.net/q/timeoutpatch * Device #2: WARNING! Kernel exec timeout is not disabled.             This may cause "CL_OUT_OF_RESOURCES" or related errors.             To disable the timeout, see: https://hashcat.net/q/timeoutpatch OpenCL Platform #1: NVIDIA Corporation ====================================== * Device #1: TITAN RTX, 6144/24576 MB allocatable, 72MCU * Device #2: TITAN RTX, 6144/24576 MB allocatable, 72MCU Benchmark relevant options: =========================== * --optimized-kernel-enable Hashmode: 0 - MD5 Speed.#1.........: 54021.6 MH/s (44.26ms) @ Accel:128 Loops:1024 Thr:256 Vec:1 Speed.#2.........: 54290.1 MH/s (44.02ms) @ Accel:128 Loops:1024 Thr:256 Vec:1 Speed.#*.........:  108.3 GH/s Hashmode: 100 - SHA1 Speed.#1.........: 19480.4 MH/s (61.53ms) @ Accel:128 Loops:512 Thr:256 Vec:1 Speed.#2.........: 19571.1 MH/s (61.23ms) @ Accel:128 Loops:512 Thr:256 Vec:1 Speed.#*.........: 39051.5 MH/s Hashmode: 1400 - SHA2-256 Speed.#1.........:  8075.2 MH/s (74.28ms) @ Accel:128 Loops:256 Thr:256 Vec:1 Speed.#2.........:  8128.5 MH/s (73.76ms) @ Accel:128 Loops:256 Thr:256 Vec:1 Speed.#*.........: 16203.7 MH/s Hashmode: 1700 - SHA2-512 Speed.#1.........:  2599.7 MH/s (57.58ms) @ Accel:128 Loops:64 Thr:256 Vec:1 Speed.#2.........:  2622.5 MH/s (57.08ms) @ Accel:128 Loops:64 Thr:256 Vec:1 Speed.#*.........:  5222.2 MH/s Hashmode: 2500 - WPA-EAPOL-PBKDF2 (Iterations: 4096) Speed.#1.........:  899.7 kH/s (40.60ms) @ Accel:128 Loops:64 Thr:256 Vec:1 Speed.#2.........:  901.9 kH/s (40.48ms) @ Accel:128 Loops:64 Thr:256 Vec:1 Speed.#*.........:  1801.6 kH/s Hashmode: 1000 - NTLM Speed.#1.........: 94411.2 MH/s (25.12ms) @ Accel:128 Loops:1024 Thr:256 Vec:1 Speed.#2.........: 94588.6 MH/s (24.99ms) @ Accel:128 Loops:1024 Thr:256 Vec:1 Speed.#*.........:  189.0 GH/s Hashmode: 3000 - LM Speed.#1.........: 50350.9 MH/s (47.34ms) @ Accel:128 Loops:1024 Thr:256 Vec:1 Speed.#2.........: 50626.8 MH/s (47.09ms) @ Accel:128 Loops:1024 Thr:256 Vec:1 Speed.#*.........:  101.0 GH/s Hashmode: 5500 - NetNTLMv1 / NetNTLMv1+ESS Speed.#1.........: 54596.3 MH/s (43.72ms) @ Accel:128 Loops:1024 Thr:256 Vec:1 Speed.#2.........: 55000.5 MH/s (43.40ms) @ Accel:128 Loops:1024 Thr:256 Vec:1 Speed.#*.........:  109.6 GH/s Hashmode: 5600 - NetNTLMv2 Speed.#1.........:  4048.2 MH/s (74.08ms) @ Accel:128 Loops:128 Thr:256 Vec:1 Speed.#2.........:  4056.3 MH/s (73.92ms) @ Accel:128 Loops:128 Thr:256 Vec:1 Speed.#*.........:  8104.4 MH/s Hashmode: 1500 - descrypt, DES (Unix), Traditional DES Speed.#1.........:  1968.0 MH/s (76.11ms) @ Accel:8 Loops:1024 Thr:256 Vec:1 Speed.#2.........:  1944.0 MH/s (77.07ms) @ Accel:8 Loops:1024 Thr:256 Vec:1 Speed.#*.........:  3912.0 MH/s Hashmode: 500 - md5crypt, MD5 (Unix), Cisco-IOS $1$ (MD5) (Iterations: 1000) Speed.#1.........: 24096.0 kH/s (43.57ms) @ Accel:1024 Loops:500 Thr:32 Vec:1 Speed.#2.........: 24571.3 kH/s (42.53ms) @ Accel:1024 Loops:500 Thr:32 Vec:1 Speed.#*.........: 48667.3 kH/s Hashmode: 3200 - bcrypt $2*$, Blowfish (Unix) (Iterations: 32) Speed.#1.........:    29266 H/s (38.28ms) @ Accel:16 Loops:4 Thr:8 Vec:1 Speed.#2.........:    29355 H/s (38.05ms) @ Accel:16 Loops:4 Thr:8 Vec:1 Speed.#*.........:    58621 H/s Hashmode: 1800 - sha512crypt $6$, SHA512 (Unix) (Iterations: 5000) Speed.#1.........:  359.5 kH/s (81.02ms) @ Accel:512 Loops:128 Thr:32 Vec:1 Speed.#2.........:  359.5 kH/s (81.04ms) @ Accel:512 Loops:128 Thr:32 Vec:1 Speed.#*.........:  719.0 kH/s Hashmode: 7500 - Kerberos 5 AS-REQ Pre-Auth etype 23 Speed.#1.........:  717.5 MH/s (52.19ms) @ Accel:128 Loops:64 Thr:64 Vec:1 Speed.#2.........:  721.6 MH/s (51.87ms) @ Accel:128 Loops:64 Thr:64 Vec:1 Speed.#*.........:  1439.1 MH/s Hashmode: 13100 - Kerberos 5 TGS-REP etype 23 Speed.#1.........:  712.5 MH/s (52.55ms) @ Accel:128 Loops:64 Thr:64 Vec:1 Speed.#2.........:  714.5 MH/s (52.40ms) @ Accel:128 Loops:64 Thr:64 Vec:1 Speed.#*.........:  1427.0 MH/s Hashmode: 15300 - DPAPI masterkey file v1 (Iterations: 23999) Speed.#1.........:  157.0 kH/s (79.63ms) @ Accel:128 Loops:128 Thr:256 Vec:1 Speed.#2.........:  156.7 kH/s (79.75ms) @ Accel:128 Loops:128 Thr:256 Vec:1 Speed.#*.........:  313.7 kH/s Hashmode: 15900 - DPAPI masterkey file v2 (Iterations: 7999) Speed.#1.........:  132.7 kH/s (70.81ms) @ Accel:256 Loops:128 Thr:32 Vec:1 Speed.#2.........:  133.9 kH/s (70.13ms) @ Accel:256 Loops:128 Thr:32 Vec:1 Speed.#*.........:  266.6 kH/s Hashmode: 7100 - macOS v10.8+ (PBKDF2-SHA512) (Iterations: 35000) Speed.#1.........:    31727 H/s (67.37ms) @ Accel:128 Loops:32 Thr:256 Vec:1 Speed.#2.........:    31960 H/s (66.88ms) @ Accel:128 Loops:32 Thr:256 Vec:1 Speed.#*.........:    63687 H/s Hashmode: 11600 - 7-Zip (Iterations: 524288) Speed.#1.........:    21950 H/s (52.06ms) @ Accel:128 Loops:256 Thr:256 Vec:1 Speed.#2.........:    22147 H/s (51.58ms) @ Accel:128 Loops:256 Thr:256 Vec:1 Speed.#*.........:    44097 H/s Hashmode: 12500 - RAR3-hp (Iterations: 262144) Speed.#1.........:    68469 H/s (66.99ms) @ Accel:4 Loops:16384 Thr:256 Vec:1 Speed.#2.........:    67441 H/s (68.04ms) @ Accel:4 Loops:16384 Thr:256 Vec:1 Speed.#*.........:  135.9 kH/s Hashmode: 13000 - RAR5 (Iterations: 32767) Speed.#1.........:    96142 H/s (47.51ms) @ Accel:128 Loops:64 Thr:256 Vec:1 Speed.#2.........:    97547 H/s (46.80ms) @ Accel:128 Loops:64 Thr:256 Vec:1 Speed.#*.........:  193.7 kH/s Hashmode: 6211 - TrueCrypt PBKDF2-HMAC-RIPEMD160 + XTS 512 bit (Iterations: 2000) Speed.#1.........:  599.2 kH/s (51.71ms) @ Accel:128 Loops:32 Thr:256 Vec:1 Speed.#2.........:  606.8 kH/s (50.95ms) @ Accel:128 Loops:32 Thr:256 Vec:1 Speed.#*.........:  1206.1 kH/s Hashmode: 13400 - KeePass 1 (AES/Twofish) and KeePass 2 (AES) (Iterations: 6000) Speed.#1.........:  237.2 kH/s (105.74ms) @ Accel:512 Loops:128 Thr:32 Vec:1 Speed.#2.........:  232.3 kH/s (107.97ms) @ Accel:512 Loops:128 Thr:32 Vec:1 Speed.#*.........:  469.5 kH/s Hashmode: 6800 - LastPass + LastPass sniffed (Iterations: 500) Speed.#1.........:  6014.0 kH/s (91.37ms) @ Accel:128 Loops:125 Thr:256 Vec:1 Speed.#2.........:  5493.7 kH/s (99.66ms) @ Accel:128 Loops:125 Thr:256 Vec:1 Speed.#*.........: 11507.7 kH/s Hashmode: 11300 - Bitcoin/Litecoin wallet.dat (Iterations: 199999) Speed.#1.........:    11698 H/s (64.08ms) @ Accel:128 Loops:64 Thr:256 Vec:1 Speed.#2.........:    11672 H/s (64.23ms) @ Accel:128 Loops:64 Thr:256 Vec:1 Speed.#*.........:    23370 H/s Started: Thu Oct 24 12:48:55 2019 Stopped: Thu Oct 24 12:57:47 2019

Actual:

Code:

hashcat64.exe -a 0 --session=2019-10-24 -m 0 -w 3 --status --status-timer=60 -p : -O --hwmon-disable -o "C:\Users\user\Desktop\results.txt" --outfile-format=3 "C:\Users\user\Desktop\hashes.txt" "C:\Users\user\Desktop\wordlists\hashes.org-2018.txt" "C:\Users\user\Desktop\wordlists\hashes.org-2019.txt" hashcat (v5.1.0) starting... * Device #1: WARNING! Kernel exec timeout is not disabled.             This may cause "CL_OUT_OF_RESOURCES" or related errors.             To disable the timeout, see: https://hashcat.net/q/timeoutpatch * Device #2: WARNING! Kernel exec timeout is not disabled.             This may cause "CL_OUT_OF_RESOURCES" or related errors.             To disable the timeout, see: https://hashcat.net/q/timeoutpatch OpenCL Platform #1: NVIDIA Corporation ====================================== * Device #1: TITAN RTX, 6144/24576 MB allocatable, 72MCU * Device #2: TITAN RTX, 6144/24576 MB allocatable, 72MCU Hashes: 8 digests; 8 unique digests, 1 unique salts Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates Rules: 1 Applicable optimizers: * Optimized-Kernel * Zero-Byte * Precompute-Init * Precompute-Merkle-Demgard * Meet-In-The-Middle * Early-Skip * Not-Salted * Not-Iterated * Single-Salt * Raw-Hash Minimum password length supported by kernel: 0 Maximum password length supported by kernel: 31 Watchdog: Hardware monitoring interface not found on your system. Watchdog: Temperature abort trigger disabled. INFO: Removed 2 hashes found in potfile. Dictionary cache hit: * Filename..: C:\Users\user\Desktop\wordlists\hashes.org-2018.txt * Passwords.: 613377895 * Bytes.....: 6429547050 * Keyspace..: 613377895 [s]tatus [p]ause [b]ypass [c]heckpoint [q]uit => Session..........: 2019-10-24 Status...........: Running Hash.Type........: MD5 Hash.Target......: C:\Users\user\Desktop\hashes.txt Time.Started.....: Thu Oct 24 13:07:19 2019 (3 secs) Time.Estimated...: Thu Oct 24 13:07:51 2019 (29 secs) Guess.Base.......: File (C:\Users\user\Desktop\wordlists\hashes.org-2018.txt) Guess.Queue......: 1/2 (50.00%) Speed.#1.........:  9548.5 kH/s (0.39ms) @ Accel:128 Loops:1 Thr:256 Vec:1 Speed.#2.........:  9476.0 kH/s (0.39ms) @ Accel:128 Loops:1 Thr:256 Vec:1 Speed.#*.........: 19024.5 kH/s Recovered........: 2/8 (25.00%) Digests, 0/1 (0.00%) Salts Progress.........: 47191472/613377895 (7.69%) Rejected.........: 5552/47191472 (0.01%) Restore.Point....: 47191160/613377895 (7.69%) Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1 Restore.Sub.#2...: Salt:0 Amplifier:0-1 Iteration:0-1 Candidates.#1....: 16_amor_yas -> 1849633311sinaweibo Candidates.#2....: 157220798927881 -> 184959591255691 [s]tatus [p]ause [b]ypass [c]heckpoint [q]uit => Session..........: 2019-10-24 Status...........: Quit Hash.Type........: MD5 Hash.Target......: C:\Users\user\Desktop\hashes.txt Time.Started.....: Thu Oct 24 13:07:19 2019 (6 secs) Time.Estimated...: Thu Oct 24 13:07:53 2019 (28 secs) Guess.Base.......: File (C:\Users\user\Desktop\wordlists\hashes.org-2018.txt) Guess.Queue......: 1/2 (50.00%) Speed.#1.........:  8991.3 kH/s (0.39ms) @ Accel:128 Loops:1 Thr:256 Vec:1 Speed.#2.........:  8997.2 kH/s (0.39ms) @ Accel:128 Loops:1 Thr:256 Vec:1 Speed.#*.........: 17988.5 kH/s Recovered........: 2/8 (25.00%) Digests, 0/1 (0.00%) Salts Progress.........: 103818998/613377895 (16.93%) Rejected.........: 9974/103818998 (0.01%) Restore.Point....: 103818948/613377895 (16.93%) Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1 Restore.Sub.#2...: Salt:0 Amplifier:0-1 Iteration:0-1 Candidates.#1....: 57600950 -> 5elmvale Candidates.#2....: 5488934558 -> 5elmerhick Started: Thu Oct 24 13:07:09 2019 Stopped: Thu Oct 24 13:07:28 2019

[undeath]

https://hashcat.net/wiki/doku.php?id=fre...full_speed

[Chick3nman]

I run hashcat on Titan RTX pretty much daily, this seems like reasonable enough speed given its just a straight wordlist attack and its dumped into the hashcat GUI(which we don't support officially). I would listen to the warnings and apply the timeout patch. Then I would stop using the GUI so you can control the flags that are being thrown better, no reason at all to disable the hardware monitoring system or set a status timer or -p unless are things you specifically need. From there, drop the -w 3 as setting that manually sometimes gives funny tuning values. A simple command is going to be easier for you to read and spot errors, as well less likely to cause issues with mixing of flags and different configurations when they arent needed.

[jason_tx]

https://hashcat.net/wiki/doku.php?id=fre...full_speed

Thanks for the link. I've obviously only dipped my toe in the hashcat possibility pool, so I appreciate the guidance.

After reading I tried a new command to see if I could generate enough 'work', yet I'm curious if anyone here could suggest better ones?! Any command suggestions for a decent number of bcrypt hashes?  
Below is the command I used, which did get me running up to around 20,000 MH/s total (2 cards combined). To be clear I'm satisfied with the speed now (knowing there isn't any actual issues), just don't know if there are better methods to generate more 'work'.  

Code:

pp64.exe "\hashcat-5.1.0\WORDLISTS\rockyou.txt\rockyou.txt" | hashcat64.exe -a 0 --session=2019-10-24 -m 0 -w 3 --status --status-timer=60 -p : -O --hwmon-disable -o "C:\Users\user\Desktop\results.txt" --outfile-format=3 -r "\hashcat-5.1.0\rules\OneRuleToRuleThemAll.rule" "C:\Users\user\Desktop\MD5_hashes.txt"

Status Result snapshot:

Code:

Session..........: 2019-10-24 Status...........: Running Hash.Type........: MD5 Hash.Target......: C:\Users\user\Desktop\MD5_hashes.txt Time.Started.....: Thu Oct 24 16:53:30 2019 (8 hours, 31 mins) Time.Estimated...: Fri Oct 25 01:25:27 2019 (0 secs) Guess.Base.......: Pipe Guess.Mod........: Rules (\hashcat-5.1.0\rules\OneRuleToRuleThemAll.rule) Speed.#1.........:  9579.7 MH/s (34.33ms) @ Accel:128 Loops:256 Thr:256 Vec:1 Speed.#2.........: 10059.8 MH/s (33.63ms) @ Accel:128 Loops:256 Thr:256 Vec:1 Speed.#*.........: 19642.6 MH/s Recovered........: 2/8 (25.00%) Digests, 0/1 (0.00%) Salts Progress.........: 597677088964608 Rejected.........: 0 Restore.Point....: 0 Restore.Sub.#1...: Salt:0 Amplifier:11008-11264 Iteration:0-256 Restore.Sub.#2...: Salt:0 Amplifier:49920-50176 Iteration:0-256 Candidates.#1....: Qbejtvc -> e15flf Candidates.#2....: mkogulomo01 -> fbejoac077

Also I'm concerned that I'm doing something wrong or skipping some easier password possibilities considering I let this run for 16 hours and I did not resolve any new hashes ('2' were from potfile). Would it be better to run a bruteforce (..and does that generate enough 'work' to get the GPUs running)?

sorry for the barrage for questions, 
Thanks in advance for any insight. 

[jason_tx]

I run hashcat on Titan RTX pretty much daily, this seems like reasonable enough speed given its just a straight wordlist attack and its dumped into the hashcat GUI(which we don't support officially). I would listen to the warnings and apply the timeout patch. Then I would stop using the GUI so you can control the flags that are being thrown better, no reason at all to disable the hardware monitoring system or set a status timer or -p unless are things you specifically need. From there, drop the -w 3 as setting that manually sometimes gives funny tuning values. A simple command is going to be easier for you to read and spot errors, as well less likely to cause issues with mixing of flags and different configurations when they arent needed.

Noted! No more hashcat GUI, thanks for the advice.

[royce]

It's OK to use hashcat GUI if it works for what you're trying to do. But many of the things you'll eventually want to do are better done with the command line directly.