06-16-2020, 05:37 PM
Welcome to hashcat v6.0.0!
Download binaries and source code from: https://hashcat.net/hashcat/
It has been a long time since the last release, and a long journey for hashcat 6.0.0 - which we are releasing today. It comes with a lot of performance improvements, new features and detailed documentations for both users and developers.
In total, we had over 1800 Git commits since the last release (5.1.0) - from 29 different contributors.
For a full list of contributors, please see: https://github.com/hashcat/hashcat/graph...2020-06-16
The previous release of hashcat was over one year ago, but hashcat constantly changes on a daily basis and has improved a lot in that time. We would like to release new hashcat versions more frequently in the future, but as you can see from the huge architectural changes below, this version is exceptional... Good things take time!
The new major features of hashcat 6.0.0:
- New plugin interface - for modular hash-modes
- New compute-backend API interface - for adding compute APIs other than OpenCL
- CUDA added as a new compute-backend API
- Comprehensive plugin developer guide
- GPU Emulation mode - for using kernel code on the host CPU
- Better GPU memory and thread management
- Improved auto-tuning based on available resources
Along with the major changes, we have added the following frequently demanded 51 new algorithms:
- AES Crypt (SHA256)
- Android Backup
- AuthMe sha256
- BitLocker
- BitShares v0.x
- Blockchain, My Wallet, Second Password (SHA256)
- Citrix NetScaler (SHA512)
- DiskCryptor
- Electrum Wallet (Salt-Type 3-5)
- Huawei Router sha1(md5($pass).$salt)
- Java Object hashCode()
- Kerberos 5 Pre-Auth etype 17 (AES128-CTS-HMAC-SHA1-96)
- Kerberos 5 Pre-Auth etype 18 (AES256-CTS-HMAC-SHA1-96)
- Kerberos 5 TGS-REP etype 17 (AES128-CTS-HMAC-SHA1-96)
- Kerberos 5 TGS-REP etype 18 (AES256-CTS-HMAC-SHA1-96)
- MultiBit Classic .key (MD5)
- MultiBit HD (scrypt)
- MySQL $A$ (sha256crypt)
- Open Document Format (ODF) 1.1 (SHA-1, Blowfish)
- Open Document Format (ODF) 1.2 (SHA-256, AES)
- Oracle Transportation Management (SHA256)
- PKZIP archive encryption
- PKZIP Master Key
- Python passlib pbkdf2-sha1
- Python passlib pbkdf2-sha256
- Python passlib pbkdf2-sha512
- QNX /etc/shadow (MD5)
- QNX /etc/shadow (SHA256)
- QNX /etc/shadow (SHA512)
- RedHat 389-DS LDAP (PBKDF2-HMAC-SHA256)
- Ruby on Rails Restful-Authentication
- SecureZIP AES-128
- SecureZIP AES-192
- SecureZIP AES-256
- SolarWinds Orion
- Telegram Desktop App Passcode (PBKDF2-HMAC-SHA1)
- Telegram Mobile App Passcode (SHA256)
- Web2py pbkdf2-sha512
- WPA-PBKDF2-PMKID+EAPOL
- WPA-PMK-PMKID+EAPOL
- md5($salt.sha1($salt.$pass))
- md5(sha1($pass).md5($pass).sha1($pass))
- md5(sha1($salt).md5($pass))
- sha1(md5(md5($pass)))
- sha1(md5($pass.$salt))
- sha1(md5($pass).$salt)
- sha1($salt1.$pass.$salt2)
- sha256(md5($pass))
- sha256($salt.$pass.$salt)
- sha256(sha256_bin($pass))
- sha256(sha256($pass).$salt)
And here's a preview of some of the performance improvements:
- MD5: 8.05%
- NTLM: 13.70%
- Domain Cached Credentials (DCC), MS Cache: 11.91%
- Domain Cached Credentials 2 (DCC2), MS Cache 2: 12.51%
- NetNTLMv1: 15.79%
- NetNTLMv2: 6.98%
- WPA/WPA2: 13.35%
- sha256crypt $5$, SHA256 (Unix): 8.77%
- sha512crypt $6$, SHA512 (Unix): 20.33%
- bcrypt: 45.58%
- IPMI2 RAKP HMAC-SHA1: 20.03%
- SAP CODVN B (BCODE): 32.37%
- Blockchain, My Wallet: 31.00%
- Electrum Wallet (Salt-Type 1-3): 109.46%
- WinZip: 119.43%
In addition to these, there are a number of other new features and changes - but in this post, we want to focus mainly on the major changes to keep the release notes to a digestible length. For those interested, the changelog and git history have a more complete list of all changes.
Most of these changes are aimed at developers. These release notes are intentionally verbose to inform current contributors and developers, as well as to catch the interest of potential future hashcat contributors.
Major Feature: Plugin Interface
One of the first things you will notice after unpacking the new hashcat version is the new modules folder. We have had modularity in mind for a long time, and have finally managed to implement it: each and every hash type is separated into its own module. This not only makes the code much easier to read, write, and maintain, but it also comes with a very nice new architecture, interface, and added flexibility.
In essence, this is actually just some overdue refactorization, but it comes with a lot of benefits for developers working on new hash types: it makes it much easier to write new host code (including parsers, decoders, encoders, hooks etc). The hash type code is 100% separated from the core code, meaning that there's no more need to edit the hashcat core sources to add a new hash type. This also enables much easier distribution of custom kernels which have not been pushed to the main repository.
During more than four months of "conversion" of the old hash types, we designed a new common interface and made all existing hash modes work with this new plugin interface. We even created a new testing framework, and converted all the old testing modules. See the tools/test_modules/ folder for more information.
A huge thanks to everyone helping to convert hash types and/or tests for these new interfaces. This has not only shown us that the new interface works great and is flexible enough to cover all the different needs from the different modules, but it also shows that contributors are able to easily write modular code.
The new fully modularized hash-type integration makes the hash-type-specific code more compact and encapsulated, but also maintains and even adds flexibility. For instance, it is now possible to easily add hash-mode specific JiT (just-in-time) compiler flags which are used at kernel compilation runtime, or easily mark hash-mode specific unstable warnings on some specific setups (for instance depending on driver and hardware). One could easily add new restriction and limitations directly to the module, without cluttering other parts of hashcat (to avoid "spaghetti code" and "special cases" everywhere).
There is a lot to say about this new architecture that we've designed and we could go into much further detail, but we will do our best to not go too far here. Fortunately, for everybody interested, we also wrote a hashcat plugin interface guide for developers. This guide is the first official "how to add a new hash type" document, it already consisting of almost 20,000 words. It does not cover every detail, but it gives you everything you need to get started adding your own hash type.
Be prepared, because reading will take a bit of time. You can find it here: https://github.com/hashcat/hashcat/blob/...t-guide.md
Major Feature: Backend Interface
Similar to the Plugin Interface feature, it took us quite some time and effort to refactor how we deal with supporting different compute devices in hashcat.
As you will notice, we have changed many command line parameters to --backend-* as replacements of the old --opencl-* parameters. The reason for this is that hashcat now has a more flexible architecture for how we deal with different backends (like CUDA/OpenCL etc). With this system, we can add additional backends in an elegant way whenever we may need to in the future.
The system is designed in such a way that backend-specific code is abstracted away from other operations (like loading the kernel source code etc) and uses a common interface which makes the code much more readable and easy to use.
Major Feature: CUDA Support
This is basically an "application" of the new Backend Interface feature. With the new architecture for hashcat backends, we were able to start supporting CUDA for NVIDIA devices. By NVIDIA devices, we mean any of their compute devices that support CUDA, not just discrete GPUs! This enables hashcat to run on chips such as the NVIDIA Jetson or NVIDIA Xavier. This also enables us to utilize CUDA on platforms where NVIDIA does not release a driver capable of OpenCL, including ARM platforms and IBM POWER9 platforms.
There are several other advantages that CUDA has over OpenCL on NVIDIA devices, but the most important one is that the entire amount of GPU memory in a single block allocation can now be unlocked by the user by installing the CUDA Toolkit.
We recommend (at the time of this writing) installing the CUDA Toolkit without the NVIDIA driver it ships with and install the latest/recommended driver from nvidia.com instead. Hashcat will actually warn you if you have an NVIDIA device, but "only" use the OpenCL driver, because you should install the CUDA Toolkit for CUDA-supported devices. This step is mandatory if you want to use CUDA instead of OpenCL backend because there's no JiT compiler for CUDA the way how it comes already built-in for the OpenCL with the NVIDIA drivers.
Hashcat will list all the devices (CUDA devices in addition to OpenCL devices) with --backend-info (short: -I) and you can easily select the devices you want with --backend-devices (short: -d). Of course, hashcat prefers the "CUDA devices" if available! (and for the curious reader: no, you can't actually use OpenCL and CUDA at the same time for the same device in hashcat - we call this an alias. The speed will NOT double this way :P)
One of the biggest advantages of CUDA compared to OpenCL is the full use of shared memory (sometimes also called Local Memory). In OpenCL, there's a minimum of 1 byte reserved by OpenCL which has bigger implications than may be apparent at first. For example, most NVIDIA cards have 48kb of shared memory. To efficiently compute bcrypt, each thread requires 4k of this shared memory pool. This means that on CUDA we are able to use 12 threads on bcrypt instead of just 11 threads with OpenCL. This and other optimizations are the reason we improved the performance of bcrypt by 46.90%.
Major Feature: Emulation Mode
This feature is basically introducing a very nice way to use kernel code within modules or host code. We came up with this strategy for the following reasons:
- For complex kernels with lots of code, it's easier to debug the code on the host side as a standalone project. This also saves long startup times and increases development speed.
- Reuse complex kernel code from within the module. A good example is WPA EAPOL/PMKID, where we compute the last steps in the module parser with the goal to find already cracked hashes in the potfile. This saves maintaining the same code on two ends.
- Reuse kernel code in order to precompute values in parsers. As a very easy example, consider some "flawed" algo that uses md5($salt) within the algorithm. In this case we could just precompute this MD5 in the host.
It is actually quite easy to use for developers. For an example of some basic hashing algos being used directly in modules with this new emulation mode, just take a look at existing modules (like -m 19100 as mentioned above) or glance at the include/emu_*.h files.
Major Feature: Memory/Thread Management
Lastly, a feature that is less directed at hashcat devs/contributors, but is still interesting and very important when it comes to performance: improvement in memory and thread management to reach maximum performance.
Hashcat 6.0.0 introduces a new way that threads and device memory (VRAM) are used and optimized: with the addition of a new automatic workload tuner, we try to guarantee maximum performance depending on the available memory, hash type, attack mode, amplifiers (e.g. rules) etc. We basically changed the thread management from a "native" thread count for GPU to maximum possible threads. We've also added a command line parameter: --kernel-threads (short: -T) if you want to play with this and set the amount of threads manually.
This obviously comes with a very nice performance gain depending on hash type, attack mode etc.
Changelog Features:
- Refactored hash-mode integration and replaced it with a fully modularized plugin interface
- Converted all existing hardwired hash-modes to hashcat plugins
- Added comprehensive plugin developer guide on adding new/custom hash-modes to hashcat
- Refactored compute backend interface to allow adding compute API other than OpenCL
- Added CUDA as a new compute backend (enables hashcat to run on NVIDIA Jetson, IBM POWER9 w/ Nvidia V100, etc.)
- Support automatic use of all available GPU memory when using CUDA backend
- Support automatic use of all available CPU cores for hash-mode-specific hooks
- Support on-the-fly loading of compressed wordlists in zip and gzip format
- Support deflate decompression for the 7-Zip hash-mode using zlib hook
- Added additional documentation on hashcat brain, slow-candidate interface and keyboard-layout mapping features
- Keep output of --show and --left in the original ordering of the input hash file
- Improved performance of many hash-modes
Changelog fixed Bugs:
- Fixed buffer overflow in build_plain() function
- Fixed buffer overflow in mp_add_cs_buf() function
- Fixed calculation of brain-session ID - only the first hash of the hashset was taken into account
- Fixed cleanup of password candidate buffers on GPU as set from autotune when -n parameter was used
- Fixed copy/paste error leading to invalid "Integer overflow detected in keyspace of mask" in attack-mode 6 and 7
- Fixed cracking multiple Office hashes (modes 9500, 9600) if hashes shared the same salt
- Fixed cracking of Blockchain, My Wallet (V1 and V2) hashes when testing decrypted data in unexpected format
- Fixed cracking of Cisco-PIX and Cisco-ASA MD5 passwords in mask-attack mode when mask > length 16
- Fixed cracking of DNSSEC (NSEC3) hashes by replacing all dots in the passwords with lengths
- Fixed cracking of Electrum Wallet Salt-Type 2 hashes
- Fixed cracking of NetNTLMv1 passwords in mask-attack mode when mask > length 16 (optimized kernels only)
- Fixed cracking of RAR3-hp hashes with pure kernel for passwords longer than 28 bytes
- Fixed cracking of VeraCrypt Streebog-512 hashes (CPU only)
- Fixed cracking raw Streebog-HMAC 256 and 512 hashes for passwords of length >= 64
- Fixed cracking raw Whirlpool hashes cracking for passwords of length >= 32
- Fixed incorrect progress-only result in a special race condition
- Fixed invalid call of mp_css_utf16le_expand()/mp_css_utf16be_expand() in slow-candidate sessions
- Fixed invalid password truncation in attack-mode 1 when the final password is longer than 32 characters
- Fixed invalid use of --hex-wordlist if encoded wordlist string is larger than length 256
- Fixed maximum password length limit which was announced as 256 but was actually 255
- Fixed out-of-boundary read in pure kernel rule engine rule 'p' when parameter was set to 2 or higher
- Fixed out-of-boundary write to decrypted[] in DPAPI masterkey file v1 kernel
- Fixed output of IKE PSK (mode 5300 and 5400) hashes to use separators in the correct position
- Fixed output password of "e" rule in pure and CPU rule engine when separator character is also the first letter
- Fixed problem with usage of hexadecimal notation (\x00-\xff) within rules
- Fixed race condition in maskfile mode by using a dedicated flag for restore execution
- Fixed some memory leaks when hashcat is shutting down due to some file error
- Fixed some memory leaks when mask-files are used in optimized mode
- Fixed --status-json to correctly escape certain characters in hashes
- Fixed the 7-Zip parser to allow the entire supported range of encrypted and decrypted data lengths
- Fixed the validation of the --brain-client-features command line argument (only values 1, 2 or 3 are allowed)
Changelog Improvements:
- Bitcoin Wallet: Be more user friendly by allowing a larger data range for ckey and public_key
- Brain: Added new parameter --brain-server-timer to specify seconds between scheduled backups
- Building: Fix for library compilation failure due to multiple defenition of sbob_xx64()
- Cracking bcrypt and Password Safe v2: Use feedback from the compute API backend to dynamically calculate optimal thread count
- Dictstat: On Windows, the st_ino attribute in the stat struct is not set, which can lead to invalid cache hits. Added the filename to the database entry.
- Documents: Added README on how to build hashcat on Cygwin, MSYS2 and WSL
- File handling: Print a truncation warning when an oversized line is detected
- My Wallet: Added additional plaintext pattern used in newer versions
- Office cracking: Support hash format with second block data for 40-bit oldoffice files (eliminates false positives)
- OpenCL Runtime: Added a warning if OpenCL runtime NEO, Beignet, POCL (v1.4 or older) or MESA is detected, and skip associated devices (override with --force)
- OpenCL Runtime: Allow the kernel to access post-48k shared memory region on CUDA. Requires both module and kernel preparation
- OpenCL Runtime: Disable OpenCL kernel cache on Apple for Intel CPU (throws CL_BUILD_PROGRAM_FAILURE for no reason)
- OpenCL Runtime: Do not run shared- or constant-memory size checks if their memory type is of type global memory (typically CPU)
- OpenCL Runtime: Improve ROCm detection and make sure to not confuse with recent AMDGPU drivers
- OpenCL Runtime: Not using amd_bytealign (amd_bitalign is fine) on AMDGPU driver drastically reduces JiT segfaults
- OpenCL Runtime: Unlocked maximum thread count for NVIDIA GPU
- OpenCL Runtime: Update unstable mode warnings for Apple and AMDGPU drivers
- OpenCL Runtime: Workaround JiT compiler error on AMDGPU driver compiling WPA-EAPOL-PBKDF2 OpenCL kernel
- OpenCL Runtime: Workaround JiT compiler error on ROCm 2.3 driver if the 'inline' keyword is used in function declaration
- OpenCL Runtime: Workaround memory allocation error on AMD driver on Windows leading to CL_MEM_OBJECT_ALLOCATION_FAILURE
- OpenCL Runtime: Removed some workarounds by calling chdir() to specific folders on startup
- Outfile: Added new system to specify the outfile format, the new --outfile-format now also supports timestamps
- Startup Checks: Improved the pidfile check: Do not just check for existing PID, but also check executable filename
- Startup Checks: Prevent the user from modifying options which are overwritten automatically in benchmark mode
- Startup Screen: Add extra warning when using --force
- Startup Screen: Add extra warning when using --keep-guessing
- Startup Screen: Provide an estimate of host memory required for the requested attack
- Status Screen: Added brain status for all compute devices
- Status Screen: Added remaining counts and changed recovered count logic
- Status Screen: Added --status-json flag for easier machine reading of hashcat status output
- Tab Completion: Allow using "make install" version of hashcat
- Tuning Database: Updated hashcat.hctune with new models and refreshed vector width values
- VeraCrypt: Added support for VeraCrypt PIM brute-force, replaced --veracrypt-pim with --veracrypt-pim-start and --veracrypt-pim-stop
- WipZip cracking: Added two byte early reject, resulting in higher cracking speed
- WPA/WPA2 cracking: In the potfile, replace password with PMK in order to detect already cracked networks across all WPA modes
Changelog Technical:
- Backend Interface: Added new options --backend-ignore-cuda and --backend-ingore-opencl to prevent CUDA and/or OpenCL API from being used
- Binary Distribution: Removed 32-bit binary executables
- Building: On macOS, switch from ar to /usr/bin/ar to improve building compatibility
- Building: Skipping Travis/Appveyor build for non-code changes
- Codebase: Cleanup of many unused rc_* variables
- Codebase: Fixed some printf() format arguments
- Codebase: Fixed some type casting to avoid truncLongCastAssignment warnings
- Codebase: Moved hc_* file functions from shared.c to filehandling.c
- Codebase: Ran through a bunch of clang-tidy checkers and updated code accordingly
- Codebase: Remove redundant calls to fclose()
- Dependencies: Updated LZMA-Headers from 18.05 to 19.00
- Dependencies: Updated OpenCL-Headers to latest version from GitHub master repository
- Hash-Mode 12500 (RAR3-hp): Allow cracking of passwords up to length 64
- Hash-mode 1460 (HMAC-SHA256 (key = $salt)): Allow up to 64 byte of salt
- Hash-Mode 1680x (WPA-PMKID) specific: Changed separator character from '*' to ':'
- Hash-Mode 8300 (DNSSEC (NSEC3)) specific: Allow empty salt
- Keep Guessing: No longer automatically activate --keep-guessing for modes 9720, 9820, 14900 and 18100
- Keep Guessing: No longer mark hashes as cracked/removed when in potfile
- Kernel Cache: Reactivate OpenCL runtime specific kernel caches
- Kernel Compile: Removed -cl-std= from all kernel build options since we're compatible to all OpenCL versions
- OpenCL Kernels: Fix OpenCL compiler warning on double precision constants
- OpenCL Kernels: Moved "gpu_decompress", "gpu_memset" and "gpu_atinit" into shared.cl in order to reduce compile time
- OpenCL Options: Removed --opencl-platforms filter in order to force backend device numbers to stay constant
- OpenCL Options: Set --spin-damp to 0 (disabled) by default. With the CUDA backend this workaround became deprecated
- Parsers: switched from strtok() to strtok_r() for thread safety
- Requirements: Add new requirement for NVIDIA GPU: CUDA Toolkit (9.0 or later)
- Requirements: Update runtime check for minimum NVIDIA driver version from 367.x to 440.64 or later
- Test Script: Switched from /bin/bash to generic /bin/sh and updated code accordingly