Hashes from clear passwords

[wallacebw]

My response is a bit late, but for anyone else needing to hash a wordlist, I published a tool on github several years ago called hashgen. It's multi-threaded (30+ million md5/sec), supports $HEX[], and currently supports over a dozen output modes such as md5, ntlm, sha*, base64 encode/decode, crc32/64... etc.

https://github.com/cyclone-github/hashgen

Haven't had a chance to read through your code yet, but I did something similar in python. (multithreaded, supports $hex[], STDIN or file input, STDOUT or file output, etc.)

https://github.com/wallacebw/hashutil 

Here's a snippet of the verbose output.   (parses ~235M sha1 hashes in 11 sec on a threadripper 3970X)

Code:

VERBOSE (process_multi): Total process pool cpu time: 0:09:54.737065
VERBOSE (process_multi):    Average process cpu time: 0:00:09.292767
VERBOSE (process_multi):                Elapsed time: 0:00:11.007720
VERBOSE (process_multi):    Process pool speed gain: 54.03x
VERBOSE (process_multi):          Process efficiency: 84.42%
VERBOSE (process_multi):          Temp File I/O time: 0:00:05.217029
VERBOSE (process_multi):      Process Pool loop time: 0:00:27.350114

Results:
      Input lines:
    skipped lines: 0
    Output lines: 236805847
======================================================
VERBOSE (main): Python init time time: 0:00:00.031992
VERBOSE (main): Total execution time: 0:00:27.353321
VERBOSE (main): Total processor time: 0:00:10.872765

Here are the parameters:

Code:

usage: hash_generator.py [-h] [-a HASH_ALGORITHMS] [-i [FILE]] [-p PARALLEL] [-t TEMP_DIRECTORY] [-s SEPARATOR] [-n] [-o OUTPUT_FILE] [-e
ERROR_FILE] [-u] [-v | -q]

Translate a file of cleartext strings (passwords) to hashes of the specified format(s)
to a text based separated file (TSV) with fields separated by -s / --separator [default ':']

options:
  -h, --help            show this help message and exit
  -a HASH_ALGORITHMS, --hash-algorithms HASH_ALGORITHMS
                        Comma separated Hash list to use (default: sha1) options are: sha1, sha224, sha256, sha384, sha512, sha3_224, sha3_256, sha3_384,, sha3_512, blake2b, blake2s, md5
  -i [FILE], --input-file [FILE]
                        The input file(s) of strings to parse, if omitted STDIN is used (comma separated)
  -p PARALLEL, --parallel PARALLEL
                        Number of processes to use or 'a' (default) for automatic detection
  -t TEMP_DIRECTORY, --temp-directory TEMP_DIRECTORY
                        Directory to use for temp files when --parallel is used default PWD)

Output Formatting:
  -s SEPARATOR, --separator SEPARATOR
                        The column separator to use in the output (default ':')
  -n, --no-header      Do not print the header line
  -o OUTPUT_FILE, --output-file OUTPUT_FILE
                        Output file, if omitted STDOUT is used
  -e ERROR_FILE, --error-file ERROR_FILE
                        Optional file to write lines that cannot be parsed
  -u, --hash-upper      Output the hash value in UPPERCASE (default is lowercase)

Output Verbosity:
  -v, --verbose        Verbose reporting of warnings (skipped lines) to STDERR (see -e) *** specify twice [-vv] for debugging (multiple messages per file line ***)
  -q, --quiet          Suppress all console output (STDOUT/STDERR)