Piping python output into hashcat continuously

[defaultusername]

Hi everybody!

I have a python script that outputs password candidates in batches, similar to this:

Code:

import time
for i in range(10):
    for j in range(10):
        print(str(i)+str(j))
    time.sleep(1)

When writing the output to a file

Code:

python generate.py > out.txt

the out.txt file is created immediately and continuously written into until the script finishes.

I want to pipe the output directly into hashcat:

Code:

python generate.py | hashcat -a 0 --stdout --outfile out.txt

However hashcat starts and then idles until generate.py has completed. Outfile out.txt only gets created after the python script has run completely.

Compare to princeprocessor:

Code:

./pp64.bin input.txt | hashcat -a 0 --stdout --outfile out.txt

creates out.txt before princeprocessor has finished and continuously writes into out.txt.


How do I make my python script feed into hashcat continuously?

I already tried

Code:

import time
import sys
for i in range(10):
    for j in range(10):
        print(str(i)+str(j))
    sys.stdout.flush()
    time.sleep(1)

which did not help.

[undeath]

Just a quick guess: your test script is not generating enough words to fill some internal buffer of hashcat used to batch together chunks of input words. Transferring every single word individually to your opencl device would probably cause very poor performance.

Test again with a larger number of generated input words.

[defaultusername]

That worked! Thanks!

Is there any documentation on the size/behaviour of this buffer? Would be very useful for my actual generation script.

EDIT: Just did a test. Hashcat starts writing at ~2,628,000 generated lines. That answers my question.

[2d4d]

hui, python will be the total bottleneck, at least use pypy

[CGretski]

Python buffering output before sending it may be compounding the problem, try running your script in unbuffered mode.
i.e. python3 -u