Piping python output into hashcat continuously

Piping python output into hashcat continuously - Printable Version

+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Support (https://hashcat.net/forum/forum-3.html)
+--- Forum: hashcat (https://hashcat.net/forum/forum-45.html)
+--- Thread: Piping python output into hashcat continuously (/thread-9709.html)

Piping python output into hashcat continuously - defaultusername - 12-16-2020

Hi everybody!

I have a python script that outputs password candidates in batches, similar to this:

Code:
import time

for i in range(10):

    for j in range(10):

        print(str(i)+str(j))

    time.sleep(1)

When writing the output to a file

Code:
python generate.py > out.txt

the out.txt file is created immediately and continuously written into until the script finishes.

I want to pipe the output directly into hashcat:

Code:
python generate.py | hashcat -a 0 --stdout --outfile out.txt

However hashcat starts and then idles until generate.py has completed. Outfile out.txt only gets created after the python script has run completely.

Compare to princeprocessor:

Code:
./pp64.bin input.txt | hashcat -a 0 --stdout --outfile out.txt

creates out.txt before princeprocessor has finished and continuously writes into out.txt.

How do I make my python script feed into hashcat continuously?

I already tried

Code:
import time

import sys

for i in range(10):

    for j in range(10):

        print(str(i)+str(j))

    sys.stdout.flush()

    time.sleep(1)

which did not help.

RE: Piping python output into hashcat continuously - undeath - 12-16-2020

Just a quick guess: your test script is not generating enough words to fill some internal buffer of hashcat used to batch together chunks of input words. Transferring every single word individually to your opencl device would probably cause very poor performance.

Test again with a larger number of generated input words.

RE: Piping python output into hashcat continuously - defaultusername - 12-16-2020

That worked! Thanks!

Is there any documentation on the size/behaviour of this buffer? Would be very useful for my actual generation script.

EDIT: Just did a test. Hashcat starts writing at ~2,628,000 generated lines. That answers my question.

RE: Piping python output into hashcat continuously - 2d4d - 12-20-2020

hui, python will be the total bottleneck, at least use pypy Wink

RE: Piping python output into hashcat continuously - CGretski - 02-24-2021

Python buffering output before sending it may be compounding the problem, try running your script in unbuffered mode.
i.e. python3 -u