Hashcat very slow between benchmarks

[pragmatic]

Hello, i'm not really sure what the problem could be here so thoughts are welcome. I'm running a benchmark on one of the new AWS EC2 P2x16 systems and it's been really slow to start and really slow between tests. To be clear, the test itself runs quite quickly, but then it just pauses for a long time until it starts the next test.

I just checked one and it sat for roughly 87 seconds from the end of one test to the start of the next. It was similarly slow at the very start as well, which i initially wrote off as just being a a startup thing but now i'm guessing it's the same issue.

Any ideas? I'd like to benchmark the entire set of suites for posterity but i'm paying by the hour here, and considering how many hash types there are and what AWS charges for one of these it's a not-insignificant amount of money.

Ideas welcome.

[Xanadrel]

Likely compiling kernels.

[royce]

Theoretically, could you generate those kernels on an otherwise identical setup, and then copy them to the kernel cache area?

Also, if it's the kernel generation, I would expect the delay before any given benchmark would only happen once, and then that kernel would be cached.

Do you have a view into what resources the system is consuming during the long pause? (CPU? GPU? I/O?)

[pragmatic]

Well that makes perfect sense. I do indeed see new kernels popping up.

I'm surprised it takes this long to compile kernels though given what a beast this machine is.

[pragmatic]

hrmm, doesn't seem to be the compilation. I stopped the benchmark and started again and even with kernels built it's still got a huge delay.

Any other ideas?

[royce]

Any view into where resources are going during the delay? Can you strace it to see what hashcat is doing in that moment?