Distributing workload in oclHashcat-lite

The goal in cloud / cluster / distributed computing is to split the workload into smaller chunks and then to distribute these chunks to the single nodes where they are computed. A server marks chunks as reserved and keeps track of the current position in they keyspace. When a node is finished it informs the server and requests a new chunk.

This goes on and on till the whole keyspace is scanned.

oclHashcat-lite, starting with v0.7, comes with two parameters that control how to pin and size such a specific chunk from the keyspace.

These parameters are:

• --pw-skip : This parameter defines the position from where to start the calcuation.
• --pw-limit : This parameter defines the position from where to stop the calcuation.

The keyspace is the result of the combination of charset ^ length (simplified), it is mandantory that this keyspace never changes across all nodes.

See the bottom of the page how to calculate the keyspace.

NOTE

• --pw-skip is also used in restore, but we can ignore this fact for now.
• Crashed sessions using chunks can get restored like normal sessions
• If you are using oclHashcat-lite v0.6 these parameters are also there, but they work different. Upgrade to v0.7!
• oclHashcat-lite can not verify all nodes use the same keyspace, you have to ensure that.
• --pw-min and --pw-max must be equal, otherwise you can not know the keyspace.

We have the following sitation:

• Keyspace with mask “?d?d?d?d”. This results in 10 ^ 4 (1000) combinations
• Four nodes, all of the same speed

OK, this is an very uncommon example, but its perfect because it is easy to explain how to work with chunks.

1. Since all nodes have the same speed, all we need to do is to divide the number of combinations by the number of nodes: 1000 / 4 = 250.
2. Each node has to process 250 combinations. This means:
   • The offset (--pw-skip) for each node is 250 added to the offset of the previous node
   • The size (--pw-limit) is added 250 from the offset.

This results in the following command sets:

• Node 1: request work from server, gets --pw-skip 0 --pw-limit 250, server marks 750 left
• Node 2: request work from server, gets --pw-skip 250 --pw-limit 500, server marks 500 left
• Node 3: request work from server, gets --pw-skip 500 --pw-limit 750, server marks 250 left
• Node 4: request work from server, gets --pw-skip 750 --pw-limit 1000, server marks 0 left
• Node 1: request work from server, stops because server 0 left
• Node 2: request work from server, stops because server 0 left
• Node 3: request work from server, stops because server 0 left
• Node 4: request work from server, stops because server 0 left

Easy, isnt it?

In a real-life situation its much more complex. For example, we have Node 1 used by the user playing some FPS game which uses the GPU a lot and Node 4 has only budget GPUs. In other words we just dont know how fast each node is, especially not over time. A good strategy is to choose a bit smaller chunks.

• Smaller chunks create more overhead, but also more scability and safety.
• Lets say our keyspace is still 1000, but we maximize the chunk size to 100
• Notice Node 1 and Node 4 are slower.

This results in the following command sets:

• Node 1: request work from server, gets --pw-skip 0 --pw-limit 100, server marks 900 left
• Node 2: request work from server, gets --pw-skip 100 --pw-limit 200, server marks 800 left
• Node 3: request work from server, gets --pw-skip 200 --pw-limit 300, server marks 700 left
• Node 4: request work from server, gets --pw-skip 300 --pw-limit 400, server marks 600 left
• Node 2: request work from server, gets --pw-skip 400 --pw-limit 500, server marks 500 left
• Node 3: request work from server, gets --pw-skip 500 --pw-limit 600, server marks 400 left
• Node 2: request work from server, gets --pw-skip 600 --pw-limit 700, server marks 300 left
• Node 3: request work from server, gets --pw-skip 700 --pw-limit 800, server marks 200 left
• Node 1: request work from server, gets --pw-skip 800 --pw-limit 900, server marks 100 left
• Node 4: request work from server, gets --pw-skip 900 --pw-limit 1000, server marks 0 left
• Node 2: request work from server, stops because server 0 left
• Node 3: request work from server, stops because server 0 left
• Node 1: request work from server, stops because server 0 left
• Node 4: request work from server, stops because server 0 left

Example 2 shows that it is not that important to know how fast each node is.

But one important question still remains. How do you know how big the keyspace is? There are two ways:

• Calculate the keyspace by the mask, see Calculating total combinations for masks
• Let oclHashcat-lite calculate it for you. If you press “s” to get the status display, the keyspace can be found in the Progress….. line.