+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Misc (https://hashcat.net/forum/forum-15.html)
+--- Forum: Hardware (https://hashcat.net/forum/forum-13.html)
+--- Thread: infiniband setup guide? (/thread-2379.html)
Pages:
1
2
infiniband setup guide? - alex - 06-18-2013
Hi,
Can anyone point me to the working infiniband setup guide for using vclhashcat?
I've tried several guide that found in google, found this one is quite easy:
http://pkg-ofed.alioth.debian.org/howto/infiniband-howto-4.html
And then 2 machines can ping each other, but when running vclhashcat, it only detect 3 cards in local computer, then kernel, and just frozen there
eg:
Device #1: Cayman, 1024MB, 800Mhz, 24MCU
Device #3: Cayman, 1024MB, 800Mhz, 24MCU
Device #3: Cayman, 1024MB, 800Mhz, 24MCU
Device #1: Kernel ./kernels/4098/m0000_a1.Cayman_938.2_CAL 1.4.1741.kernel (490316 bytes)
then it does nothing.
Help please
Will reward if required.
RE: infiniband setup guide? - blandyuk - 06-18-2013
epixoip is the man u need
RE: infiniband setup guide? - epixoip - 06-19-2013
VCL only works with TCP/IP, so to use Infiniband with VCL, you need to do IP over Infiniband.
If you're using Debian/Ubuntu, it's very simple.
Add the following to /etc/modules
Code:
## ipoib
ib_sa
ib_cm
ib_umad
ib_addr
ib_uverbs
ib_ipoib
ib_ipath
ib_qibThen add the following to /etc/network/interfaces (making appropriate changes for each host):
Code:
auto ib0
iface ib0 inet static
address 192.168.1.1
netmask 255.255.255.0
post-up echo connected >`find /sys -name mode | grep ib0`
post-up echo 65520 >`find /sys -name mtu | grep ib0`Then you can bring everything up without rebooting:
Code:
for m in ib_sa ib_cm ib_umad ib_addr ib_uverbs ib_ipoib ib_ipath ib_qib; do modprobe $m; done
ifup ib0Now you should be able to ping the IPv4 addresses using your normal ICMP ping, scp files back and forth over Infiniband, etc. And most importantly use VCL.
Oh, and I need to mention that the ip space that you use for your Infiniband network needs to be completely separate from your Ethernet network's ip space. Else you'll run into massive issues like Rurapenthe did.
So for example, if your Ethernet network is 192.168.0.0/16, then your Infiniband network should be anything but that. Maybe make it 10.0.0.0/24 or 172.16.0.0/12 or something.
It will NOT work if you use the same address space for both Ethernet and Infiniband.
RE: infiniband setup guide? - alex - 06-27-2013
Hi, epixoip
Everything seem to be okay now. I can ping, can transfer files between nodes.
But when I run oclhashcat-plus or oclhashcat-lite, It only detect cards, and then freeze there, doing nothing.
Device #1: Cayman, 1024MB, 800Mhz, 24MCU
Device #2: Cayman, 1024MB, 800Mhz, 24MCU
Device #3: Cayman, 1024MB, 800Mhz, 24MCU
Device #4: Cayman, 1024MB, 800Mhz, 24MCU
Device #5: Cayman, 1024MB, 800Mhz, 24MCU
Device #6: Cayman, 1024MB, 800Mhz, 24MCU
Please help
RE: infiniband setup guide? - epixoip - 06-27-2013
please provide the following:
- description of physical network topology
- /etc/network/interfaces from broker & one compute node
- /etc/vcl/nodes from the broker node
RE: infiniband setup guide? - alex - 06-27-2013
Thanks for answer.
- physical network topology:
3 PCs, 1 broker, 2 nodes, all of them connect to an ASUS router via LAN to access internet, IP is 192.168.1.11 , .12 , .13 and connect to a infiniband switch , address 10.10.10.11, .12, .13
ibstat give interface is active , all machines can ping each other on infiniband IPv4
Broker & node are same as I clone them BEFORE setup vcl
of course on nodes, address on /etc/network/interfaces is 10.10.10.12 , other node is 10.10.10.13
/etc/network/interfaces on
Broker:
auto lo
iface lo inet loopback
auto ib0
iface ib0 inet static
address 10.10.10.11
netmask 255.255.255.0
post-up echo connected >`find /sys -name mode | grep ib0`
post-up echo 65520 >`find /sys -name mtu | grep ib0`
/etc/vcl/nodes:
10.10.10.12
10.10.10.13
RE: infiniband setup guide? - epixoip - 06-28-2013
looks good from here.
what algorithm are you cracking? how many hashes are you targeting? what is your -n value? how much memory does your broker have?
RE: infiniband setup guide? - alex - 06-29-2013
I use the example in guide
http://hashcat.net/wiki/doku.php?id=vcl_cluster_howto
./vclHashcat-plus64.bin -t 32 -a 7 example0.hash ?a?a?a?a example.dict
and
./vclHashcat-lite64.bin -b --benchmark-mode 1 -m 1000 --force
both of them have the same result.
RE: infiniband setup guide? - epixoip - 06-29-2013
what version of vcl are you using?
RE: infiniband setup guide? - alex - 06-30-2013
Hi,
I'm using the latest one, v1.21. can't find v1.17.
Last time I run with v1.20 , with normal router, not Infiniband, it works, but about 900MB/s only