Help Troubleshooting (8) R9 290X Hashcat Sever

[Professor]

All,

 Could use the community's help.  I'm a professor who had a student start a hashcat server (higher end with 8-dual GPUs), but was unable to finish it before he had to leave.  Now I am picking up from where he left off.  What I need help with is finishing the install to identify ALL 8 R9 290X cards.
  Know I am not a gamer, or a Linux guru, by any sense but I get by.  Where I need the help is trying to get all 8 card recognized by the OS'. 

Here is what has been done so far:
1) Installed Ubuntu Server 14.04 LTS on one drive, Windows 8.1 on the another
For the Ubuntu:
2) Installed fglrx
3) I believe the correct AMD drivers are loaded…  But could use the commands to confirm (aticonfig doesn't seem to have a --ver command)
4) Currently running oclhashcat 2.01 but no reason I couldn't bump up to 3.0 (just released)
For Windows:
5) Removed/installed drivers per the wiki page HOWTO: Upgrading AMD Drivers on Windows
6) Couldn't get 8.1 to see more then 4 cards

First Q:
Is there any limitation to running more then 4 graphic cards, in either Ubuntu LTS or Windows (8.1)?  Reason I ask is that neither seem to be identifying more then 4 cards. I Plan on having a dual-boot setup where I can run Hashcat 3.0 from both, for testing purposes…

Second Q:
Are the (8) R9 290X cards being seen by the system?
On Ubuntu here is what I am seeing:
sensors: (I think this is showing me the (8) R9 290X cards, am I correct?)
$ sensors
i350bb-pci-0d00
Adapter: PCI adapter
loc1:         +40.0°C  (high = +120.0°C, crit = +110.0°C)

power_meter-acpi-0
Adapter: ACPI interface
power1:      286.00 W  (interval =   1.00 s)

coretemp-isa-0000
Adapter: ISA adapter
Physical id 0:  +33.0°C  (high = +77.0°C, crit = +87.0°C)
Core 0:         +25.0°C  (high = +77.0°C, crit = +87.0°C)
Core 1:         +25.0°C  (high = +77.0°C, crit = +87.0°C)
Core 2:         +26.0°C  (high = +77.0°C, crit = +87.0°C)
Core 3:         +27.0°C  (high = +77.0°C, crit = +87.0°C)
Core 4:         +27.0°C  (high = +77.0°C, crit = +87.0°C)
Core 5:         +26.0°C  (high = +77.0°C, crit = +87.0°C)
Core 6:         +25.0°C  (high = +77.0°C, crit = +87.0°C)
Core 7:         +27.0°C  (high = +77.0°C, crit = +87.0°C)

coretemp-isa-0001
Adapter: ISA adapter
Physical id 1:  +32.0°C  (high = +77.0°C, crit = +87.0°C)
Core 0:         +24.0°C  (high = +77.0°C, crit = +87.0°C)
Core 1:         +25.0°C  (high = +77.0°C, crit = +87.0°C)
Core 2:         +23.0°C  (high = +77.0°C, crit = +87.0°C)
Core 3:         +26.0°C  (high = +77.0°C, crit = +87.0°C)
Core 4:         +24.0°C  (high = +77.0°C, crit = +87.0°C)
Core 5:         +26.0°C  (high = +77.0°C, crit = +87.0°C)
Core 6:         +25.0°C  (high = +77.0°C, crit = +87.0°C)
Core 7:         +25.0°C  (high = +77.0°C, crit = +87.0°C)

aticonfig --list-adapters:
$ sudo aticonfig --list-adapters
* 0. 04:00.0 Supported device 67B0
  1. 05:00.0 Supported device 67B0
  2. 08:00.0 Supported device 67B0
  3. 09:00.0 Supported device 67B0

* - Default adapter

Here is the system specifics:
Tyan B7079F77CV10HR server - DUAL-SOCKET 8GPGPU PLATFORM
Motherboard S7079GM2NR-2T S7079GM2NR
(8) SAPPHIRE 100361- 8GL Radeon R9 290X 8GB 512-Bit GDDR5 PCI Express 3.0 (Model 100361-8GL) (AMD)

[darkseid4nk]

Do you have 4g decoding enabled in the bios? Are you running 32 bit or 64 bit?

Q1: There is a limit with AMD gpus with the AMD driver. The AMd driver in linux sees 8 GPUs and 6 in windows. Currently right now I have a box on my desk with 9 Nvidia 1080s in it. Sagitta HPC sells a commercial solution with 8 GPUs in it.

Q2: Doesnt look like it. You could do 'lspci -vvv | grep AMD' to see if 8 entries show up.

Have you followed this: https://hashcat.net/wiki/doku.php?id=linux_server_howto

[Professor]

Do you have 4g decoding enabled in the bios? Are you running 32 bit or 64 bit?

Q1: There is no limit really. Currently right now I have a box on my desk with 9 Nvidia 1080s in it. Sagitta HPC sells a commercial solution with 8 GPUs in it.

Q2: Doesnt look like it. You could do 'lspci -vvv | grep AMD' to see if 8 entries show up.

You'll probably want to install AMDAPPSDK.

Have you followed this: https://hashcat.net/wiki/doku.php?id=linux_server_howto

darkseid4nnk, thanks for the quick reply!

Above 4G Decoding is enabled.

Here is the output of the lspci
$ sudo lspci -vvv | grep AMD

04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT [Radeon R9 290X] (prog-if 00 [VGA controller])
04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio
05:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT [Radeon R9 290X] (prog-if 00 [VGA controller])
05:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio
08:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT [Radeon R9 290X] (prog-if 00 [VGA controller])
08:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio
09:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT [Radeon R9 290X] (prog-if 00 [VGA controller])
09:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio

I believe this is showing only 4 of the cards (slots 4,5,8,9?)

I will see about installing AMD's SDK...  will report back when it's done.


And the link supplied was the one I started with...  no joy after doing that.

[darkseid4nk]

I think I would also like to question your hardware. What PSU do you have? The rigs I build with 8 x GPUs in them have 3 x 1600W 80Plus Platinum PSUs. Are you sure all of your GPUs are getting enough power? Ive personally built a good number of systems with 8x290x and never had a problem with ubuntu detecting less than the correct number of GPUs unless there was a problem with a GPU itself or a problem with the pcie slot. All the servers I built have all been ubuntu 14.04 + newest fglrx driver.

[darkseid4nk]

So I read right over your hardware on your first post. I would suggest to double check your BIOS. There has to be a bios misconfiguration somewhere.

[mrleau]

Do you have 1 or 2 cpu in it?

[Professor]

I think I would also like to question your hardware. What PSU do you have? The rigs I build with 8 x GPUs in them have 3 x 1600W 80Plus Platinum PSUs. Are you sure all of your GPUs are getting enough power? Ive personally built a good number of systems with 8x290x and never had a problem with ubuntu detecting less than the correct number of GPUs unless there was a problem with a GPU itself or a problem with the pcie slot. All the servers I built have all been ubuntu 14.04 + newest fglrx driver.

That's something I hadn't thought about...  I think you are on to something!!
The rig is getting 6,400 watts, plus a failover

RPSU
Output Watts
3,200 Watts (200-240Vac input)
Redundancy 2+1

BUT forced me to go back in and look at the cards physically!  The student had set up 4/8 cards with 2x4 power cables and 4/8 with 2x3 power cables (I hadn't realized that)!!  I think the second set is NOT getting the power they need!!

I will get the correct cables and check it! 

THANKS!!!

[darkseid4nk]

You're welcome.