04-29-2014, 02:02 PM
I'm looking to expand a render farm that uses Octane Render on a bunch of NVidia GPUs. Right now, the most cost effective card for our workload is the GTX 690, and our most efficient render node is an Intel P4304CR2LFKN with four GTX 690 cards and a single Radeon 5450 PCIe x1 for display output (the NVidia Windows driver can only handle eight GPUs when something else is providing display). A Tyan FT77A-B7059 with seven GTX 780Ti cards is about 20-25% faster, but costs twice as much. However, the P4304 system costs over $4k before GPUs, and I'm looking for something more affordable. Otoy is about to release Octane Render 2.0, which, among other things, adds support for network rendering, which means that we won't be tied to Windows anymore, and as far as I can tell, the NVidia Linux driver doesn't suffer from the same GPU limit as the Windows one - I haven't had an opportunity yet to try it on Intel P4304, but on the Tyan FT77A, it was able to handle eight cards rather than Windows' maximum of seven.
There are more than a few boards with 4-7 PCIe slots currently on the market, and it shouldn't be hard to put together a sub-$2k base system (not including GPUs), but I'm afraid into running into compatibility issues/resource exhaustion with this many GPUs in a single system, and manufacturers typically don't have any info regarding such edge case configurations - typically, large GPU systems are only certified with Tesla, which is prohibitively expensive for our use case.
So, my question - does anyone have positive experience running 4x GTX 690 cards on a desktop class (single socket) motherboard?
While at it, I'm also trying to locate a 3U/4U rackmount case that will accept a 1500-1600W PSU (4x GTX 690 pull about 1100-1200W in operation), an ATX motherboard, 2-4 120x38mm high airflow (6k+ rpm) fans, and, most critically, at least an inch of clearance on the left side (looking from the front) of the motherboard, allowing a dual-slot card in slot #7 to overhang the motherboard edge.
Thanks in advance.
There are more than a few boards with 4-7 PCIe slots currently on the market, and it shouldn't be hard to put together a sub-$2k base system (not including GPUs), but I'm afraid into running into compatibility issues/resource exhaustion with this many GPUs in a single system, and manufacturers typically don't have any info regarding such edge case configurations - typically, large GPU systems are only certified with Tesla, which is prohibitively expensive for our use case.
So, my question - does anyone have positive experience running 4x GTX 690 cards on a desktop class (single socket) motherboard?
While at it, I'm also trying to locate a 3U/4U rackmount case that will accept a 1500-1600W PSU (4x GTX 690 pull about 1100-1200W in operation), an ATX motherboard, 2-4 120x38mm high airflow (6k+ rpm) fans, and, most critically, at least an inch of clearance on the left side (looking from the front) of the motherboard, allowing a dual-slot card in slot #7 to overhang the motherboard edge.
Thanks in advance.