GPU servers?

Having your own datacenter gives you great flexibility. Especially when you have a highly efficient datacenter with low PUE and decent electrical pricing.

Lately we have been thinking about GPU servers, as we already have some questions for them and several potential users for them.

Looking at the current GPU market, miners are taking the great bulk of the AMD GPUs, driving their prices upwards. Still, there are deals to be had with older gen AMD GPUs if emphasizing more on the lower acquiring cost vs. electrical consumption. For example, HD 5970s which are Dual GPU cards which are rather old now can be had for very cheap compared to their computing power, sometimes even new. Same goes for the first gen Tahiti GPU, HD7970 which was a beast of a card. Current gen RX560-580 GPUs are still fetching a premium however.

How about Nvidia cards? Well, miners are not looking to get them in as great numbers due to their lower performance per cost. So some great deals can be acquired from there as well. However, the current 1070, 1080 are fairly expensive cards to acquire and will quickly be the determining factor in system cost. Some older generation Tesla systems are occasionally on the market for extremely cheap as semi barebone systems, but comparing their processing power is very hard and are likely going to have a premium versus using gaming GPUs for computing. At least for many tasks. Older gen GPUs can be had fairly cheapily here as well, such as GTX 960-980 are fairly inexpensive.

However, server systems for great number of PCI-E x16 3.0 slots and bandwidth is fairly limited market, and acquiring those can be hard for non-Nvidia systems and require that we build something off commodity hardware in order to keep the total system cost sensible.

Such a chassis could be off the shelf commodity 4U chassis for ATX / EATX motherboards, a AM3+ or AM4 (Ryzen) system to utilize ECC RAM. or perhaps even an AMD FM2 APU platform? With any luck there is a suitable IPMI motherboard from a vendor like ASRock. Supermicro would be optimal but these tend to cost a great deal more. Moderate amount of CPU power, RAM capacity, and SSD storage.

Power consumption is very important, as it will be the driving factor for operational expense. A single 4 GPU system could easily take 1000W or more. As such, fairly expensive high efficiency PSUs needs to be utilized. A rack full of GPU servers can easily consume 10kw, so heat dissipation will be a major factor as well. Especially so if built really high density (6+ GPUs per chassis), we will fairly quickly see 15-20kw per rack.

Building systems like these always have great considerations to balance the capital expenditure (purchase cost) vs operation expenses (electricity, housing etc.). The typical way to do things is “do not care about capital expenditure at all, as long as the systems sell”, which is why a GPU server from the market currently easily costs at least 1000€ a month.

We have never been like that, we believe that our job is to maximize the value proposition for the users. Quickly thinking a 4×5970 system with FX-series CPU could be priced perhaps even as low as each of these cards are rated for roughly 240W, so we are looking at north of 1000W for the system, roughly 1150W all in, which is likely “off the wall” around 1300W after PSU efficiency. Add cooling to this and you are looking at 1600-1800W total consumption. A system like this could be priced around the 500-600€ per month mark in that case.

Let us know what you think about all of this? What would be Your use case, which elements matter the most?