Second Build: ESXi 5.0 AMD Whitebox Server with Passthrough (IOMMU)

Last Updated - February 27, 2019

ESXi Whitebox IOMMUOn one of our recent posts, we have talked about a white box server which can be built around $500 budget. The best thing about that server was that it was affordable and at the same time it was powerful enough as a home server. On that build, we have used the Gigabyte GA-990FXA-UD3 motherboard. It did a great job, to be honest.

However, you know that there are other options when it comes to choosing the motherboard. For instance, another AMD motherboard has caught our attention while we were building that server. And, that is the 970 Extreme 3 from ASRock. We have already told you that the reason behind choosing the previous motherboard. Because it has more ports and PCIe/PCI slots than the 970 Extreme 3.

Even though the 970 Extreme 3 has 1 less PCI-e x1 slot than the GA-990FXA-UD3 motherboard, we think it can be a good alternative motherboard for a home server. Thus we have gone and built a second server with the 970 Extreme 3 motherboard this time. Our main priority about this server was to build a white box server which comes with (IOMMU) pass-through technology. Added to that, it would have to be cheap.

Now we will share the entire server configuration with you. Then we will be explaining why we choose the particular parts later in this post.

The config of the second build:

  • CPU: “AMD FX-8120 Zambezi” @ 3.1GHz, 125W, Octa-Core, Socket AM3+
  • Mobo (Motherboard): ASRock 970 Extreme3 (you can go with the same Mobo of other brands)
  • RAM: 8GBx4 = 32GB DDR3 @1333 MHz bus speed
  • Case: 2U Rackmount from iStar, D Value D-213-MATX
  • GPU (Video Card): AMD Radeon HD6670 Low Profile from HIS, PCI-e x16
  • Power Supply Unit: Logisys PS550E12BK 550W or any 550W ones (branded would be a plus)
  • ESXi Host Video Card: Any PCI supported video card will do e.g. ATI Rage PCI card
  • NIC cards: 1xPCI Gigabit card, 2xPCI-e Gigabit cards
  • SATA Controller card (optional): 4-Port SAS/SATA PCI-e x4, LSI SAS3041E model

So, let’s see the slot setup for this server.

Slot Population of the White Box server:

  • PCI: Dual Gigabit card from Intel – Intel Pro/1000 MT NIC card
  • PCI-e x1: Realtek 8168 Gigabit NIC card, used by ESXi host
  • PCI-e x1: USB Port- Ours is 5 Port. It is used for Pass-through
  • PCI-e x4: SATA PCI-e x4/4-Port SAS, LSI SAS3041E 
  • PCI-e x16: AMD Radeon HD6670 (Passthrough to Virtual Machine)
  • PCI: Any PCI supported video card will do e.g. ATI Rage PCI card

As you know about the server setup and configuration. Lets’ get to know the reason behind choosing this setup.

Why we went for this particular setup?

The most crucial thing we wanted to achieve with our server is to get the ability to use advanced features like fault tolerance, clustering or ESXi DRS, ESXi HA etc. on our server. The most challenging part was to find a consumer level motherboard which supports all these features.

Fortunately, with the ASRock 970 Extreme3, we can enable all of these advanced features. This motherboard supports all the processors of the AMD FX8120/8150 (Octa-Core), AMD FX-6100/6200, and AMD FX-8320/8350 series. So, you can see that it has a wide range of compatible processors.

About Graphics Card Passthrough:

You should already know that it is a hectic thing to use graphics card Passthrough with ESXi. However, you can be safe if you are aware of some of the best practices about this topic. The most essential thing is that you need to reserve your RAM for the VM (Virtual Machine). It is a must. Because ESXi re-assigns the RAM from time to time. Due to this, your graphics card might die anytime. Thus, you must reserve the RAM to solve any issues regarding video card pass-through.

A Blessing in Disguise:

There is a problem with this motherboard. However, for us, it was a blessing. The thing is when we have tried to use the pass-through technology for the SATA controller, 4 ports were active from 5 of them. Now, for a lot of people, this could be a turn-off. However, in our case, we have used the unused 5th port as a local data storage unit for the ESXi host. You can also follow our path.

More about this build:

First thing is that it is a sample of a generic white box server. So, you don’t have to copy this build entirely. You can include or exclude any parts from/to this server. For instance, you don’t have to use 32 GB RAM, you can use less than that. Another one is that it has a SATA controller card. This is also an optional addition. You don’t have to use a SATA controller at all.

We are using a Cooler Master hard drive bay (converter), which is capable of making 3, 5.25″ hard drive bays into 4, 3.5″ drive bays. This has saved us from a lot of hassle as we are using a lot of hard drives. Another interesting part is that you can use any 4 port NIC card on the build to make sure to separate different traffics for our home lab.

But there’s a problem with that, you would need at least a PCI-e 4 slot. So, if you want to use it then you can’t use the SATA controller (the SATA controller needs PCI-e 4 slot as well). But as this is optional, you can ultimately use any 4 port gigabit card instead of using the SATA controller.

For your information, if you haven’t got an idea about a white box server, you can read all about it on our previous post. We have talked about it there.

For your convenience, we will share some of the screenshots below.

Check Images:

Final Verdict:

Building a home server is not easy. On top of these, most of the time, it is not cheap as well. However, we have wanted to give you something which is affordable and also powerful as a home server. The above-mentioned configuration is just an idea.

You can obviously mix it up by customizing the configuration according to your taste. You have to remember that the cost might increase depending on the configuration. However, you must need a motherboard with IOMMU support. Other than that, you can get any other compatible parts of your choice.

Reference: Donald Fountain, TheHomeServerBlog

I am Sanforde, who is interested in server technologies and home server DIY. I share useful tips and news about the latest server technologies.