As you know my lab got an addition this year, the Supermicro's Single CPU board a X10SRH-CLN4F model. In this post, we will be upgrading LSI 3008 HBA on the X10SRH-CLN4F. It is a board with four 1-Gigs Intel NICs, IPMI, and lots of other features which I previously missed on consumer based boards. It's certainly the memory limit which I needed to blow.
Also, I wanted to go as energy efficient as possible but still to have something “future proof” with a good performance as well. The board supports up to 512 Gb RAM, but DDR4 is still very expensive so there is 64Gb of RAM for the moment. You might want to check the series from the start – Efficient Home Server – Start with an Efficient Power Supply.
Update 3: I just found out that my LSI 3008 controller which is used by VSAN is still “Green” via VMware HCL health checks. This controller is a SAS/SATA controller, 12GBps which can drive up to 8 disks. In my case, I use it for vSAN and right after the upgrade, the controller seems to fall off the HCL, but a few months later, it's back again.
Note that while this controller can do software raid, I'm not using it. I'm passinng through the disks into my vSAN so it' is vSAN cluster which has a full management of the devices. You might be interested in reading this post about the latest lab upgrade to ESXi 6.5 and vSAN 6.6.
It seems that while VMware does not have priority on this Supermicro Motherboard (which is understandable), the controller itself is still pretty valuable.
I also tried to use the new vSAN CLI command:
esxcli vsan debug controller list
Which basically gives you not only the details about the controller, driver and firmware but also the queue depth… -:) Pretty cool. This is the way VMware is going forward. Latest news from VMworld 2017 shows that Ruby vSphere Console will be phased out in the future and VMware will pack more CLI commands and PowerCLI commandlets in order to interract with vSAN clusters.
Updated view the September 24th:
The board's built-in LSI Logic Fusion-MPT 12GSAS SAS3008 PCI-Express which has been recently added on VMware HCL for ESXi 6.0 has queue depth 600 (same as my other Dell Perc H310 flashed with IT firmware…). The difference is that the LSI 3008 is 12 GbpS SAS where the H310 is 6Gbps only… You can check yours too – how to check queue depth of your storage adapter.
This is the first time that i'll be flashing a board from UEFI mode and since I had to seek hard for the solutions I thought that spreading the knowledge is a good way to do. Now you might say, why would you bother to flash via UEFI if it can be done via bootable DOS USB stick? It can't. I tried several times and each time it throws an error….
If you receive a “Failed to initialize PAL” message while trying to flash an LSI controller card, it is most likely due to your motherboard having limited Option ROM. And the advice I found of few sites was “You will need to find another motherboard to use to flash the controller card”. Since it wasn't possible as the LSI 3008 is directly on the board…
The UEFI was something new and only to find out how to change to the USB stick within the UI it wasn't easy as the USB drives starts with fsx… like fs0, fs1 etc… so to change to the USB stick from within the UEFI you just do for example:
And then simply dir to the directory where you have copied your firmware upgrade files.
Upgrading LSI 3008 HBA on the X10SRH-CLN4F – the necessary steps:
1. Format a USB stick and copy there the firmware upgrade files. I have downloaded the files from Supermicro's ftp site:
2. Boot your system by hitting CTRL-C to enter the LSI Bios. Once there write down the last 9 digits of the card's
3. Reboot and this time hit F11 to choose the UEFI: Built-in EFI Shell
4. You'll get to the (famous) EFI shell where you need to change to the USB and run a single command to upgrade the firmware… Yes single command, so it's actually more simple than the upgrade I've done to my Dell Perc H310 via DOS.
The EFI shell on the Supermicro board looks like this
5. Then according to the documentation, just run SMC3008T.nsh and follow the instruction to enter the last 9 digits/characters of the SAS address.
I have also recorded a video which details the whole process. So this time you'll see it going live… -:)
Watch in HD and Full Screen…
There are 8 RAM slots on the motherboard which allows boosting the single server up to a total of 512 GB of RAM when using DDR4 LRDIMMs. Or when used with lower capacity DDR4 1.2V-2133 RAM ECC sticks (8 or 16Gb RAM sticks) you can go up to 64Gb or 128Gb of RAM respectively.
In the same category at Supermicro, you can find a board less expensive, with only 2 Intel 1Gb NICs, or a board without the storage controller from LSI. Here are the 3 Single CPU boards for 2011-3 architecture with DDR4 8 memory slots, (all boards are ATX format):
- Supermicro X10SRH-CLN4F-O (with LSI 3008 SAS, SATA, IPMi) – the board I use
- Supermicro X10SRH-CF-O (with LSI 3008 SAS, SATA, 2 NICs, IPMi)
- Supermicro X10SRI-F-O ( with 2 NICs, IPMi)
The whole journey:
The 3 lab hosts I have now in the lab (https://www.vladan.fr/lab) are all based on X10SRH-CLN4F board which can go up to 512Gb RAM…
- Efficient Home Server – Start with an Efficient Power Supply
- ESXi Home lab – Get a quality and comfortable case
- Supermicro Single CPU Board for ESXi Home lab – X10SRH-CLN4F
- Supermicro Single CPU Board for ESXi Home lab – X10SRH-CLN4F – Part 2
- Supermicro Single CPU Board for ESXi Home lab – Upgrading LSI 3008 HBA on the X10SRH-CLN4F – Part 3 – this post
- Building Energy Efficient ESXi Home Lab – parts list
- Homelab – Airflow Solar System
I have learned a new way to patch via UEFI. In fact, it's same (or easier) than through DOS-based bootable USB. The IT firmware can be reverted back to IR firmware as in the ZIP package there are both versions there. So in case you need a server with hardware RAID, you can use the IR version. I was actually wondering what it means the IT and IR and here is what I have found at LSI (Avago) website:
“IT” firmware maximizes the connectivity and performance aspects of the HBA. “IR” firmware offers RAID functionality via RAID 0, 1, and 10 capabilities.
Exactly what I thought.
More from ESX Virtualization:
- Free Tools
- Dedicated vSphere 6.5 Page
- What’s New in vSphere 6.5 – New Training Course from PluralSight
- How-to unlock the VMware VCSA root password?
- What is The Difference between VMware vSphere, ESXi and vCenter
- What is VMware Platform Service Controller (PSC)?
- VMware Mirage Server Infrastructure [Guide]