Intel NUC 7th Generation with Thunderbolt Ethernet on ESXi 7

My 3 host ESXi 6.5 home lab was built a couple of years ago now in order to develop some vSAN test scenarios that I was assisting a client with. Now that lab is in the process of being repurposed to become an NSX-T / vSAN lab running ESXi 7.0.

My three NUCs are based on the 7th Generation Intel NUC7i5BNH with 32GB RAM, onboard M2 NVMe SSD cache disk (256GB) and a 512GB SSD capacity disk.

– By the way, in case you’re wondering – it flies, even over 1Gbit ethernet with all-flash vSAN and deduplication/compression turned on. However – don’t use something like this for production as you’ll need to be prepared to lose ~14GB RAM per host for the dedupe in-memory object map and all-flash requires 10Gbit ethernet!

However, back to the issue in hand. One of the first issues I ran in to when assessing the requirements for NSX-T in a fully collapsed cluster (running vSAN, vCenter and NSX Manager) was a need to have two physical network interfaces (pNICs) but my Intel NUC 7th Gen hosts only have a single on-board gigabit ethernet adapter. This isn’t front page news of course as William Lam has been documenting the use of USB based ethernet in lab scenarios for quite some time, originally resulting in a VMware Fling he coauthored with another device driver engineer (Songtao Zheng) at VMware.

Now I began to question with recent releases such as VMware ESXi 7.0 whether any of the drivers or settings mentioned would even be required in order to get a second ethernet adapter working. This post is really just a signpost for people who might be doing similar things in their own labs.

I decided to jump in with both feet by purchasing three Startech USB-C 1-Gbit Ethernet adapters from Amazon UK. These devices use the Realtek RTL8153 chipset. My NUC devices have Thunderbolt 3 interfaces (with the lightning bolt marking) but I was pretty sure that the USB-C connector would work as the interface supports USB 3.1 Gen2 devices.

UPDATE YOUR BIOS – mine hadn’t been touched since 2017 so I went online first and downloaded the latest update for my NUCs, then flashed using a USB key with the .bio file and F7 key during boot.

UPDATE YOUR ESXi release – reading one of William Lam’s posts I found that there was a recent patch release of ESXi 7.0.0b which includes an updated USB driver:

VMware-vmkusb_0.1-1vmw.700.1.25.16324942

As far as I can tell this includes the ability to detect Thunderbolt connected devices amongst other improvements, but this awareness certainly negates the need to disable any existing VMware USB driver (which older posts I’d read had discussed prior to installing the USB ethernet fling).

INSTALL the VMware ESXi 7.0.0 release of the USB Fling:

esxcli software vib install -d '/vmfs/volumes/QNAP_VMFS_DS01/tmp/ESXi700-VMKUSB-NIC-FLING-34491022-component-15873236.zip'

USB Native Driver Fling for VMware ESXi | 0.1-4vmw.700.1.0.34491022

3. Shut down the ESXi host, you’ll need to go into the BIOS at next boot

ENABLE THUNDERBOLT BOOT in BIOS -until you do this you won’t be able to see any USB 3.x network devices. William Lam again has the lead again, with this linked post concerning Thunderbolt 10Gbit adapters on Intel Skull Canyon devices. Enter the BIOS and enable THUNDERBOLT BOOT.

Before enabling this feature you’ll find that 'lsusb -tv' will only show a single USB XHCI root HUB:

4. Save your BIOS settings, connect the Startech USB device and boot ESXi. Once reloaded compare the ‘lsusb -tv’ result with the previous version.

NB – If you find that your USB adapters are only connected at 100Mbit/s then it’s likely that the default ESXi 7.0.0 drivers have been loaded instead of the ones provided in the Fling. You’ll also see that the adapter name is detected as ‘cdce’ instead of ‘uether’. In this case make sure that the drivers are installed correctly and try a reboot with the adapter connected.

[OPTIONAL] if you have any 10 Gbit Thunderbolt adapters you could also use the following steps to add the Marvell drivers. I haven’t actually acquired any of these yet, but the instructions should be good as I’ve tested the installation process itself.

INSTALL the VMware release of the Marvell Atlantic USB driver:

  1. Download the .zip file and upload it to a datastore that your hosts can access.. e.g. /vmfs/volumes/58134191-c9bf8fe8-d464-d067e5e666da/tmp/MRVL-Atlantic-Driver-Bundle_1.0.2.0-1OEM.670.0.0.8169922-offline-bundle-16081713.zip
  2. Enter maintenance mode and vacate the ESXi host
  3. Install the offline bundle VIB using:
esxcli software component apply -d /vmfs/volumes/QNAP_VMFS_DS01/tmp/MRVL-Atlantic-Driver-Bundle_1.0.2.0-1OEM.670.0.0.8169922-offline-bundle-16081713.zip

Native atlantic network driver for VMware ESXi | 1.0.2.0-1OEM.670.0.0.8169922