I was playing a lot with Nested ESXi 6 in a lab recently. But it does not matter, we can do quite a lot with a 64 Gigs of RAM and bunch of SSDs… What's new in this release is the fact that VM tools are installed by default so the nested ESXi behaves like any other VM with VM tools installed right when you finish your ESXi 6 Setup. There is nothing else to do. Previously one had to install small vib called VMware Tools for Nested ESXi (used in ESXi 5.x).
Folks willing to run All-in-one nested lab with bare metal ESXi can now do a simple install of ESXi and then just few simple tweaks to start building complete nested ESXi environment on the top. Since it's been a while (I believe I've done it for v 5.0 in the past) and the UI has changed over time, let's recap those steps which are necessary to build a Nested ESXi 6 in a Lab.
Nested ESXi 6 in a Lab
Step 1: Install the physical server with ESXi first. Simple enough and few days to do it….. You can do the installation to or from an USB key (use USB as a destination OR use USB to boot the installer), OR you can use Workstation or Player to pre-install the ESXi 6 hypervisor (and then plug this pre-installed USB into your server). If you're upgrading from 5.x follow this simple guide.
Step 2: Connect to your ESXi host via vSphere client and go to the Configuration tab > Networking > double click vSwitch properties > Security Tab > change Promiscuous mode to Accept. Make sue that Promiscuous mode is set to Accept and also Forged Transmits is also set to accept (everything is set to accept, actually). If you don't change the setting to accept you won't be able to pass traffic through to the nested ESXi VM's.
Step 3: Deploy your VM(s) to assure the core network services for your lab like DNS, AD, DHCP … and Install VMware vCenter server 6 (VCSA). Note that there is also simplified scripted installation of vCenter Server (VCSA) available and it's really simple – check my post on it! Once you have vCenter you can start build the nested ESXi 6 hosts.
Start new wizard for creating new VM > choose meaningful name (ESXi6 in my case) > Select host > select storage > and select Guest OS family, choose Other.
Click finish and then modify the VM's properties with 2 CPU and 8Gb of RAM and also check the Expose hardware assisted virtualization to the guest OS
You'll certainly want to use thin provisioned disks (if you don't expand the disk properties and don't changes, by default it will create thick format disk). And another change you'll want to do is to change to VMXNET3 which has performance and has less overhead then E1000 based NIC.
Oh, don't forget to attach your ISO to the ESXi and click the connect check box, otherwise it won't boot the iso.. -:).
That's about it. You have deployed vCenter server and your first ESXi 6 host (as a VM). You now have a solid base to start building a virtual datacenter, play with VSAN and other cool features that vSphere 6 provides even if you only have single ESXi host.
On the image below you can see the ESXi 6 VM running with VMware tools…
Nested lab does not have a speed of real lab (everything is about cost), but to discover and learn VMware technology, to prepare for certification exam, nested lab is a good start. Unless you go and take Online lab (remote). RAM is the first resource which gets exhausted and it's also the RAM which you need to do anything. If you building nested lab, get at least 64Gb! Make sure that you run everything on SSD. This should not be a problem today as prices are falling and price per Gb is getting more and more affordable.
Check back often for my Lab page where you'll find the full setup of the 2015 lab and also check back for vSphere 6 page where new how-to, videos, and news articles about vSphere 6 are located.
Once the vm is created, if you edit the settings, you have the option to change the guest from “other” to “ESXi”.
No the VM is created with Other as Guest OS AND with choice ESXi 6 for Guest OS version… Note that ESXi 5.x or 4.x are other options there too. (Web client only)…
Could I run everything in a VM using VMware Player or Virtualbox from a Windows host? I dont’ have a dedicated machine to act as my initial host, but was looking to setup multiple virtual hosts to get more vSphere 6 experience
Depends on what you mean “everything”. But yes you can basically use Workstation/player to create virtual instances of ESXi hosts and create a fully virtual cluster-in-a-box. You should check http://www.vladan.fr/nested-lab/ to download the e-book where I’m detailing the steps. (btw. for now it’s not been updated for vSphere 6, but the basics remains the same)…
Great. Subscribed.
Was wondering if I could solicit some feedback/advise on how the best way to setup a VMware lab. I have the following servers that I will be installing into my lab:
1) Two servers with the following specs – 4C/8M, 2.4GHz, 8GB, 1.8TB, 1x Broadcom 5716 2x1GbxRedundant
2) Two servers with the following specs – 4C/8M, 2.4GHz, 16GB, 3TB, 2x Broadcom 5709C 2x1GbxRedundant
Any advice would be greatly appreciated
Thx
Dear Sir,
I only have one ESXi 6.0 and no vSphere web client.
Would it be possible for you to share the setting in vmx file about enable hardware virtualization (Expose hardware assisted virtualization to the guest OS)?
Thanks.
Best regards,
Swooshx
Hey gurus,
I have a box with CPU VT Virtualization enabled on, installed ESXI 6 + VCSA 6 and trying to nested ESX5.5. I have installed Windows 7 64 (ESX 5.5) bit and I cannot make it boot, in fact any other 64bit instances within the ESX 5.5.
secondly, I had installed within the VCSA 6 Windows 2012 Datacenter and I tried to boot an Windows 7 64bit within the HyperV 2012 instance and it cannot boot. So what I’m missing ?
I followed the article http://www.virtuallyghetto.com/2012/08/how-to-enable-nested-esxi-other.html
http://www.virtuallyghetto.com/2011/07/how-to-enable-support-for-nested-64bit.html
but it didn’t help
any ideas?
Well I found out for windows 2012 server but now i will try the ESXi5.5
ESXi 6.0
Virtualized HV is fully supported for virtual hardware version 9 or later VMs on hosts that support Intel VT-x and EPT or AMD-V and RVI. To enable virtualized HV, use the web client and navigate to the processor settings screen. Check the box next to “Expose hardware-assisted virtualization to the guest operating system.” This setting is not available under the traditional C# client.
Great description and definetly not the only one who safes a LOT of time for me. Many thanks.
Great guide!
With 6.0U1 I was having a problem that the nested ESXi did not detect the NIC when choosing vmxnet3. If you select E1000 everything works fine.
Thanks. This is good to know!
Hi Vladan ,
I had installed ESXi 6.0 on top of ESXi 6.0. everything went right but guest is not able to get ip address from dhcp server. all other VMs (other than ESXi) are getting IP address. What may be the issue? Please help me.
Give the vESXi fixed IP and try to ping other VMs first…. to check the connectivity.
I have this same issue! I’ve done a lot of testing the past few days and I found I’m unable to get a dhcp address on the second or more vmk interface. I’ve attached the nic vmk0 is on to multiple subnets with dhcp servers, it gets an address without issue. I attach a second nic, use that on a new vswitch or same vswith with portgroup override on the nic, and the vmk interface created fails to get a second dhcp address. This doesn’t happen when using a physical server. For some reason, more than one vmk getting DCHP fails regardless of vswitch or portgroup assigned. I’ve made sure to enable forged addresses, etc on the vmnic backing too. Very strange, I’m troubleshooting still because I want to figure this out!
don’t supposed you ever figured this one out. I’m just now running into it.
Is it possible to deploy esxi 6 nested on esxi5.x base ?
Thank this document.
But there is a problem to add host.
I installed esxi 6 on physical machine, and install on vcsa6, windows 2008 as AD,DNS.
Also, 3 esxi6 VM for lab.
But tried a lot, change Promiscuous mode to Accept in all esxi6.
Could not add a host to vCenter.
When I clicked finish button, error said “Cannot contact special host 10.0.0.101………”
I can ping esxi vm by ip and name (esxi101.company.local).
Please help, thanks.
Start new wizard for creating new VM > choose meaningful name (ESXi6 in my case) > Select host > select storage > and select Guest OS family, choose Other.
Before creating new VM, you need add host.
How you added host?
Thank.
David,
before, you must create the ESXi VM from scratch. Then only you can add it to your cluster (as ESXi host…). Does it make sense?
I have Dell physical server, installed ESXi6, presented 500G iSCSI storage from Equallogic array.
Installed vCenter 6.
when trying to create 3xESXi6, which storage and I should select? and how datastores will be shared among ESXi6?
Do I need to create solarwind iscsi storage for nested ESXi?
Thanks
Are the virtual machines of the nested ESXi host able to run in 64-bit mode, i.e. could I run Windows Server 2012/R2 in the nested host? (Windows Server 2012/R2 is available only in 64-bit.)
Thanks.
Yes, you can do use x64 VMs in nested, however your underlying CPU hardware must support the Intel-VT… Most of them does.
i have installed ESXi 6 Hypervisor on Physical Server , On ESXi 6 Hypervisor created a virtual machine as ESXi 6.0 . On ESXi 6.0 created Windows 2008 VM Os but when i try to ping to Windows 2008 it is not pinging , Kindly guide , Promiscuous mode and Forged Transmit is selected Accept on Physical ESXi Box. Kindly guide what setting will make the Windows 2008 virtual machine ping
I will explain my test lab , I have HP server installed vsphere 6.0 Hypervisor On Phyical server created Guest Virtual machine as ESXi 6.0. On Guest ESXi 6 virtual machine created Windows 2008 VM which i am not able to ping. Promiscous and forget transmit is set to Accept , Using 192.168.20.x ipaddress for virtual machine and Host, Windows firewall is also disabled
Hi
I have ESXI host LAB ver 6.5 with vCenter and 5 Nested ESXI (ESXI-1,ESXI-2,ESXI-3,ESXI-4 and ESXI-5) and 1 Nested vCenter installed on NESTED ESXI-1 and cisco SW L3
All Nested ESXIs on specific port group called (NESTED-ESXI Network) and all option in security options for this port group is Accept .
My problim there is no connection between Nested ESXIs for that I can’t add nested host to nested vcenter
plz advice me.