Creating a Nested ESXi 5 Environment

Since vSphere 5 was released earlier this year, it is now time to get ready to pass the VCP exam.  I am a VCP for both 3.5 and 4, and decided I would post some updates along the way as I study before taking the VCP 5 exam.

The first thing I did was create a lab environment. I currently use 2 whitebox ESXi 5 servers, attached to iSCSI storage.

However, since I am studying VMware products, I don’t want to configure, and reconfigure these 2 ESXi servers constantly.  Thankfully, I can run Nested ESXi Servers – or, run ESXi 5 inside of ESXi 5.  I can even run 64bit guests inside of my virtualized ESXi servers!  See below:

Now, I can run more than 2 ESXi servers, and blow them away and recreate if needed. (yes, it will be slow).

In order to run ESXi 5 inside of ESXi 5, there are a couple of steps we need to do:

First, we will need to add a line to the etc/vmware/config file.  Using SSH to connect to your ESXi server, type the following:

echo ‘vhv.allow = “true” ‘ >> /etc/vmware/config

Second, we need to make sure the networking is setup properly.  On each host, for your VM Network, enable promiscuous mode for the port group.

• click Configuration
• click Networking
• click Properties for your vSwitch with the VM Network (assuming your haven’t renamed”
• Click “VM Network” and choose Edit.
• Click Security
• Check Promiscuous Mode and choose Enable.

Next, create a VM for each ESXi host with 2CPUs, and at least 4GB of RAM. When creating the VM, make sure that it is version 8 HW, and that for the operating system you choose “other” and select “Other (64 bit)”.  Once the VM is created, we will choose to edit it’s settings.  In the Options tab, you can now change the OS to “VMware ESXi 5.x”

Last, we need to make sure that under “CPUID Mask” that we expose the NX/XD Flag to the guest if it is not already.

Now, you can power on the VM, and install ESXi 5 from an ISO image.

Once installed, you can then create 64bit guests inside of your virtual ESXi server.

 VMware Workstation 8

The best thing about VMware, is I can also virtualize ESXi inside VMware Workstation!  So on my laptop, in VMware Workstation, I can run multiple ESXi servers, that in turn have VMs inside them.  Even better is the fact that I can have 64bit guests inside my ESXi servers running inside VMware Workstation.  Sounds confusing as I type, so here’s a diagram:

In order to run ESXi inside VMware workstation, we simply need to select “VMware ESXi 5″ as the operation system of the guest VM we are creating.

Then, Edit the VM’s Settings, and under the Hardware tab, and processors, check the box to Virtualize Intel VT.

The option to run ESXi inside of VMware workstation is great, especially for when you don’t have a lot of physical hardware to test with.