Lately I was working on the redesign of my lab at home and I purchased a Cisco L3 capable switch. In my quest for a VMware vSphere homelab, I recently purchased my first Cisco switch. You could see it in my recent post called My switch adventures where I bought the wrong switch because of similarities in the device's model numbers. If you're seeking cheap L3 switch make sure you read it. The Cisco SG 300 – 10 port, a gigabit port model is layer 3 switch. It’s a switch for the SMB market with the capability to enable Layer 3 functionality via the CLI. By default the switch comes in L2 mode and the L2 to L3 change can be done only through the CLI, not through the GUI. When in L3 mode, you can assign up to 32 static routes and up to 32 IP interfaces, this means 31 routed VLANs (1 route per VLAN plus default GW) providing inter routing VLAN functionality.
This was the feature which I wanted. Creating VLANs activates the routing leaving only to configure my ESXi hosts for them. (I have 2 physical ESXi hosts and small homemade NAS device which you could read about in my article about how it’s been built for about a half of the price of commercial NAS device).
Update: The switch was replaced by his bigger brother – the SG300-28. Check the latest lab builds at https://www.vladan.fr/lab
The ESXi hosts have 3 NICs each. The motherboard’s built-in NIC is not recognized and I didn’t want to bother to hack the ISO to try to make it work. But there are built-in NIC cards which works (or) the drivers works, but they're not in the original VMware ISO, so you ‘ll have to “hack” the ISO in order your built-in NIC card to be recognized. So, the first thing to do is to connect the console cable to the serial port on the switch. I did not have a Hyper terminal on my W7 so I went an fired up one of my Windows XP VMs through a VMware Workstation and connected through the COM port with a serial cable. The default login/password combination is cisco/cisco.
Note that when you put the switch from L2 to L3 all other configuration is lost, so don't bother trying to configure anything else before changing to L3 mode. After the restart of the switch, while still in CLI you can configure the login/password and IP address of the default VLAN which is VLAN1. The rest of the operations can be done through the GUI. The GUI is nice done. Clean and sharp. Read the rest of the setup on next page.
The creation of VLANs through the GUI on the Cisco SG300 is not difficult, assigning the IP address to each VLAN too. But the hard part is the creating of static routes on the VLANs. Those routes should be created (populated) automatically when you go to the menu “Port to VLAN” and change the radio buttons. Without an understanding of networking terminology the GUI can be a bit tricky and I was struggling to get the routes to populate (there is a way to manually create those routes). I restarted several times from scratch 🙂 The screenshots below walk through the process of creating a VLAN and associated IP interface.
Be sure that you save your work before you reboot the switch. On the Cisco's interface there is little link and when not saved configuration is detected, there is a little red cross inside a circle. I did not see this sign and after reboot the conf was lost several times… -:(. I was learning the hard way… But wait it's not over.
I was facing a problem. My internet access is going through my ISP router which I rent from my ISP. The device is not my property and I don’t have a CLI or GUI access to be able to create a routes back for my newly created VLANs. You need to create a route back from your router for each VLAN used on the Cisco, otherwise VMs will not be able to communicate outside the SG300. I’m not a routing expert, but with the help of Andy Grant, he gave me the idea to place another router in between the ISP box and the Cisco SG 300 to configure the route back on that device. This is not normally required if you have access to your home router and it provides static routing functionality. I had a router which was not used in my lab at the moment. A Linksys WRT 54 GL. I went and flashed the device with the DD-WRT firmware and configured the routes back with Andy’s help. I also needed to configure this router in “mode gateway” and not “router”. This is something I discovered when testing, but DD-WRT is well documented on that as
well, and also on creating a static routes too.
So now I had 2 routers in my lab, with the traffic going from my ISP box on the 192.168.1.X network back through the Linksys WRT 54GL to my Cisco SG 300 at 10.10.1.x lan. Perfect. I was ready to go and start configure my 2 ESXi hosts and I put the network layout like this. I’m not saying that this is the perfect network desing, but it’s the one I did for my homelab needs.
Note from Andy: We created the link between the SG300 and the router as an access port to provide a configuration that a broader user-base could use. I had 3 NICs available in each ESXi host and I knew that my NAS box had only one NIC (no redundancy there… on the NAS side). So I dedicated one nic on each ESXi host to the iSCSI storage traffic.
I put the storage traffic on VLAN 8. The other 2 LAN cards on each ESXi host are used with VLANs for:
– Management Network, VM network, FT, vMotion
I have the first NIC in Active mode configured for Management Network. The second NIC is configured for Management Network NIC too, but in standby mode with failover set to no. The First NIC is in standby mode for VM network traffic, for redundancy. The second NIC is in active mode for VM network traffic.
All those networks are separated in VLANs to achieve the best performance and isolation. So that is the story of my home lab network upgrade, it took me some time and some sweat too…, but with help of Andy I could manage to go deeper in routing and Cisco in general. In fact, the experience you get by building your own homelab is enormous. If you haven't read the How to build a cheap Do it Yourself NAS system, you might want to have a look at this series of articles, where I'm building a Home made NAS system for a bit more than €300.
- How to build a low cost NAS for VMware Lab – introduction
- How to build low cost shared storage for vSphere lab – assembling the parts
- VMware Home Lab: building NAS at home to keep the costs down – installing FreeNAS
- Performance tests with FreeNAS 7.2 in my homelab
- Installation Openfiler 2.99 and configuring NFS share
- Installing FreeNAS 8 and taking it for a spin
- My homelab – The Network design with Cisco SG 300 – a Layer 3 switch for €199. – this post
- Video of my VMware vSphere HomeLAB
- How to configure FreeNAS 8 for iSCSI and connect to ESX(i)
- Haswell ESXi Whitebox