VMware has released a new whitepaper helping out for VSAN network design. The paper provides an outline of VSAN functions and then focuses on different network scenarios.
It walks through the possibilities and best practices for configuring networking with putting an accent on 10GbE which is actually not required but strongly recommended.
Quote from the paper:
The hosts in a Virtual SAN cluster must be part of a Virtual SAN network and must be on the same subnet regardless whether the hosts contribute storage or not. Virtual SAN requires a dedicated VMkernel port type and uses a proprietary transport protocol for traffic between the hosts.
The paper has some NIOC (Network IO control) scenarios with QoS on single 10GbE NIC and recommendations. Also switches in stacked or uplinked scenarios are discussed with benefits and inconvenients. What happens when there is a failure at the physical NIC, what happens and where the traffic will flow.
There are some good screenshots with those scenarios shown as well. Here switches configured with uplinks.
VSAN dos not require 10GbE but it's strongly recommended.
Virtual SAN requires a 1GbE network at the minimum. As a best practice, VMware strongly recommends 10GbE network for Virtual SAN to avoid the possibility of the network being the bottleneck. As demonstrated in this paper, 1GbE network can easily be saturated by Virtual SAN traffic and teaming of multiple NICs can only provide availability benefits in most cases. If 1GbE network is used, VMware recommends it be used for smaller clusters, and be to be dedicated to Virtual SAN traffic.
Fault tolerance is currently not supported on VSAN, but it's possible to run FT enabled VMs on external storage though.
In fact there are 2 cases for physical switch config:
The stacked switches operates as single switch:
A stackable network switch can both operate fully functionally standalone and operate together with one or more other network switches. A group of switches that have been set up to operate together is normally termed as “stack”. This group of switches shows the characteristics of a single switch but contains the port capacity of the sum of the combined switches. The stack members behave and work together as a unified system.
The document can be found on this post. Enjoy..