Skip to content

August 30, 2011

VMworld: Distributed vSwitch Best Practices (VSP2894)

Speaker: Vyenkatesh Deshpande (VMware)

Agenda:
– Overview of VDS
– vSphere 5 New Features
– VDS Best Practices
– VDS Myths

# Overview
– unified network virtualization management in dependent of physical fabric
– manage datacenter-wide switch vs. individual switches per host
– vMotion-aware: statistics and policies follow the VM simplifying debugging and troubleshooting

Network I/O control and load balancing features
– why Network I/O control?
– – situation 1: multiple dedicated 1GigE pNICs, bandwidth assured, but less efficient utilization
– – situation 2: two 10GigE converged pNICs, better efficiency, but “noise” sensitive (i.e. VMotion)
– – Network I/O control addresses the “noise” in situation 2

Network I/O Control: Parameters
– limits
– – limits specify the absolute maximum bandwidth for a traffic type
– – specified in Mbps; traffic will never exceed its specified limit
– shares
– – shares specify the relative importance of an egress traffic on a vmnic
– – specified in abstract units; bandwidth divided between types based on shares
– controls apply to output from ESXi host
– shares apply to a given vmnic or uplink
– limits apply across the team

Network I/O Control: Benefits
– isolation: one flow should not dominate others
– flexible partitioning

Load-based teaming: Basics
– teaming algorithm based on pNIC load
– avoids congestion on one pNIC
– algorithm
– – every 30 secs pNIC load check is performed
– – if greater than 75% mean utilization on Tx or Rx is detected, LBT is invoked
– – based on utilization of other NICs, load reallocated

Advantages of Distributed Switch
– One single virtual switch to manage and configure
– Operational Simplicity
– Ability to manage IO resources efficiently through LBT and NIOC

# vSphere New Features
– Network Discovery and Visibility/Monitoring Features
– – LLDP
– – NetFlow
– – Port Mirror
– Enhanced Network I/O Control features
– – new traffic types: User Defined Network Resource pool
– – end-to-end QoS support through 802.1p tagging

What is Discovery Protocol?
– data link layer network protocol used to discover capabilities of network devices
– helps automate the deployment process in a complex environment
– vSphere supports CDP and LLDP

What is NetFlow?
– collects IP traffic info as records and sends them
– collector/analyzer report on various information
– – current top flows consuming the most bandwidth
– – which flows are behaving irregularly
– – number of bytes a particular flow has sent/received in past 24hrs

NetFlow Usage
– helps customers monitor the application flows and measure application performance over time
– helps in capacity planning and optimizing IO and network resource usage
– capability in vSphere infrastructure provides complete visibility into virtual infrastructure traffic
– – inter-host VM traffic
– – intra-host VM traffic
– – VM-to-physical infrastructure traffic

Port Mirroring
– capability to send copy of network packets sent to one switch port to a network monitoring device connected to another switch port
– also referee to SPAN
– overcomes the limitation of promiscuous mode
– granular control on which traffic can be monitoring: ingress & egress sources
– helps in troubleshooting intra- and inter-host traffic
– you can specify a VM on the same host as a destination or an external uplink port (to a physical switch)

Enhanced Network I/O Control (NETIOC)
– user defined network resource pools (up to 64 different traffic types)
– new vSphere replication traffic type (for DR and SRM deployments)
– QoS tagging to provide end to end service guarantees
– usage: tier 1 applications, service providers of shared platforms and public clouds
– (images)

# VDS Best Practices

Example deployment with rack servers and two 10GigE interfaces per host

– separate your infrastructure traffic from your VM traffic
– methods:
– – separate logical networks (VLANs)
– – – create one VDS with all pNICs
– – – create port groups with different VLANs with vmknic and VMs on different port groups
– – – 2 pNICs is sufficient
– – separate physical networks
– – – create one VDS for each physical network

2) Avoid single point of failure
– connect links to different physical switches

3) Understand your virtual infrastructure traffic flows
– make use of NetFlow to monitor flows over time
– use the data to come up with appropriate traffic shares to help NIOC configuration

4) Prioritize your traffic with QoS and 802.1p tagging

Physical Switch Setup Consideration
– VLAN setup: trunk all VLANs that might be used on all pNIC ports
– spanning-tree related
– – use portfast on ESXi facing ports
– – use bpduguard to enforce STP boundary
– link aggregation
– – static only; no LACP (dynamically negotiated etherchannel)
– – static etherchannel must go with IP hash on VDS
– link-state tracking
– – signal pNIC link down when downstream link fails
– – always enable link state tracking if available
– – consider beacon probing on virtual switch if link state tracking is unavailable on physical switch
– make sure MTU setting match VDS

VDS Configuration Steps
– configure uplink port groups
– define port groups based on number of traffic types
– configure following parameters per port group
– – teaming
– – VLAN setting
– assign NIOC shares to different traffic types
– (image)

Advantages of using NIOC and LBT
– dynamic scheduling based on utilization
– resiliency through active-active paths
– better utilized I/O resources
– no manual intervention

Example of Blade Deployment
– same as prior example except that the blade chassis acts as the access layer

Scalability and Extensibility of Distributed Switch
– scaling numbers
– – hosts per VDS: 350
– – distributed ports: 20,000
– – distributed port groups: 5,000
– automation in physical switch configuration
– – vCenter API
– – different vendor integration (Arista VMtracer, Juniper virtual control, IBM VMready)

# VDS Myths
– failure of VC stops all traffic
– – vCenter is used to create and manage VDS, but all data flow happens at host level
– one VDS is required per cluster
– – you can span VDS across multiple clusters

Share your thoughts, post a comment.

(required)
(required)

Note: HTML is allowed. Your email address will never be published.

Subscribe to comments