Port Channels and vPCs

In early Layer 2 Ethernet network environments, Spanning Tree Protocol (STP) was the primary solution to limit the disastrous effects of a topology loop in the network. STP has one suboptimal principle: to break loops in a network, only one active path is allowed from one device to another, regardless of how many actual connections might exist in the network. The single logical link creates two problems: one problem is that half (or more) of the available system bandwidth is off limits to data traffic, and the other problem is that a failure of the active link tends to cause multiple seconds of systemwide data loss while the network re-evaluates the new “best” solution for network forwarding in the Layer 2 network. In addition, no efficient dynamic mechanism exists for using all the available bandwidth in a robust network with STP loop management. To overcome these challenges, enhancements to Layer 2 Ethernet networks were made in the form of port channel and virtual port channel (vPC) technologies. Port Channel technology allows multiple links between two participating devices to be used to forward traffic by using a load-balancing algorithm that equally balances traffic across the available Inter-Switch Links (ISLs) while also managing the loop problem by bundling the links as one logical link. vPC technology allows multiple devices to form a port channel. In vPC, a pair of switches acting as a vPC peer endpoint looks like a single logical entity to port channel–attached devices; the two devices that act as the logical port channel endpoint are still two separate devices. The vPC environment combines the benefits of hardware redundancy with the benefits of port channel loop management.

In this chapter, we will discuss Ethernet port channels, including port channel modes, port channel compatibility requirements, and port channel load balancing. We will also discuss virtual port channels, including various vPC topology implementations, vPC components, vPC control and data planes, vPC failure scenarios, and vPC configuration and verification.

Ethernet Port Channels

A port channel bundles physical links into a channel group to create a single logical link that provides the aggregate bandwidth of up to 32 physical links. A port channel logically bonds several physical connections into one logical connection. Port channeling also load-balances traffic across these physical interfaces. From the perspective of a Nexus switch, the data plane of the receiving Cisco Nexus device, in a port channel setup, treats packets that travel through physical links as coming through several links, while the management plane treats the packets as a single data flow. If a member port within a port channel fails, the traffic previously carried over the failed link switches to the remaining member ports within the port channel, and the port channel stays operational as long as at least one physical interface within the port channel is operational. Each port can be in only one port channel.

Figure 4-1 illustrates port channel physical and logical views.

You can create a Layer 2 port channel by bundling compatible Layer 2 interfaces, or you can create Layer 3 port channels by bundling compatible Layer 3 interfaces. You cannot combine Layer 2 and Layer 3 interfaces in the same port channel. You can also change the port channel from Layer 3 to Layer 2. You can create port channels directly by creating the port channel interface, or you can create a channel group that acts to aggregate individual ports into a bundle. When you associate an interface with a channel group, the Cisco NX-OS software creates a matching port channel automatically if the port channel does not already exist. In this instance, the port channel assumes the Layer 2 or Layer 3 configuration of the first interface. You can also create the port channel first. In this instance, the Cisco NX-OS software creates an empty channel group with the same channel number as the port channel and takes the default Layer 2 or Layer 3 configuration as well as the compatibility configuration.

You can configure Layer 2 port channels in either access or trunk mode. A Layer 2 port channel interface and its member ports can have different STP parameters. Changing the STP parameters of the port channel does not impact the STP parameters of the member ports because a port channel interface takes precedence if the member ports are bundled. After a Layer 2 port becomes part of a port channel, all switchport configurations must be done on the port channel; you can no longer apply switchport configurations to individual port channel members. Layer 3 port channel interfaces have routed ports as channel members. You cannot apply Layer 3 configurations to an individual port channel member either; you must apply the configuration to the entire port channel. You can configure a Layer 3 port channel with a static MAC address. If you do not configure this value, the Layer 3 port channel uses the router MAC of the first channel member to come up.

Figure 4-2 illustrates Layer 2 (access and trunk) and Layer 3 (routed) port channel interfaces. Port channel 20 is the L2 access port channel, with only VLAN 1 allowed on the port channel. Port channel 21 is the L2 trunk port channel, with VLAN 1 and VLAN 2 allowed on the port channel. Port channel 22 is the L3 routed port channel, whereas Eth 2/3 is the regular Ethernet routed interface.

For simplified port channel configuration, you can use static port channels with no associated aggregation protocol. For more flexibility, you can use the Link Aggregation Control Protocol (LACP), which is defined in the IEEE 802.1AX and IEEE 802.3ad standards. LACP controls how physical ports are bundled together to form one logical channel; for example, you can control the maximum number of bundled ports allowed. You cannot configure LACP on shared interfaces.

The port channel is operationally up when at least one of the member ports is up and that port’s status is channeling. The port channel is operationally down when all member ports are operationally down. On Cisco Nexus 7000 Series switches, all ports in a port channel must be in the same virtual device context (VDC).

Port Channel Modes

Individual interfaces in port channels are configured with channel modes. When you run static port channels with no aggregation protocol, the channel mode is always set to on. When you run LACP as the aggregation protocol, you must first enable the LACP feature. By default, LACP is disabled. The system automatically takes a checkpoint before disabling the feature, and you can roll back to this checkpoint. You cannot disable LACP while any LACP configurations are present. After you enable LACP globally on the device, you enable LACP for each channel by setting the channel mode for each interface to either active or passive. You can configure channel mode for individual links in the LACP channel group when you are adding the links to the channel group. When you delete the port channel, the software automatically deletes the associated channel group. All member interfaces revert to their original configuration.

Table 4-1 describes the channel modes.

Table 4-1 Channel Modes

Both the passive and active modes allow LACP to negotiate between ports to determine if they can form a port channel based on criteria such as the port speed and the trunking state. The passive mode is useful when you do not know whether the remote system, or partner, supports LACP.

Two devices can form an LACP port channel, even when their ports are in different LACP modes, if the modes are compatible.

Table 4-2 shows various compatible channel modes for port channels.

Table 4-2 Channel Modes Compatibility

Port Channel Compatibility Requirements

When you add an interface to a channel group, the NX-OS software checks certain interface and operational attributes to ensure that the interface is compatible with the channel group. If you configure a member port with an incompatible attribute, the software suspends that port in the port channel. You can use the show port-channel compatibility-parameters command to see the full list of compatibility checks that Cisco NX-OS uses. You can force ports with incompatible parameters to join the port channel if the following parameters are the same:

• (Link) Speed capability and speed configuration

• Duplex capability and duplex configuration

• Flow-control capability and flow-control configuration

When the interface joins a port channel, some of its individual parameters are removed and replaced with the values on the port channel. The following list provides some of these individual parameters:

• Bandwidth

• Delay

• VRF

• IP address

• MAC address

• Spanning Tree Protocol (STP)

• Service policy

• Access control lists (ACLs)

All the QoS service policies on the port channel are implicitly applied on the member ports when they join the port channel. You will not see QoS service policies in the running-config of the member ports. When you delete the port channel, the software sets all member interfaces as if they were removed from the port channel.

Many interface parameters remain unaffected when the interface joins or leaves a port channel, including those in the following list:

• Description

• CDP

• LACP port priority

• UDLD

• Rate mode

• Shutdown

• SNMP trap

Each port that is configured to use LACP has an LACP port priority. LACP uses the port priority to decide which ports should be put in standby mode when there is a limitation that prevents all compatible ports from aggregating and which ports should be put into active mode. You can accept the default value of 32768 for the LACP port priority, or you can configure a value between 1 and 65535. A higher port priority value means a lower priority for LACP. You can configure the port priority so that specified ports have a lower priority for LACP and are most likely to be chosen as active links rather than as hot-standby links.

Port Channel Load Balancing

Because a port channel uses several links to transport packets through physical infrastructure, the packets must be distributed between the physical links through some load-balancing algorithm. The Cisco NX-OS software load-balances traffic across all operational interfaces in a port channel by hashing the addresses in the frame to a numerical value that selects one of the links in the channel.

You can configure the load-balancing mode to apply to all port channels that are configured on the entire device or on specified modules. The per-module configuration takes precedence over the load-balancing configuration for the entire device. You cannot configure the load-balancing method per port channel. The default load-balancing method for Layer 2 packets is src-dst-mac. The default method for Layer 3 packets is src-dst ip-l4port.

You can configure the device to use one of the following methods to load-balance across the port channel:

• Destination MAC address

• Source MAC address

• Source and destination MAC address

• Destination IP address

• Source IP address

• Source and destination IP address

• Source TCP/UDP port number

• Destination TCP/UDP port number

• Source and destination TCP/UDP port number

• GRE inner IP headers with source, destination, and source-destination