What is your LAN uplink strategy?
Uplinks are an important component of LAN design since they provide the upstream pathway to other devices within the network, and are the method by which traffic is transported out of the network. In this example, I’d like to look at a 2960 access-layer switch and work through an uplink design strategy and implementation. Ideally, you would gather all of your design requirements before purchasing your switching hardware since the requirements you set forth can determine what port features are necessary. In this scenario, I will look at the design capabilities of my lab switch and come up with the best design available, while notating possible upgrades to consider to improve the design.
Lab Access Switch: Cisco 2960
Model: WS-2960-8TC-S
This switch has eight FastEthernet interfaces and one Gigabit Ethernet interface. The gigabit port is a combo port meaning there is a female rj-45 port for copper ethernet and also an SFP port for a GBIC fiber ethernet module representing the same port number. The two physical ports cannot be active at the same time forcing you to choose a path. This is clearly meant to be the uplink port for this switch since it is the only port that provides both copper and fiber capability. This port also provides a good bandwidth subscription ratio since the eight FastEthernet ports won’t oversubscribe the gigabit uplink. All is good then, right? From a bandwidth perspective, yes, this is a good design but we’re missing something. Can you think about what’s missing? Redundancy! Let’s explore that next.
With this low port density and small form factor switch, the desire for redundant uplinks may not have been anticipated and therefore was not included as a feature by Cisco. If my switch has to support production user traffic then I want to ensure uplink connectivity is persistent in the event of a structured cabling or upstream device failure. With those requirements in mind, I need a minimum of two uplinks; each uplink must terminate to a separate physical device. I mention physical devices specifically because those two upstream devices could represent one logical device as it will in this design example. With our lab access switch, we only have one designated uplink port so what do we do? Unfortunately are forced to trade bandwidth for link redundancy if we want to meet our resiliency requirements. We’ll take two of the fast ethernet ports and use link aggregation to pool bandwidth and reduce the dependency on spanning-tree for failover to our redundant link. This will of course reduce the amount of designated access ports from 8 to 6 but we can re-classify the gigabit uplink port as an access port to give us a total of 7 access ports. We’ll have to be careful here because using the gigabit interface we’re now using for access more than doubles our over-subscription ratio. The two uplink ports will go into a port-channel providing a total of 200Mbps uplink speed. Let’s look at our upstream switches next.
My upstream switches are a pair of Cisco Catalyst 3750’s configured as a switch stack. Both switches contain Etherswitch modules and are connected through their StackWise ports. This configuration allows the two switches to be seen as a single logical device and is the reason I can use a port-channel on my access switch uplink even though the individual links will be connected to separate physical switches. One interface from each switch in the stack will be placed in a port-channel forming a single logical layer 2 interface that spans both switches.
Lab Distribution Switches: Cisco 3750’s
Model: WS-C3750V2-24TS-S
Pro Tip: Add any LAG configuration to your interfaces while they’re in a ‘down’ state, then bring them up when the configuration is complete. This is easily accomplished if the configuration is new, however, if you have to make changes to existing port-channel config you can run into odd order of operations issues. Be mindful of that if you have experienced odd behavior after adding or changing port-channel configuration to active interfaces. A good shut/no-shut on the port-channel can usually fix these issues if they arise.
I’ll issue the below command on my uplink interfaces to enable Link Aggregation Control Protocol (802.3ad), commonly called LACP.
channel-group 1 mode active
I’m using LACP active mode which sets up the ports to actively try to negotiate the port-channel by sending LACP packets. I’m using a LAG (link-aggregation) protocol instead of configuring static trunks so that I have a mechanism to detect a port misconfiguration and prevent a scenario where I’m black-holing packets because of configuration inconsistencies on either side of the link. Once either interface stops receiving lacp packets the port is invalidated in some fashion and the link will not be used to transport traffic in the bundle. You can use the verification command below to monitor the status of the port-channel member links.
show etherchannel summary
There is a flag that is assigned to each port-channel member interface to reflect the link status within the port-channel. You want to see the P flag which indicates the interface is bundled in the port-channel
Fa1/0/24 (P)
A logical port-channel interface is created that corresponds with the integer used in the channel-group command. In my example Port-channel1 was created as a layer 2 interface. This port-channel interface also is flagged in the etherchannel output to identify its status.
Po1 (SU)
The flags are case sensitive which is important to note because the capital S indicates a layer 2 port-channel and the lower-case s indicates a suspended state. The U indicates the port-channel is in use, so it looks like I’m in good shape so far!
Next we’ll look at spanning-tree on the access switch
show spanning-tree vlan 1
Partial output:
Interface Role
————— ——
Po1 Root
We can see here that the new layer 2 port-channel interface is showing as the spanning-tree root port for VLAN 1 instead of the individual fast ethernet physical interfaces. The port-channel allows us to utilize both uplinks instead of the active/standby state instantiated by spanning-tree on the individual links. The below commands show us the load-balancing method the port-channel is using.
show etherchannel load-balance
Partial output:
IPv4: Source MAC address
This shows that the port-channel is using the source mac address as the load-balancing method. A hash is generated using the source mac-address and a specific link is utilized for the assigned hash. You can change the load-balancing method to use destination mac, source ip, destination ip, or a combination of src and dst mac or ip. The default is source mac.
I’d like to note that CDP will still use the member links, not the port-channel interface to display neighbors. When running this command on my access switch, I see the same distribution switch name on the two port-channel members which is expected.
Let’s review what we’ve achieved with this uplink design. Using multiple links in a port-channel allows us to load-balance traffic across each of the uplinks. Our switch stack at the distribution layer allows us to terminate each member link of the port-channel to a different switch providing link level and switch level redundancy upstream. Using LACP to negotiate port-channel state allows us to identify issues with the port-channel member links beyond layer 1 failures. This design gives us the resiliency we want for our access-layer uplinks. In our initial setup, we had to make bandwidth trade-offs to achieve this desired resiliency. It’s important to identify how you will uplink your switches to determine your port requirements when ordering switches. You’ll also need to identify your uplink transport medium or cable type as well to ensure you can support the uplink physical handoff. Optical transceivers can be costly and can wreck your budget if it’s an afterthought. The design of your distribution layer also has an impact on what you can achieve in your access layer uplink strategy. My uplink capabilities would have been limited without the switch stack configuration used in this example. There are obviously many things to consider regarding your LAN design, hopefully, this article and design scenario helps you on your journey!
~Eric Perkins