Understanding Network Infrastructure · Part 2 · How does data reach nearby devices?
Do four bundled links make one connection four times faster?
How LACP forms one logical link and hashing assigns traffic flows to members
Combining physical links into a port channel can distribute the traffic of multiple connections. It does not multiply one connection's speed by the number of links. LACP checks both ends' port parameters, and the switch hashes each flow onto one member link. We examine negotiation, load distribution, member-link failure, and counters to explain why a bottleneck can remain even with unused link capacity.
Why do bundled physical links appear to the switch as one logical link?↑
A link aggregation group treats parallel point-to-point links as one logical link. The switch's STP and MAC learning normally operate on the logical port channel, while the data plane selects a member link to send each frame.
If one link fails, the remaining member links can carry the traffic. Multiple flows can also share the bundle's total bandwidth. Both endpoints and the intervening paths must support the same logical bundle.
With four separate 1 Gbps connections between two switches, each link acts as an independent trunk, and STP may keep some on standby to prevent loops. In a LAG, other protocols see one port channel. The data plane chooses one of its four members for each frame. Multiple flows together can approach 4 Gbps, but actual throughput in each direction depends on traffic distribution and device limits.
Physical ports need compatible speed, duplex, Layer 2/Layer 3 mode, access or trunk configuration, and allowed VLANs to join the same bundle. If only one end forms a port channel while the other keeps independent ports, loops, duplicate frames, or packet loss can result. A stack or MLAG that spans chassis also depends on its control plane and inter-device connection; its failure conditions differ from link aggregation on one device.
Members must satisfy the bundle's requirements for speed, duplex, Layer 2/Layer 3 mode, and VLAN configuration. Forcing incompatible ports into a bundle can cause one-way forwarding or missing VLANs, so a device may exclude them in a suspended or individual state. Capacity planning must account for the minimum number of working links and the independence of each link's physical path, not just the sum of member rates.
Sources for this section IEEE 802.1AX-2020
LACP: how both ends verify the same bundle↑
LACP interfaces exchange LACPDUs carrying system IDs, keys, port states, and priorities. Only compatible links join the aggregation and enter forwarding.
Modes such as active and passive determine whether an endpoint proactively sends LACPDUs. At least one end must initiate negotiation. Static aggregation skips that exchange and assumes the configurations match.
Each LACPDU carries actor and partner system IDs, port IDs, keys, and state bits. Each device checks whether its own information returns as the other's partner information, confirming that they see the same connection. A synchronized member in collecting state can receive frames into the aggregate; in distributing state, it can transmit them. A physical link being up therefore does not prove that it is forwarding data as a member.
If a port's key or trunk setting differs, the link may remain in an individual or suspended state. The port-channel interface can remain up and carry traffic on other members, hiding the capacity loss. Inspect each member's actor and partner information and collecting/distributing states, not just the aggregate summary. A static LAG has no such protocol check, making change control and configuration verification at both ends even more important.
A logical interface can be up while some members are not collecting or distributing. For each member, check the actor and partner system IDs and keys, synchronization, collecting, and distributing states to verify bidirectional aggregation. With MLAG, where two physical devices appear as one logical system, also check the peer link and any split condition.
Sources for this section IEEE 802.1AX-2020 · Cisco Layer 2 EtherChannel and Trunking
Why a flow stays on one link, chosen by its hash↑
The device hashes supported fields—source and destination MAC addresses, IP addresses, and TCP/UDP ports, for example—to select a member link. Keeping a flow on one link reduces the chance of packet reordering.
One TCP flow therefore usually cannot exceed a single member's speed. Enough diversity among clients, servers, and flows is needed to use the bundle's total capacity. The actual hash fields and algorithm depend on the ASIC, software, and configuration.
Four members do not mean flows are assigned to them in numerical order. Selected source and destination IP and port bits map a flow to a bucket, which maps to a member. The same 5-tuple keeps a flow's packets in one bucket to preserve order. The opposite device hashes the return traffic independently, so requests and responses need not use the same physical member.
If a single TCP transfer uses one 1 Gbps member, the other three links can sit idle: the flow is normally not split packet by packet. Multiple TCP connections used by a file transfer, or traffic from several clients, provide different tuples that may use the aggregate capacity. ASIC support for hash fields and inner header inspection varies. For overlay traffic, check whether hashing only the outer tunnel addresses concentrates traffic on one member.
A single large TCP flow is thus usually limited to one member's speed. The bundle's capacity becomes useful when users and sessions spread across different hash buckets. Some devices offer resilient hashing or per-packet modes. Do not assume EtherChannel always uses a 5-tuple hash; check the actual algorithm and supported fields.
Sources for this section IEEE 802.1AX-2020
Four links are up—why is only one busy?↑
If all connections go to the same server IP and port, and the hash uses only that destination information, identical inputs can concentrate traffic on one member. That may reflect the traffic and hash algorithm, not a failed LACP negotiation.
Compare per-interface byte and packet counters with the distribution of flows. A different hash can make other traffic worse, so do not change the overall policy based on one speed test.
Even when one backup server receives connections from hundreds of clients, all their flows can land on one member if the hash uses only the destination MAC and IP address. Including Layer 4 ports may improve distribution, but fragments, encrypted tunnels, and hardware support limit the available fields. First inspect the header diversity of the flows that produce most of the traffic.
Average interface utilization can hide short bursts and queue drops on one member. Collect member byte and packet counts, drops, queue statistics, and errors at the same intervals, then compare them with the port-channel total. A hash change can move existing flows to different links and temporarily reorder packets, so schedule the change and prepare a rollback. The goal is to find bottlenecked flows and measure loss, not to make every link equally busy at every instant.
Start with each member's byte, packet, and drop counters measured over the same interval. Identify heavy flows' source and destination values and the current hash inputs. Adding Layer 4 ports may improve distribution, but limited input diversity—fragments, encrypted tunnels, or few endpoints—can limit the benefit. Test changes for flow reassignment and possible reordering.
Sources for this section IEEE 802.1AX-2020 · Cisco Layer 2 EtherChannel and Trunking
What to look for in the diagram The left side shows one flow assigned to a member link; the right side shows how flows with multiple 5-tuples can be distributed across member links. Compare the hash-input explanation with the bucket assignments, keeping in mind that equal bucket counts do not necessarily mean equal traffic volumes on the links.
↔ Scroll horizontally to explore the diagram.
Member link failure: moving flows to the remaining links↑
When a physical link goes down or LACP loses its partner state, the member is removed from the forwarding set. Flows using it are reassigned to one of the remaining links.
Switchover can cause packet loss and TCP retransmissions. With minimum links configured, too few remaining members can take the entire LAG down, allowing routing or redundancy mechanisms to choose another path.
Loss of an optical signal can trigger a physical down indication quickly. A one-way failure or a partner that stops sending LACPDUs may instead require an LACP timeout before removal, losing some frames in the meantime. The removed member's hash buckets are reassigned to remaining links. Depending on the implementation, many flows may move or only the affected buckets may change. Existing TCP connections may recover through retransmission, while real-time UDP applications experience the loss directly.
Minimum links tells other protocols that the remaining capacity is below the service requirement. If only one of four links survives but the port channel stays up, routing may keep using it and overload the remaining 1 Gbps. Requiring at least two members takes the logical interface down below that count, allowing another route to be chosen. Set the threshold from the minimum required capacity and the alternate path's ability to carry the load, not availability alone.
min-links can take the entire port channel down when available members fall below the configured minimum. Set for the required capacity, it can trigger a move to another connection. That alternate must have sufficient capacity, and routing must converge. When a member returns, the reassignment of existing flows to links depends on the device and hashing method. Test failover and restoration separately.
Sources for this section IEEE 802.1AX-2020
What to look for in the diagram The left side shows normal operation with four member links; the right side shows an example bucket assignment after Port 4 is removed. Compare how buckets are assigned to the three remaining links. Bucket counts do not represent traffic volumes, and switchover time and loss must be tested separately.
↔ Scroll horizontally to explore the diagram.
Port-Channel Up? Check each member's state and counters too↑
Scenario: A 4×1 Gbps port channel cannot push one large transfer beyond about 1 Gbps.
- Verify that all four members are in LACP collecting and distributing states.
- Distinguish a single TCP flow from multiple independent flows in the test.
- Check the current hash fields and each member's byte counters.
- Measure aggregate capacity again with flows using multiple sources and destinations.
- Disconnect one member link, verify flow reassignment and effects on other protocols, then restore it.
Test single-flow and multi-flow traffic separately. Use one TCP connection to measure a member link's speed limit and packet loss. Then vary source, destination, and port values across multiple flows to measure distribution over the whole port channel. Collect per-member counters and LACP state from both devices at the same time. If outgoing traffic concentrates on one link, do not assume replies use that same link.
In failure tests, disconnect member links one at a time and record port-channel state, flow reassignment, packet loss, and changes in STP or routing. Check that a backup path is selected below the configured minimum member count. Restore each link and verify that it rejoins without repeatedly disconnecting. After configuration changes, also confirm that allowed VLANs, MTU, and other logical interface settings apply consistently to all members.
Four green port indicators do not prove that all four links are usable. Verify each link's membership, the distribution of flows, and which links carry traffic after a failure to establish the capacity you can actually use.
Sources for this section IEEE 802.1AX-2020 · Cisco Layer 2 EtherChannel and Trunking
FAQ↑
Are LACP and EtherChannel the same thing?
Glossary↑
Sources and verification
- IEEE 802.1AX-2020 — The current link aggregation standard
- Cisco Layer 2 EtherChannel and Trunking — EtherChannel and trunk configuration examples without LACP
- IEEE 802.1AX-2014 Project — Link aggregation standards project information
Hash fields, member counts, minimum-links settings, and failover behavior depend on the device and software. Adding several links' bandwidth does not turn that total into one connection's speed.