imbizcorp
Close menu

Understanding Network Infrastructure · Part 4 · How do we use these principles to design a real network?

How should switches across multiple wiring closets connect?

BY IM Business Platform Co., Ltd. PUB 2026-09-06 UPD 2026-10-24 VERIFY Standards and official sources

With several wiring closets, deciding where to group switches starts with the paths and resources those closets will share. In this example, the access layer handles routing, and two distribution devices provide upstream connectivity. After drawing the normal topology, we check the paths and capacity left after one device fails, using the same demand assumptions.

Back to contents

Map wiring closets and service paths first↑

Three wiring closets may give you three groups of switches, but the closet count does not determine the architecture. To define the uplinks' roles, first identify which users in each closet communicate with which services.

Call the three closets' access devices A1, A2, and A3. Users connect to their local access device; shared servers and external connectivity sit in the service segment beyond D1 and D2. Record addressing and demand separately for traffic that stays within one access segment and traffic that goes upstream to shared services.

D1 and D2 both aggregate access segments and forward traffic between segments. The access layer has no separate core above it in this design. This is a two-tier architecture, also called a collapsed core because it combines distribution and core roles. It does not mean there are only two devices.

Show actual closet locations and fiber routes on the diagram. A1 may have a line to both D1 and D2, yet both fibers could be cut in the same conduit. If service-segment connections share a device or power source, document that common failure as another constraint.

Check whether all services are in one location. A local server in a closet creates different uplink demand from a shared server elsewhere. Record which traffic is included in D1/D2 link calculations alongside service locations. Then, when a server moves, update capacity planning as part of the same change.

For the rest of this example, each access device performs routing. L2 access is another option, but mixing both approaches in one drawing can obscure the VLAN boundaries. We are applying one design selected in the L2/L3 boundary comparison to the building's physical layout.

Place the three closets in one topology as shown below. A1, A2, and A3 each have L3 connections to D1 and D2, with no separate core above them. Use this diagram to distinguish traffic that stays in a closet from traffic that goes upstream to shared services.

↔ Scroll horizontally to explore the diagram.

Three closets connect to D1 and D2, with no separate coreA1, A2, and A3 are access devices in separate wiring closets, each providing its endpoints' gateway. Each has separate L3 links to D1 and D2. D1 and D2 combine access aggregation with core forwarding to shared services. Orange shows one request path from A1 through D1 to those services; the other lines show the remaining connections.Three closets connect to D1 and D2, with no separate coreAggregation and core forwarding · No separate coreCloset 1Closet 2Closet 3Shared servicesD1D2A1 · Routed AccessA2 · Routed AccessA3 · Routed AccessD1/D2: aggregation + coreDevices and roles in the diagramDevice shapes identify types and roles. A grouped symbol does not specify a device count or a required set of products.Shared servicesInternal servers · External linksCheck return paths and shared faultsD1Access aggregation + Core forwardingD2Access aggregation + Core forwardingA1 · Routed AccessLocal access · Endpoint gatewaySeparate L3 links to D1 and D2A2 · Routed AccessLocal access · Endpoint gatewaySeparate L3 links to D1 and D2A3 · Routed AccessLocal access · Endpoint gatewaySeparate L3 links to D1 and D2Crossing lines are not junctions. Separate L3 uplinks do not form one EtherChannel across independent D1 and D2.
A1, A2, and A3 are access devices in separate wiring closets, each providing its endpoints' gateway. Each has separate L3 links to D1 and D2. D1 and D2 combine access aggregation with core forwarding to shared services. Orange shows one request path from A1 through D1 to those services; the other lines show the remaining connections.

Distinguish endpoint gateways from upstream routing paths ↑

A1's business endpoints use a gateway on A1 itself. Their VLAN ends in A1's access segment; the links from A1 to D1 and D2 are separate L3 connections. This design does not extend the endpoints' VLAN unchanged to both distribution devices.

To reach a shared server, PC-A first sends a frame to A1's gateway MAC address. A1 selects a route for the destination IP address and forwards toward D1 or D2. Each router then builds a new frame for the next hop on its outgoing link.

Whether both paths can carry traffic simultaneously depends on the routing design and installed routes. Two connected cables do not guarantee an exactly equal split. Even with equal-cost routes, the device's forwarding behavior and traffic mix affect how flows are distributed.

Here, D1 and D2 are independent L3 neighbors. Unless a specific technology makes them operate as one logical system, links to these separate devices cannot be combined into one EtherChannel. This example uses no link aggregation—just two routed paths.

If ECMP uses both paths, observe which flows are reassigned when one disappears. A change in the path count can change where a flow is forwarded. Test new connections separately from existing sessions to see whether users must reconnect after failover.

Draw the server's return path to PC-A on the same diagram. Sending a request through D1 does not guarantee that its reply also uses D1. If stateful security devices sit in between, verify that their configuration supports the resulting asymmetry so both directions can pass.

A1, A2, and A3 each have separate L3 links to D1 and D2. Locate the endpoints' gateways and the point where upstream paths to the servers split.

↔ Scroll horizontally to explore the diagram.

Three access devices have L3 paths to two independent distribution devicesA1, A2, and A3 each have a 10 Gbps L3 link to D1 and another to D2. Each access device provides its endpoints' gateway. These links to independent distribution devices do not form one EtherChannel. Shared services sit upstream of D1 and D2.Three access devices have L3 paths to two independent distribution devicesA1A2A3D1D2Each line is a separate 10 Gbps L3 linkForward to services beyond D1/D2 · Check each return pathDevices and roles in the diagramDevice shapes identify types and roles. A grouped symbol does not specify a device count or a required set of products.A1Endpoint gatewayA2Endpoint gatewayA3Endpoint gatewayD1Aggregation · Upstream routingD2Aggregation · Upstream routingLine crossings are not junctions. This illustrative topology gives each access device two L3 paths.
A1, A2, and A3 each have a 10 Gbps L3 link to D1 and another to D2. Each access device provides its endpoints' gateway. These links to independent distribution devices do not form one EtherChannel. Shared services sit upstream of D1 and D2.

Calculate access and aggregate bottlenecks separately ↑

Each access device has one 10 Gbps link to D1 and one to D2. Assume each access device sends 8 Gbps upstream. This is not a field measurement; it is a fixed teaching assumption for comparing normal and failure conditions.

Normally, each access device has 20 Gbps of nominal uplink speed. Those link speeds can be added only if both routing paths can actually be used. A single flow does not thereby reach 20 Gbps. Path distribution and brief traffic concentrations need separate measurement.

If D1 fails, each access device has one 10 Gbps link left to D2. With the same 8 Gbps demand, the simple utilization ratio is 80%. Protocol overhead and additional traffic also consume capacity, so verify the actual headroom separately.

If demand from all three access devices uses the same service link on D2, the total is 24 Gbps. With an assumed 40 Gbps service link, the simple ratio is 60%. Spare aggregate capacity does not prevent an individual access device's 10 Gbps link from becoming the first constraint. That is why both calculations matter.

A 20 Gbps service link, by contrast, cannot carry the full 24 Gbps demand simultaneously. A connection that looks sufficient normally can become inadequate when a failure concentrates demand on one side. Carry the same direction, observation segment, and demand assumptions from the uplink-capacity calculation into this building diagram.

After D1 fails, calculate capacity separately for each access uplink and D2's service link. An access link can congest before the aggregate service link runs out of capacity.

↔ Scroll horizontally to explore the diagram.

After D1 fails, check access and aggregate bottlenecksEach access device's 8 Gbps demand uses one 10 Gbps link to D2: a simple 80% ratio. Together, the three devices send 24 Gbps over D2's 40 Gbps service link: 60%. Assess the headroom of access links and the aggregate link separately.After D1 fails, check access and aggregate bottlenecksObservation point without D1Assumed demand / Nominal rate · One directionSimple utilizationA1 → D28Gbps / 10Gbps80%A2 → D28Gbps / 10Gbps80%A3 → D28Gbps / 10Gbps80%D2 → Shared services24Gbps / 40Gbps60%Illustrative assumptions. Check overhead, traffic bursts, flow distribution, and reverse traffic to assess real headroom.
Each access device's 8 Gbps demand uses one 10 Gbps link to D2: a simple 80% ratio. Together, the three devices send 24 Gbps over D2's 40 Gbps service link: 60%. Assess the headroom of access links and the aggregate link separately.

Remove D1 and retrace communication in both directions↑

For failure analysis, do more than gray out D1. Remove its links and the functions it provided, then trace a complete path from PC-A to the server and from the server back, using only surviving devices.

Once A1 determines that the path through D1 is unusable, it must forward over an available path through D2. Detection, route selection, and forwarding updates do not happen instantaneously. Observe any lost packets and application retries separately, using the criteria in redundancy and recovery time.

D2 may have a route toward the server, but communication still fails if the server side has no return path to A1's business subnet. A default route does not prove that replies to PC-A's subnet take the correct path. Record forwarding state in both directions before and after the failure.

Here, the endpoints' gateway stays on A1. D1's failure alone does not mean the endpoints need a different gateway device. If A1 fails instead, endpoints connected only to A1 lose access even while D2 is healthy. This design can accommodate a D1 failure, but it cannot preserve those endpoints' access when A1 fails.

Planned maintenance and sudden failure may remove a path in different ways. Record a test that moves traffic before maintenance separately from a test that shuts a device down without warning. The failure method must be known to compare failover time and packet loss under equivalent conditions.

Test D1's return as well. As its routes are reintroduced, check traffic distribution, existing sessions, and whether new policies or stale configuration enter the network with it. Even if D2 is operating normally, verify the repaired D1's state before reconnecting it.

Hand over diagrams, addressing, policy, and tests as one consistent design↑

A two-tier handover needs operational documents as well as a device diagram. Physical and logical diagrams, address plans, service policies, and normal and failure test results must use the same device names and connections, so operators can reconstruct the actual network.

The physical diagram should show closets, devices, ports, cable routes, and power dependencies. The logical diagram should show each access device's gateway and subnet, its two L3 connections, and routes toward the servers. Mark where every VLAN ends so operators can identify the L2/L3 boundary.

Configuration documentation should explain the design before presenting a full command listing. Why is the gateway at access? If D1 disappears, how much demand moves to each surviving link? Which traffic is allowed or blocked? These decisions must remain understandable even if device syntax changes later.

Record how D1's failure and return were triggered, the two-way communication observed, actual measurement times, surviving load, and results that differed from expectations. Mark untested services as unverified, not healthy. Keep teaching calculations separate from actual measurements.

Use port labels and device identifiers that operators can read on site. They must be able to match the logical diagram's A1 to a physical device in a specific closet. Provide a mapping if the names differ, and require labels and documents to be updated together after a cable break or equipment replacement.

Using this topology, the next article on switch and router redundancy examines which failures are detected and which device takes over. An alternate path may exist while the application still waits for detection, failover, or reconnection. After separating those steps, we consider when several distribution blocks need a core between them.

Sources for this sectionCisco Validated · Troubleshooting Overview

Glossary↑

This is a list of key terms used in this article; their definitions are available in the full glossary.

capacity planningcapacity planning
Collapsed Corecollapsed core
distribution blockdistribution block
L2 AccessLayer 2 access
L2/L3 boundaryLayer 2 / Layer 3 boundary
Routed Accessrouted access
Two-Tiertwo-tier design
serverserver
MAC addressMAC address
VLANVLAN
Network Accessnetwork access
Core Networkcore network
Distributiondistribution layer
endpointendpoint
gatewaygateway
IPIP
packetpacket
L3Layer 3
ECMPECMP
Next Hopnext hop
routingrouting
Port Aggregationlink aggregation
switchswitch
device portdevice port

Scope and references

The technical principles draw on the official documents below. Equipment layouts and loads identified as examples are illustrative, not field measurements or guaranteed product performance.

Before applying this to a live network, check support conditions for your hardware and software, and test the behavior your site needs.