Understanding Network Infrastructure · Part 2 · How does data reach nearby devices?
How do devices on different VLANs communicate?
At a gateway, crossing a VLAN boundary requires a route lookup and a new Layer 2 header for the packet
Across different VLANs, endpoints cannot communicate directly through Layer 2 switching alone. An endpoint sends its packet to the default gateway. An SVI or router-on-a-stick applies the route and policy, then forwards it into the destination VLAN with a new Layer 2 header. We compare both designs' paths in each direction, track what stays and what changes at the VLAN boundary, and identify troubleshooting checkpoints.
For another VLAN's server, the PC first needs the gateway's MAC address↑
A PC at 10.10.10.25/24 recognizes server 10.20.20.80/24 as being on another network. The PC does not ask for the server's MAC through ARP. It resolves the default gateway 10.10.10.1 to its MAC address instead.
Ethernet Src PC MAC
Ethernet Dst VLAN 10 Gateway MAC
IP Src 10.10.10.25
IP Dst 10.20.20.80The access switch forwards this frame within VLAN 10 toward the gateway's port or logical interface.
The PC's first frame contains destinations at two layers. The Ethernet destination is VLAN 10's gateway MAC, while the IP destination is VLAN 20's server, 10.20.20.80. The switch uses the gateway MAC for Layer 2 forwarding. The gateway then examines the IP header for a route lookup. The PC needs neither VLAN 20's tag nor the server's MAC. It delegates forwarding beyond its broadcast domain to the gateway.
The PC's ARP cache should contain an entry not for the server's IP, but for 10.10.10.1. An attempt to resolve the server IP through ARP may indicate a wrong mask or a more-specific on-link route. Resolving the gateway MAC is not enough if the VLAN 10 uplink fails or a virtual gateway moves to another MAC: the first frame may never arrive. Before investigating a route failure, check the client's on-link decision and the gateway's Layer 2 reachability.
Without the gateway MAC, routing cannot begin: traffic stops in the local VLAN. Starting with the server VLAN's MAC address table would miss the first failure. Check the PC's address, mask, route result, and ARP target, then follow requests and replies through VLAN 10 to the gateway SVI. Once ARP completes, inspect the SVI's Layer 3 counters and route lookup.
Sources for this section IEEE 802.1Q-2022 · IETF RFC 1812
An SVI removes the incoming frame, looks up a route, and builds another VLAN's frame↑
An SVI is a switch's logical connection to a specific VLAN, implemented as a Layer 3 interface. When the VLAN 10 SVI receives a frame, it processes the Ethernet header and uses the IP destination to look up the routing table.
If the route points to the connected network in VLAN 20, the device resolves the VLAN 20 server's IP to a MAC through ARP. It sets the source MAC to the VLAN 20 SVI's address and the destination MAC to the server's. This new frame carries the existing IP packet.
When the SVI receives a VLAN 10 frame, it checks that the destination MAC is its own MAC and removes the Ethernet header. It processes the IP header, decrements TTL, and looks up destination 10.20.20.80 in the routing table. If VLAN 20 is connected, that SVI becomes the egress interface. If the server's MAC is absent from the neighbor table, the device starts ARP. Once resolved, the IP packet receives a new VLAN 20 Ethernet header.
In ordinary IP routing, source and destination addresses stay the same. The new Layer 2 source is the VLAN 20 SVI's MAC, and the Layer 2 destination is the server's MAC. TTL and the IPv4 header checksum also reflect router processing. Compare captures in both VLANs for matching IP IDs and TCP information but changed MAC addresses to identify the Layer 3 boundary. Here, the SVI is more than a management IP: across the two broadcast domains it performs header replacement and route selection as a routing interface.
Ordinary routing preserves the IP source and destination while changing the Layer 2 header on each link. Features such as ACLs, NAT, and policy-based routing can affect the result further. Distinguish differences caused by basic routing from those caused by policy.
Sources for this section IEEE 802.1Q-2022 · IETF RFC 1812
What to look for in the diagram Compare the SVI used for Layer 2 switch management on the left with the Layer 3 interface acting as the endpoints' default gateway on the right. Distinguish the example ARP mappings for management traffic from TTL decrement, checksum recalculation, and MAC rewriting in inter-VLAN packet processing.
↔ Scroll horizontally to explore the diagram.
Router-on-a-Stick: why do inter-VLAN packets cross one link twice?↑
A Layer 2 switch without Layer 3 routing can use a trunk to carry multiple VLANs to an external router. For each VLAN, the router assigns a VLAN-specific 802.1Q subinterface a gateway IP. Each frame's tag identifies the appropriate Layer 3 interface.
This keeps device roles clear in a small network, but multiple VLANs share one physical link and the router's throughput. SVIs and router-on-a-stick use the same basic routing principles. What differs is the location of the Layer 3 boundary—on which device and link it sits.
The switch sends a VLAN 10 frame over the trunk with VID 10 to the router. The router's VLAN 10 subinterface receives it and performs a route lookup. The VLAN 20 subinterface then sends it back with VID 20. The switch forwards the returned tagged frame to a VLAN 20 access port. Traffic travels both ways on one physical interface, but inside the router it crosses distinct Layer 3 subinterfaces through routing.
All inter-VLAN traffic crosses one link twice, concentrating bandwidth demands and failure impact. A wrong trunk allowed list, native VLAN, or subinterface encapsulation can break only one VLAN. Simplicity and cost favor small sites, but growing traffic may call for Layer 3 switch SVIs or distributed gateways. Choose based on throughput, policy placement, redundancy, and alternate paths—not the design's name.
This can be simple for a small or temporary network, but a trunk failure disconnects all VLAN gateways together. Inter-VLAN traffic traversing the link in both directions can also create a bottleneck. Check the router subinterfaces' encapsulation VLANs, native settings, trunk allowed list, and counters in both directions. As single-link risk and throughput requirements grow, compare Layer 3 switch SVIs and distributed gateways.
Sources for this section IEEE 802.1Q-2022 · Cisco Inter-VLAN Routing
What to look for in the diagram Compare a Layer 3 switch's SVI hardware forwarding with router-on-a-stick subinterfaces. Trace the frames in each design, comparing the per-VLAN interfaces on the left with the trunk and subinterfaces on the right to identify the routing location and the links traversed.
↔ Scroll horizontally to explore the diagram.
Between VLANs, the L3 boundary is a useful place to check access permissions and logs↑
Ordinary VLAN-local unicast traffic does not pass through the gateway, so the gateway ACL does not inspect it directly. Traffic between VLANs crosses a Layer 3 boundary, making it a useful point to apply policy by source and destination prefix and Layer 4 port.
One SVI ACL is not the answer to every policy requirement. If you need stateful inspection, user identification, or application control, you can design a path through a firewall. When policy moves, check for asymmetric paths and distinguish original from translated addresses in logs.
A VLAN 10 to VLAN 20 packet crosses the gateway's route and policy checkpoint. An ACL can match source 10.10.10.25, destination 10.20.20.80, and Layer 4 ports. That ACL must be placed correctly: an ingress ACL and an egress ACL inspect the packet on different interfaces and in different directions, so the same rule may match differently. Check platform behavior for implicit deny, the established keyword, and fragments too.
Traffic sent directly between hosts in the same VLAN bypasses this gateway. Assuming that VLAN 10 users are isolated from each other by an SVI ACL is therefore wrong for ordinary Layer 2 forwarding. East-west inspection needs a control point on the actual path: private VLANs, a host firewall, an inline firewall, or fabric policy. First match the policy goal to a boundary the packet must cross; then select the feature.
Not every policy resides on an SVI. You may use hairpin routing through a central firewall, leak VRF routes, or use a distributed firewall. Trace the actual packet path to identify where lookup and policy occur. Correlate each log's original tuple with the return state to distinguish a one-way ACL restriction from a server response problem.
Sources for this section IETF RFC 1812 · Cisco Inter-VLAN Routing
SVI Up but no connection? Check return routes and host settings↑
An SVI's state can depend on its VLAN's Layer 2 activity and platform rules. Even if the SVI is up and a route exists, the server's default gateway may point to another device, sending the response down a different path and breaking communication.
Different VRFs or an ACL that permits only one direction can produce the same symptom. Follow the path from the source PC's gateway ARP entry to the Layer 3 device's route and adjacency, then the server's gateway and return route.
On some switches, the VLAN must exist and at least one VLAN forwarding port must be up for the SVI's line protocol to remain up. Other platforms or virtual environments may use different conditions. Reaching the SVI IP therefore does not prove that VLAN 20 server ARP resolution works. Check the SVI state, connected route, neighbor entry, and actual access port state separately.
A request can reach the server without a response returning if the server's mask or default gateway is wrong. If the server uses another router as its gateway, the asymmetric path can cause a stateful firewall to reject the response. Different VRFs use separate prefix lookups and routing tables, even when the prefix values match. Trace both hosts' routes, both SVIs' VRFs, and intervening policy in both directions to find problems hidden by the single observation that the gateway is up.
If a packet reaches the server, check the server's IP, prefix, gateway, and return route. If the server incorrectly treats the source PC as on-link, it tries to resolve that host using ARP, even though the host is not on the local link. A response sent through another gateway may also be rejected as asymmetric by a stateful firewall. Correlate both SVIs' ARP/adjacency information, FIB entries, and ACL counters using the same 5-tuple.
Sources for this section IETF RFC 1812 · Cisco Inter-VLAN Routing
In both VLAN captures, find the same IP packet and compare changed L2 headers↑
Scenario: A VLAN 10 PC can reach its gateway but cannot connect to a VLAN 20 server.
- Check that the PC treats the server as remote and sends to the VLAN 10 gateway MAC.
- On the Layer 3 device, check the
10.20.20.0/24route and VRF. - Check the VLAN 20 SVI state and the server IP's ARP and adjacency information.
- Check ACL and firewall hits, plus the server's default gateway and local firewall.
- Capture traffic in both VLANs to verify that the same IP packet has the expected MAC header changes.
Choose one reference packet—a PC's TCP SYN—and capture it in VLAN 10, at the Layer 3 gateway, and in VLAN 20. The first capture should show PC MAC → gateway MAC; the gateway's output should show SVI MAC → server MAC. The IP source and destination and TCP ports should stay the same. If the gateway drops the packet, inspect the route, VRF, and ACL. If it leaves for VLAN 20 but no reply appears, inspect the server configuration and local firewall.
Check that the response SYN-ACK crosses the same boundaries in reverse. Collect ACL hits, SVI input/output, ARP, and drop counters for the same time window. After a fix, test more than ping: open a new session on the actual application port and rule out effects from existing cache entries and sessions. Record both sides' headers before and after the change so another engineer can distinguish VLAN, routing, and server problems.
Find the first packet present in the upstream capture but missing downstream to narrow the investigation to gateway lookup, policy, or adjacency. If both captures contain the request, examine server processing and the return path. A changed MAC address alone does not identify a different communication flow.
Sources for this section IETF RFC 1812 · Cisco Inter-VLAN Routing
Glossary↑
Sources and verification
- IEEE 802.1Q-2022 — the bridge and VLAN standards family
- IETF RFC 1812 — IPv4 router forwarding and route lookup
- Cisco Inter-VLAN Routing — vendor configuration examples for SVIs. For router-on-a-stick configuration, follow the related link at the end of the document.
SVI activation conditions, ACL processing order, and hardware forwarding architecture vary by platform. This article explains the common principles: Layer 2 header replacement and Layer 3 forwarding decisions.