SONiC L2 Leaf–Spine Migration Eliminates STP and Uses MCLAG

A data centre case study outlines a move from an STP-based Layer-2 design to a Layer-2 leaf–spine fabric using SONiC and MCLAG, with firewall-based Layer-3 termination kept in place. The change targets predictable failure handling, removal of STP from the data plane, and active-active server uplinks.

Research Overview

The post describes a real-world data centre network migration that replaces a traditional Layer-2-only architecture built on Spanning Tree Protocol with a leaf–spine design running SONiC as the network operating system. The legacy approach relied on VLANs and trunk links and faced issues around blocked links and operational complexity.

As part of the migration, the implementation deploys a Layer-2 leaf–spine (L2LS) fabric that uses Multi-Chassis Link Aggregation (MCLAG) to support server connectivity. The vendor frames the overall goal as eliminating STP in the data plane while enabling active-active forwarding for server uplinks.

Key Findings

The authors report that post-migration operations removed STP events from the network. They also report improved bandwidth utilisation through active-active links and faster, deterministic failure convergence.

The case study states that troubleshooting and visibility became simpler compared with the legacy design. It also notes a reduction in operational overhead relative to the earlier STP-based approach.

Technical Breakdown

The legacy environment is described as a pure Layer-2 network using STP for loop prevention, extensive VLAN usage across multiple switches, trunk links between aggregation and access layers, and single-homed or active-standby server connections. The paper lists observed challenges including STP convergence delays during topology changes, blocked redundant paths, time-consuming troubleshooting around MAC learning and loops, and limited scalability from VLAN sprawl and broadcast domains.

The target architecture uses a Layer-2 leaf–spine structure where spine switches provide a non-blocking Layer-2 fabric between leaf switches. Leaf switches connect servers and form MCLAG pairs at the access layer, with server dual homing to two leaf switches using port-channels.

Layer-3 Gateway and Firewall Placement

The SONiC leaf–spine fabric is described as operating purely as a Layer-2 switching domain. The default gateway and Layer-3 termination are intentionally placed on the firewall so that routing logic and security enforcement rules can remain unchanged during the migration.

According to the post, server VLANs are extended across the SONiC leaf switches using MCLAG, while the firewall performs Layer-3 gateway functions, inter-VLAN routing, security policy enforcement, and NAT and inspection policies. This placement is described as avoiding changes to the existing security architecture and routing policies.

Operational Impact

The migration strategy avoids a “big bang” cutover and instead uses a parallel build approach. It racked and cabled new SONiC spine and leaf switches alongside the legacy equipment, then created a temporary high-bandwidth Layer-2 trunk between the legacy core and the new SONiC spines to extend the L2 domain for VM migration.

Servers were migrated rack-by-rack by moving cables from legacy access switches to SONiC leaf switches and updating server-side bonding to LACP (802.3ad). The authors state that after each batch they verified connectivity and application health, then after all hosts were moved they severed the bridge link and powered down legacy hardware.

Leadership Perspective

The design goals listed in the post include removing STP dependency by using a loop-free topology without a spanning tree, improving link utilisation via active-active forwarding, maintaining high availability through dual-homed server connectivity, and achieving operational simplicity with predictable failure behaviour. It also states a future readiness objective that positions the design as a path toward Layer-3 and EVPN-based designs.

The paper ties these goals to the selection of a leaf–spine topology with MCLAG and to SONiC’s described technical attributes, including an open, vendor-neutral network operating system, modular services, FRRouting integration, SAI-based hardware abstraction, and automation-friendly interfaces such as CLI, REST, and telemetry.

Overall, the case study reports that migrating from an STP-based Layer-2 network to a SONiC-powered Layer-2 leaf–spine design with MCLAG removed STP from the data plane, enabled active-active server uplinks, kept Layer-3 termination on the firewall, and produced faster and more predictable failure convergence. This Blog Signals brief is a fact-based summary of the vendor blog.