Arista Leaf-Spine Architecture – Blog 2: L2LS and L3LS Designs

Layer 2 Leaf-Spine – L2LS design

In this design, 2 spine switches are paired as an MLAG domain and are presented to the leaf switches as a single switch. All links in the topology are used for forwarding with no blocked ports. In simple terms, this is a Layer-2 data center design. The rest of the network connects to the Spine switches which perform both Layer-2 and Layer-3 functions and act as the Inter-VLAN gateway for the data center VLANs.

Below are the main advantages of L2LS design:

• Allows for continuity during migration from Legacy 3-tier design to Leaf-spine design with minimal reconfiguration of the server VLANs.
• The spine layer is presented as a single switch to the leaf switches allowing for deterministic layer 2 paths.
• No STP problems as MLAG removes Layer-2 loops in the topology.
• The main advantage is the interoperability in multi-vendor deployments due to open-standard protocols used.

Layer 3 Leaf-Spine – L3LS design

In this design, we do not have the 2 switch limitation on spine switches. Any number of spine switches can be used subject to limitations on ECMP scalability and port availability on the switch hardware of the leaf switches. 

Using ECMP (Equal Cost Multipathing), all available uplinks from a leaf are used in forwarding. The actual cost of reaching the destination leaf switch is determined by the underlay routing protocol. All links in the topology are used for forwarding. There is no STP required as the topology is completely Layer-3. In simple terms, this is a Layer-3 data center design. 

The rest of the network typically connects via dedicated Leaf switches called Border-leaf switches.

Layer-3 IP Fabric with a Layer-2 Overlay

The L3LS design requires 2 protocols for route exchange. One routing protocol is needed for the Underlay to provide reachability between all the switches (Leaf and Spine) in the topology. For this purpose, typically eBGP is used between the Leafs and Spines with the Spines in a separate private AS number and the Leafs in their own POD specific/single leaf AS number. Physical interface IPs are used for the eBGP peering to ensure route withdrawal in case of link failures.

A second routing protocol (BGP) is needed to exchange EVPN routes using MP-BGP. EVPN is deployed in conjunction with VXLAN where EVPN performs the control-plane function for the VXLAN data-plane. While it is possible to use VXLAN flooding using Head-end replication (HER), it is not recommended as it is a sub-optimal means of learning MAC-IP routes needed for East-West traffic forwarding.

Another important point to note is that it is possible to support dual-homed compute scenarios in L3LS designs using MLAG on a pair of Leaf switches. In this case, the leaf switches are logically configured as a single VTEP by using an anycast VTEP IP on their loopback interfaces which participates in the EVPN overlay routing protocol.

VXLAN is used as the L2 overlay on top of this L3 topology for actual data forwarding.

Below are the main advantages of L3LS design:

• Allows for Layer 2 communication across layer 3 boundaries.
• High levels of horizontal scaling can be achieved by increasing the number of spine switches
• No STP problems as it is a complete Layer-3 network.
• Interoperability in multi-vendor deployments due to open-standard protocols (EVPN, VXLAN) used.
• ECMP routing
• Deterministic failover paths

Conclusion – 

Rahi can help enterprises identify and deploy the latest leaf-spine solutions available in the market from a multitude of vendors. Rahi has extensive experience in deploying highly scalable data center networks across the globe and experienced professional services and managed services teams for Day 1 configuration and Day 2 support.

If you want to learn more please read part one.