VXLAN Overlays and Underlays

You can use VXLAN as an overlay, a bridge, and a router.

VXLAN uses overlay technology to improve network extensibility and flexibility.

Welcome to this video on Cisco NDFC and the concept of VXLAN underlays and overlays. You will learn the purpose and benefits of VXLAN and how VXLAN underlays and overlays work together to provide network extensibility and flexibility. VXLAN, which stands for virtual extensible LAN, is a technology that enables us to create overlaid networks on top of physical underlay networks. It provides us with greater network extensibility and flexibility compared to traditional VLANs. An underlay network forms the foundation of the overall network infrastructure and is responsible for packet transmissions across networks. It serves as the physical layer upon which the overlay virtual network is built. VXLAN uses overlay technology to provide the same Ethernet layer two network services as VLANs, but with enhanced extensibility and flexibility. At the core of VXLAN overlays is the VXLAN network identifier, or VN ID, which is a twenty four bit segment ID that defines the broadcast domain, also known as the VXLAN segment ID. Here is how the VXLAN overlay functions. First, the server transmits traffic to the ingress VTEP virtual tunnel endpoint. The ingress VTEP encapsulates the traffic into VXLAN packets and signals the underlay IP network. Based on the underlay routing protocol, the packet traverses from the ingress VTEP to the egress VTEP through the IP network. Finally, the egress VTEP removes the VXLAN encapsulation and delivers the traffic to the intended destination server. VXLAN can also act as a bridge for packets. This is how it works. The source end host, host A, and target end host, host B, are associated with VNI thirty thousand. Host A transmits traffic to the directly attached VTEP V1. VTEP V1 performs a lookup based on the packet's destination MAC address, DMAC. VTEP V1 bridges the packets and forwards them to VTEP V2 with a VXLAN header stamped with the VNI thirty thousand. VTEP V2 receives the packets, decapsulates them, and performs a lookup, and then bridges them to the intended destination, host B. VXLAN can also operate as a router within a virtualized network in the following way. If the destination MACDMAK is the default gateway, host A sends traffic to its directly attached VTAP V1. VTEP V1 performs a forwarding information based FIB lookup based on the destination IP address in the packet header. VTEP V1 routes the packets towards VTEP V2 using a VXLAN header with the VRF layer three VNI fifty thousand. VTEP V2 receives the packets, decapsulates them, and performs a lookup, and then rewrites the packets before forwarding them to the destination host F. VTEPs provide end host detection in VXLAN. Here's a breakdown of the steps. When a new end host, host A, attaches to VTEP V1, V1 learns the MAC and IP address of host A via ARP. VTEP V1 advertises the MAC and IP address of host A along with the encapsulation choice, VXLAN, to other VTEPs, V2 and V3, through the BGP route reflector in the spine. The information about Host A is propagated across the VXLAN network. When a host, like Host A, moves from one VTEP, for example, V1, to another, for example, V3, here is what happens. VTEP V3 detects host A and advertises it with a new sequence number, updating the previous instance. VTEP V1 detects the more recent route and withdraws its advertisement. ARP suppression is an advanced function in VXLAN that helps manage ARP requests. Here's how it works. Locally learned IP MAC pairs through ARP and those learned over BGP EVPN are stored in a local ARP suppression cache at each top of rack TOR switch. If an ARP request is received by the source TOR for an IP MAC pair in the cache, the TOR proxies the request on behalf of the destination by using the DMAC. If the destination IP MAC pair is unknown or is a silent end host, the TOR reinjects the ARP request, broadcasting it within the Layer two VNI. Receiving VTEPS decapsulate the VXLAN encapsulated packet and forwards the inner ARP frame to the server facing ports. As long as the destination remains alive, the ARP request reaches the destination, which in turn transmits an ARP response toward the sender. The destination TOR traps the ARP response even though the ARP response is a unicast packet that is directed to the source VM. The destination TOR learns the destination IP and MAC, advertises it over BGP EVPN to all the other TORs, and reinjects the ARP response packet into the IP core network, allowing it to reach the original requester.

An underlay network is the underlying network infrastructure that is in charge of packet transmissions across the networks. The overlay virtual network is constructed on top of the physical underlay network.

VXLAN uses overlay technology to provide the same Ethernet Layer 2 network services that VLAN provides but with greater extensibility and flexibility. This technology uses a VXLAN Network Identifier (VNID or sometimes VNI), which is a 24-bit segment ID that defines the broadcast domain—also called the VXLAN Segment ID. VNID facilitates optimal scaling when you compare it to a VLAN.

The VXLAN Tunnel Endpoint (VTEP) encapsulates and de-encapsulates through the Network Virtual Interface (NVE), the logical interface for those processes to occur. VXLAN may use Intermediate System-to-Intermediate System (IS-IS), OSPF, BGP, and Protocol Independent Multicast (PIM) for the underlay.

VXLAN Overlay

The following VXLAN fabric overview shows how the technology works.

VXLAN fabric operates by the following steps:

The server transmits traffic to the ingress VTEP. The VTEP communicates based on the destination MAC (DMAC) and derives the next hop (or destination VTEP). If the DMAC is the default gateway MAC, first-hop routing will be performed.

Note

When a packet bridges, the target end host's MAC address is stamped in the DMAC field of the inner frame. During packet routing, the default gateway MAC address is stamped in the DMAC field of the inner frame.

The VTEP encapsulates the traffic (frames) into VXLAN packets (depicted in the overlay function shown in the figure) and signals the underlay IP network.
Based on the underlay routing protocol, the packet transmits from the ingress VTEP to the egress VTEP through the IP network.
The destination VTEP removes the VXLAN encapsulation (overlay function) and transmits traffic to the intended server.

VXLAN as a Bridge

VXLAN can act as a bridge for packets.

Each step number corresponds to a number in the following figure:

The VNID of the source end host, Host A, and the target end host, Host B, is 30000. Host A transmits traffic to the directly attached VTEP V1.
V1 performs a lookup based on the DMAC address in the packet header. (For bridged communication, the target end host's MAC address updates in the DMAC field.)
VTEP V1 bridges the packets and transmits toward VTEP V2 with a VXLAN header that has a Layer 2 VNID 30000 stamp.
VTEP V2 receives the packets. After de-encapsulation and lookup, it bridges them to Host B.

VXLAN as a Router

A VXLAN also can serve as a router.

Each step number corresponds to a number in the following figure:

If the DMAC is the default gateway, then host A transmits traffic to its default gateway, after using the Address Resolution Protocol (ARP), which configures on the directly attached VTEP V1.
V1 performs a Forwarding Information Base (FIB) lookup that is based on the destination IP address in the packet header.
VTEP V1 routes the packets and transmits them toward VTEP V2 with a VXLAN header stamped with the virtual routing and forwarding (VRF [Layer 3]) VNID 50000.
VTEP V2 receives the packets. After de-encapsulation, routing lookup, and rewrite, it transmits them to Host F.

VTEP End-Host Detection

VTEP provides end-host detection.

When a new end host (Host A) attaches to VTEP V1, the following actions occur:

VTEP V1 learns the Host A MAC and IP address via ARP (MAC_A and IP_A).
V1 advertises MAC_A and IP_A to the other VTEPs V2 and V3 through the BGP route reflector in the spine.
The encapsulation choice (VXLAN) also is advertised.

VXLAN: Host Moves from VTEP V1 to VTEP V3

If Host A moves from VTEP V1 to V3, the following actions occur:

V3 detects Host A and advertises it with Sequence 1 (updating the previous instance of the sequence, 0). The next hop IP address reassigns it to that of VTEP 3.
VTEP V1 detects a more recent route and withdraws its advertisement.

VXLAN ARP Suppression

ARP suppression is an enhanced function that is configured under the Layer 2 VNID (with the suppress-arp command). Essentially, the IP-MACs learned locally through ARP and the IP-MACs learned over BGP EVPN are stored in a local ARP suppression cache at each top-of-rack (ToR) switch. The source ToR switch traps an ARP request that the end host transmits. The ARP suppression cache performs a lookup with the destination IP as the key. If a hit occurs, then the ToR switch proxies on behalf of the destination host, which is illustrated in the following figure.

If the destination is unknown or the end host is silent, the lookup results in a miss. When this happens, the ToR switch reinjects the ARP request that it received from the requesting end host and broadcasts it within the Layer 2 VNID. The process entails transmitting the ARP request locally over the server-facing ports and transmitting a VXLAN-encapsulated packet with the Layer 2 VNID over the IP core. Every receiving VTEP with membership within the same Layer 2 VNID will de-encapsulate the VXLAN-encapsulated packet.

The receiving VTEPs then forward the inner ARP frame toward the server-facing ports. As long as the destination remains alive, the ARP request reaches the destination, which then transmits an ARP response toward the sender. The receiving ToR switch traps the ARP response, even though that ARP response is a unicast packet that is directed to the source virtual machine (VM), since the ARP suppression feature is enabled.

The ToR switch learns about the destination IP and MAC and advertises it over the BGP EVPN to all the other ToR switches. The ToR switch also reinjects the ARP response packet into the network so that it reaches the original requestor. VXLAN encapsulates it toward the IP core because the original requestor was remote.