Virtual & Physical Network Connectivity

Virtual & Physical Network Connectivity

Logical Router VLAN LIF

The logical router’s LIFs may connect to VXLAN backed dvPortgroups (logical switches) or VLAN backed vPortgroups. Any logical router may have a direct Layer 3 connection to the physical world if the following condition is met: It is a distributed logical router and all ESXi hosts running the instance of the logical router have access to the same VLAN backed dvPortgroup.

If any of the ESXi hosts in the transport zone did not have access to the VLAN backed dvPortgroup, there would be no way to configure the VLAN LIF for the logical router copy in that ESXi host.

vMAC is the same across all ESXi hosts and the pMAC is different in each ESXi host, while the LIF IP remains the same in all ESXi hosts running the same copy of the logical router. The vMAC is seen by the physical switches when the logical router has VLAN LIFs.

Let’s do some packet walks to see what happens when the ESXi hosts running a copy of the same logical router send and respond to ARP requests over VLAN LIFs.

Above figure shows a logical router named Physical Access that has a VLAN LIF in dvPortgroup Physical-PG and the following configurations:

• Virtual machines L3-T-VM1 and L3-T-VM2 and logical router Physical Access are connected to the same logical switch.
• Virtual machines L3-T-VM1 and L3-T-VM2 have an ARP entry for their default gateway, which is the IP of logical router’s Physical Access VXLAN LIF.
• Virtual machine L3-T-VM1 is running in ESXi host COM-B1-ESXi01.
• Virtual machine L3-T-VM2 is running in ESXi host COM-B1-ESXi02.
• dvPortgroup Physical-PG is in VLAN 10.
• Physical server L3-Server-P1 is connected to a physical switch in VLAN 10.
• Physical server L3-Server-P1’s default gateway IP is the IP of the logical router’s Physical Router VLAN LIF.

Step 1. Virtual machine L3-T-VM1 sends traffic destined for L3-Server-P1.The destination MAC is L3-T-VM1’s default gateway, the vMAC.

Step 2. Physical Access, in COM-B1-ESXi01, receives the frame, does a routing table lookup, and determines the egress interface to be the VLAN LIF.

Step 3. Physical Access sends an ARP request out of VLAN LIF with a source MAC of the vMAC, 00:50:56:56:44:52.

Step 4. The frame is received by the physical switch L2-SWITCH, which learns MAC 02:50:56:56:44:52 is coming from port 1/1.L2-SWITCH adds the entry to its MAC table.

Step 5. L2-SWITCH forwards the frame out of port 1/3 as shown in below figure

Step 6. Now virtual machine L3-T-VM2 sends traffic destined for L3-Server-P1, JUST after L3-T-VM1 had sent its packet.The destination MAC is L3-T-VM2’s default gateway, the vMAC.

Step 7. Physical Access, in COM-B1-ESXi02, receives the frame, does a routing table lookup, and determines the egress interface to be the VLAN LIF.

Step 8. Physical Access sends an ARP request out of VLAN LIF with a source MAC of the vMAC, 02:50:56:56:44:52.This is the same MAC from step 3.

Step 9. The frame is received by L2-SWITCH, which learns MAC 02:50:56:56:44:52 is coming from port 1/2.

Do you see the problem? L2-SWITCH sees the same vMAC bouncing back and forth between ports 1/1 and 1/2 and this is called a MAC flap, and it is BAD. MAC flaps can cause Layer 2 switches to start flooding frames. In this packet walk we just concluded, only L2-SWITCH experienced the MAC flap, but in practice, the MAC flap may be experienced by many of the physical switches in the same Layer 2 domain, including the Top of Rack switches.

Designated Instance

The Designated Instance is the ESXi host, randomly selected by the NSX Controller responsible for the logical router, that responds to ARP requests for the VLAN LIF IP and that sends out ARP requests over the VLAN LIF. No other ESXi host running a copy of the logical router sends an ARP request over the VLAN LIF nor responds to an ARP request for the IP of the VLAN LIF. By assigning the Designated Instance to send ARP requests over the VLAN LIF and to respond to ARP requests for the IP of the VLAN LIF, the vMAC will only be visible to physical switches over the port leading back to the Designated Instance. There is a single Designated Instance per VLAN LIF.