Bits & Bytes of Remote Leaf

Remote leaf Discovery

When Ever Remote leafs are connected and is powered ON, they are automatically discovered and configured. In order to achieve this discovery, APIC controller and IPN must be configured and ready.

This discover is done in two process Step.

Step1:  IP address allocation to uplink interfaces and configuration push to Remote leaf.

• As soon as Remote leaf is powered on, its sends the DCHP discover out of its uplink interfaces.
• Uplink Router Upon receipt, relays this DHCP discover to APIC Controller.
• APIC controller then send the DHCP Offer message for Uplink interface of remote leaf, along with bootstrap configuration in DHCP offer message.
• Upon receipt of DHCP Offer and Bootstrap file, Remote leaf send the DHCP request to APIC for its Uplink Interface IP address.
• APIC Controller Finally sends the DHCP ACK to complete IP address allocation for Uplink IP address.
• Inorder to get the Bootstrap Configuration rom APIC, Remote leaf automatically configures the static route with next hop of upstream router.
• Once the Configuration file is received and remote leaf is configured as per file, this static route gets removed.

Below figure shows IP address assignment for remote leaf Uplink and configuration push to Remote leaf.

Now Once Remote leaf is configured, it will go to next step.

Step2: TEP Address assignment to remote Leaf.

• Once Remote leaf gets IP connectivity via its Uplink interface to APIC controller, its send the DHCP discover message to APIC to receive TEP address.
• APIC Controller upon receipt, will send DHCP offer for TEP address.
• Remote Leaf, picks up one of the APIC controller and sends the DHCP request for TEP address
• APIC controller, finally sends the DHCP ACK to complete the DHCP discovery process.
• Once this process is completed, Remote leaf is properly discovered by APIC and now part of ACI fabric.

Below figure describes the TEP address assignment of remote Leaf.

Endpoint Connectivity Option

• In order to connect the Endpoint to Remote leaf, there are following options available
• End point can be connected to remote leaf via vPC port-channel.
• End points can be connected to single Remote Leaf as orphan Port.
• End Points can be connected to remote leaf by using the NIC redundancy option (Active/Active), Active/Standby.

Option 2 and Option 3 is only supported from ACI 3.2 release. 

Refer Below figure to explore options for End Point connectivity.

Remote Leaf Control Plane Concept

As soon as Endpoint are discovered by remote Leaf, Remote leaf builds the COOP session with Spine on main DC and share this Endpoint information and its reachability Information. For Control plane working, there are some specific TEP address that are defined on Spine or Remote for control plane and data plane information.

RL-DP-TEP: This is the TEP IP address assigned to remote Leaf from TEP pool. It is used as Source IP address for VXLAN packet, sourced from remote leaf when endpoint are not connected via vPC.

RL-vPC-TEP: This is Anycast IP address assigned to vPC domain of remote Leaf from TEP pool. It is used as Source IP address for VXLAN packet, sourced from remote leaf when endpoint are connected via vPC.

 RL-Ucast-TEP: This is the one Anycast IP address assigned to all Spine switches from routable TEP pool if configured else from ACI Pod TEP pool. When Endpoint traffic from remote leaf is sent to endpoint connected to ACI main DC, VXLAN uses this RL-Ucast-TEP as Destination and RL-DP-TEP or RL-vPC-TEP as source Address.

RL-Mcast-TEP: This is the one Anycast IP address assigned to all Spine switches from routable TEP pool if configured else from ACI Pod TEP pool. When Endpoint BUM traffic from remote leaf is sent to endpoint connected to ACI main DC, VXLAN uses this RL-Mcast-TEP as Destination and RL-DP-TEP or RL-vPC-TEP as source Address.