Routing over Cisco VPC pairs

Multipod prerequisites

To be able to extend a Cisco ACI Fabric into new pods, it is mandatory to take care of following prerequisites

the maximum latency supported between Pods is 50 msec RTT
minimum of 1700 bytes for MTU (a MTU 9150 is recommended)
OSPF protocol between spines and IPN in all Pods
full PIM BiDir system between all pods

VPC pairs in IPN network

The design

In case you need to extend L2 domains between pods in multi-pod for Cisco ACI deployment, you have the possible option to deploy in each pod a VPC pair as IPN devices, as in the following diagram

POD 1
- IPN_p1_1 and IPN_p1_2 are joined into a VPC pair
POD 2
- IPN_p2_1 and IPN_p2_2 are joined into a VPC pair
po10 => 2 physical links are joined into a port-channel between the two pods
po1 => 2 physical links are joined into o port-channel to ensure the VPC peer-link feature with all required vlans

To match the IPN network prerequisites, you have to deploy OSPF neighborings between IPN devices, as in the following diagram.

Po1 : OSPF neighbors between IPN of each site
- OSPF between IPN_S1_1 and IPN_S1_2
- OSPF between IPN_S2_1 and IPN_S2_2
Po10 : OSPF neighbors between IPN of the two sites
- OSPF between IPN_S1_1 and IPN_S1_2
- OSPF between IPN_S2_1 and IPN_S2_2

The issue

If you deploy the above design, you will face the following issue

All the flows routed from the first IPN of a site to the second IPN of the second site will go through the peer-link to be forwarded to a spine. Unfortunately, traffic coming from a vpc port-channel from the first IPN is not allowed to go to the second IPN to be forwarded further. That is why the routed flow will be dropped as shown in the following diagram

The solution

To avoid this problem, you can simply add a (red) link (or an aggregation of links) between the members of the IPN pair. This additional link will be dedicated to routing. In this case, the routed flows will no longer pass via the routing link but via the peer link, as shown in the following diagram

Vincent GROSDOY

Vincent is a network engineer who has been working for several years on corporate networks in various sectors. He has in-depth knowledge of routing, new SDN datacenter solutions and automation tools.