Deploy cross-connect to extend you Cisco ACI interpod network (IPN)

Multipod prerequisites

To be able to extend a Cisco ACI Fabric into new pods, it is mandatory to take care of following prerequisites

• the maximum latency supported between Pods is 50 msec RTT
• minimum of 1700 bytes for MTU (a MTU 9150 is recommended)
• OSPF protocol between spines and IPN in all Pods
• full PIM BiDir system between all pods

Note — for more information on Cisco ACI multipod, please refer to the official design guide : Cisco ACI multipod design guide

why could we need cross-connect ?

It may happen that, in the L3 network that connects all your pods, the above prerequisites are not met. The case of PIM BiDir is representative of this problem, as it must be extended on all the paths of the IPN network. This is generally not supported by existing MPLS backbones or firewalls that may be located on these paths.

In these cases, the cross-connect feature allows you to create VxLAN tunnels that mask the entire network between your pods.

In the following diagram

• your backbone allow IP connectivity between xconnect_31 and xconnect_32 (OSPF in this case).
• BGP-EVPN will extend vxlan between them
• xconnect_31 and xconnect_32 will deliver layer 2 link to each IPN.
• Finally, IPN_31 and IPN_32 will act as if they would be connected to each other. And you can setup a direct point-to-point OSPF neighbor with PIM BiDir neighboring

In that design the xconnect tunnel start on interface eth1/60 of xconnect_31 device and ends in interface eth1/61 of xconnect_32 device.

In addition

• MP-BGP between xconnect_31 and xconnect_32 devices requires a simple IP connectivity thru the provider network
• MP-BGP ensures the exchange of VNI information
• All traffic (OSPF, PIM BiDir, data and so on) between IPN_131 and IPN_132 will be encapsulated in vxlan tunnel.

configuration

the xconnect feature configuration looks as follows

BGP-EVPN

router bgp 650zz

router-id x.x.x.x
log-neighbor-changes
address-family l2vpn evpn
neighbor y.y.y.y
remote-as 650zz
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended

VXLAN

vlan 100

vn-segment 100000
xconnect

interface nve1s

no shutdown
host-reachability protocol bgp
source-interface loopback1
global ingress-replication protocol bgp
member vni 100000

Link to IPN

interface Ethernet1/54

description To_IPN01_e1/57
switchport
switchport mode dot1q-tunnel
switchport access vlan 100