Cisco ACI Best Practices
When people starting out with Cisco ACI, they often struggle to find the correct or best settings for their ACI Fabric. Further they look for the correct settings, to make sure that the fabric is working properly.
To help people out, we line out some ACI best practices here, which should help people to set some important settings and configuration correctly. We will update this page in the future, to be sure to include new additions as well.
We try to reference to Cisco wherever possible, but some Best Practices are also based on our own experience of nearly 8 years with ACI.
Cisco ACI Global Settings
Cisco ACI AEP & VLAN Pool Best Practice
An AEP or Attachable Entity Profile, stitches together VLAN Pools & Physical Interface. It’s important to make sure that you have no overlapping VLANs within a single AEP.
With the help of a VLAN Pool, ACI decides which VXLAN ID (VNID) is assigned for each VLAN. Each AEP uses a different VNID Pool, which means that VLAN 20 attached to AEP A has a different VNID then VLAN 20 attached to AEP B.
For that reason it’s Best Practice for VLAN Pools to try to not overlap them. Less is definitely more here, we always try to go with one or two (If we do VMM integration).
The same goes for the AEP, it’s Best Practice there as well to try to have only the minimum required AEPs configured. In a lot of cases one can be enough. If you really need the scale and multiple AEPs and overlapping VLAN Pools, be careful!
Since Version 3.2.6 there is an option to enable Enforce EPG VLAN Validation, which prevents you from configuring overlapping VLANs that are attached to the same EPG.
It can be configured under System -> System Settings -> Fabric Wide Setting.
Issues when not following the Best Practices for AEP & VLAN Pool configuration
If you configure overlapping VLANs and then link them together through the same AEP, you can face various issues in the ACI Fabric.
The VNID assignments can become indeterministic and this leads to issues with STP BPDU flooding, endpoint learning and so on.
Cisco ACI Bridge Domain Best Practices
The Best Practices for the Bridge Domain Depends on different factors, there is a distinction between a Layer 2 Bridge Domain (Where the default gateway is outside ACI) and a Layer 3 Bridge Domain (Where ACI is the default gateway).
Layer 3 Bridge Domain
If the Bridge Domain is the Layer 3 default gateway for the Clients in the EPG, we call it Layer 3 Bridge Domain. In this case you have to decide if you need ARP flooding or not. If you have clustered servers, firewalls with some sort of active/standby mechanism, or Loadbalancers in this Bridge Domain, it’s best practice to enable ARP flooding.
These systems and their failover mechanism often rely on GARP to announce a failover from one of the members to the others. With ARP flooding, these messages are always flooded to all clients in the Bridge Domain.
Personal Recommendation: Enable ARP flooding, never had an issue with it.
Layer 2 Bridge Domain
In case that the default gateway is outside ACI (e.g. a Firewall or a Router), then we talk about a Layer 2 Bridge Domain. Maybe there isn’t even a default gateway and the Bridge Domain only allows connectivity between some systems to sync/replicate data without ever leaving the network.
For a Layer 2 Bridge Domain, it’s Best Practice to set L2 Unknown Unicast to Flood.
On a Bridge Domain with L2 Unkown Unicast set to Flood, you can’t disable ARP flooding. So this Best Practices is enforced ;)
Cisco ACI Contracts Best Practices
Over the years of deploying and operating ACI Fabrics for and with our customers, we learned one important lesson regarding ACI & Contracts.
Keep it simple!
In the beginning we tried different Designs and levels of Contract complexity, but ACI is no Firewall! It’s nearly impossible to manage larger sets of Contracts in production and the benefit they provide don’t justify this operational burden.
ACI is a great Networking Platform, but leave the Security to other Devices.
We have a great success with PBR designs, where we use a simple redirect Contract to forward all the Traffic to a DC Firewall. This Firewall then makes the security decision, is the traffic allowed or not, and send it back to ACI.
Best Practice for Contracts, keep it simple! A single allow (or PBR) is the best thing most of the time.
Cisco ACI Naming Best Practices
Prefixes in Naming of Objects
This is another topic where experience is key, at least it feels to me like that. In the beginning we often included the object type in the name, as example “T-Network” or “TN-Network” for a Tenant called Network. We prefixed the Name itself with the Type.
But the longer we worked with ACI, the more we had to realize that this just pollutes the names and doesn’t provide any real benefit in the end. We came to this conclusion after realizing where and how we use these names in the end.
Working with the API
If you are working with the API, then it’s anyhow always clear with what type of object you are working.
Here you can see the type of the object from the
fvTenant class name and also in the DN
Working in the GUI
In the GUI, it’s always given which object you can choose, so there is never a confusion between a Tenant called “Network” and a Bridge Domain called “Network”.
Best Practice regarding Prefix in names
Don’t use type prefixes in your object names. Just go with a short and clear Name.
A setup like that is completely valid for a lot of cases, if you have a 1 Zone = 1 Tenant Setup, you normally only need one VRF.
Access Port selectors & Port Policy Groups
This is another topic where we made quite a pivot from the original naming we used. It’s tempting to name your Access Port selectors and the respective Policy Groups after the Devices you connect to them. But what if these servers are renamed? You have to delete and recreate new Objects with different Names.
Probably you will just leave it as it is, and the name is just wrong. Been there, done that ;-)
For that reason we propose generic naming and then just work with the Description (Which is the information you see on the Leafs itself).
Best Practice for Access Port selectors
For Access Port selectors, use something in the style of
eth-1-01 or maybe just
1-01. But with that, the information should always
be correct. For VPCs we normally just use the first port in the name.
Not saying this is perfect, but for us, it works better than system names.
Best Practice for Policy Groups
Policy Groups are normally required for the connection of a VPC to the ACI
Fabric. Here we tend to do the same thing as we are doing for the Access Port
Selectors. Use something generic in the format
VPC-L111-ETH-1-01 which would
be the channel on Leaf 111 (and 112) on Port 1.
Just some lines ago I told you to not use prefixes in names and now that? This is a bit a special case, because with this naming its clear that we are looking at a VPC in the Access Port Selector configuration :)
Every Rule has its exception, right?!
Open Best Practices
There are more to cover, especially the following which are hopefully covered soon:
- Cisco ACI Multi Site Best Practices
- Cisco ACI Migration Best Practices
- Cisco ACI Monitoring Best Practices