Overview

Cisco ACI nodes behave a bit special in regards to memory usage, it’s not unusual to see high memory values of up to 90%. Some monitoring tools are not tuned correct and report the high memory values as error.

Even Cisco himself colors these values wrong in the ACI GUI (Bug: CSCvu05236). Values over 80% are displayed with a red color, even though they are inside the limits.

There was a Cisco ACI operating Guide, which provided information about which values to expect. But it’s no longer available. In the Cisco community is a reference of this document.

Switch memory utilization of 70-75% is normal, even for leaf and spine switches in a new deployment where no configuration has been pushed. Switch memory utilization in ACI mode should not be compared to utilization in standalone NX-OS mode due to the additional ACI-specific processes, which consume additional memory.

You should be concerned only if memory utilization exceeds 90%. Open a TAC case for proactive troubleshooting if memory utilization exceeds 85%."

This matches with what I saw at different customers.

How to check the memory utilization

As always, there are many ways to check the current memory usage of a ACI Node (Leaf or Spine). As long as you don’t find values above 85%, there is no need to worry. In the case of higher memory values, it’s advised to open a TAC case and double check that there is no real issue.

GUI

An easy way to check the current memory situation is in the ACI GUI. After you logged in, navigate to Fabric -> Inventory -> Pod X -> Select a Node.

In the summary tab of the Node you then can see the Hardware usage tab, which includes Memory. As mentioned before, you should ignore the color and check the value.

CLI

On the CLI you have different approaches, my favorite is show system resources. It not only shows the absolut values, it also provides an evaluation of the memory status. Current memory status gives you a good understanding of the current usage and if it’s an issue.

1
2
3
4
5
6

SPINE# show system resources
Load average:   1 minute: 1.77   5 minutes: 1.61   15 minutes: 1.06
Processes   :   741 total, 2 running
CPU states  :   1.7% user,   2.9% kernel,   95.3% idle
Memory usage:   16204908K total,   12505812K used,   3699096K free
Current memory status: OK

You can also leverage one of the many Linux commands to the get memory usage, e.g. cat /proc/meminfo

1
2
3
4

SPINE# cat /proc/meminfo
MemTotal:       16204908 kB
MemFree:         2509876 kB
MemAvailable:    3708012 kB

Or top

1
2
3
4
5
6

SPINE# top
top - 07:33:10 up 221 days,  4:18,  2 users,  load average: 1.15, 0.54, 0.41
Tasks: 330 total,   1 running, 329 sleeping,   0 stopped,   0 zombie
Cpu(s):  1.7%us,  1.8%sy,  0.0%ni, 95.4%id,  0.0%wa,  0.6%hi,  0.6%si,  0.0%st
Mem:  16204908k total, 10510784k used,  2504052k free,     5136k buffers
Swap:        0k total,        0k used,        0k free,  3184936k cached

API

If you prefer the API way, you can also get the information there! :)
There are different classes which provide a different time span of the memory utilization.
E.g. procSysMem1d for 1 day averages. The call itself looks like this:

1

https://{{ip}}/api/class/procSysMem1d.json

All the related classes:

• procSysMem5min
• procSysMem15min
• procSysMem1h
• procSysMem1d
• procSysMem1w
• procSysMem1mo
• procSysMem1qtr
• procSysMem1year

The result will include a list with entries for each node that is part of the fabric. It’s a good way to easy get an overview of the memory usage.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

...
{
 "procSysMem1d": {
  "attributes": {
   "childAction": "",
   "cnt": "7",
   "dn": "topology/pod-2/node-212/sys/procsys/CDprocSysMem1d",
   "freeAvg": "10525492",
   "freeLast": "10515313",
   "freeMax": "10533356",
   "freeMin": "10515313",
   "freeSpct": "0",
   "freeThr": "",
   "freeTr": "0",
   "freeTrBase": "10546052",
   "freeTtl": "73678448",
   "lastCollOffset": "25200",
   "repIntvEnd": "2022-09-22T08:59:58.128+02:00",
   "repIntvStart": "2022-09-22T01:59:58.102+02:00",
   "status": "",
   "totalAvg": "24499856",
   "totalLast": "24499856",
   "totalMax": "24499856",
   "totalMin": "24499856",
   "totalSpct": "0",
   "totalThr": "",
   "totalTr": "0",
   "totalTrBase": "24499856",
   "totalTtl": "171498992",
   "usedAvg": "13974361",
   "usedLast": "13984540",
   "usedMax": "13984540",
   "usedMin": "13966497",
   "usedSpct": "0",
   "usedThr": "",
   "usedTr": "0",
   "usedTrBase": "13953800",
   "usedTtl": "97820527"
  }
 }
},
...

Summary

In this post you learned what it’s important to know about memory utilization on ACI (everything < 85% is fine) and how to check the memory usage through GUI, CLI and API.