BFD and ip redirects

We faced some strange ICMP redirect messages today on one of our devices after we configured BFD for BGP.

Device1

ICMP: bogus redirect from 192.168.100.1 - for 192.168.100.2 use gw 192.168.100.2
      gateway address is one of our addresses
ICMP: bogus redirect from 192.168.100.1 - for 192.168.100.2 use gw 192.168.100.2
      gateway address is one of our addresses
ICMP: bogus redirect from 192.168.100.1 - for 192.168.100.2 use gw 192.168.100.2
      gateway address is one of our addresses

So we checked the device that was sending these redirects and did a short ethanalyzer capture
Device2

ethanalyzer local interface inband-in vdc vdc2 capture-filter "host 192.168.100.2" limit-captured-frames 0
Capturing on inband
192.168.200.2 -> 192.168.200.2  UDP 60 Source port: 49152  Destination port: bfd-echo
192.168.200.2 -> 192.168.200.2  UDP 60 Source port: 49152  Destination port: bfd-echo
192.168.200.2 -> 192.168.200.2  UDP 60 Source port: 49152  Destination port: bfd-echo
192.168.200.2 -> 192.168.200.2  UDP 60 Source port: 49152  Destination port: bfd-echo

So these redirect messages where triggered from the BFD Echo packets that Device2 received from Device1.
We simply forgot to disable `ip redirects` on the interface between Device2 and Device1, after we changed this the ICMP bogus redirect messages where gone.

interface port-channel1
  <strong>no ip redirects</strong>

This is documented on various points on the cisco page, for example here.

Before using BFD echo mode, you must disable the sending of Internet Control Message Protocol (ICMP) redirect messages by entering the no ip redirects command, in order to avoid high CPU utilization.

Error: Error executing command on leaf02. Error Code: 255

I had a issue today with running remote commands on one of my fabric switches, always generated an error.

apic1# fabric leaf02 show switchname
----------------------------------------------------------------
 Node 102 (leaf02)
----------------------------------------------------------------
Error: Error executing command on leaf02. Error Code: 255

I tried to directly connect and found the error

admin@apic1:attach leaf02
This command is being deprecated on APIC controller, please use NXOS-style equivalent command
# Executing command: ssh leaf02 -b 10.127.240.1
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
d1:f1:c4:8a:3e:a7:df:4a:76:bf:ec:01:bb:0d:28:99.
Please contact your system administrator.
Add correct host key in /home/admin/.ssh/known_hosts to get rid of this message.
Offending key in /home/admin/.ssh/known_hosts:2
Password authentication is disabled to avoid man-in-the-middle attacks.
Keyboard-interactive authentication is disabled to avoid man-in-the-middle attacks.

Permission denied (publickey,password,keyboard-interactive).

I think this was the switch that i converted to NX-OS standalone mode, and thus it changed the host key.

The fix is easy, open the file and delete the offending key

vi /home/admin/.ssh/known_hosts

After that i was able to run commands from the apic and also connect to the fabric switch again.

Cisco ACS – Patch Install error (% Manifest file not found in the bundle)

I’m sure that it’s not the first time i felt for this …
Tried to install a ACS Patch this morning and got an error:
% Manifest file not found in the bundle

First i used the wrong command:
patch install 5-4-0-46-5.tar.gpg FTP

ACS01/admin# patch install 5-4-0-46-5.tar.gpg FTP
Save the current ADE-OS running configuration? (yes/no) [yes] ?
Generating configuration...
Saved the ADE-OS running configuration to startup successfully
Initiating Application Patch installation...
% Manifest file not found in the bundle

Easy solution, use the right command! 🙂
acs patch install 5-4-0-46-5.tar.gpg repository FTP

ACS01/admin# acs patch install 5-4-0-46-5.tar.gpg repository FTP
Installing ACS patch requires a restart of ACS services. Continue?  (yes/no) yes
Calculating disk size for /opt/CSCOacs/patches
Total size of patch files are 1079 M.
Max Size defined for patch files are 1000 M.
WARNING: Patch of size 1079 M exceeds the allowed quota of 1000 M.
This will not prohibit patch installation process as long as there is enough disk space.
Please note that this indicates you should consider moving ACS to a higher disk space machine
Stopping ACS.
Stopping Management and View...............................................................
Stopping Runtime.........................
Stopping Database.....
Stopping Ntpd....
Cleanup..
Stopping log forwarding .....
Installing patch version '5.4.0.46.5'
Installing ADE-OS 1.2 patch.  Please wait...
Decompressing patch files 5.4.0.46.5 ...
About to install files
Removing old war
Removing old war
Removing old war
Removing old war
Removing old war
Removing old war
Removing old war
Removing old war
Removing old war
/opt/CSCOacs/patches/5-4-0-46-5
Patch '5-4-0-46-5' version '5.4.0.46.5' successfully installed
Starting ACS ....

To verify that ACS processes are running, use the
'show application status acs' command.