AVI Networks – Reset Controller

Sometimes it’s required to reset a controller, this can be done from the cli.

First login to the controller you wish to reset

→ ssh admin@192.168.128.8
Avi Cloud Controller

Avi Networks software, Copyright (C) 2013-2017 by Avi Networks, Inc.
All rights reserved.

Version:      18.2.2
Date:         2019-03-06 09:07:37 UTC
Build:        9224
Management:   192.168.128.8/24                UP
Gateway:      192.168.128.1                   UP



admin@192.168.128.8's password:


The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
admin@192-168-128-8:~$

After that you need to switch to the AVI Shell and there you can issue the ‘reboot clean’ command. Credentials for the AVI Shell are the same as for the SSH Login.

admin@192-168-128-8:~$ shell
Login: admin
Password:

[admin:192-168-128-8]: > reboot clean

This will erase the entire configuration and reboot the cluster

Would you like to proceed? (yes/no): yes

This may take 2-3 minutes to complete. Please wait...

Lost connectivity to the controller -- Retrying to connect
Re-established connectivity -- Please retry the command

Broadcast message from root@192-168-128-8 (somewhere) (Fri Mar  8 16:46:51 2019)

Rebooting this VM because of the cluster event 'clean reboot'

Cisco ACI – Convert Leaf Ports (Uplink to Downlink)

Since ACI release 3.1(1) it’s now possible to use some of the Leaf Uplinks as Downlink Ports. This could help you out if there is a need for some 40/100G ports but you are currently only running 10G Leafs.
Currently the following Leafs support the conversion:

  • N9K-C9348GC-FXP
  • N9K-C93180LC-EX and N9K-C93180YC-FX
  • N9K-93180YC-EX, N9K-C93180YC-EX, and N9K-C93180YC-EXU
  • N9K-C93108TC-EX and N9K-C93108TC-FX
  • N9K-C9336C-FX2 (only downlink to uplink conversion supported)

There are some limitations, check them out on the Cisco Page.

Our use case was to use some of the 40/100G Ports as Downlinks on a N9K-C93180YC-FX. The main limitation there is that the last 2 ports (53 and 54) don’t support conversion, no issue if you use them as Uplinks anyway.

(more…)

ACI/N9K – How to convert a Nexus 9000 from NX-OS to ACI

Some time ago i posted how to convert a ACI switch to NX-OS, now the other way around.

Copy the file to the NX-OS Mode N9k

switch# copy scp: bootflash:
Enter source filename: Downloads/aci-n9000-dk9.12.2.1o.bin
Enter vrf (If no input, current vrf 'default' is considered): management
Enter hostname for the scp server: 192.168.0.5
Enter username: USER
Password:
aci-n9000-dk9.12.2.1o.bin                       1%   15MB   3.8MB/s   04:34 ETA

Change the boot mode to aci

switch(config)# boot aci bootflash:///aci-n9000-dk9.12.2.1o.bin
Warning: Please check list of all ACI supported hardware before doing this operation, not all hardware are supported.
Warning: Booting to an ACI image will remove all nxos configuration and format bootflash. Do you want to continue (y/n)?[n] y
Performing image verification and compatibility check, please wait....

Image verification successful.

That’s it, just boot the switch and you can join the fabric 🙂
Be aware that the first reload can take quite some time! (15minutes+)

switch# reload
!!!WARNING! there is unsaved configuration!!!
This command will reboot the system. (y/n)?  [n] y

Unable to create San-Port-Channel Between Nexus 5548UP and UCS(-Mini)

The Issue

We implemented a new UCS-Mini for a customer with existing Nexus 5548UP (5.1(3)N1(1a)), on the SAN Part we faced some strange issues:

2017 Mar 25 12:11:30 NEX5548-2 %PORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: %$VSAN 300%$ Interface san-port-channel 200 is down (No operational members)
2017 Mar 25 12:11:31 NEX5548-2 Mar 25 12:11:31 %KERN-3-SYSTEM_MSG: fc2_nsh_tx_frame: FC2 s_id/d_id/vsan error: sid=0xfffffe,did=0x0,vsan=300,rctl:0x23,type:0x1,oxid 0x4d,rxid:0xff25 - kernel
2017 Mar 25 12:12:10 NEX5548-2 %PORT-5-IF_PORT_QUIESCE_FAILED: Interface fc1/20 port quiesce failed due to failure reason: Force Abort Due to Link Failure (NOS/LOS) (0x119)
2017 Mar 25 12:12:10 NEX5548-2 %PORT-5-IF_DOWN_OLS_RCVD: %$VSAN 300%$ Interface fc1/20 is down (OLS received) san-port-channel 200
2017 Mar 25 12:12:10 NEX5548-2 Mar 25 12:12:10 %KERN-3-SYSTEM_MSG: fc2_nsh_tx_frame: FC2 s_id/d_id/vsan error: sid=0xfffffe,did=0x0,vsan=300,rctl:0x23,type:0x1,oxid 0x5a,rxid:0xff32 - kernel

The san-port-channel was really basic and added to just one VSAN

interface san-port-channel 200
  channel mode active
  switchport mode F
  switchport trunk mode off

vsan 220 interfaces:
    san-port-channel 100 san-port-channel 200

There was also an existing UCS where the san-port-channel worked without any issue

san-port-channel 100 is up
    Hardware is Fibre Channel

Solution

After some looking around i found a bug that matched pretty good on the cisco page.
I checked the MAC OUI on the UCS Mini

UCS-Mini-A# connect nxos
.
.
UCS-Mini-A(nxos)# show int fc1/1
fc1/1 is down
    Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
    Port WWN is XX:XX:00:de:fb:XX:XX:XX

These matches the OUIs described in the bug

Add MAC OUI “002a6a”, “8c604f”, “00defb” for 5k/UCS-FI

After upgrading the Nexus 5548UP to 5.2.1.N1.9b i was finally able to bring the san-port-channel up between the Nexus and the UCS-Mini.

Software
  BIOS:      version 3.6.0
  loader:    version N/A
  kickstart: version 5.2(1)N1(9b)
  system:    version 5.2(1)N1(9b)

2017 Mar 26 07:52:12 NEX5548-2 %PORT-5-IF_UP: %$VSAN 300%$ Interface san-port-channel 200 is up in mode F

BFD and ip redirects

We faced some strange ICMP redirect messages today on one of our devices after we configured BFD for BGP.

Device1

ICMP: bogus redirect from 192.168.100.1 - for 192.168.100.2 use gw 192.168.100.2
      gateway address is one of our addresses
ICMP: bogus redirect from 192.168.100.1 - for 192.168.100.2 use gw 192.168.100.2
      gateway address is one of our addresses
ICMP: bogus redirect from 192.168.100.1 - for 192.168.100.2 use gw 192.168.100.2
      gateway address is one of our addresses

So we checked the device that was sending these redirects and did a short ethanalyzer capture
Device2

ethanalyzer local interface inband-in vdc vdc2 capture-filter "host 192.168.100.2" limit-captured-frames 0
Capturing on inband
192.168.200.2 -> 192.168.200.2  UDP 60 Source port: 49152  Destination port: bfd-echo
192.168.200.2 -> 192.168.200.2  UDP 60 Source port: 49152  Destination port: bfd-echo
192.168.200.2 -> 192.168.200.2  UDP 60 Source port: 49152  Destination port: bfd-echo
192.168.200.2 -> 192.168.200.2  UDP 60 Source port: 49152  Destination port: bfd-echo

So these redirect messages where triggered from the BFD Echo packets that Device2 received from Device1.
We simply forgot to disable `ip redirects` on the interface between Device2 and Device1, after we changed this the ICMP bogus redirect messages where gone.

interface port-channel1
  <strong>no ip redirects</strong>

This is documented on various points on the cisco page, for example here.

Before using BFD echo mode, you must disable the sending of Internet Control Message Protocol (ICMP) redirect messages by entering the no ip redirects command, in order to avoid high CPU utilization.

Cisco Champion 2017

I just got the mail that i was accepted to the Cisco Champions 2017 program, this is the first year for me!

What makes a Cisco Champion? Quote from Cisco.com:
Passion, plus a desire to share their perspectives with the community. There are Cisco Champions all over the world. They represent a variety of segments across the IT industry. And they offer their time to help others learn about Cisco and connect with Cisco in unique ways.

Thanks to Cisco for the opportunity to be member of this program!

Happy New Year & Resolutions for 2017

2016 is over, at least here in switzerland. It was an interesting year for me, i attend my first Cisco Live (@Vegas) and had to recertify my CCIE for the first time. As the new year starts i’m taking a new challenge @Netcloud as Network Engineer. I have different things that i want to accomplish in 2017.

Short Description
Run on 200 Days About 4 Days a week, this will be hard but should be doable
First Marathon Already booked, Berlin in September
Read 26 Books I will create a list and keep track of this on a separate post
Improve my Spanish I learned some basic Spanish skills last year and put them in use on my trip to central america, but i definitely want improve them!
Get my CCIE Datacenter I'm already working on this
Blog Redesign An easy one, but it's time for a new look