ACI/N9K – How to convert a Nexus 9000 from NX-OS to ACI

Some time ago i posted how to convert a ACI switch to NX-OS, now the other way around.

Copy the file to the NX-OS Mode N9k

switch# copy scp: bootflash:
Enter source filename: Downloads/aci-n9000-dk9.12.2.1o.bin
Enter vrf (If no input, current vrf 'default' is considered): management
Enter hostname for the scp server: 192.168.0.5
Enter username: USER
Password:
aci-n9000-dk9.12.2.1o.bin                       1%   15MB   3.8MB/s   04:34 ETA

Change the boot mode to aci

switch(config)# boot aci bootflash:///aci-n9000-dk9.12.2.1o.bin
Warning: Please check list of all ACI supported hardware before doing this operation, not all hardware are supported.
Warning: Booting to an ACI image will remove all nxos configuration and format bootflash. Do you want to continue (y/n)?[n] y
Performing image verification and compatibility check, please wait....

Image verification successful.

That’s it, just boot the switch and you can join the fabric 🙂
Be aware that the first reload can take quite some time! (15minutes+)

switch# reload
!!!WARNING! there is unsaved configuration!!!
This command will reboot the system. (y/n)?  [n] y

Unable to create San-Port-Channel Between Nexus 5548UP and UCS(-Mini)

The Issue

We implemented a new UCS-Mini for a customer with existing Nexus 5548UP (5.1(3)N1(1a)), on the SAN Part we faced some strange issues:

2017 Mar 25 12:11:30 NEX5548-2 %PORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: %$VSAN 300%$ Interface san-port-channel 200 is down (No operational members)
2017 Mar 25 12:11:31 NEX5548-2 Mar 25 12:11:31 %KERN-3-SYSTEM_MSG: fc2_nsh_tx_frame: FC2 s_id/d_id/vsan error: sid=0xfffffe,did=0x0,vsan=300,rctl:0x23,type:0x1,oxid 0x4d,rxid:0xff25 - kernel
2017 Mar 25 12:12:10 NEX5548-2 %PORT-5-IF_PORT_QUIESCE_FAILED: Interface fc1/20 port quiesce failed due to failure reason: Force Abort Due to Link Failure (NOS/LOS) (0x119)
2017 Mar 25 12:12:10 NEX5548-2 %PORT-5-IF_DOWN_OLS_RCVD: %$VSAN 300%$ Interface fc1/20 is down (OLS received) san-port-channel 200
2017 Mar 25 12:12:10 NEX5548-2 Mar 25 12:12:10 %KERN-3-SYSTEM_MSG: fc2_nsh_tx_frame: FC2 s_id/d_id/vsan error: sid=0xfffffe,did=0x0,vsan=300,rctl:0x23,type:0x1,oxid 0x5a,rxid:0xff32 - kernel

The san-port-channel was really basic and added to just one VSAN

interface san-port-channel 200
  channel mode active
  switchport mode F
  switchport trunk mode off

vsan 220 interfaces:
    san-port-channel 100 san-port-channel 200

There was also an existing UCS where the san-port-channel worked without any issue

san-port-channel 100 is up
    Hardware is Fibre Channel

Solution

After some looking around i found a bug that matched pretty good on the cisco page.
I checked the MAC OUI on the UCS Mini

UCS-Mini-A# connect nxos
.
.
UCS-Mini-A(nxos)# show int fc1/1
fc1/1 is down
    Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
    Port WWN is XX:XX:00:de:fb:XX:XX:XX

These matches the OUIs described in the bug

Add MAC OUI “002a6a”, “8c604f”, “00defb” for 5k/UCS-FI

After upgrading the Nexus 5548UP to 5.2.1.N1.9b i was finally able to bring the san-port-channel up between the Nexus and the UCS-Mini.

Software
  BIOS:      version 3.6.0
  loader:    version N/A
  kickstart: version 5.2(1)N1(9b)
  system:    version 5.2(1)N1(9b)

2017 Mar 26 07:52:12 NEX5548-2 %PORT-5-IF_UP: %$VSAN 300%$ Interface san-port-channel 200 is up in mode F

Nexus 1000v – Port-Profile Error ‘MSP-5-PP_UPDATE_FAILED’

I tried to create a new port-profile on a Nexus 1000V and got the error

2016 Oct 14 10:33:35 N1Kv %MSP-5-PP_UPDATE_FAILED: Update of port-profile 'New-Port-Group' on the vCenter Server failed. Please  verify port-profile config.

This error can appear if you configure more max-ports on the port-profiles than you specified in the ‘svs connection vcenter’. In my case i had overprovisioned the port-profiles with ‘max-group 512’, so i just reduced the max-port on some port-profiles and this solved the issue.

ACI/N9K – How to convert a Nexus 9000 from ACI Mode to NX-OS (Standalone)

Get the standalone software here

First you have to copy the new nexus 9000 standalone firmware to the APIC:

admin@apic1:~> scp richy@YOURIP:Downloads/nxos.7.0.3.I2.2a.bin .
nxos.7.0.3.I2.2a.bin                          100%  513MB   9.2MB/s   00:56

And now you can push it to the appropriate Nexus 9000. (The user/password matches the APIC)

admin@apic1:~> scp nxos.7.0.3.I2.2a.bin admin@leaf02:bootflash
Password:
nxos.7.0.3.I2.2a.bin                          100%  513MB   7.0MB/s   01:13

Now reboot the switch and break into the load prompt, this can be done with Control+C in Putty.

loader >
loader > boot nxos.7.0.3.I2.2a.bin

After the switch is booted up you got the default prompts (POAP, Secure Admin PW, etc..) When you finaliy reach the CLI you have to Set the Boot Path!

switch(config)# boot nxos bootflash:///nxos.7.0.3.I2.2a.bin
Performing image verification and compatibility check, please wait....

Also save the config!

switch# copy running-config startup-config
[########################################] 100%
Copy complete.

Now is time to verify that everything is fine with the new image (should, as it already booted ;-))

switch# show boot
Current Boot Variables:

sup-1
NXOS variable = bootflash:/nxos.7.0.3.I2.2a.bin <---- Good
No module boot variable set

Boot Variables on next reload:

sup-1
NXOS variable = bootflash:/nxos.7.0.3.I2.2a.bin <---- Good
No module boot variable set


switch# show install all impact

.
.
.

Compatibility check is done:
Module  bootable          Impact  Install-type  Reason
------  --------  --------------  ------------  ------
     1       yes  non-disruptive          none



Images will be upgraded according to following table:
Module       Image                  Running-Version(pri:alt)           New-Version  Upg-Required
------  ----------  ----------------------------------------  --------------------  ------------
     1        nxos                              7.0(3)I2(2a)          7.0(3)I2(2a)            no
     1        bios     v07.41(10/12/2015):v07.17(09/10/2014)    v07.34(08/11/2015)            no

That’s it, make a final boot and your switch is now ACI-Free!
Maybe there is the time you want to go back to ACI, check out my new post!