The Issue
We implemented a new UCS-Mini for a customer with existing Nexus 5548UP (5.1(3)N1(1a)), on the SAN Part we faced some strange issues:
2017 Mar 25 12:11:30 NEX5548-2 %PORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: %$VSAN 300%$ Interface san-port-channel 200 is down (No operational members) 2017 Mar 25 12:11:31 NEX5548-2 Mar 25 12:11:31 %KERN-3-SYSTEM_MSG: fc2_nsh_tx_frame: FC2 s_id/d_id/vsan error: sid=0xfffffe,did=0x0,vsan=300,rctl:0x23,type:0x1,oxid 0x4d,rxid:0xff25 - kernel 2017 Mar 25 12:12:10 NEX5548-2 %PORT-5-IF_PORT_QUIESCE_FAILED: Interface fc1/20 port quiesce failed due to failure reason: Force Abort Due to Link Failure (NOS/LOS) (0x119) 2017 Mar 25 12:12:10 NEX5548-2 %PORT-5-IF_DOWN_OLS_RCVD: %$VSAN 300%$ Interface fc1/20 is down (OLS received) san-port-channel 200 2017 Mar 25 12:12:10 NEX5548-2 Mar 25 12:12:10 %KERN-3-SYSTEM_MSG: fc2_nsh_tx_frame: FC2 s_id/d_id/vsan error: sid=0xfffffe,did=0x0,vsan=300,rctl:0x23,type:0x1,oxid 0x5a,rxid:0xff32 - kernel
The san-port-channel was really basic and added to just one VSAN
interface san-port-channel 200 channel mode active switchport mode F switchport trunk mode off vsan 220 interfaces: san-port-channel 100 san-port-channel 200
There was also an existing UCS where the san-port-channel worked without any issue
san-port-channel 100 is up Hardware is Fibre Channel
Solution
After some looking around i found a bug that matched pretty good on the cisco page.
I checked the MAC OUI on the UCS Mini
UCS-Mini-A# connect nxos . . UCS-Mini-A(nxos)# show int fc1/1 fc1/1 is down Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN) Port WWN is XX:XX:00:de:fb:XX:XX:XX
These matches the OUIs described in the bug
Add MAC OUI “002a6a”, “8c604f”, “00defb” for 5k/UCS-FI
After upgrading the Nexus 5548UP to 5.2.1.N1.9b i was finally able to bring the san-port-channel up between the Nexus and the UCS-Mini.
Software BIOS: version 3.6.0 loader: version N/A kickstart: version 5.2(1)N1(9b) system: version 5.2(1)N1(9b) 2017 Mar 26 07:52:12 NEX5548-2 %PORT-5-IF_UP: %$VSAN 300%$ Interface san-port-channel 200 is up in mode F