UCS FC uplinks can be bound to vHBA’s by VSAN or Pining. Each has pro’s and con’s.
VSAN
- UCS balanced (round robin FLOGI’s to each uplink).
- If a path fails round robin kicks in and FLOGI’s to next uplink
- VSAN’s are “Best Practice” from Cisco
- Less configuration
- New uplinks can be added dynamically but FLOGI’s will NOT redistribute immediately. A reboot of Blades is needed to redistribute over new FC uplink
Pining
- Control Freak friendly.
- Servers per FC uplink can be managed
- If a path fails round robin does not happen. Link stays down until restored. This is an advantage in ensuring QoS bandwidth per vHBA/Blade
VSAN uplink example
Multiple FC uplinks are round robin load balanced with a common VSAN
Best practice for VSAN’s is a unique VSAN ID per uplink VSAN. Also the VSAN ID and the FCoE VLAN ID should not be the same. I prefer VSAN 1000 on FIA, and VSAN 2000 on FIB. If there are multiple fabrics being connected to I increment by 100. 1000, 1100, 1200,etc. Remember VSAN’s should Never be common between FI’s. Dual Fabric is the only way, if you want to keep failover and controller port login balance a certainty etc.
Bind each group of FC uplink to corresponding VSAN. If FC uplinks are added later they are simply added to VSAN. Easy peasy huh?
Bind each vHBA to corresponding VSAN.
Pining/VSAN uplink example
Pinning and unique VSAN’s can co-exist. A Brocade switch upstream has completely no idea what a VSAN is. So realize that the VSAN is strictly taking place on the FI. So for Brocade upstram VSAN 1 could be used. BUT best practice is unique VSAN’s (an no pining technically). The example below is only using VSAN 1. Needs to be fixed to proper VSANs 1000 & 2000, but other than that, example is 100%.
Basically steps are the same with added Pining. Create FC uplink ports (best to use the same on each FI if possible but not mandatory).
In vHBA template select proper VSAN and Pin group.
With most UCS builds I use the VSAN method. But the control freak in my likes Pining better. Following the “Rules of SAN” and my Brocade background Pining is a no-brainer. But the VSAN method fits UCS best practices and does allow adding and removing uplinks easy. Pining has made needed changes very difficult for me.
Unless there is a reason to hardcose pining, go VSAN. Even if you have some traditional SAN experience and it is hard to deal with.
Some useful CLI commands
To see what port vHBA’s are logged into fabric
connect nxos b
show npv external-interface-usage
NPV Traffic Usage Information:
—————————————-
Server-If External-If
—————————————-
vfc911 fc1/31
vfc919 fc1/31
vfc927 fc1/32
vfc935 fc1/32
vfc943 fc1/31
vfc951 fc1/32
vfc959 fc1/32
vfc967 fc1/31
vfc975 fc1/32
vfc983 fc1/32
vfc991 fc1/31
vfc999 fc1/32
vfc1007 fc1/31
vfc1015 fc1/31
vfc1023 fc1/32
vfc1031 fc1/32
See what VLAN a vHBA is in
connect nxos b
show npv status
npiv is enabled
disruptive load balancing is disabled
External Interfaces:
====================
Interface: fc1/29, VSAN: 2100, FCID: 0x027e00, State: Up
Interface: fc1/30, VSAN: 2100, FCID: 0x02e600, State: Up
Interface: fc1/31, VSAN: 1100, FCID: 0x010600, State: Up
Interface: fc1/32, VSAN: 1100, FCID: 0x010700, State: Up
Number of External Interfaces: 4
Server Interfaces:
==================
Interface: vfc911, VSAN: 1100, State: Up
Interface: vfc919, VSAN: 1100, State: Up
Interface: vfc927, VSAN: 1100, State: Up
Interface: vfc935, VSAN: 1100, State: Up
Interface: vfc943, VSAN: 1100, State: Up
Interface: vfc951, VSAN: 1100, State: Up
Interface: vfc959, VSAN: 1100, State: Up
More related articles below.
UCSGuru has a Great Post on “Understanding UCS VIF Paths” that is a good next step in this study.
Be First to Comment