Cisco UCS Firmware Upgrade Procedure – Manual Steps
XSJ from 2.2 (1c) -> 2.2 (3f)
XIR from 2.2 (2c) -> 2.2 (3f)
What's Required for a Firmware Update?
There are 2 bundles that are required for our environment:
- Infrastructure Bundle:ucs-k9-bundle-infra.2.2.3f.A.bin
- B Series Blade Server Bundle: ucs-k9-bundle-b-series.2.2.3f.B.bin
Check List (Check off events as they are completed)
Item No. / Task / Task Started / Task Completed1 / Obtaining Software Bundles from Cisco
2 / Downloading Firmware Images to the Fabric Interconnect from a Local Disk
3 / Determine the Contents of the Firmware Package
4 / Review Release Notes
5 / Back Up the Configuration into an All Configuration Backup File
6 / Disable Call Home
7 / Verify and Enable the User Acknowledgement Option in Maintenance Plan for all Service Profiles and Templates if not already done.
8 / Verify the High Availability Status and Roles of the Cluster Configuration
9 / Verify the Status of the IO Modules
10 / Verify the State of Servers
11 / Verify the Status of Adapters on Servers in the Chassis
12 / Verify There are No Major Faults in UCS
13 / Verify the Number of Ethernet Data Paths
14 / Verify the Data Path for Fibre Channel End-Host Mode
15 / Enumerate the Current Running/Startup and Backup Configs
16 / Activate the Cisco UCS Manager Software
17 / Update the Firmware on the IOMs for each Chassis and both IOM’s in each Chassis
18 / Activate the Firmware on the IOMs for each Chassis and both IOM’s in each Chassis
19 / Activate the Firmware on the Subordinate Fabric Interconnect
20 / Force a Fabric Interconnect Failover
21 / Verify that the Ethernet Data Path has been Restored
22 / Verify the Data Path for Fibre Channel End-Host Mode Has Been Restored
23 / Activate the Firmware on the Primary Fabric Interconnect
24 / Once the FI A upgrade is completed, then promote FI A the primary Fabric Interconnect
25 / Create a Host Firmware Package
26 / Activate the Host Firmware Package
27 / This is where you evacuate guests from each ESXi server and then reboot servers accordingly
28 / Verify Firmware Versions on Components after reboot
29 / Enable Call Home
30 / Final verification of all: UCS Infrastructure, ESXi, Servers, VM Guests, and Datastores
NOTE: From here on onwards the photo snapshots are examples for your reference only
Order to Upgrade Components
The Infrastructure must be upgraded first and then the Servers. The specific order within each bundle should be:
Infrastructure Bundle:
a.UCS Manager
b.IOMs - The new firmware will be flagged as the "Startup Version". They will be rebooted once the Interconnect Switches are upgraded and rebooted
c.Fabric Interconnect Switches
B-Series Blade Server Bundle:
a.CIMC firmware
b.BIOS firmware
c.Adapter firmware
d.Board controller firmware
Method of Upgrade
a.Infrastructure Bundle - Direct upgrade (follow the steps in this procedure)
b.B-Series Server Bundle - Host Firmware Packages (Follow the steps in this procedure)
1.Obtaining Software Bundles from Cisco
Determine which of the following software bundles you need to update the Cisco UCS domain:
- Cisco UCS Infrastructure Software Bundle—Required for all Cisco UCS domains. Ours is 2.2(3F)
ucs-k9-bundle-infra.2.2.3f.A.bin
- Cisco UCS B-Series Blade Server Software Bundle—Required for all Cisco UCS domains that include blade servers.
ucs-k9-bundle-b-series.2.2.3f.B.bin
- Verify that the software integrity by running the checksum md5sum command against each bin you downloaded:
# md5sum ucs-k9-bundle-infra.2.2.3f.A.bin
3036a71476fea1aeb2f6c72453e4de3d ucs-k9-bundle-infra.2.2.3f.A.bin
# md5sum ucs-k9-bundle-b-series.2.2.3f.B.bin
e54c56a1e92365e6de0b341bb8b13d71 ucs-k9-bundle-b-series.2.2.3f.B.bin
Step 1 In a web browser, navigate to Cisco.com.
Step 2 Under Support, click All Downloads.
Step 3 In the center pane, click "Servers - Unified Computing"
Step 4 If prompted, enter your Cisco.com username and password to log in.
Step 5 Click "Cisco UCS Infrastructure and UCS Manager Software"
Step 6 Click "Unified Computing System (UCS) Infrastructure Software"
Step 7 Select the Version you are interested in from the Navigation Pane. Ours is 2.2(3F)
Step 7 Click the Release Notes link to download the latest version of the Release Notes.
Step 8 Download:
a."Infrastructure Software Bundle"
b."Software for the B-Series Blades"
Save the bundles to a location on the network accessible to Cisco UCS
2.Downloading Firmware Images to the Fabric Interconnect from a Local Disk
Equipment --> Equipment
Step 1 In the Work pane, click the Firmware Management tab
Step 2 Click the Installed Firmware tab
Step 3 Click the "Download Task” TAB
Step 4 Select “Download Firmware” link
a.Select the "Local File System" radio button
b.Browse to the location of the firmware bundle
c.Click "OK"
Repeat the process to upload the Infrastructure and the B-Series Blade Server Bundle
3.Determine the Contents of the Firmware Package
Equipment --> Equipment
Step 1 In the Work pane, click the Firmware Management tab.
Step 2 On the Packages sub tab, click the + icon next to a package to view its contents.
Step 3 You can also see the version numbers by clicking the "Images" tab
4.Review Release Notes
5.Back Up the Configuration into an All Configuration Backup File
Admin --> General --> Backup Configuration
Step 1 Click the All option in the Filter dropdown box
Step 2 In the Backup Configuration dialog box, click Create Backup Operation.
Step 3 In the Create Backup Operation dialog box, do the following:
a) Complete the following fields:
a.Admin State field—Click the Enabled radio button to run the backup operation as soon as you click OK.
b.Type field—Click the All Configuration radio button to create an XML backup file that includes all system and logical configuration information.
c.Preserve Identities check box—If the Cisco UCS domain includes any identities derived from pools that you need to preserve, check this check box. Identities such as MAC addresses, WWNNs, WWPNs, or UUIDS are assigned at runtime. If you do not want these identities to change after you import the backup file, you must check this check box. If you do not, these identities may be changed after the import and operations such as a PXE boot or a SAN boot
d.Location of the Backup File: select "Local File System"
e.Browse to a location for the file and give the file a name
6.Disable Call Home
Call Home has not been configured in our environment yet on the Cisco Vendor side as firewall rules need to be opened up on Xilinx, so at the moment it is configured on UCS, but not enabled yet.
7.Enable the User Acknowledgement Option in Maintenance Plan for all Service Profiles and Templates
NOTE:This configuration is should already be in place a standard policy. This is just an additional verification as the default Maintenance Policy has been configured with the Reboot Policy set to "User Acknowledgement". The policy is bound to all Templates and is part of all Service profiles.
8.Verify the High Availability Status and Roles of the Cluster Configuration
Equipment --> Equipment --> Fabric Interconnects
Step 1 Click the node for one of the fabric interconnects in the cluster.
Step 2 In the Work pane, click the General tab.
Step 3 If the fields in the "High Availability Details" area are not displayed, click the Expand icon to the right of the heading.
Step 4 Verify that the following fields display the following values:
a.Ready = Yes
b.State = Up
c.Leadership = Either Primary or Subordinate depending on role of Interconnect
d.Cluster Link Status = Full
Step 5 Repeat for both Fabric Interconnect nodes
If the values are different, create and download a Tech Support file, and contact Cisco Technical Support. Do not proceed with the firmware upgrade. For more information about Tech Support files, see the Cisco UCS Manager B-Series Troubleshooting Guide.
Step 6 Note the value in the Leadership field to determine whether the fabric interconnect is the primary or subordinate. You need to know this information to upgrade the firmware on the fabric interconnects.
Step 7 Make a note of which Fabric Interconnect Switch is the Primary and which is the Secondary
9.Verify the Status of the IO Modules
Equipment --> Equipment --> Chassis
Step 1 Click on the chassis for which you want to verify the status of the I/O modules.
Step 2 In the Work pane, click the IO Modules tab.
Step 3 For each I/O module, verify that the following columns display the following values:
a.Overall Status column = ok
If the values are different, create and download a Tech Support file, and contact Cisco Technical Support. Do
not proceed with the firmware upgrade. For more information about Tech Support files, see the Cisco UCS
Manager B-Series Troubleshooting Guide.
Step 4 Repeat to verify the status of the I/O modules in each chassis.
10.Verify the State of Servers
Equipment --> Equipment --> Chassis
If a server is inoperable, you can proceed with the upgrade for other servers in the Cisco UCS domain. However, you cannot upgrade the inoperable server.
Step 1 In the Navigation Pane Select a Chassis
Step 2 In the Work pane, click the Servers tab to display a list of all servers in all chassis.
Step 3 For each server, verify that the following columns display the following values:
a.Overall Status = ok, unassociated, or any value that does not indicate a failure. If the value indicates a failure, such as discovery-failed, the endpoints on that server cannot be upgraded.
b.Operability = Operable
Step 4 If you need to verify that a server has been discovered, do the following:
a.Right-click the server for which you want to verify the discovery status and choose Show Navigator.
b.In the Status Details area of the General tab, verify that the Discovery State field displays a value of complete.
If the fields in the Status Details area are not displayed, click the Expand icon to the right of the heading.
Step 5 Repeat for every chassis full of servers
11.Verify the Status of Adapters on Servers in the Chassis
Equipment --> Equipment --> Chassis --> {Chassis Number } --> Servers
Step 1 Click the server for which you want to verify the status of the adapters.
Step 2 In the Work pane, click the Inventory tab.
Step 3 In the Inventory tab, click the Adapters subtab.
Step 4 For each adapter, verify that the following columns display the following values:
Overall Status column = Operable
Operability column = Operable
If the fields show a different value and the adapter is inoperable, you can proceed with the upgrade for other adapters on the servers in the Cisco UCS domain. However, you cannot upgrade the inoperable adapter.
12.Verify There are No Major Faults in UCS
Check all faults to ensure that there isn't anything that will prevent an upgrade. Resolve all Major Faults where possible. Servers with Major faults may not be upgraded
13.Verify the Number of Ethernet Data Paths
Command or Action / PurposeStep1 / UCS-A /fabric-interconnect # connect nxos {a | b} / Enters NX-OS mode for the fabric interconnect.
Step2 / UCS-A(nxos)# show int br | grep -v down | wc –l / Returns the number of active Ethernet interfaces.
Step3 / UCS-A(nxos)# show platform fwm info hw-stm | grep '1.' | wc –l / Returns the total number of MAC addresses.
The following example returns the number of active Ethernet interfaces and MAC addresses for subordinate fabric interconnect B so that you can verify that the Ethernet data path for that fabric interconnect is up and running:
UCS-A /fabric-interconnect # connect nxos b
UCS-A(nxos)# show platform fwm info hw-stm | grep '1.' | wc -l
14.Verify the Data Path for Fibre Channel End-Host Mode
For best results when upgrading a Cisco UCS domain, we recommend that you perform this task before you begin the upgrade and after you activate the subordinate fabric interconnect, and then compare the two results.
Command or Action / PurposeStep1 / UCS-B /fabric-interconnect # connect nxos {a | b} / Enters NX-OS mode for the fabric interconnect.
Step2 / UCS-B(nxos)# show npv flogi-table / Displays a table of flogi sessions.
Step3 / UCS-B(nxos)# show npv flogi-table | grep fc | wc -l / Returns the number of servers logged into the fabric interconnect.
The following example displays the flogi-table and number of servers logged into subordinate fabric interconnect B so that you can verify that the Fibre Channel data path for that fabric interconnect in Fibre Channel End-Host mode is up and running:
ssh into your vippcfiucs01
pcfiucs01 # connect nxos b
pcfiucs01-B(nxos)# show npvflogi-table
pcfiucs01-B(nxos)# show npvflogi-table
------
SERVER EXTERNAL
INTERFACE VSAN FCID PORT NAME NODE NAME INTERFACE
------
vfc895 1 0x631507 20:00:00:25:31:05:02:06 20:00:00:25:31:01:00:06 fc2/13
vfc1006 1 0x631c02 20:00:00:25:31:05:02:01 20:00:00:25:31:01:00:01 fc2/14
vfc1014 1 0x631c03 20:00:00:25:31:05:02:02 20:00:00:25:31:01:00:02 fc2/14
………….
Total number of flogi = 55.
pcfiucs01-B(nxos)# show npvflogi-table | grep fc | wc –l
55
Reference:
15. Enumerate the Current Running/Startup and Backup Configs
Equipment --> Equipment --> Firmware Management --> Installed Firmware
16.Activate the Cisco UCS Manager Software
Equipment --> Equipment --> Firmware Management --> Installed Firmware
.Step 1 / Click Activate Firmware.
Step2 / On the UCS Manager row of the Activate Firmware dialog box, do the following:
a.From the drop-down list in the Startup Version column, select the version to which you want to update the software. We are upgrading to 2.2(3F).
b.Click OK.
Cisco UCS Manager disconnects all active sessions, logs out all users, and activates the software. When the upgrade is complete, you are prompted to log back in. If you are prompted to re-login immediately after being disconnected, the login will fail. You must wait until the activation of Cisco UCS Manager is completed, which takes a few minutes.
Cisco UCS Manager makes the selected version the startup version and schedules the activation to occur when the fabric interconnects are upgraded.
Reference:
17.Update the Firmware on the IOMs
[This process will need to be done for each chassis’s IOM’s]
Equipment --> Equipment --> Chassis --> Chassis Number --> IO Modules
Step 3 Click the I/O module that you want to update.
Step 4 In the General tab, click Update Firmware.
Step 5 In the Update Firmware dialog box, do the following:
a.From the Version drop-down list, select the firmware version to which you want to update the endpoint
b.Click OK
Cisco UCS Manager copies the selected firmware package to the backup memory slot, where it remains until
you explicitly activate it.
Step 6 (Optional) Monitor the status of the update in the Update Status area.
The update process can take several minutes. Do not activate the firmware until the selected firmware package
displays in the Backup Version field in the Firmware area of the General tab.
18.Activate the Firmware on the IOMs
[This process will need to be done for each chassis’s IOM’s]
Equipment --> Equipment --> Chassis --> Chassis Number --> IO Modules
Step1 / Select the IO Module node that includes the I/O module for which you want to activate the updated firmware.Step2 / In the General tab, click Activate Firmware.
Step3 / In the Activate Firmware dialog box, do the following:
a.Set the startup version and check the Set Startup Version Only check box.
If you configure Set Startup Version Only, the activated firmware moves into the pending-next-reboot state and the endpoint is not immediately rebooted. The activated firmware does not become the running version of firmware until the endpoint is rebooted.
b.Click OK.
c.Repeat for all IOMs on all Chassis
********** The IOMs get booted when the Subordinate Fabric Interconnect boots *************
Reference:
19.Activate the Firmware on the Subordinate Fabric Interconnect
Equipment --> Equipment --> Firmware Management --> Installed Firmware
Step1 / Determine which fabric interconnect in the cluster is the subordinate fabric interconnect.Step2 / On the Installed Firmware tab, click Activate Firmware.
Step3 / From the Filter drop-down list on the menu bar, choose Fabric Interconnects.
Step4 / On the row of the Activate Firmware dialog box for the subordinate fabric interconnect, do the following:
a.In the Kernel row, choose the firmware version to which you want to upgrade from the drop-down list in the Startup Version column.
b.In the System row, choose the firmware version to which you want to upgrade from the drop-down list in the Startup Version column.
Step5 / Click Apply.
Cisco UCS Manager updates and activates the firmware and reboots the fabric interconnect and any I/O module in the data path to that fabric interconnect, disrupting data traffic to and from that fabric interconnect. However, assuming the Cisco UCS domain is configured to permit traffic and port failover, data traffic fails over to the primary fabric interconnect and is not disrupted.
Step6 / Verify the high availability status of the subordinate fabric interconnect.
If the High Availability Details area for the fabric interconnect does not show the following values, contact Cisco Technical Support immediately. Do not continue to update the primary fabric interconnect.
Field Name / Required Value
Ready field / Yes
State field / Up
Step7 Check the high availability status of the subordinate fabric interconnect. Ensure it is operable and working correctly