Just to share my experience when working on an IOS upgrade for the Cisco Catalyst 3850 Switch and encounter a IOS-XE installation corruption.
Logs below is the actual IOS upgrade process which it has indicated a successful installation of the new version
Switch#$stall file flash:cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin
Preparing install operation ...
[1]: Starting install operation
[1]: Expanding bundle flash:cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin
[1]: Copying package files
[1]: Package files copied
[1]: Finished expanding bundle flash:cat3k_caa-universalk9.SPA.03.03.01.SE.150-1.EZ1.bin
[1]: Verifying and copying expanded package files to flash:
[1]: Verified and copied expanded package files to flash:
[1]: Starting compatibility checks
[1]: Finished compatibility checks
[1]: Starting application pre-installation processing
[1]: Finished application pre-installation processing
[1]: Old files list:
Removed cat3k_caa-base.SPA.03.03.04SE.pkg
Removed cat3k_caa-drivers.SPA.03.03.04SE.pkg
Removed cat3k_caa-infra.SPA.03.03.04SE.pkg
Removed cat3k_caa-iosd-universalk9.SPA.150-1.EZ4.pkg
Removed cat3k_caa-platform.SPA.03.03.04SE.pkg
Removed cat3k_caa-wcm.SPA.10.1.140.0.pkg
[1]: New files list:
Added cat3k_caa-base.SPA.03.03.01SE.pkg
Added cat3k_caa-drivers.SPA.03.03.01SE.pkg
Added cat3k_caa-infra.SPA.03.03.01SE.pkg
Added cat3k_caa-iosd-universalk9.SPA.150-1.EZ1.pkg
Added cat3k_caa-platform.SPA.03.03.01SE.pkg
Added cat3k_caa-wcm.SPA.10.1.110.0.pkg
[1]: Creating pending provisioning file
[1]: Finished installing software. New software will load on reboot.
[1]: Committing provisioning file
[1]: Do you want to proceed with reload? [yes/no]: yes
As you have seen the IOS-XE have indicated the successful installation of the new version but the next log indicated a corruption of the IOS-XE installation and cause a boot loop
System configuration has been modified. Save? [yes/no]: yes
Building configuration...
Compressed configuration from 2484 bytes to 1398 bytes[OK]
[1]: Reloading
Switch#
fri Feb 13 16:00:58 2015> Message from sysmgr: Reason Code:[3] Reset Reason:Reset/Reload requested by [stack-manager]. [User requested reload]
umount: /proc/fs/nfsd: not mounted
Unmounting ng3k filesystems...
Unmounted /dev/sda3...
Warning! - some ng3k filesystems may not have unmounted cleanly...
Please stand by while rebooting the system...
Restarting system.
Booting...(use DDR clock 667 MHz)Initializing RAM +++++++@@@@@@@@...++++++++++++++++++++++++++++++++
Base ethernet MAC Address: e0:89:9d:26:ba:00
Interface GE 0 link down***ERROR: PHY link is down
Initializing Flash...
flashfs[7]: 0 files, 1 directories
flashfs[7]: 0 orphaned files, 0 orphaned directories
flashfs[7]: Total bytes: 6784000
flashfs[7]: Bytes used: 1024
flashfs[7]: Bytes available: 6782976
flashfs[7]: flashfs fsck took 2 seconds....done Initializing Flash.
Getting rest of image
Reading full image into memory....done
Reading full base package into memory...: done = 79113792
Bundle Image
--------------------------------------
Kernel Address : 0x6042d350
Kernel Size : 0x402ecf/4206287
Initramfs Address : 0x60830220
Initramfs Size : 0xdb9420/14390304
Compression Format: .mzip
Bootable image at @ ram:0x6042d350
Bootable image segment 0 address range [0x81100000, 0x82110000] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@boot_system: 377
Loading Linux kernel with entry point 0x81653a10 ...
Bootloader: Done loading app on core_mask: 0x3f
### Launching Linux Kernel (flags = 0x5)
All packages are Digitally Signed
Starting System Services
Feb 13 16:03:55 %PLATFORM_MGR-1-PLATMGR_INIT_FAIL: Platform ManaFeb 13 16:03:55 %IOSXE-2-PLATFORM: process ng_dumper: Process platform_mgr: terminated abnormally.
Feb 13 16:03:56 %IOSXE-2-PLATFORM: process platform_mgr: ===================== Exception Data ====================
Feb 13 16:03:56 %IOSXE-2-PLATFORM: process platform_mgr: Exception in 'platform_mgr' with Signal=SIGABRT(6)
Feb 13 16:03:56 %IOSXE-2-PLATFORM: process platform_mgr: Frame Pointer=0x7F7A7390, Program Counter=0x2C4733D0
Feb 13 16:03:56 %IOSXE-2-PLATFORM: process platform_mgr: Thread ID=5745: -Traceback= 1#250697ece830f1be269faef4a03993b8 pthread:2C461000+123D0
Feb 13 16:03:56 %IOSXE-2-PLATFORM: process platform_mgr:
Feb 13 16:03:55 2015> Message from sysmgr: Reason Code:[2] Reset Reason:Service [platform_mgr] pid:[5745] terminated abnormally [6].
Details:
--------
Service: platform_mgr
Description: 3K Platform Manager
Executable: /tmp/sw/mount/cat3k_caa-platform.SPA.03.03.01SE.pkg//usr/binos/bin/platform_mgr
Started at Fri Feb 13 16:03:54 2015 (47649 us)
Stopped at Fri Feb 13 16:03:55 2015 (614391 us)
Uptime: 1 seconds
Start type: SRV_OPTION_RESTART_STATELESS (23)
Death reason: SYSMGR_DEATH_REASON_FAILURE_SIGNAL (2)
Last heartbeat 0.00 secs ago
PID: 5745
Exit code: signal 6 (no core)
CWD: /var/sysmgr/work
PID: 5745
UUID: 3008
get_local: failed: No such file or directory
Unmounting ng3k filesystems...
Unmounted /dev/sda3...
Warning! - some ng3k filesystems may not have unmounted cleanly...
Please stand by while rebooting the system...
Restarting system.
Booting...(use DDR clock 667 MHz)Initializing RAM +++++++@@@@@@@@...++++++++++++++++++++++++++++++++
Base ethernet MAC Address: e0:89:9d:26:ba:00
Interface GE 0 link down***ERROR: PHY link is down
Initializing Flash...
flashfs[7]: 0 files, 1 directories
flashfs[7]: 0 orphaned files, 0 orphaned directories
flashfs[7]: Total bytes: 6784000
flashfs[7]: Bytes used: 1024
flashfs[7]: Bytes available: 6782976
flashfs[7]: flashfs fsck took 2 seconds....done Initializing Flash.
Getting rest of image
Reading full image into memory....done
Reading full base package into memory...: done = 79113792
Bundle Image
--------------------------------------
Kernel Address : 0x6042d350
Kernel Size : 0x402ecf/4206287
Initramfs Address : 0x60830220
Initramfs Size : 0xdb9420/14390304
Compression Format: .mzip
Bootable image at @ ram:0x6042d350
Bootable image segment 0 address range [0x81100000, 0x82110000] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@boot_system: 377
Loading Linux kernel with entry point 0x81653a10 ...
Bootloader: Done loading app on core_mask: 0x3f
### Launching Linux Kernel (flags = 0x5)
All packages are Digitally Signed
Starting System Services
Feb 13 16:07:04 %PLATFORM_MGR-1-PLATMGR_INIT_FAIL: Platform ManaFeb 13 16:07:04 %IOSXE-2-PLATFORM: process ng_dumper: Process platform_mgr: terminated abnormally.
Feb 13 16:07:05 %IOSXE-2-PLATFORM: process platform_mgr: ===================== Exception Data ====================
Feb 13 16:07:05 %IOSXE-2-PLATFORM: process platform_mgr: Exception in 'platform_mgr' with Signal=SIGABRT(6)
Feb 13 16:07:05 %IOSXE-2-PLATFORM: process platform_mgr: Frame Pointer=0x7F6CDE80, Program Counter=0x2BEC83D0
Feb 13 16:07:05 %IOSXE-2-PLATFORM: process platform_mgr: Thread ID=5758: -Traceback= 1#250697ece830f1be269faef4a03993b8 pthread:2BEB6000+123D0
Feb 13 16:07:05 %IOSXE-2-PLATFORM: process platform_mgr:
To recovery from this boot loop we need to wait until 5 reload which the system will disable the auto-reload and present a prompt into rommon
Switch rebooted continously for atleast 5 times. Disabling autoreboot
When we are on the rommon mode we could now revert back to the old IOS-XE by doing the following steps
1. Backup the “packages.conf”
Switch#copy flash: flash:
Source filename []? packages.conf
Destination filename [packages.conf]? packages.conf.bad
Copy in progress...C
1247 bytes copied in 0.000 secs
2. Overwrite “packages.conf” with “packages.conf.00-“
Switch#copy flash: flash:
Source filename []? packages.conf.00-
Destination filename [packages.conf]? packages.conf
Copy in progress...C
1247 bytes copied in 0.000 secs
3. Boot into the replace “packages.conf”
switch: boot flash:packages.conf
Getting rest of image
Reading full image into memory....done
Reading full base package into memory...: done = 79110340
Bundle Image
--------------------------------------
Kernel Address : 0x6042f350
Kernel Size : 0x402ecf/4206287
Initramfs Address : 0x60832220
Initramfs Size : 0xdb86a3/14386851
Compression Format: .mzip
Bootable image at @ ram:0x6042f350
Bootable image segment 0 address range [0x81100000, 0x82110000] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@boot_system: 377
Loading Linux kernel with entry point 0x81653a10 ...
Bootloader: Done loading app on core_mask: 0x3f
### Launching Linux Kernel (flags = 0x5)
All packages are Digitally Signed
Starting System Services
Restricted Rights Legend
Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.
cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706
Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.03.04SE RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Fri 29-Aug-14 22:22 by prod_rel_team
Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0. The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.
FIPS: Flash Key Check : Begin
FIPS: Flash Key Check : End, Not Found,FIPS Mode Not Enabled
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.
A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html
If you require further assistance please contact us by sending email to
export@cisco.com.
cisco WS-C3850-12S (MIPS) processor with 4194304K bytes of physical memory.
Processor board ID XXXXXXX
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
257008K bytes of Crash Files at crashinfo:.
1550272K bytes of Flash at flash:.
0K bytes of Dummy USB Flash at usbflash0:.
0K bytes of at webui:.
Base Ethernet MAC Address : XX:XX:XX:XX:XX:XX
Motherboard Assembly Number : XX-XX-XXXX
Motherboard Serial Number : XXXXXXX
Model Revision Number : A0
Motherboard Revision Number : A0
Model Number : WS-C3850-12S
System Serial Number : XXXXXXX
With this step we are able to recover from the Catalyst 3850 bootloop cause by corrupted IOS-XE installation.
Jamsud's Blogs 