Today I manually updated one server via remote SSH connection. During update, the SSH tunnel collapsed, the update was not complete. Logging in again, I tried to continue the update, but I dismissed to clean pending transactions i.e. to run yum-complete-transaction .
After the update, the server tried to boot the brand new kernel vmlinuz-2.6.32-696.10.2.el6.i686, and this resulted in a kernel panic:
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
Pid: 1, comm: swapper Not tainted 2.6.32-696.10.2.el6.i686 #1
Call Trace:
Using the Grub boot menu, I tried an old kernel, which started without problems. Since I had to bring back the system up and running, I have decided to manually modify the grub.conf setting default=1 (instead of default=0):
[root@mysystem grub]# cat grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro rd_NO_PLYMOUTH root=/dev/mapper/main-root
# initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=1
timeout=5
splashimage=(hd0,0)/grub/smeserver.xpm.gz
foreground 000000
background 4E95D3
title SME Server 9.2 (2.6.32-696.10.2.el6.i686)
root (hd0,0)
kernel /vmlinuz-2.6.32-696.10.2.el6.i686 ro rd_NO_PLYMOUTH root=/dev/mapper/main-root rd_NO_LUKS rd_LVM_LV=main/root KEYBOARDTYPE=pc KEYTABLE=de-latin1-nodeadkeys nodmraid rd_LVM_LV=main/swap SYSFONT=latarcyrheb-sun16 rd_MD_UUID=88c49d56:3eb48d5f:62af662b:99f835e1 LANG=de_DE.UTF-8 rd_NO_DM rhgb quiet crashkernel=auto
title SME Server 9.2 (2.6.32-696.6.3.el6.i686)
root (hd0,0)
kernel /vmlinuz-2.6.32-696.6.3.el6.i686 ro rd_NO_PLYMOUTH root=/dev/mapper/main-root rd_NO_LUKS rd_LVM_LV=main/root KEYBOARDTYPE=pc KEYTABLE=de-latin1-nodeadkeys nodmraid rd_LVM_LV=main/swap SYSFONT=latarcyrheb-sun16 rd_MD_UUID=88c49d56:3eb48d5f:62af662b:99f835e1 LANG=de_DE.UTF-8 rd_NO_DM rhgb quiet crashkernel=auto
initrd /initramfs-2.6.32-696.6.3.el6.i686.img
title SME Server 9.2 (2.6.32-696.3.2.el6.i686)
root (hd0,0)
kernel /vmlinuz-2.6.32-696.3.2.el6.i686 ro rd_NO_PLYMOUTH root=/dev/mapper/main-root rd_NO_LUKS rd_LVM_LV=main/root KEYBOARDTYPE=pc KEYTABLE=de-latin1-nodeadkeys nodmraid rd_LVM_LV=main/swap SYSFONT=latarcyrheb-sun16 rd_MD_UUID=88c49d56:3eb48d5f:62af662b:99f835e1 LANG=de_DE.UTF-8 rd_NO_DM rhgb quiet crashkernel=auto
initrd /initramfs-2.6.32-696.3.2.el6.i686.img
[root@mysystem grub]#
I do understand that this is not the best way, but at least the system is up and running again at the moment.
I am not sure whether this issue is linked to the problems during update, or whether the kernel does not work on my system. It is an Intel NUC, which is officially not supported. Therefore I get the the message
Detected CPU family 6 model 76
Warning: Intel CPU model - this hardware has not undergone testing by Red Hat an
d might not be certified. Please consult https://hardware.redhat.com for certified hardware.
while booting. This message is also shown when older kernels boot successfully.
So where do I now start best with troubleshooting: is this a problem linked to the kernel itself, or did I create issues while updating the system?
While writing this up, I have just recognized that grub.conf does not point to initrd /initramfs-2.6.32-696.10.2.el6.i686.img. This file is also missing:
[root@mysystem boot]# ls -l
insgesamt 52861
-rw-r--r-- 1 root root 112821 12. Sep 15:59 config-2.6.32-696.10.2.el6.i686
-rw-r--r-- 1 root root 112820 20. Jun 02:53 config-2.6.32-696.3.2.el6.i686
-rw-r--r-- 1 root root 112820 12. Jul 15:44 config-2.6.32-696.6.3.el6.i686
drwxr-xr-x 3 root root 1024 3. Dez 2016 efi
drwxr-xr-x 2 root root 1024 22. Sep 21:20 grub
-rw------- 1 root root 17265279 21. Jun 20:26 initramfs-2.6.32-696.3.2.el6.i686.img
-rw------- 1 root root 17261064 18. Jul 22:14 initramfs-2.6.32-696.6.3.el6.i686.img
drwx------ 2 root root 12288 3. Dez 2016 lost+found
-rw-r--r-- 1 root root 211993 12. Sep 16:00 symvers-2.6.32-696.10.2.el6.i686.gz
-rw-r--r-- 1 root root 211993 20. Jun 02:54 symvers-2.6.32-696.3.2.el6.i686.gz
-rw-r--r-- 1 root root 211993 12. Jul 15:44 symvers-2.6.32-696.6.3.el6.i686.gz
-rw-r--r-- 1 root root 2064350 12. Sep 15:59 System.map-2.6.32-696.10.2.el6.i686
-rw-r--r-- 1 root root 2064268 20. Jun 02:53 System.map-2.6.32-696.3.2.el6.i686
-rw-r--r-- 1 root root 2064411 12. Jul 15:44 System.map-2.6.32-696.6.3.el6.i686
-rwxr-xr-x 1 root root 4137952 12. Sep 15:59 vmlinuz-2.6.32-696.10.2.el6.i686
-rwxr-xr-x 1 root root 4137600 20. Jun 02:53 vmlinuz-2.6.32-696.3.2.el6.i686
-rwxr-xr-x 1 root root 4137632 12. Jul 15:44 vmlinuz-2.6.32-696.6.3.el6.i686
[root@mysystem boot]#
One of the problems is that it is a remote system. To manually change the used kernel on the boot menu, I need to sit down at the local konsole...
What should I do?
Thank you,
turandot