Topic: Kernel panic - not syncing: Attempted to kill init!

[fredd]

Hi,

I wanted to clone one non-raid data-disk mounted with something like this http://wiki.contribs.org/AddExtraHardDisk

I didn't had shell access at the moment of doing this. So I stopped the server (SME 8.1), hooked an usb-disk for holding the clone and another usb-stick with clonezilla on it and started the server with clonezilla live. I've choosen clone disk to disk, waited to the end, stopped the server and removed disk and stick. But now after restarting the server stops with this:

Code: [Select]

Scanning logical volumes
  Reading all physical volumes.  This may take a while...
  No volume groups found
Activating logical volumes
  Volume group "main" not found
Trying to resume from /dev/main/swap
Unable to access resume device (/dev/main/swap)
Creating root device.
Mounting root filesystem.
mount: could not find filesystem '/dev/root'
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!

I booted with SME 8.1 disk in rescue-mode. When I choose continue it tells me that only a part of my system had been mounted under /mnt/sysimage.

Without chrooting, cat /proc/mdstat gives:

Code: [Select]

Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md126 : active raid1 sda2[0]
156183808 blocks [2/1] [U_]

md127 : active raid1 sda1[0]
104320 blocks [2/1] [U_]

and fdisk -l

Code: [Select]

Disk /dev/sda: 160GB, 160040803840 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280

/dev/sda1 * 1 13 104391 fd Linux raid autodetect
/dev/sda2 14 19457 156183930 fd Linux raid autodetect


after chrooting, cat /proc/mdstat gives:
cat: /proc/mdstat: No file or directory

and fdisk -l:
cannot open /proc/partitions

/boot is empty.


Symptoms are close to this post: RESOLVED - Upgrading to new hardware with SME 8.1

but if I do: mkinitrd -f /boot/initrd-$(uname -r).img $(uname -r) it shows No modules available for Kernel “2.6.18-371.4.1.el5”.

If I do mkinitrd -f /boot/initrd-2.6.18-398.el5.img  2.6.18-398.el5 which was the last Kernel installed it shows:
error opening /sys/block: No such file or directory
error opening /sys/block: No such file or directory

In bios Sata is set to IDE.

If someone has an idea

[TerryF]

Looks like the SME8.1 system on the 160gb drive has been nuked.

Are you sure you attempted the clone procedure in the right direction?

You may have only one option - reinstall system and use the backup you made before using clonezilla.

Good Luck

[guest22]

I didn't had shell access at the moment of doing this. So I stopped the server (SME 8.1),...

What procedure did you follow for this action if you had no shell access?

[fredd]

In first, thank's for spending some of your time for answering.

@TerryF

Are you sure you attempted the clone procedure in the right direction?

I was so sure about this, that I even didn't checked it right after. I'm worrying much more about this disk than the server. But now after verifying (it's right, your comment made me a little afraid for a moment) all is how it should with one hick. Inside it's a 1.5 GB and extern a 2 GB disk and I advised Clonzilla to extend it after cloning. It didn't do so, df -h shows 2 disks whith the same size. May be a sign for what could have went wrong.

@RequestedDeletion

What procedure did you follow for this action if you had no shell access?

LOL. Good point. I pressed the power button.  But seriously, I wanted to say no shell access to the (SME) server, the password is to complicate to remember; it's a server I'm used to manage by network. Don't worry, I was authorized to do so.

[TerryF]

@RequestedDeletionLOL. Good point. I pressed the power button. 

This raises a flag

After entering rescue mode did you try "mount -n -o remount /" as described in the wiki.

[fredd]

@TerryF

This raises a flag

Can you please explain what this mean? English is actually only my third language and I'm not familar with asking for help in forums. If it was my comment for RequestedDeletion, it was just for honoring his percipience. He is absolutely right, how can one use clonzilla without shell access! This made me laughing.

[fredd]

@TerryF

After entering rescue mode did you try "mount -n -o remount /" as described in the wiki.

I am a little confused.

http://wiki.contribs.org/Booting#Rescue_Mode_Help
says

Code: [Select]

mount -n -o remount /
and http://wiki.contribs.org/Recovering_SME_Server_with_lvm_drives#Method_with_the_official_SME_Server_CDROM

Code: [Select]

chroot /mnt/sysimage
su -

Both are giving completely different results. Without the latter for example it's impossible for me to start midnight-commander; starting mc gives a segmentation fault.

[TerryF]

Raises a flag ie a warning, an indication that something is wrong - downing a system and restarting using the power button may cause damage to the filesystem, in other words trash the system.

Re the rescue mode just reading the wiki entries, another way/starting point.

[fredd]

@TerryF

downing a system and restarting using the power button may cause damage to the filesystem, in other words trash the system.

I don't like to start an ot in my own thread, but I'm not sure if you're right.
Pfloor states here http://forums.contribs.org/index.php/topic,41695.msg197005.html#msg197005

Short (1 second) press of the power button should cleanly shut down your server, this is the default behavior of SME.

And

I have setup dozens of SME and CentOS servers and they ALL shut down cleanly when I press the power button.

And Shad L. Lords here http://bugs.contribs.org/show_bug.cgi?id=6735#c22

On the other hand if you push the power button you should expect the system to receive a message and shut down nicely.

Has something changed inbetween?

[janet]

fredd

You are correct to say that the system SHOULD shutdown cleanly with a short press of the power button, as long as your system is configured correctly, otherwise file system damage may occur (ie in the event of a hard forced shutdown).

So did your machine shutdown cleanly in a controlled manner, you were there, what did you see ?

[TerryF]

When I read anything that uses words like "should expect" and " should cleanly" I am always wary because should is not a certainty and when should ends up as did not its to late. Hope it all works out.

[fredd]

If I remember right, this is a “feature” since SME 6. It's certainly not the reason for the failing server.
@TerryF

"should expect" and " should cleanly"

it's because i've taken them out of their context. If you read the posts you will understand what I mean.

@janet

as long as your system is configured correctly

Bios → Power Management  → Power Button set to Soft-Off and I haven't disabled ACPI-support in SME's boot options.

So did your machine shutdown cleanly in a controlled manner, you were there, what did you see?

I don't remember exactly, something like:
Receiving Term signal . . .
. . .
Shutting down server
. . .
You cannot know, but I started playing with computers a long time before ACPI and ATX-Power supplies came out and I would never take down a RUNNING computer in cutting the electric current.

[janet]

fredd

If I remember right, this is a “feature” since SME 6.

Yes IIRC.

It's certainly not the reason for the failing server.

I think the concern was that it may have contributed, so best to check.
With your many years of experience you would know it is a mistake to overlook the most simple things when troubleshooting &/or diagnosing.

something like:
Receiving Term signal . . .
. . .
Shutting down server
. . .

Well that sounds like a clean shutdown.

You cannot know, but I started playing with computers a long time before ACPI and ATX-Power supplies came out and I would never take down a RUNNING computer in cutting the electric current.

That's correct we would not know that until you told us, so you will excuse basic questions that you are well experienced with.

[fredd]

I have spend the last days in reading (and trying to understand) http://wiki.contribs.org/Recovering_SME_Server_with_lvm_drives and http://wiki.contribs.org/Raid . I think, I even found something strange in latter page, at least it sounds strange for my understanding:

Note:
the addition of another drive is restricted to a Raid1 that is degraded, i.e. when the system has been installed with a single drive (/dev/hda and /dev/hdc or their SATA equivalent).

Makes no sense for me. I would have written

has been installed with a single drive (/dev/hdx or it's SATA equivalent /dev/sdx).

Badly there is realy nothing helpful for someone with a one-drive degraded Raid1. I don't understand what I should see and do in my case.

I resume:
I have a one-drive degraded Raid1. When I start with SME 8.1 disk in rescue-mode and choose continue, I will get nearly the complet tree structure with files and subfolders in /mnt/sysimage.
The only folders which are empty are:
/boot
/proc
/selinux
/srv
/sys

After issuing “mkinitrd -f /boot/initrd-2.6.18-398.el5.img  2.6.18-398.el5“ (see my first post) there is now in /boot “initrd-2.6.18-398.el5.img”, a file with 2244099 Bytes.
There is no grub or anything else in /boot.

In /dev there is main, mapper, null

in /dev/main
root → /dev/mapper/main-root
swap → /dev/mapper/main-swap

in /dev/mapper
control
main-root
main-swap

When I started to use SME (v5 or v5.5) there was no Raid and, if I remember right, no LVM. I'm not against evolution, but I was never in a situation where I had to update the gap in my knowledge.

If someone could tell me in which direction I have to search, or isn't there any hope for my case. I would have liked to understand what had happened.

By the way, the first I did was saving (from /mnt/sysimage) ibays, users, /etc, /var/lib/mysql to another disk, so important files should be at a save place.

[janet]

fredd

Have you ran a short & long test on your hard disk drive(s), ie using smartctl or UBCD (Ultimate Boot CD freely downloadable).

You should do so to check that the drive(s) is/are OK & not the reason for file system problems, BEFORE attempting to fix the system.