Linux server does not boot: emergency mode, fsck and kernel fixes

A server that does not come back after a reboot usually tells you why on its screen. SSH cannot help here, because the network and the SSH service start late in the boot. This guide covers the cases we see most often and the exact steps to fix them from the console.

Before you start

• Open the console. Use the console on your service page where your plan offers one (see manage your services), or the IPMI virtual console on a dedicated server that includes it (see IPMI console and virtual media). If you have no console, open a ticket with the service name; we can look at the screen and help.
• Rule out the simple causes first with server unreachable: is the server switched on, is there a network incident?
• Know your root password, or the password of a user with sudo. Emergency mode asks for the root password.

Case 1: "You are in emergency mode"

The boot stops with a message that you are in emergency mode, and asks for the root password.

Usual cause: a line in /etc/fstab points to a disk, partition or UUID that does not exist (for example a removed data disk or a typo), or names a file system the system cannot mount.

1. Log in with the root password and read the reason:

journalctl -xb | grep -iE "mount|fstab|dependency" | head -40
findmnt --verify

2. If the root file system is mounted read-only in this mode, make it writable so you can edit:

mount -o remount,rw /

3. Open /etc/fstab with nano /etc/fstab. Put # in front of the line that fails, or fix its UUID (compare with blkid). For disks that may be missing in future, add the option nofail so the boot continues without them.

4. Check the file again and boot normally:

findmnt --verify
systemctl daemon-reload
systemctl default

Verify: the server finishes booting and SSH works again. Fix the data disk's line properly afterwards; see use new disk space.

Case 2: "Read-only file system" errors

The server runs, but writes fail with Read-only file system, services crash and logs stop.

Usual cause: the kernel detected file system errors and remounted the file system read-only to protect it. Ubuntu and Debian mount the root file system with errors=remount-ro.

1. Look at the kernel messages for the reason:

dmesg | grep -iE "ext4|xfs|i/o error|remount" | tail -30

2. Copy anything important off the server now, while you can still read it.

3. Reboot. A file system with recorded errors is checked automatically at the next boot:

sudo reboot

Verify: after the boot, findmnt / shows rw and writes work. Repeated I/O error messages point to the storage: open a ticket with the dmesg lines, and on dedicated servers check the disks as in replace a failed RAID disk.

Case 3: the "(initramfs)" prompt

The boot stops at a BusyBox shell with the prompt (initramfs), often after a message that the root file system could not be found or needs a manual check.

Usual cause: the root file system has errors the automatic check would not fix on its own.

1. Find the root device in the message on the screen (for example /dev/sda1 or /dev/mapper/ubuntu--vg-ubuntu--lv). At this point it is not mounted.

2. Check and repair it, answering yes to the repairs:

fsck -y /dev/sda1

3. Continue the boot:

exit

Case 4: a kernel update broke the boot

The boot stops with a kernel panic, or the server stops responding right after a kernel update.

1. Restart the server from the console and open the GRUB menu: press Esc (or hold Shift) while the server starts, if the menu is hidden.
2. Choose Advanced options and the previous kernel version.
3. Once the server runs, see what was installed and remove or hold the faulty kernel until a fixed one is released:

uname -r
dpkg -l 'linux-image*' | grep ^ii

On AlmaLinux and Rocky Linux, sudo grubby --default-kernel shows the default and rpm -q kernel the installed kernels.

Case 5: forcing a full check at the next boot

If you suspect file system damage but the server still runs, schedule a full check: in the GRUB menu press e on the default entry, add fsck.mode=force to the end of the line that starts with linux, and boot with Ctrl + X. The change applies only to this one boot.

Troubleshooting

Emergency mode says Cannot open access to console, the root account is locked. The image has no root password. Use the console of a rescue system to fix /etc/fstab, or open a ticket. Once the server runs again, set a root password with sudo passwd root if you want emergency mode to work next time.

The console shows a login prompt but SSH does not work. The system booted; the problem is the network or the firewall. See server unreachable and locked out after a firewall change.

The boot stops with "No space left on device". Free space as in disk full; old kernels in /boot are a common culprit.

Nothing helps. If your data is backed up, a reinstall is often the fastest way back. Otherwise open a ticket with a photo or a copy of the console message.

Next steps

• Avoid the next surprise: test changes to /etc/fstab with findmnt --verify and sudo mount -a before you reboot.
• Keep restores possible: backup strategy.