Skip to main content

debugging

Pre-requisites​

Disable secureboot​

If you are using Secure Boot, you will need to disable it in order to enable debugging options in the OS.

Enable Boot Menu​

When booting the machine, you can hold the Spacebar to see the persistent Boot Menu that comes from systemd-boot. You can press ? or h to see all the available options. Pressing e will allow you to edit the kernel command line (cli) arguments (also commonly referred to as boot arguments).

Failed boot​

Here are some tips and tricks for diagnosing and debugging a failed boot.

First you should check if you can get a VT (Virtual Terminal) by using the control-alt-f2 shortcut. You can also use f3-f6 to get additional terminals. control-alt-f1 and control-alt-f7 are reserved for the display server.

Boot to the working version and read the logs of the failing one​

If the latest version does not boot, you can inspect the logs by restarting and selecting the older one.

GNOME OS keeps two versions of the system around at all times. You can select the previous one from the boot menu as mentioned above.

Once you are in a working version you can inspect the failed build with the following command:

journalctl --boot=-1

GDM does not start / crashes on a loop​

If GDM or Shell is crashing or you don't get into a session for some reason you can override the systemd target unit with the following kernel cli argument.

systemd.unit=multi-user.target

Boot without System Extensions​

If you have installed a System Extension (sysext) that is causing issues, you can disable all the extensions with the following cli argument.

systemd.mask=systemd-sysext.service

Enable Debug Shell​

Enabling the debug-shell will give you a root console you can drop into. By default, it will appear in tty9 (control + alt + f9).

systemd.debug-shell=1

If the system fails to get into the debug shell, you can add the following to get a shell inside the initrd.

systemd.debug-shell=1 rd.systemd.debug-shell=1

You can also connect the debug shell on a serial console. (Useful for VMs).

ttyS0 is usually the default serial console name. Depending on the Virtualization solution you are using, the name might be different (ex: ttyAMA0 or ttyhvc0).

systemd.debug-shell=ttyS0 rd.systemd.debug-shell=ttyS0

Disable plymouth​

If plymouth is causing issues for whatever reason you can disable it in the following way.

First remove quiet and splash from the default kernel command line arguments and then add the following:

plymouth.enable=0

Enable systemd debug logging​

To enable more verbose logging of systemd itself add the following.

systemd.log_level=debug systemd.log_target=console

You can also forward the journal to the console

systemd.journald.forward_to_console=1

For more detailed and systemd specific information, refer to the detailed documentation.

Disable gnome-initial-setup​

You can do this by either adding InitialSetupEnable=false in /etc/gdm/custom.conf or by adding gnome.initial-setup=0 to the kernel command line arguments

Other​

Force Toggle GNOME Shell animations​

Animations may be disabled in certain situations, such as when using software rendering for Graphics, such as llvmpipe.

However, its sometimes needed to be able to toggle them on demand for testing. This can be done easily from the lookingglass interface in GNOME Shell.

Simply type alt+f2 to bring up the command runner, then type lg. Then enter the following global.force_animations = true

Show the Tour dialog again​

If you want to trigger the Tour dialog on the next login, you can do so by reseting its settings key.

gsettings reset org.gnome.shell welcome-dialog-last-shown-version

Inspect the Kernel Config​

You can inspect the config of the kernel with the following command

zcat /proc/config.gz | less

Inspect the System​

Here are a couple of files with metadata that can help you identify the versions of the software running. They can also be attached in bug reports.

The /usr/lib/os-release file will print the version of the OS currently running, along with the commit it was build from and date.

The /usr/manifest.json file contains the version of the source of every module in the system.

With updatectl features you will get a list of all the available features and their state.

Last, you can find the installed versions of the OS with updatectl list host.