At the Dread Prompt Grub

This is not a grub tutorial or reference. It's a collection of expressions you can evaluate to track down root-on-zfs boot problems.

Make sure the pager is on:

set pager=1

Tab completion

In all contexts where a file path is expected, you can use <tab> to complete the unique path. If the path prefix isn't unique, you'll get a prompt to enter more characters.

List all environment variables:

set

Show value of an individual environment variable:

echo $boot

Variables associated with ZFS boot

boot=hd0,gpt1
fw_path=(hd0,gpt1)/EFI/fedora
kernelopts=root=ZFS=Magoo/fedora ro
prefix=(hd0,gpt1)/EFI/fedora
root=hd0,gpt2

At boot time, grub searches for grub.cfg at $fw_path and failing that, at $prefix. In this case they are the same.

Path variables:

You will notice that some environment variables are in "path notation" that you can list directly:

ls $fg_path/

(But you do have to add the "/" suffix by hand.)

Other variables like "root" need more punctuation:

ls ($root)/

This inconsistency is designed to thwart the unwashed. Examples:

List the boot directory:

ls ($boot)/

List fw_path directory:

ls $fw_path

The files should match what you see in chroot in the directory:

/boot/efi/EFI/fedora

If you followed the guide, these files will be present:

shimx64-fedora.efi
bootx64.csv
mmx64.efi
shim.efi
grubenv
shimx64.efi
grub.cfg
fonts/
grubx64.efi

If you're one of those enthusiasts who puts /boot in the root dataset, you should also see:

x86_64-efi/ 
    zfs.mod
    zfsinfo.mod
    zfscrypt.mod

Grubenv

The grubenv file defines the effective kernelopts:

cat $fw_path/grubenv

Output:

# GRUB Environment Block
kernelopts=root=ZFS=Magoo/fedora ro
boot_success=0
boot_indeterminate=0
######...

The #### business is to pad the file to exactly 1024 bytes. This prevents tinkerers from successfully editing the file using a text editor.

Kernel and initramfs files

They should be here:

ls /

The "/" symbol maps to whatever path is defined by the "root" environment variable. So this shows the same listing:

ls ($root)/

You should see everything expected in the runtime /boot directory. Example:

config-5.7.16-200.fc32.x86_64
efi/
elf-memtest86+-5.01
extlinux/
grub2/
initramfs-0-rescue-d28c1d8f9c3647b5922b9f2fb3ccf927.img
initramfs-5.7.16-200.fc32.x86_64.img
loader/
memtest86+-5.01
System.map-5.7.16-200.fc32.x86_64
vmlinuz-0-rescue-d28c1d8f9c3647b5922b9f2fb3ccf927
vmlinuz-5.7.16-200.fc32.x86_64

Essential files

BLS menu directory: (see later section for details)

loader/

The boot partition mountpoint:

efi

The kernel:

vmlinuz-5.7.15-200.fc32.x86_64

The initramfs:

initramfs-5.7.15-200.fc32.x86_64.img

List the BLS menu configuration directory:

cd ($root)/loader/entries/

Dump the file that matches your kernel (after the machine_id goop)

cat ($root)/loader/entries/<YOUR_MACHINE_ID>-5.7.15-200.fc32.x86_64.conf

(remember that works)

It should look something like this;

title Fedora (5.7.15-200.fc32.x86_64) 32 (Thirty Two)
version 5.7.15-200.fc32.x86_64
linux /vmlinuz-5.7.15-200.fc32.x86_64
initrd /initramfs-5.7.15-200.fc32.x86_64.img
options $kernelopts
grub_users $grub_users
grub_arg --unrestricted
grub_class kernel

Forcing a boot:

linux /vmlinuz-5.7.15-200.fc32.x86_64 $kernelopts
initrd /initramfs-5.7.15-200.fc32.x86_64.img
boot

Using dataset names in grub> context expressions

If you disregarded my advice and put /boot in a ZFS dataset, the BLS entry for your kernel and initramfs will contain these expressions:

linux /fedora@/boot/vmlinuz-5.7.15-200.fc32.x86_64
initrd /fedora@/boot/initramfs-5.7.15-200.fc32.x86_64.img

Here "fedora@" refers to a zfs dataset. Grub can boot from this kind of path (if "insmod zfs" was successful) but you can't inspect the path from the grub prompt. This won't work:

ls /fedora@/

"...incorrect dnode type..."

Consequently you can't do a command line boot from grub because that requires using the commands "linux" and "initrd" which check their paramters as valid paths. And that will fail at the grub prompt.

I'm not sure if turning off compression would help with this or not.