Moving partitions around underneath LVM

It's like nothing ever happened

March 15th, 2020

I recently installed a new VM host for SOWN, and managing to confuse both myself and the installer, ended up with a dumb partition layout - LVM on a single large partition, with a MBR partition table.

root@vms-b53-1:~# fdisk -l /dev/sda
Device     Boot Start        End    Sectors  Size Id Type
/dev/sda1  *     2048 1952446463 1952444416  931G 8e Linux LVM

This machine BIOS boots, and the RAID array may be extended past 2TB in the future - so a GPT partition table with a separate /boot partition near the start of the disk would be ideal.

The sensible option at this point would be to simply reinstall with a more sensible layout. However, moving the start of the LVM partition forwards to make room for a /boot partition is far more fun!

LVM has excellent tooling for moving LVs between PVs, and for growing or shrinking a PV, but not for moving it by an offset, so this needs some manual work.

To start, there's a LV at the start of the disk where /boot would ideally go:

root@vms-b53-1:~# pvdisplay -m
  Physical extent 0 to 4767:
    Logical volume      /dev/vms-b53-1-vg/root
    Logical extents     0 to 4767

Luckily though, LVM's pvmove can move back to the same PV it's copying from, so this will allow moving a LV to a different section of free space on the same drive. So, to get some free space at the start of the drive:

root@vms-b53-1:~# pvmove -v --alloc anywhere -n root /dev/sda1 /dev/sda1
  /dev/sda1: Moved: 100.00%
    Polling finished successfully.

And now there's free space!

root@vms-b53-1:~# pvdisplay -m
  --- Physical Segments ---
  Physical extent 0 to 4511:
  Physical extent 14355 to 19122:
    Logical volume      /dev/vms-b53-1-vg/root
    Logical extents     0 to 4767

At this point, with the exact position of LVs noted down, the machine can be repartitioned. I netbooted into an ubuntu installer in "rescue mode" for this.

A plain new GPT partition table can be created, with the LVM partition offset carefully by a whole number of extents. In this case I picked 1GiB, as this is plenty for a /boot. After some fiddling in fdisk, I ended up with a partition table like this:

Device       Start        End    Sectors  Size Type
/dev/sda1     2048       4095       2048    1M BIOS boot
/dev/sda2     4096    2099199    2095104 1023M Linux filesystem
/dev/sda3  2099200 1952446463 1950347264  930G Linux LVM

Now all the LVs are still on the disk, but missing any LVM metadata. After the usual pvcreate and vgcreate to get a new VG on sda3, the LVs can be put back.

In this case, the root LV was originally on extents 14355-19122, so shifted by 1GiB (256 4MiB extents), this ends up at extents 14099-18866.

The LV can be re-created with lvcreate. Note that --zero n is very important - by default LVM will wipe the first 4KiB of the LV, leaving behind a filesystem that can't (without extra work) be mounted.

~ # lvcreate --extents 4768 -n root --zero n vms-b53-1-vg /dev/sda3:14099-18866
  WARNING: Logical volume vms-b53-1-vg/root not zeroed.
  Logical volume "root" created.

By forcing a specific number of extents and also a specific range of extents, the new LV covers exactly the same blocks as the original - so this filesystem can be mounted exactly the same as before.

At this point, it's just a case of chrooting in and copying the contents of the original /boot to the new /boot, appending it to fstab, and reinstalling grub.

Once complete, the VM host booted back up with a far more sane partition layout, and the rootfs and VM disks as if they were untouched!