Moving partitions around underneath LVM
It's like nothing ever happened
March 15th, 2020
I recently installed a new VM host for SOWN, and managing to confuse both myself and the installer, ended up with a dumb partition layout - LVM on a single large partition, with a MBR partition table.
root@vms-b53-1:~# fdisk -l /dev/sda ... Device Boot Start End Sectors Size Id Type /dev/sda1 * 2048 1952446463 1952444416 931G 8e Linux LVM
This machine BIOS boots, and the RAID array may be extended past 2TB in the future - so a GPT partition table with a separate /boot partition near the start of the disk would be ideal.
The sensible option at this point would be to simply reinstall with a more sensible layout. However, moving the start of the LVM partition forwards to make room for a /boot partition is far more fun!
LVM has excellent tooling for moving LVs between PVs, and for growing or shrinking a PV, but not for moving it by an offset, so this needs some manual work.
To start, there's a LV at the start of the disk where /boot would ideally go:
root@vms-b53-1:~# pvdisplay -m ... Physical extent 0 to 4767: Logical volume /dev/vms-b53-1-vg/root Logical extents 0 to 4767
Luckily though, LVM's pvmove can move back to the same PV it's copying from, so this will allow moving a LV to a different section of free space on the same drive. So, to get some free space at the start of the drive:
root@vms-b53-1:~# pvmove -v --alloc anywhere -n root /dev/sda1 /dev/sda1 ... /dev/sda1: Moved: 100.00% Polling finished successfully.
And now there's free space!
root@vms-b53-1:~# pvdisplay -m ... --- Physical Segments --- Physical extent 0 to 4511: FREE ... Physical extent 14355 to 19122: Logical volume /dev/vms-b53-1-vg/root Logical extents 0 to 4767
At this point, with the exact position of LVs noted down, the machine can be repartitioned. I netbooted into an ubuntu installer in "rescue mode" for this.
A plain new GPT partition table can be created, with the LVM partition offset carefully by a whole number of extents. In this case I picked 1GiB, as this is plenty for a /boot. After some fiddling in fdisk, I ended up with a partition table like this:
Device Start End Sectors Size Type /dev/sda1 2048 4095 2048 1M BIOS boot /dev/sda2 4096 2099199 2095104 1023M Linux filesystem /dev/sda3 2099200 1952446463 1950347264 930G Linux LVM
Now all the LVs are still on the disk, but missing any LVM metadata. After the usual pvcreate and vgcreate to get a new VG on sda3, the LVs can be put back.
In this case, the root LV was originally on extents 14355-19122, so shifted by 1GiB (256 4MiB extents), this ends up at extents 14099-18866.
The LV can be re-created with lvcreate. Note that --zero n is very important - by default LVM will wipe the first 4KiB of the LV, leaving behind a filesystem that can't (without extra work) be mounted.
~ # lvcreate --extents 4768 -n root --zero n vms-b53-1-vg /dev/sda3:14099-18866 WARNING: Logical volume vms-b53-1-vg/root not zeroed. Logical volume "root" created.
By forcing a specific number of extents and also a specific range of extents, the new LV covers exactly the same blocks as the original - so this filesystem can be mounted exactly the same as before.
At this point, it's just a case of chrooting in and copying the contents of the original /boot to the new /boot, appending it to fstab, and reinstalling grub.
Once complete, the VM host booted back up with a far more sane partition layout, and the rootfs and VM disks as if they were untouched!