boot - Transferring Linux OS from one drive to another with different partitions

First of all, some related questions:

My situation is as follows:

1. Create a virtual machine (VBox) to install Linux (Debian testing, 3.14 kernel).


2. One BTRFS boot partition, one LUKS BTRFS root partition.


3. Make IO intensive modifications in the VM for speed.


4. Transfer the installation to an USB flash drive.

My initial attempt was to use a harddisk in the VM of the same size as the flash drive and just dd the whole thing. For some reason, the boot partition's UUID cannot be retrieved and therefore used for booting on the flash drive.

If I partition the flash drive manually with a boot partition of EXT4 and a root partition of LUKS BTRFS, the UUID retrieval works fine all around. I then used rsync -aEXS --progress to transfer the files from each mounted partition (VM image -> flash drive). I chroot-ed into the flash drive and ran grub-install on the flash drive device, along with update-grub to update the filesystem UUIDs. This did not work. It was not able to correctly detect/use the encrypted partition to get root. I tried update-initramfs -u without success.

What did I miss or could have done better to perform this scenario?

Answer

The original steps were mostly correct, but for some reason, I kept ending up with a corrupt initrd. Summarized below are all the steps that should result in a working scenario for anyone interested in doing the same:

1. Create a virtual machine as desired.


2. Install Linux using desired partition layout and make all desired changes. In this case, it consists of a BTRFS boot partition and a LUKS encrypted BTRFS root partition. Other layouts should work if care is taken to properly update references during the transfer.


3. Boot the previous VM using a Linux live ISO so that you can access the installed system offline.


4. Attach a USB flash drive to the VM for the transfer. (Note: You can create a dd image of the installed drive and save outside the VM, which you can then use from a normal Linux system.)


5. Prepare the flash drive partitions.
   
   
   
   1. Option: dd the original image onto the flash drive and skip to step 7, then to step 10. (Note: It appears to have been necessary in my case to recreate the partitions for an unknown reason where blkid was unable to enumerate the UUID of the boot BTRFS partition.)
   
   
   2. Option: Create new partitions using the same UUIDs as the original ones. Assuming /dev/sda1 for boot and /dev/sda2 for root.
      
      
      
      1. For BTRFS boot partition: sudo mkfs.btrfs -f -L BOOT -U /dev/sda1
      
      
      2. For LUKS root partition: sudo cryptsetup --uuid= luksFormat /dev/sda2
      
      
      3. Map root partition: sudo cryptsetup --allow-discards open --type luks /dev/sda2 flash_luks
      
      
      4. Format root using BTRFS: sudo mkfs.btrfs -f -L ROOT -U /dev/mapper/flash_luks
      
      
   
   
   3. Option: Create new partitions using new UUIDs. Repeat the same steps as Option 2, but remove arguments relating to UUID/uuid. Take note of the new UUIDs using sudo blkid.
   
   


6. Mount original image partitions.
   
   
   
   1. Create read-only loop-back devices for each partition: sudo kpartx -ar . Assuming kpartx attaches to loop0.
   
   
   2. Create dirs for mount points: sudo mkdir -p /media/orig_boot /media/orig_root
   
   
   3. Boot: sudo mount -r /dev/mapper/loop0p1 /media/orig_boot
   
   
   4. Root LUKS: sudo cryptsetup -r open --type luks /dev/mapper/loop0p2 orig_luks
   
   
   5. Root: sudo mount -r /dev/mapper/orig_luks /media/orig_root
   
   


7. Mount flash drive partitions. Assuming /dev/sda1 for boot.
   
   
   
   1. Create dirs for mount points: sudo mkdir -p /media/flash_boot /media/flash_root
   
   
   2. Boot: sudo mount -t btrfs -o nodiratime,noatime,compress=zlib,discard,ssd,space_cache /dev/sda1 /media/flash_boot
   
   
   3. Root (assuming it was already mapped using cryptsetup earlier): sudo mount -t btrfs -o relatime,compress=zlib,discard,ssd,space_cache /dev/mapper/flash_luks /media/flash_root
   
   


8. Copy contents:
   
   
   
   1. Boot: sudo rsync -aEXS --progress /media/orig_boot/ /media/flash_boot
   
   
   2. Root: sudo rsync -aEXS --progress /media/orig_root/ /media/flash_root
   
   


9. Unmount original image.
   
   
   
   1. Root: sudo umount /media/orig_root
   
   
   2. Root LUKS: sudo cryptsetup close orig_luks
   
   
   3. Boot: sudo umount /media/orig_boot
   
   
   4. kpartx: sudo kpartx -d
   
   


10. Chroot into flash drive.
    
    
    
    1. Bind /sys: sudo mount --bind /sys /media/flash_root/sys
    
    
    2. Bind /dev: sudo mount --bind /dev /media/flash_root/dev
    
    
    3. Bind /dev/pts: sudo mount --bind /dev/pts /media/flash_root/dev/pts
    
    
    4. Bind /proc: sudo mount --bind /proc /media/flash_root/proc
    
    
    5. Unmount boot: sudo umount /media/flash_boot
    
    
    6. Remount inside root: sudo mount -t btrfs -o relatime,compress=zlib,discard,ssd,space_cache /dev/sda1 /media/flash_root/boot
    
    
    7. Chroot: sudo chroot /media/flash_root
    
    


11. Update /etc/fstab.
    
    
    
    1. Boot line: UUID= /boot btrfs nodiratime,noatime,compress=zlib,discard,ssd,space_cache 0 2
    
    
    2. Root line: UUID= / btrfs nodiratime,noatime,compress=zlib,discard,ssd,space_cache 0 1
    
    


12. Update /etc/crypttab: flash_luks UUID= none discard,luks


13. Install btrfs-tools (if not installed): sudo apt-get install btrfs-tools


14. Update boot stuff.
    
    
    
    1. Initrd (makes use of fstab/crypttab to decide what is needed prior to mount and to prompt for LUKS password on boot): sudo update-initramfs -u -k all (Note: You can recreate all initrds using sudo update-initramfs -c -k all, but I didn't try that to be sure.)
    
    
    2. Grub (makes use of fstab to update kernel boot options and mount correct partitions): sudo update-grub
    
    
    3. Install grub to MBR (assuming /dev/sda): sudo grub-install --recheck /dev/sda
    
    
    4. Exit chroot: exit
    
    


15. Unmount everything.
    
    
    
    1. sudo umount /media/flash_root/boot
    
    
    2. sudo umount /media/flash_root/proc
    
    
    3. sudo umount /media/flash_root/dev/pts
    
    
    4. sudo umount /media/flash_root/dev
    
    
    5. sudo umount /media/flash_root/sys
    
    
    6. sudo umount /media/flash_root
    
    
    7. sudo cryptsetup close flash_luks