{"id":1742,"date":"2013-09-05T21:18:30","date_gmt":"2013-09-05T21:18:30","guid":{"rendered":"http:\/\/davstott.me.uk\/?p=1742"},"modified":"2013-09-05T21:21:11","modified_gmt":"2013-09-05T21:21:11","slug":"ubuntu-13-04-on-a-usb-flash-drive-and-merging-its-persistent-storage","status":"publish","type":"post","link":"https:\/\/davstott.me.uk\/index.php\/2013\/09\/05\/ubuntu-13-04-on-a-usb-flash-drive-and-merging-its-persistent-storage\/","title":{"rendered":"Ubuntu 13.04 on a USB flash drive, and merging its persistent storage"},"content":{"rendered":"

I’ve got a new whizzy-fast USB3 flash drive and one of the best bits about it is booting into a live Ubuntu in about 12 seconds flat. Carrying around a full desktop environment is great, especially one that takes up very little space and can persist its storage, but one of its potential pitfalls is that it only has a limited amount of disk space that it can write to.<\/p>\n

<\/p>\n

Pitfalls<\/h2>\n

With the version of the 13.04 iso I downloaded, I had to take a couple of unusual steps to get Ubuntu working properly from USB.<\/p>\n

The first was when using usb-creator-gtk to create a bootable USB drive from the Ubuntu iso, I had to run it as root and allow writing to system internal drives, otherwise it segfaulted after doing the bulk of the copying. (https:\/\/bugs.launchpad.net\/ubuntu\/+source\/usb-creator\/+bug\/859539<\/a>)<\/p>\n

\r\n$ sudo usb-creator-gtk --allow-system-internal \r\n<\/pre>\n
It might not be obvious, but the bootable USB drive actually has more than one bootloader. When it boots using UEFI, it uses grub, when it uses the older style BIOS booting, then it uses syslinux. At the version I was running, I had to manually edit grub’s boot menu to pass in the ‘persistent’ kernel parameter, which is easy enough but not entirely obvious. (https:\/\/bugs.launchpad.net\/ubuntu\/+source\/usb-creator\/+bug\/1159016<\/a>)<\/p>\n
Merging persistent storage<\/h2>\n
The root filesystem that boots into the live environment is a read only squashfs that takes up about 740Mb of space on the drive in a file called casper\/filesystem.squashfs. squashfs is very good, but it is read-only, so to store your changes, Ubuntu uses overlayfs<\/a> to layer a normal writable filesystem on top of the read-only one, that’s stored in \/casper-rw.<\/p>\n
This works really well for casual use, but it can use up writable space at a surprising rate, especially if you delete a big package from the read-only filesystem (such as LibreOffice) or patch your system up to date with an apt-get upgrade.<\/p>\n
The good news is that you can use another Ubuntu installation to merge the changes from the writable filesystem back into the read-only squashfs, then start afresh. <\/p>\n
This all seems entirely scriptable, but you wouldn’t just download a random script from a stranger on the internet and run it as root would you? So for now, this is how I did it:<\/p>\n
Change to root, copy the files from USB to a temporary area and mount them as filesystems<\/p>\n
\r\n$ sudo -s\r\n# cd \/var\/tmp\r\n# mkdir usblive\r\n# cd usblive\r\n# cp \/media\/dav\/C35B-7E7E\/casper-rw .\r\n# cp \/media\/dav\/C35B-7E7E\/casper\/filesystem.squashfs .\r\n# mkdir readonly readwrite merged\r\n# mount filesystem.squashfs readonly -o loop,ro\r\n# mount casper-rw readwrite -o loop,rw\r\n# mount none merged -o lowerdir=readonly,upperdir=readwrite -t overlayfs\r\n<\/pre>\n
If this has all worked then you should now see something like this:<\/p>\n
\r\n# df -h\r\nFilesystem          Size  Used Avail Use% Mounted on\r\n< snip >\r\n\/dev\/sde1            30G  4.8G   26G  17% \/media\/dav\/C35B-7E7E\r\n\/dev\/loop0          738M  738M     0 100% \/var\/tmp\/usblive\/readonly\r\n\/dev\/loop1          4.0G  1.2G  2.6G  33% \/var\/tmp\/usblive\/readwrite\r\nnone                4.0G  1.2G  2.6G  33% \/var\/tmp\/usblive\/merged\r\n<\/pre>\n
Create a new squashfs based on the merged filesystem, this can take a little time depending on your CPU.<\/p>\n
\r\n# apt-get install squashfs-tools\r\n# mksquashfs merged newfilesystem.squashfs\r\nCreating 4.0 filesystem on newfilesystem.squashfs, block size 131072.\r\n[=========================================================\\] 108641\/108641 100%\r\nExportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072\r\n\tcompressed data, compressed metadata, compressed fragments, compressed xattrs\r\n\tduplicates are removed\r\nFilesystem size 1069333.62 Kbytes (1044.27 Mbytes)\r\n\t39.06% of uncompressed filesystem size (2737455.95 Kbytes)\r\n< snip >\r\n<\/pre>\n
If you’re patient, or have a very fast cpu (hello Core i7),  you can use a higher compression technique to save some space, at the cost of it taking longer to create:<\/p>\n
\r\n# mksquashfs merged newfilesystem.squashfs -comp xz\r\nParallel mksquashfs: Using 8 processors\r\nCreating 4.0 filesystem on newfilesystem.squashfs, block size 131072.\r\n[=========================================================|] 108641\/108641 100%\r\nExportable Squashfs 4.0 filesystem, xz compressed, data block size 131072\r\n\tcompressed data, compressed metadata, compressed fragments, compressed xattrs\r\n\tduplicates are removed\r\nFilesystem size 934018.40 Kbytes (912.13 Mbytes)\r\n\t34.12% of uncompressed filesystem size (2737455.95 Kbytes)\r\n< snip >\r\n<\/pre>\n
OK, we’re nearly there, we just need to regenerate the filesystem’s size and manifest just in case we want to install from this USB stick<\/p>\n
\r\n# umount merged readwrite readonly\r\n# mount newfilesystem.squashfs readonly -o loop,ro\r\n# chroot readonly dpkg-query -W --showformat='${Package} ${Version}\\n' > newfilesystem.manifest\r\n# printf $(du -sx --block-size=1 readonly | cut -f1) > newfilesystem.size \r\n\r\n<\/pre>\n
Now we copy the new files back onto the USB flash drive and reset the writable overlay filesystem<\/p>\n
\r\n# cp newfilesystem.squashfs \/media\/dav\/C35B-7E7E\/casper\/filesystem.squashfs \r\n# cp newfilesystem.size \/media\/dav\/C35B-7E7E\/casper\/filesystem.size \r\n# cp newfilesystem.manifest \/media\/dav\/C35B-7E7E\/casper\/filesystem.manifest\r\nroot@newt:\/var\/tmp\/usblive# mkfs -t ext3 \/media\/dav\/C35B-7E7E\/casper-rw \r\nmke2fs 1.42.5 (29-Jul-2012)\r\n\/media\/dav\/C35B-7E7E\/casper-rw is not a block special device.\r\nProceed anyway? (y,n) y\r\nFilesystem label=\r\nOS type: Linux\r\n< snip >\r\n<\/pre>\n
and remember to clean up after ourselves<\/p>\n
\r\n# cd\r\n# rm -rf \/var\/tmp\/usblive\r\n<\/pre>\n
then rebooting back into the live USB drive, we can see that we’ve now got the full size to use for writing again<\/p>\n
\r\nubuntu@ubuntu:~$ df -h\r\nFilesystem      Size  Used Avail Use% Mounted on\r\n\/cow            4.0G   92M  3.7G   3% \/\r\n\/dev\/sdi1        30G  5.0G   25G  17% \/cdrom\r\n\/dev\/loop0      913M  913M     0 100% \/rofs\r\n< snip >\r\n<\/pre>\n
References<\/h3>\n
I got most of the way there myself by starting with seeing that the usb environment used overlayfs, then had a look around the flash drive to see what file system images were where, but I wouldn’t have got the package manifest without the Ubuntu docs, the example mount for overlayfs saved some time from the Ubuntu forum post, and it was good to see that I could have googled a little bit harder when I started this afternoon and just used’s Jan Varho’s excellent method instead. Oops. <\/p>\n