The few people out there who have slogged through the process of building OXT know how painful it is / can be. One of the most significant contributors to the difficulty here is the amount of technical debt that has built up in the build system over the years. We always “got it to work” when doing build stuff, but with the rate of change in the OE project / community we never kept pace with “doing it right”.
The reasons for this are many and every developer out there is familiar with the “ship it” pressures that tip the scales from “do it right” to “just make it work”. I won’t get into that here because it’s just the way things are now. What’s interesting and useful IMHO is coming up with a way to tip the scales back now that the pressures / incentives are different. So this post will cover some of my work to pay down some of the technical debt that I think is particularly important.
File Systems and Xattrs
One of the unique opportunities afforded us by the OE build system is the ability to produce complete file systems. When we do an install of OXT we don’t have to muck around in the file systems of any of the service VMs (including dom0) because they’re all ready to go straight from the build system. This means we can boot them read only from first boot. With a little bit of work, we could even do a measured launch from first boot complete with predictable measurements.
One of the hurdles we had to overcome was the need for xattr support in the file system creation code. Refusing to have a bunch of “first boot” code to put band aids on the shortcomings of our build system means that we had to come up with a way to create xattrs in the build and have them preserved through the creation of the service VM file systems. This is a problem that the usptream meta-selinux OE layer suffers from currently as anyone who’s built the core-image-selinux image will tell you. They have first boot code that detects the unlabeled file system, relabels it accordingly and then reboots. Only after this first reboot will the system come up with SELinux functioning properly.
Upstreaming xattrs into e2fsprogs
Having some common ground with an upstream like this seems like a good first step. Nothing like a shared problem / need to motivate work. For the OXT community getting this upstream will get us one step closer to being able to build service VMs for OXT (including our own) that don’t depend on a pile of old build metadata and an ancient version of bitbake.
The bits that need to go upstream are actually pretty small. We had to hack up a patch to Pseudo to get that to support xattrs but upstream Pseudo has already implemented this functionality and they did it right. When I saw that Pseudo had already implemented this I was relieved as it probably cut the amount of effort required in half. This left getting the actual disk image produced by the build to support xattrs.
gene3fs was the OXT tool that we used to get the build to spit out ext3 file systems. This is a departure from upstream where they have patches added to the e2fsprogs mke2fs utility. This work wasn’t mine but gene3fs looks like a fork of the unmaintained gene2fs project adding support for the ext3 file system, xattrs and VHDs. Even if this approach is technically superior to the usptream method it’s a moot point. Upstreaming is about longevity.
So ignoring the VHD stuff for now I’ve implemented a set of patches for OE that add xattr support to the build. With these patches in place we can apply SELinux labels to a root file system and boot the resulting image into enforcing mode without having to do first boot fixups. Stage one complete!
Getting patches merged
The hard part about sending stuff upstream is motivating maintainers to merge the patches. meta-selinux is pretty sparsely maintained unfortunately and 3 weeks after having sent my patches to the Yocto mailing list I haven’t had anyone test or even respond to the post. I’m not sure which is worst: being ignored or having someone tear apart the patches because they suck.
Trying to keep from being too persistent / obnoxious I’ve left these patches alone for a bit. Thankfully the e2fsprogs package is one of those critical pieces of the build system and so the maintainers seem to favor stability over frequent updates. This means the likelihood of having to rebase my patches is pretty low. If I don’t get some attention from the meta-selinux maintainers soon I’ll likely rebase them into the openembedded-core layer and send them there. I think the need for this functionality is less understood there but they’re much more active and likely to provide feedback (even if it’s “no thanks”).
Tracking work
We haven’t had a lot of upstreaming work done for OXT yet so there’s not really a process in place for tracking such efforts. As such I’ve created a place on the confluence wiki where I’ll track this stuff. The SELinux / xattr specific stuff is here: https://openxt.atlassian.net/wiki/display/~flihp/SELinux+file+system+labeling+in+build
I’ve set up a page describing the general process I’m following as well and that’s here: https://openxt.atlassian.net/wiki/display/~flihp/Upstreaming
If anyone else is working on sending functionality upstream regardless of where feel free to work off these pages. Once we’ve got a few examples to work from we may decide to formalize the process but for now I’m just trying to pay down some technical debt and make some progress both for OXT and one of the usptreams we share common goals (and hopefully code) with.
technomasochism 