xattrs in OE rootfs part 2

This is an update on some work from a previous post. My xattr patches sat on the yocto mailing list waiting for meta-selinux maintainers to review them for about a month. I got a bit testy and even commented to another contributor off list about the possible need to fork given the lack of response from maintainers. Turns out that this contributor was using his gmail account but actually works for the same company as the maintainer. So about 24 hours later the maintainers magically reappeared and merged my patches!

I really have no interest in forking and maintaining meta-selinux so this was a win. Hopefully I didn’t look like too much of a jerk in the process … ¯_(ツ)_/¯ After a quick stint in a feature branch my xattr code was merged into meta-selinux master. This is only a step in the process though. These patches really belong in the openembedded-core repository. They can likely be moved with minimal effort on the technical side. Getting traction for these sorts of fringe features upstream however takes a bunch of time though, as demonstrated by the 2 months it took to get this into meta-selinux.

As a strategy I should probably prove out an additional use case beyond SELinux file labels. I did some work on meta-measured a while back to add basic IMA support. I really want to expand this to support IMA appraisal. I’m not a huge fan of local enforcement policies but this use case is a great example of build-time xattr usage that would demonstrate the utility of my xattr patches. This is all food for thought though at this point.

Building and Booting Grub2 for a UEFI System

I’ve been playing around with Grub2 a bit while trying to learn something about UEFI. Long story short: I’m really interested in the current measurement gap between the UEFI environment and the Linux OS. Before I go too far into those details I just wanted to get a quick note up on building Grub2 for a UEFI target, specifcally the x86_64 (aka amd64) target.

WARNING: This post won’t help you get Linux or any other OS booted from Grub. I’m only covering the steps necessary to build Grub for the efi target and setting up a UEFI boot disk to run it. That’s all.

Build

Doing the actual build is the easy part. First things first: get the source from the GNU project website. Then RTFM in the INSTALL file. For my setup, specifying --target and --with-platform. The build goes something like this:

./autogen.sh
./configure --target=x86_64 --with-platform=efi
make

UEFI bootable USB

UEFI is a huge departure from the legacy BIOS world. To keep this post short we won’t spend any time discussing BIOS but if you’ve ever wrestled with fixing a broken grub-pc install you’ll know what I mean. Installing or fixing a UEFI Grub system by comparison is pretty simple:

  1. Format a disk (USB in my case) with a GPT using your favorite partitioning tool.
  2. Create a partition with the appropriate type for UEFI. Unfortunately the name for this type is different depending on which partitioning tool you’re using. I use fdisk and the type is ‘EFI System’. It doesn’t need to be large, just enough to hold the grub executable. I made mine 100M which is extremely large. Make note of the UUID of the partition.
  3. Format the partition as FAT16 or FAT32.

Create the UEFI Executable

This builds everything but we don’t yet have a UEFI executable. To create one you’ll need to run the grub-mkimage utility. This is the root of the build directory and it’s responsible for stitching the grub kernel, core modules and a builtin config to bootstrap grub. Generally you’d only build the minimally necessary grub modules into the UEFI executable but since this is for my debug setup I’m going to build every module in. This makes the executable very large but it’s easier.

Before we can do this we need to grok the idea of the builtin config. Grub needs a hint when it comes to setting up the system. For my purposes all I care about is getting the FAT partition mounted so my config just tells Grub how to find the root partition and what to set as the prefix.

(
  cat < grub-buildin.cfg

The --fs-uuid is pretty self explanitory. The UUID in the above code fragment isn’t the actual UUID of my UEFI system partition. If you’re following along at home you can get the UUID for the FS you created above using the blkid utility. The prefix tells Grub where to look for the grub.cfg configuration. We won’t even supply one here so the system will boot straight to the grub shell.

Since we’ve specified the prefix in the builtin config the -p option below is useless but grub-mkimage throws an error if it’s not provided. The value we pass is obviously wrong but the image produced will still function.

Finally execute the command:

./grub-mkimage -d ./grub-core -o bootx64.efi -O x86_64-efi -c ./grub-buildin.cfg -p /foobar $(find . -name '*.mod' | tr 'n' ' ' | sed -e 's/.mod//g')

Setup the Boot Disk

By default UEFI firmware will look for the bootloader on a disk at /EFI/BOOT/bootx64.efi. So all we need to do is mount our EFI system partition, create the /EFI/BOOT directory and copy the bootx64.efi executable there. In the above command we built in ever module that Grub has so we don’t need to worry about copying over any modules.

And that should do it. Point your system to this USB device, boot up the grub efi executable and you should go straight to a Grub shell. If you want to get an OS up and running on top of this you’ve still got a long way to go. But if you’re like me and just screwing around with grub (more on this later) then you’ve got a pretty good test environment. Now all you need to do is write a quick config to get Grub logging to some serial hardware and crank up the debug output!

Purism, the TPM and the FSF

Alright, I’ll come out and say it: I’m optimistic about the Purism / Librem project. The impending release of their first product is really exciting. Their goals are extremely ambitious (the hallmark of a worthwhile project) and despite having come up short of their stated goals and receiving some harsh though justified critique on the web, they’ve still produced a laptop that’s shipping soon. And so my optimism is holding out.

This is my first post on this topic and is mostly introductory: basically why I care about the project and the subject in general. My goal is to develop a series of posts discussing freedom and security (two things that I’m convinced are related) in the context of general purpose computing and the Purism Librem laptop. If you love freedom, free software and computer security, read on … if not, read on anyways.

Criticism from the Web

Not everyone is as impressed with Purism as I am, and the best articulation I’ve found is from Alexandru Gagniuc on the coreboot blog. The TL;DR is that people have been working in the free hardware space for a long time. The impression is that Purism showed up and claimed that they could do in 6 months what others have been working towards for years. Many have taken this as a sign that something is amiss. Those working in OSS see this sort of stuff all the time: someone shows up on your mailinglist with big plans, lots of enthusiasm … and then they figure out how much work it will take and you never hear from them again. I’m hopeful that, despite falling short of their initial goals, the folks at Purism stick around and continue to chip away at their stated goals.

Something that I find particularly troubling is the increasing rhetoric on the Purism website around “fighting for your freedom” all while hedging on past promises. The new video on their homepage with the young cartoon woman flying their flag and weeping cartoon tears for her lost freedoms has set my instinctive negative reaction to over-the-top marketing into high gear. Add to this some genuine and public concerns over the claims made by Purism and I’d say that they’ve got a bit of digging to do if they want to get out of this hole.

Lesson to learn: Grand or unrealistic claims about security, privacy-preserving and software freedom won’t go unnoticed or unchallenged when your target audience knows what’s up. Purism is catering to a niche market. Their target audience is extremely savvy about these things and they’re not going to be shy about “calling bullshit”. Skeptics are going to be skeptical, they’re going to demand proof. This is a good thing.

Continued Optimism

Despite all of this, I’m still optimistic about the project. Even better, the folks at Purism seem to be paying attention to their critics. They’ve published a road map detailing the steps necessary to reach their stated goal of FSF RYF (respects your freedom) certification. No doubt it’s going to be an uphill battle and the folks over at coreboot may be right: the entirety of the firmware may never be OSS. But having an OEM who actually wants to fight for the user will never be a bad thing so long as they don’t alienate their target audience in the process. In a time when OEMs are getting busted for integrating ad/spy-ware into their firmware we need a change.

The Future: Freedom and Security

This is probably going to get dangerously close to an attempt to predict the future, but I’m very hopeful for Purism and their products. I haven’t been following too closely but when I looked the other day, sure enough the specs for the Librem 15 list a TPM and it’s a version 2. Sweet. More on this in a future post.

Additionally, the FSF has endorsed CrowdSupply (the crowd funding platform used by Purism). I’m trying not to read too far into this but I can’t help but think that this represents an organized push from free software movement into hardware while adopting the pro-security-and-privacy rhetoric that’s become so relevant in the wake of the Snowden revelations.

I generally think of myself as a pragmatist when it comes to running purely open source software. This may just be a result of the need to load proprietary / binary firmware to get the wireless card on my laptop to function properly. It also could be related to my love of security technologies and the fact that the FSF has taken such a hard-line in their labeling of the TPM as malicious, a position I’ve always viewed as misinformed. I’m hopeful that the Purism project can help mend the rift between the free software movement and the security technologies that are essential to preserving our freedoms in an increasingly hostile computing environment.

This post was a sort of introduction, just some background about why I care about the Purism project. In my next post I’m hoping to get into why I think freedom and security have become so closely related. Stay tuned.

Nagasawa

My posts have been very sparse recently. Even less frequent have been posts on non-technical topics. To stay true to the title of my blog I’ve gotta step up my posts on “two-wheeled vehicles”. That or change the name. Changing the name would require digging through WordPress menus and since that’s my least favorite thing to do I ran out and got another bicycle so I could blog about it … and ride it.

Nagasawa makes a pretty serious track frame. More serious than me for sure but when I saw this thing on CraigsList I couldn’t resist. It was meant to be. Nearly everything on it is NJS and it’s in very good shape. A few chips in the paint here and there but nothing beyond standard wear. Even better, it’s the right size. You don’t find very many Japanese track bikes that fit someone over 6 feet tall.

The guy I bought it from had it set up for track riding. Since I’ll be riding it on the street I’m in the process of taming it a bit. First things first: get rid of the NJS track seat (aka the “ass-hatchet”). Then I’ll probably have to drop the gear ratio a bit. The wheels are super nice but they’re not beefy enough for every day street riding. I’ll have to start saving before I can get a new set.

Till my next post, enjoy the obligatory bike porn:
Nagasawa-headtubeNagasawa_front-rightNagasawa_right-rear

OXT and Paying Down Technical Debt

The few people out there who have slogged through the process of building OXT know how painful it is / can be. One of the most significant contributors to the difficulty here is the amount of technical debt that has built up in the build system over the years. We always “got it to work” when doing build stuff, but with the rate of change in the OE project / community we never kept pace with “doing it right”.

The reasons for this are many and every developer out there is familiar with the “ship it” pressures that tip the scales from “do it right” to “just make it work”. I won’t get into that here because it’s just the way things are now. What’s interesting and useful IMHO is coming up with a way to tip the scales back now that the pressures / incentives are different. So this post will cover some of my work to pay down some of the technical debt that I think is particularly important.

File Systems and Xattrs

One of the unique opportunities afforded us by the OE build system is the ability to produce complete file systems. When we do an install of OXT we don’t have to muck around in the file systems of any of the service VMs (including dom0) because they’re all ready to go straight from the build system. This means we can boot them read only from first boot. With a little bit of work, we could even do a measured launch from first boot complete with predictable measurements.

One of the hurdles we had to overcome was the need for xattr support in the file system creation code. Refusing to have a bunch of “first boot” code to put band aids on the shortcomings of our build system means that we had to come up with a way to create xattrs in the build and have them preserved through the creation of the service VM file systems. This is a problem that the usptream meta-selinux OE layer suffers from currently as anyone who’s built the core-image-selinux image will tell you. They have first boot code that detects the unlabeled file system, relabels it accordingly and then reboots. Only after this first reboot will the system come up with SELinux functioning properly.

Upstreaming xattrs into e2fsprogs

Having some common ground with an upstream like this seems like a good first step. Nothing like a shared problem / need to motivate work. For the OXT community getting this upstream will get us one step closer to being able to build service VMs for OXT (including our own) that don’t depend on a pile of old build metadata and an ancient version of bitbake.

The bits that need to go upstream are actually pretty small. We had to hack up a patch to Pseudo to get that to support xattrs but upstream Pseudo has already implemented this functionality and they did it right. When I saw that Pseudo had already implemented this I was relieved as it probably cut the amount of effort required in half. This left getting the actual disk image produced by the build to support xattrs.

gene3fs was the OXT tool that we used to get the build to spit out ext3 file systems. This is a departure from upstream where they have patches added to the e2fsprogs mke2fs utility. This work wasn’t mine but gene3fs looks like a fork of the unmaintained gene2fs project adding support for the ext3 file system, xattrs and VHDs. Even if this approach is technically superior to the usptream method it’s a moot point. Upstreaming is about longevity.

So ignoring the VHD stuff for now I’ve implemented a set of patches for OE that add xattr support to the build. With these patches in place we can apply SELinux labels to a root file system and boot the resulting image into enforcing mode without having to do first boot fixups. Stage one complete!

Getting patches merged

The hard part about sending stuff upstream is motivating maintainers to merge the patches. meta-selinux is pretty sparsely maintained unfortunately and 3 weeks after having sent my patches to the Yocto mailing list I haven’t had anyone test or even respond to the post. I’m not sure which is worst: being ignored or having someone tear apart the patches because they suck.

Trying to keep from being too persistent / obnoxious I’ve left these patches alone for a bit. Thankfully the e2fsprogs package is one of those critical pieces of the build system and so the maintainers seem to favor stability over frequent updates. This means the likelihood of having to rebase my patches is pretty low. If I don’t get some attention from the meta-selinux maintainers soon I’ll likely rebase them into the openembedded-core layer and send them there. I think the need for this functionality is less understood there but they’re much more active and likely to provide feedback (even if it’s “no thanks”).

Tracking work

We haven’t had a lot of upstreaming work done for OXT yet so there’s not really a process in place for tracking such efforts. As such I’ve created a place on the confluence wiki where I’ll track this stuff. The SELinux / xattr specific stuff is here: https://openxt.atlassian.net/wiki/display/~flihp/SELinux+file+system+labeling+in+build
I’ve set up a page describing the general process I’m following as well and that’s here: https://openxt.atlassian.net/wiki/display/~flihp/Upstreaming

If anyone else is working on sending functionality upstream regardless of where feel free to work off these pages. Once we’ve got a few examples to work from we may decide to formalize the process but for now I’m just trying to pay down some technical debt and make some progress both for OXT and one of the usptreams we share common goals (and hopefully code) with.

PGP links

I wish PGP was easier to use. When it comes to having to invoke gpg directly I usually spend some time with the man page and that’s enough. But when it comes to the more detailed stuff like setting algorithm preference or identities I always fall back to the collective internet knowledge. I think I’ve looked up the following links a few times now and I’m leaving them here for reference:

https://wiki.debian.org/Subkeys
https://www.debian-administration.org/users/dkg/weblog/48
http://blog.infertux.com/2013/11/03/how-to-remove-an-email-address-from-a-gpg-key/

HP Elitebook 820 G1 laptop dock script

Upgraded my laptop from an ancient HP 2760p so it’s time to write a new script to handle plug / unplug events from the docking station. Wait, what? Write a new script? Haven’t I done this before? Sure, I wrotea docking script for my Thinkpad x61s, but that was 6 years ago. No way that script still works … no wait it does!

Minor tweaks to sort out some differences between the udev events for the thinkpad / elitebook and changes to the xrandr command for my current monitor setup and it’s good to go!

The thinkpad had better integration with the kernel ‘dock’ driver so it generated different events for ‘dock’ and ‘undock’ events. For the EliteBook I had to trigger on the DRM kernel subsystem getting a ‘change’ event on ‘card0’. Not sure what that means but it’s consistent for plug / unplug. To determine whether the ‘change’ is docked or undocked I’m parsing xrandr output to see which monitor is present where. It’s not as nice but by parsing the output of xrandr it’s pretty simple. So the udev rule looks like this:

SUBSYSTEM=="drm", KERNEL=="card0", ACTION=="change", RUN+="/usr/local/bin/dock.sh"

And the script I’m now using for dock/undocking my EliteBook 820 G1 on Debian 8.0 is here.

Thanks for doing me a solid past me. Well done.

on-break

awesome battery widget

I spent a few hours last night customizing the awesome window manager (wm) on my new laptop install. I’ve been using awesome for a while now (years) but I haven’t done much by way of customizing my setup. The default config from Debian, with a few small tweeks to default tags and layouts, has been sufficient. But having a battery gauge on my laptop is pretty important so I carved out a few minutes to set this up.

As always I’m not the first person to have this problem. Luckily those that came before me put their work up on github so all I had to do was clone awesome-batteryinfo, copy battery.lua into my ~/.config/awesome directory and integrate the battery widget into my rc.lua.

Integrating this widget is pretty painless. There are four steps: First you have lua pull in battery.lua:

require("battery")

Second you instantiate the widget:

mybatterywidget = widget({type = "textbox", name = "batterywidget", align = "right" })

Third you place the widget in a wibox. Debian has a wibox positioned across the top of each screen. I took the batterywidget created above and added it to the wibox widget list along side the layoutbox, textclock etc. My final mywibox.widgets looks like this:

mywibox[s].widgets = {
    {
        mylauncher,
        mytaglist[s],
        mypromptbox[s],
        layout = awful.widget.layout.horizontal.leftright
    },
    mylayoutbox[s],
    mytextclock,
    batterywidget,
    s == 1 and mysystray or nil,
    mytasklist[s],
    layout = awful.widget.layout.horizontal.rightleft
}

Finally I set up a timed event to update the widget every 20 seconds. I also seeded the widget text to show the battery data at the time the widget is created. This means that the widget will come up with meaningful data before the first even fires (20 seconds after init):

-- seed the battery widget: don't wait for first timer
mybatterywidget.text = batteryInfo("BAT0")
-- timer to update battery widget
mybatterywidget_timer = timer({timeout = 20})
mybatterywidget_timer:add_signal("timeout", function()
    mybatterywidget.text = batteryInfo("BAT0")
end)
mybatterywidget_timer:start()

That’s all there is to it. Thanks to koenwtje for the great widget. I should probably collect my awesome configs into a git repo so I don’t have to go back and rediscover how to do this every time I build a new system …

Cleaning up my TPM hack

A few weeks back I posted some stuff about hacking up an Asus TPM daughter card to hook it up to a PCEngines APU system. That was the first I’ve soldered in pretty much forever. It was ugly. Seriously ugly. But it worked. Still, we can do better.

Desoldering

In my first pass at this I left the pins in the board after cutting off the connector. To hook up the new connector I just soldered the wires directly to the existing pins. The right way to do this is to desolder the existing pins, pull them out, and run the new wires into the holes left by the old pins.

After discovering what a solder pump is this isn’t the impossible task I thought it was. There were two pins that wouldn’t budge though. These two were hooked up to the ground plate on the card so when I applied heat the whole card absorbed it. Since I didn’t need these pins I just left them alone. This is what the card looks like with the pins desoldered:

IMG_20150303_152024

IMG_20150303_152006

New Wiring

Now we just add some new wires following the mapping from the last post. Business as usual. I did up ground in the same way running it directly to the plate on the daughter card. The sanding job I did this time was much better since I used real sand paper. Actually I picked up a few of those nail files at the grocery store and they worked out great.

IMG_20150308_184619

Last time I ran all the wires over the top and they piled up something awful. This time I ran the problematic ones under the card to keep things clean. I’m pretty happy with the end result.

IMG_20150309_195408

IMG_20150309_195358

Programming the DS1077L Oscillator

My last post documented my first bit of hardware hacking in probably 10 years. I had hoped to have a full post for a related bit of work ready for posting by now but, to stick with our theme, this stuff is hard so I’ve had to break it down into more than one post. Hopefully this won’t be too much of a spoiler but my current project requires a 33MHz oscillator. Here’s a bit of data about the programmable once I picked up along with a few tools I wrote to program it from Linux.

DS1077L Oscillator

Fixed frequency crystal oscillators are pretty easy to buy, but if you’re prototyping on a breadboard these are a bit of a PITA to hook up. As an alternative, there are surfance mount programmable oscillators available and the DS1077L from Maxim seems to be the most popular in the “maker” community. Sparkfun sells these pre-packaged on breakout boards so if you just just want to buy something and start using it this is the route to go.

I ordered the parts individually so I could practice my soldering but the end result is the same. Take a few DS1077L+66s from Mouser Electronics along with a few 8-pin SparkFun SOIC to DIP Adapters, solder them together and then plug them into your breadboard. The only difference is that the breakout board from Sparkfun comes with a filter capacitor so we’ll have to supply our own. This oscillator is programmable via the I2C bus so to program the oscillator to the frequency we need we need to wire it up to the I2C bus on a Linux system. In the end the circuit looks something like this.

IMG_20150307_115107

Top quality CAD drawing for you there. If you’re interested in the details of how this chip works the spec sheet is your friend. The values for the pull up resistors on the I2C lines and the bypass capacitor are specifically defined there. My purposes are simple in that I just need a 33MHz clock so the CTRL pins can be ignored, as can the reference output OUT0. OUT1 will output the clock and SCL/SDA are the two I2C clock and data pins. Initially the DS1077L+66 is programmed to output a 66MHz clock so we’ll need to program the chip to cut this in half.

Programming the DS1077L

The first step in this process is wiring the circuit and attaching it to the I2C bus on a Linux system. I soldered an I2C header on to my PCEngines ALIX3d2 and hooked it up like so:

IMG_20150225_200211

This chip is pretty sophisticated so I won’t cover all possible configurations. The data sheet is readily available here so if you want the details have a read (you should anyways if you’re using this chip). All I’m going to cover here is scaling down the clock frequency and the tool I’ve written to do so.

The primary output (OUT1) has two mechanisms for controlling its output. First is a prescaler circuit that can divide the native frequency (66MHz) by 1, 2, 4 or 8. The second is a divider circuit that can divide the frequency by integers between 2 and 1025. The divider circuit can be bypassed completely if we wish so we can get a 33MHz clock in two ways: 1) set the prescaler to 2 and bypass the divider or 2) set the prescalar to 1 and the divider to 2. The DS1077L defines a low power state where much of it’s internal circuitry is disabled including the divider so I opted for this low power state and the first option above.

To aid in programming this thing I’ve put together a small set of tools I’m calling ds1077l-ctrl. These tools expose a simple command interface for modifying the state of the control registers in the DS1077l. As an example we would program this chip to output the 33MHz clock that I need like so:

$ ./ds1077l-mux --bus-dev=/dev/i2c-0 --set --pdn0=0 --sel0=0 --en0=0 --div=1 --p1=2

That’s it. The –help / –usage messages for the tools combined with data from the spec sheet should be all that you need to set the DS1077L to any possible state. In the command above we’ve set (–set) the DS1077L device on I2C bus 0 (–bus-dev=/dev/i2c-0) such that the divider is disabled (–div=1), the prescalar to 2 (–p1=2). The other options (–pdn0=0 –sel0=0 –en0=0) set the oscillator to a low power state disabling the circuitry we’re not using.

To get a sort of before and after I hooked up this chip to an oscilloscope to capture the default 66MHz waveform and the 33MHz one we get after executing the command above:

IMG_20150225_200236
IMG_20150225_193909

Bugs

This software is not without bugs (as always). I couldn’t get the WRITEE2 command to work properly but it’s only necessary if you change the default behavior of the chip and set the WC bit to 1. This causes the oscillator to write register state to EEPROM only when the WRITEE2 command is invoked. By default it will save the register state every time it changes so just leave the WC bit alone and everything should work as expected. Alternatively feel free to fix this bug and send me a pull request.