Using tmpfs to Minimize Disk IO

Now that I’ve got my ALIX system up and running Lenny, it’s time to tweak the configuration. One of the things I liked best about the Voyage distribution is its use of tmpfs for the directories that receive a lot of writes to minimize the IO on the compact flash (CF) card. The reason for doing this is there’s a maximum number of write cycles that can be made to the CF card. Not that I’ve actually worn out a CF card before but I don’t intend to either.

I want to have /tmp, /var/run, /var/lock and /var/log mounted as tmpfs. There’s a few resources out there that provide scripts and methods for doing this but I’m not a big fan of any of them (see references section below). Debian has almost all of the necessary machinery to perform this task with minimal custom scripting. We’ll be be mucking around in the /etc/init.d and /etc/rcS directories but as little as possible.

/var/run and /var/lock

A significant portion of what we want can be achieved using the features of the mountkernfs.sh script. There are two variables called RAMRUN and RAMLOCK that control whether or not /var/run and /var/lock are mounted as tmpfs respectively. These variables are set in /etc/default/rcS and the mount points are created in the /etc/init.d/mountkernfs.sh script if the associated variable is set to “yes”.

There does seem to be a small bug in this script however. It does not import the variables it needs from /etc/defaults/rcS. I’m pretty sure this is a bug and can be fixed with a very small patch

--- ./mountkernfs.sh.old	2010-01-02 22:32:44.000000000 -0500
+++ ./mountkernfs.sh	2010-01-02 22:33:09.000000000 -0500
@@ -18,6 +18,7 @@
 . /lib/init/mount-functions.sh
 
 [ -f /etc/default/tmpfs ] && . /etc/default/tmpfs
+[ -f /etc/default/rcS ] && . /etc/default/rcS
 
 do_start () {
 	#

/tmp and /var/log

After this we’re half way to achieving our goal. It would be nice if the /var/log directory could be mounted as easily but most people will tell you that having log files reside on non-persistent storage is a very bad idea. If something goes wrong and your system goes down you won’t be able to analyze your log files. This is a very real concern which we will address shortly. First the remaining two mount points need to be mounted through /etc/fstab with the following two entries:

tmpfs  /tmp     tmpfs   defaults,noexec,nosuid,mode=1777         0   0
tmpfs  /var/log tmpfs   defaults,noexec,nosuid,nodev,mode=755  0   0

This solves the issue of mounting /tmp but /var/log requires a little more work. Debian (and LInux in general I think) expects that some files and directories will exist in the logging directory. To account for this, after the mount scripts run we want to create the necessary file structure. I’ve done this by creating a tar archive of the expected structure and extract it to the newly mounted tmpfs /var/log directory on each system boot. The following script: logskel.sh.gz does exactly this:

PATH=/sbin:/bin
. /lib/init/vars.sh
. /lib/lsb/init-functions

# get configuration info for this script
[ -e /etc/default/log-skel ] && . /etc/default/log-skel

case "$1" in
	start|"")
		log_begin_msg $@
                # select defaults if the configured options don't make sense
		[[ ! -f $SKEL ]] && SKEL=/lib/init/log-skel.tar.gz
		[[ ! -d $LOG_DIR ]] && LOG_DIR=/var/log
		/bin/tar -zxf ${SKEL} -C ${LOG_DIR} 2>&1 > /dev/null
		log_end_msg $?
		;;
	restart|reload|force-reload)
		echo "Error: argument '$1' not supported" >&2
		exit 3
		;;
	stop)
		# No-op
		;;
	*)
		echo "Usage: $NAME [start|stop]" >&2
		exit 3
		;;
esac

You’ll need to put the archive that’s being extracted into /lib/init or specify a different location through the /etc/default/log-skel file. I’m using this structure on a system with very few daemons running: log-skel.tar.gz. You may want to build one specific to your systems needs. The above script should be run after all file systems are mounted. On a Lenny system this is done by linking the script to /etc/rcS.d/S36log-skel.sh

Persistent logging of “serious” errors

Finally we still want to log “serious” error messages from syslog to persistent storage so they aren’t lost if the system reboots. This is a single rsyslog rule that can be put in the rsyslog.conf file directly or a separate file in the /etc/rsyslog.d directory. I chose the latter: persistent.conf

*.err    /var/persistent.log

Now cross your fingers and reboot. Any messages you see during boot indicating missing log files can be fixed by adding the file to the template archive we extract in the init script above. After a successful reboot you should be able to see that these four directories are tmpfs mount points by executing the mount command. This is the full output on my ALIX system.

/dev/hda2 on / type ext2 (rw,errors=remount-ro)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
varrun on /var/run type tmpfs (rw,nosuid,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
procbususb on /proc/bus/usb type usbfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
tmpfs on /tmp type tmpfs (rw,mode=1777)
tmpfs on /var/log type tmpfs (rw,noexec,nosuid,nodev,mode=755)

We’re interested in lines 5, 6, 11 and 12. Success!

References

Installing Lenny on ALIX 2d3 over Serial Console

In my last post I linked to the howto forge article that gives detailed instructions for installing Debian on a PCEngines WRAP system. After playing with Voyage Linux on my new ALIX system (the successor to the WRAP) I decided that I would be better of going with the stock Debian Lenny (5.0). The cool thing is, I didn’t follow the howto 🙂 Instead I decided to exercise the PXE-boot support in the ALIX bios and learn something new in the process. This system will be a VPN gateway to a management network that I need to access remotely.

The install requires two connections between my laptop (itself running Lenny) and the ALIX system. First the ethernet connection between the two for the PXE-boot and subsequent network install and a null-modem / serial cable to act as a console for the installer. That’s right, no VGA on this thing … old school. Here’s what it looks like:
ALIX-Serial
The red box is just the housing for a retractable ethernet cable.

First off get minicom up and running. My laptop has no serial port so I broke out my new fangled USB one. Grab the ALIX manual and look up the default comport settings: 38400 8N1, flow control = none. I had a problem with minicom having the following failed assertion:

minicom "Assertion `inptr - bytebuf > (state->__count & 7)' failed"

It seems the way to solve this is by setting the LANG environment variable to “C”. You can do this by executing minicom like this:

LANG=C minicom -c on

Plug the ALIX board in now and you should see the BIOS initializing then failing to find a disk to boot from. Reset it and this time when the BIOS is performing the memory test press the ‘S’ key to enter the BIOS settings. Change the serial baud to 9600 and while you’re in the menu enable PXE-boot by pressing ‘E’.

I recommend you change the baud setting because all documentation I found on installing Linux using a serial console used this baud rate and I wanted to keep things consistent. Likely you can chose any rate you want as long as you’re consistent in the settings you chose … YMMV

Next we track down the directions for installing Debian using netboot. This is well documented on the Debian websites but naturally there are a few catches. I’ll cover those here. Specifically netboot doesn’t support serial console installs. First we’ll worry about getting PXE-boot going, then worry about the installer.

I used the tftpd-hpa tftp server as recommended and the CMU bootp server. inetd.conf already had the necessary configuration lines for these two servers, they only need to be uncommented:

tftp           dgram   udp     wait    root  /usr/sbin/in.tftpd /usr/sbin/in.tftpd -s /var/lib/tftpboot
bootps          dgram   udp     wait    root    /usr/sbin/bootpd        bootpd -i -t 120

Take note of the root directory for the tftp server. This had me scratching my head for a while. The ‘-s’ option on the first line is the root directory for the tftp server (see the man page for more). This is where we extract the the netboot.tar.gz archive. This affects the configuration we’ll use for bootpd. Note the hd option:

# /etc/bootptab: database for bootp server (/usr/sbin/bootpd)
mgmtvpn:
  hd=/:
  bf=pxelinux.0:
  ip=10.1.0.1:
  sm=255.255.255.0:
  sa=10.1.0.2:
  ha=XXXXXXXXXXXX:

hd is set to / since we’ve told the tftp server that /var/lib/tftpboot is it’s root directory. ha needs to be the MAC address of the NIC on the ALIX board you’re using.

Next comes the patch to add serial console support to the syslinux configuration used in the netboot. The lack of serial console in the installer is documented in bug 309223. There’s a patch posted as a work around but it’s for the amd64 installer and has a lot of options we don’t need (the GTK installer won’t do us much good over the serial console). The patch isn’t short so I won’t include it in it’s entirety. It can be downloaded here: installer.diff.gz. Copy this file to the root of the installer directory and apply the patch:

zcat installer.diff.gz | patch -p1

Notice that we’ve set the serial console to 9600 baud just like we did in the ALIX BIOS menu.

From here the installer should work just like it would using VGA. The serial console is slower (though we may be able to speed it up a bit using a higher baud rate) and the Geode CPU is only 500Mhz but the install didn’t take long. Now the last detail: I’m using my laptop to NAT traffic from the ALIX system to my wireless network when doing the install. This isn’t a requirement and if you’ve got a wired network available then you may want to just use that as is.

Next we need to configure some odds and ends specific to the ALIX system. That’s coming up next.

ThinkPad x61s UltraBay docking script

UPDATE: I’ve updated this script with a much better design because SELinux wouldn’t let me muck around with X’s tmp files. dock.sh.

Over a year ago I decided to pick up a new ThinkPad, this time the ultra portable x61s. My desktop was aging and I needed mobility more than anything else. I had always intended to get the X6 UltraBay so I could set it up on my desk but I never seemed to get around to it (probably on account of the phobia I developed while trying to get RedHat 9 to suspend and resume on my old T40).

Tonight, after getting my new (well new to me) UltraBay from ebay in the mail I took on setting up a script to run when docking and undocking the laptop. First off, I’m impressed with how well Linux works with the docking station “out of the box”. Even without udev rules to handle the dock/undock event the CD-RW/DVD combo drive hot plugs perfectly (I’m running a vanilla 2.6.31 kernel on Debian Lenny).

The udev rules and docking scripts for this set-up are well documented out there on the web but there were some details I had to assemble from a few different places. This post is mostly to collect the details in one place. If it’s useful to someone else out there even better.

First lets define what we’re trying to do: I’ve hooked up an external monitor to the UltraBase and I want to distribute my desktop across the external monitor and the LVDS display on the laptop when it’s docked. When it’s undocked the laptop should return to using only the LVDS display. We can script all that using a little bit of control logic and some very basic xrandr commands.

Let’s run through getting xrandr working first, then worry about when and how the script should be run. ThinkWiki has some great info for using xrandr to configure an external monitor. But when I ran some of these commands manually, nothing happened?

Turns out that when I first installed Debian on this laptop there wasn’t an external monitor attached (big surprise) so when X was configured it generated a configuration file that couldn’t handle the external monitor. This means xrandr could query properties from the connected monitor just fine, but any attempt to change the configuration did nothing. The command gave a successful status code but nothing happened.

The suggested fix is to attach the second monitor and reconfigure X. On Debian we’d expect this to be done through dpkg:

dpkg-reconfigure xserver-xorg

Which does nothing. Turns out you can tell X to generate a config file directly which does the trick this time:

sudo X -configure

I know nothing about X configuration files so I can’t say why this works but it does because now when we send X commands through xrandr it responds.

Start small by just turning on the external display:

xrandr --output LVDS --auto --output VGA --auto

Your displays may be named a bit differently which you can check using the query option (-q). The external monitor turned on but the screens overlapped in a funny way … progress.

The effect I want is to have my desktop extend across both screens so they’re side by side. xrandr does this without any hassle:

xrandr --output LVDS --mode 1024x768 --output VGA --mode 1024x768 --left-of LVDS

The command above makes the layout pretty obvious: both screens are 1024 by 768 and the VGA screen is positioned to the left of the built in LVDS. Some people want their external screen to have a higher resolution but it’s easy to change the configuration so I’m going for symmetry to start off. That and the larger the screens get the more RAM the video card borrows from the rest of the system.

This is the command we want to run when the laptop is docked, now the command when it’s undocked:

xrandr --output VGA --off

The two commands can be combined into a script which we can pass a parameter, maybe a 1 when the system is being docked, a 0 when it’s undocked. Throw in a case statement and then we can test it:

#!/bin/sh
case $1 in
    0)
        echo "undock" | logger -t $0
        xrandr --output VGA --off
        ;;
    1)
        echo "dock" | logger -t $0
        xrandr --output LVDS --mode 1024x768 --output VGA --mode 1024x768 --left-of LVDS
        ;;
    *)
        echo "unexpected input" | logger -t $0
        exit 1
        ;;
esac
exit 0

Make this file executable and put it in /etc/thinkpad, call it dock.sh

Now to get the system to run the script for us when the laptop is docked and undocked. Debian Lenny uses udev so this is as simple as adding the following in a file in /etc/udev/rules.d/

KERNEL=="dock.0", ATTR{docked}=="1", RUN+="/etc/thinkpad/dock.sh 1"
KERNEL=="dock.0", ATTR{docked}=="0", RUN+="/etc/thinkpad/dock.sh 0"

I named this file 55-thinkpad-local.rules based on someone’s reported success on the linux thinkpad mailing list. The order in which udev rules are run, why the order is important and how to write them is still a mystery to me and will remain that way for now.

Now we put the two together. The log messages that are sent to the syslog should be verified to be sure the script is running right because when the laptop is docked/undocked … the screen layout won’t change! Great right? We can get the error message by redirecting the output of the xrandr commands to syslog too like so:

xrandr --output LVDS --mode 1024x768 --output VGA --mode 1024x768 --left-of LVDS 2>&1 | logger -t $0

The error message tells us that the script “Can’t open display”. Wait, root (the script is run as roor) doesn’t have permission to open the display? This turns out to be some X magic that’s explained on the ThinkWiki Fn-F7 page. The important part is way down at the end of the script where root enumerates the X sessions and then one by one exports the server identifiers and the Xauthority cookie. After this root can change the display all it wants. We’ve gotta include this in the original script and while we’re at it throw in some extra stuff to make it pretty. The final script is here. Works pretty good for me, but YMMV.

Minimal GStreamer App

So in my last post (way back when) I was talking about transcoding some video. I wanted to use GStreamer to do the job but HandBreak was so much easier (pretty cool app too). So after I got the transcoding working with HandBreak I started playing around with GStreamer to see what it would take to code up a small app to do the job.

The first thing is just to code up a little bit of C to get the video playing. The GStreamer part here is pretty simple. It’s just like the gst-launch command I laid out in my last post. I’ve added a few bells and whistles to get command line parameters and to do some input validation. I’m using as much glib as I can for practice.

The result is a pretty short (for C) application that is a bare-bones GStreamer media player. Naturally you’d want a few additional things from a media player but that’s not our end goal. We want to take the playbin (or something like it) and hook it up to another pipeline of our own design that will encode the video & audio for our specific purposes. This is what’s coming up … soon hopefully. Honestly with the class I’m currently taking at SU it may be a while.

For now read the code and enjoy. It’s available for download below. I even made a Makefile for ya.

Download: gst-play-min.tar.gz

Video transcoding for the N97

A big part of why I bought the N97 was the media player. It has some pretty impressive specs: supports the h264 and MPEG4 codecs, the mp4 and 3gp containers, AAC audio … so yeah that’s pretty sweet. First thing I did when I got the phone all charged up was go into the old media library, copy over a file I thought met there specs and load it up into the media player. Yeah it definitely didn’t work.

Upon closer look there are some constraints I wasn’t meeting and unfortunately some bugs I didn’t know about. Let’s start with what I was doing wrong.

First off the constraints on the video dimensions are 630 by 320 pixels. Anything over this won’t play, the phone won’t scale the image down. So we’ve gotta convert the video, no getting around that. First thing I did was look into using GStreamer. I wanted to mock up a pipeline and throw it at gst-launch to do the transcoding. First thing I did was break out the playbin to do the decoding:


gst-launch-0.10 playbin uri:file:///path/to/vid

This does some GStreamer magic detecting all of the media file’s attributes and builds a pipeline in the bin to do the decoding. If you execute the above the bin will build default sinks for both the audio and video streams which will basically play the file.

Now it’s a matter of linking the audio and video streams from the playbin to a pipeline, simple right? I couldn’t figure out a way to do this from gst-launch. In fact I found a mailing list post indicating that this can’t be done through gst-launch. So now it’s build my own application or find one that already does this. As much as I like rolling my own, this is no small feat.

Coincidentally I’m at LinuxCon this week and I ran into another guy with the same phone. He pointed me to HandBrake. That was the ticket. This app has a great CLI interface and supports the codecs we need. It also has some built in profiles for popular devices.

My goal was to produce transcode to h264 at 630×320, AAC (don’t really care about bitrate etc) in an mp4 container. The HandBrake iPhone profile comes pretty close with a few minor modifications:


HandBrakeCLI -i ./infile.avi -o outfile.mp4 -e x264 -q 0.589999973773956 -a 1 -E faac -B 128 -R 48 -6 dpl2 -f mp4 --maxWidth 640 --maxHeight 320 -m -x level=30:cabac=0:ref=2:mixed-refs:analyse=all:me=umh:no-fast-pskip=1

This is a pretty compute intensive task. On my laptop it maxed out both CPUs for about a half hour for a 24 minute TV episode. In the end I had an mp4 file that was about 150MB with the desired attributes. Playing it with Totem was successful. The properties menu showed all the attributes were as expected and it looked great. Success?

Not quite. Turns out my N97 still couldn’t play it. The media player just threw up an error, wouldn’t play audio or video. I spent a while playing around with the parameters for the x264 encoder (on a much smaller file) but wasn’t able to get the phone to play any h264 encoded video. On a few occasions it would play the audio but never the video.

So h264’s out and MPEG4’s in. After getting rid of the h264 stuff from the above command I ended up settling on something like this:


HandBrakeCLI --input ./infile.avi --output outfile.mp4 --encoder ffmpeg --rate 24 --audio 1 --aencoder faac --ab 128 --arate 48 --mixdown dpl2 --format mp4 --maxWidth 640 --maxHeight 320 --markers

That’s the trick. Originally I had included the quality flag form the h264 command line. This produced a super small video file, 3.5 times smaller than the original but the video produced looked kinda grainy. Since I’m not pressed for space on my phone yet (32GB of storage is a lot of space) I dropped the quality flag and let it run presumably at 100%. This produced a file about 25MB smaller than the original (likely attributable to the decreased image resolution) with excellent picture quality. Better yet my N97 played it perfectly.

Conclusion: looks like the h264 implementation on the N97 isn’t all it’s cracked up to be. This is likely a bug. So for now, stick to MPEG4 through HandBrake and enjoy some videos on your N97. The new firmware update (v20) has been in the Nokia press recently but I haven’t seen anything about fixing h264 support. Meh.