About a month back I installed the latest mono release on on one of my servers. A friend of a friend needed a place to do some ASP.Net development. He’d previously had an account with a Windows ASP.Net web host but wasn’t happy with the support he was getting and was trying to cut costs (as everyone always is).
I had the space and the know how to get him up and running so I pointed him to the mono-project ASP.Net page and told him to be sure all of the .Net features he was using were supported by Mono. On my end I installed all of the necessary mono stuff from source which, on Debian 5.0 (Lenny) was very simple. I then set him up with an account just like everyone else that uses this system.
Typically I use Unix DAC Groups to separate users from eachother on the shared host. I create a group for each user (call it www-username) and add both the user and the apache user to this group. This way users can make files they wish to be served up by the web server readable by this group. Nothing fancy, standard DAC stuff.
Funny thing though: this set-up works for static web pages, php scripts etc but didn’t work for the mod-mono-server. The apache error logs showed a stack trace leading back to a permission denied exception on the directory serving up the test ASP web page I was using. This directory had the following permissions as showed by ls -l
drwxr-s--- 3 username www-username 4096 2009-11-01 16:50 web_dir
The DAC permissions are set up to keep everyone out except the owner and the web server. The server is given read access through the shared group and I’ve also set the sticky bit on the directory to keep files created by the user readable by this group. This last bit isn’t relevant to the denial but it’s useful and apparent in the output above.
Time to debug! We know the apache user (www-data) is in this group and can access files owned by the shared group (www-username) but why not mono? First off the Apache2 server only starts the mono server, it doesn’t process the ASP.Net pages itself. Through the modules interface Apache2 forks off the mod-mono-server2 process which it then feeds requests for the ASP.Net pages. Looking at the output of
ps we can check to see that the mod-mono-server2 process has the right effective user and group (which were both correct) but I don’t know of a tool to inspect the supplementary groups for a running process. So we turn to the mod_mono source code for details. Note: Wikipedia has a good explanation of what Unix supplementary groups are if you need more background.
I’m a pretty good C coder but I’ve never looked at the mod_mono code in my life and I know nothing about writing modules for Apache2. But to debug this all I needed was a basic understanding of Unix DAC users and groups. The first thing I looked for was the place in the code where the mono server is started: either a
fork or an
exec system call is where we should start. These calls are both made in a function called ‘fork_mod_mono_server’ in the mod_mono.c file … yeah that’s probably where we should start looking.
This function also makes calls to
setgid which is the forked process dropping the root privileges of its parent and taking on the identity of a lesser privileged user and group (specified in the apache configuration). On Debian this is the www-data user and group. As I say above we’ve already verified that the uid and gid are being set properly, it’s the supplementary groups that seem to be missing. We can check to see what group membership the forked process has by calling
getgroups. Add a few debug statements (ouput the group ids from getgroups to the apache log file), rebuild the module, install it and restart the server and we can see that our suspicions have been confirmed: the only group id returned is 0, the root group?
Yeah, that’s not right. After flipping through a few man pages it looks like the
initgroups has the functionality we’re looking for. All we need to do now is pass it the group name for the apache group and it does all the hard work for us. A quick test shows that after rebuilding the module and restarting the server it now has the expected behavior: the mod-mono-server2 process is able to access files that are readable by this shared group!
Now that the problem’s been fixed we want to contribute the it back to the Mono project. I’ve never submitted a patch to a project as high profile as Mono before so I was very conscious about making this patch clean and easy. This wasn’t hard since it was only one system call 🙂 Still it’s important to use formatting and logging statements that are consistent with the rest of the code. After looking at the code around my small fix I added a debug statement to output some information about the group membership initialization and some error handling to output an error message should the system call fail.
After running this fix on two of my servers for a few days I sent the patch to the mono-devel mailing list. Kinda sucks that the mail archive thinks the attachment is a binary. ::shrug:: Either way I got an off-list reply the same day from Marek Harbersack saying that the patch had bee accepted and applied to the mod_mono svn tree! The patch is revision #145618 and they even gave me credit in the svn log!