Ted’s Mozilla Blog

I often find myself landing patches on our 1.9.2 and 1.9.1 branches, both of which are in Mercurial repositories. This generally involves getting a changeset from the mozilla-central repository into one of these repositories, and also amending the changeset message to include the name of the person who gave me approval to land. There was some discussion recently on how it’s kind of a pain to do that. I’ve cooked up an easy solution for my own needs, perhaps it will serve yours as well.

I use “hg transplant” to get changesets from one repository to another. This assumes that you have local clones of both the source repository (mozilla-central in this case) and the destination repository (mozilla-1.9.2 or 1.9.1, usually). Assuming you had both clones side-by-side in a directory, you could run the transplant command in the destination repository’s working directory like so (where xxx is the changeset identifier of the changeset you want transplanted, you can also specify more than one changeset):

hg transplant -s ../mozilla-central xxx

The transplant command conveniently includes a “–filter” option that will let you alter the commit message or patch while transplanting. This requires you to have some sort of script for transplant to run. Here’s what I’m using (on Linux):

#!/bin/sh

if test -n "$APPEND"; then
 echo " $APPEND" >> "$1";
else
 if test -n "$EDITOR"; then
 $EDITOR "$1";
 else
 editor "$1";
 fi
fi

Save this as “transplant.sh” somewhere (and ensure that it’s executable), then in your ~/.hgrc, add a section:

[transplant]
filter = /path/to/transplant.sh

Now, when you run “hg transplant”, by default it will open an editor to edit the commit message for each changeset, allowing you to add approval information. But, even better, transplant.sh will append the contents of the “APPEND” variable if set, so you can run transplant like so to quickly append approval information:

APPEND="a=someone" hg transplant -s ../mozilla-central xxx

I find that this saves me a bunch of time, so hopefully it’s useful to someone else!

Some time ago, Lukas Blakk implemented support for a source server on our Windows builds as a class project in Dave Humphrey’s class at Seneca College. Of course, soon after that we switched our main VCS from CVS to Mercurial, which broke all of her hard work. Thankfully, we got another one of Dave’s students, Jesse Valianes, to fix things to make it work with Mercurial. We landed his patch, but as it turns out we never enabled a setting on our build machines to make it actually work. However, when we finally tried to do so, I found out that another patch we had landed in the interim had broken things. I finally landed a fix for that, and we flipped it back on, and so today’s trunk build is source-enabled again.

If you have no idea what any of this means, it means you can download a Windows nightly build, attach a debugger, have it download the debug symbols automatically from our symbol server, and the debugger will download the matching source for you automatically.

I hope to get this backported to our 1.9.2 and 1.9.1 branches ASAP, so that our 3.5.x and 3.6 release builds will be similarly debuggable.

I recently landed some changes (on trunk and 1.9.2) to the way Firefox packaging works. There are two immediate consequences of this you should be aware of:

1. Mac builds now use a packaging manifest just like Windows and Linux. If you add a file that you intend to ship on Mac, it needs to wind up in a packaging manifest. (bug 463605)
2. All the  packaging manifest files have been combined into one single file: browser/installer/package-manifest.in. This should save everyone some time and annoyance. (bug 511642)

These changes had no effect on applications other than Firefox.

MozillaBuild 1.4 is now available for download. This release focused primarily on compatibility with systems running the x64 edition of Windows. It also contains a few other niceties such as a newer version of Mercurial and other tools, and support for all known Microsoft SDKs. You can view the full list of changes from version 1.3 here.

For a long time when our unit tests or Talos performance tests encountered a crash, the result was nothing but frustration. If you were lucky, you could tell that it crashed, but you had no idea where. Poor Blake spent weeks tracing down a crash from his speculative-parsing patch that only seemed to occur on Talos. Up until recently I figured the only way to make this happen was going to involve a fair amount of work that only I was going to be able to do. A few weeks ago it was determined that this was becoming a significant impact on development, as patches would get checked in, cause a crash and be backed out, leaving the developer with nothing to go on.

Benjamin Smedberg has been hard at work making it possible to get stacks in this situation, using the same Breakpad utilities we use on our Socorro server, but locally on the machine running the tests. Practically all of the pieces were in place this afternoon when #developers cornered Alice and closed the tree while she landed the final patch to make Talos produce stack traces. Boris then committed a test crash, and as a result we were able to see crash stacks in Mochitest (OS X, Linux) as well as Talos (OS X, Linux).

Thanks to Benjamin for doing most of the heavy lifting here, and for
Alice for taking the Talos part across the finish line. The Talos work
was mostly in bug 480577, and the unit test work was bug 481732. Note
that currently this only works in Mochitest (all 4 varieties), it will
work in Reftest/Crashtest after bug 479225 is fixed (which should be soon).

(Cross posted in dev.tree-management, but posting here for a wider audience.)

The New Yorker has a great article about reforming the US health care system entitled “Getting There from Here.” It’s not terribly long and if you’re at all interested in the topic I’d recommend reading it. It discusses how other nations with universal health care arrived at their present systems, which is a topic that seems completely absent from debate on the subject in the USA. Anyway, I’m not really posting here to talk about health care, but one part of this article rang true with me in other ways:

There is no dry-docking health care for a few months, or even for an afternoon, while we rebuild it. Grand plans admit no possibility of mistakes or failures, or the chance to learn from them. If we get things wrong, people will die. This doesn’t mean that ambitious reform is beyond us. But we have to start with what we have.

As owner of the Mozilla build system, I hear a lot of complaints. This is understandable, our build system is showing its age, and we are certainly straining against the limits of Autoconf and GNU make on a regular basis. Along with complaints, I hear a lot of suggestions of the form “why don’t you just use X“, where X is any of a number of alternative tools such as CMake or SCons. The basic answer parallels the quote above. Because we don’t have time to stop and rewrite everything. Our build system contains tens of thousands of lines of makefiles, as well as a configure.in that’s over 8,000 lines. Converting this much build junk by hand is doomed to failure. Converting it through automated tools might be possible, but we would need a smart plan, and it would likely involve testing the tools in parallel with the existing build system for a while in order to make for a smooth transition. In any case, it’s clear that if we are to find a way forward, it will require building on what we have, and not burning it to the ground and starting over from scratch.

[via]

Our Release Engineering group provides a VMWare VM image of the Linux Reference Platform, which is the VM upon which all of the official Linux builds happen. This is very handy, as you can trade some download time (it’s about 1.2 GB) for the time it would take you to install Linux and setup all the build dependencies. I’m currently running Ubuntu 8.04 64-bit on one of my home machines, and I’ve been using VirtualBox for running VMs on the machine because it was super easy to install in Ubuntu (via apt-get). I found out today that VirtualBox can use VMWare disk images, so you can run the Linux Reference Platform VM pretty much out-of-the-box in VirtualBox.

The steps I took were:

1. Download the reference platform VM from the link above and unzip it somewhere.
2. Run VirtualBox, and go to the “Virtual Disk Manager”. Click “Add”, and browse to the directory where you unzipped the VM. Add “CentOS-5.0-ref-tools-vm.vmdk” and “CentOS-5.0-ref-tools-vm_1.vmdk” to the “Hard Disks” list in the manager, then click “Ok”.
3. Click “New” to create a new VM. Name it whatever you want, and select “Linux 2.6″ as the OS. Set the base memory size to something usable on your hardware (but not so small that it can’t compile Mozilla). For the “Boot Hard Disk (Primary Master)”, click “Existing…” and select “CentOS-5.0-ref-tools-vm.vmdk”. Click “Ok”, then “Finish”.
4. Click on your new VM in the list on the left, and click “Settings”. Click on the “Hard Disks” entry in the list on the left of the Settings dialog. Check “Primary Slave”, click the “Select” button to the right of the drop-down, and choose “CentOS-5.0-ref-tools-vm_1.vmdk”. Click “Ok”.
5. You should now be able to click “Start” and see your new VM boot. It will complain about a missing disk for the /builds mount, this is normal and shouldn’t be a problem.

You should read the wiki page linked in the first paragraph, as by default the VM is not configured to boot into X windows, but does provide a VNC Server.

I was reminded of bug 414049 yesterday, a bug I filed about getting screenshots from our unit test machines after every run so we could see if there was obviously something wrong with the machine (like error dialogs covering the screen). Linux and OS X tend to have built-in tools to grab screenshots (as mentioned in the bug), but Windows does not. I searched around for a free tool to do the job, but all I could find was shareware. It’s possible there’s a free tool out there that I just couldn’t find, but I figured I would just write one. After a bit of poking around on MSDN, I wrote screenshot.cpp. It’s only about 70 lines of C++, hard to believe people pay money for stuff like that. I’ve placed it under a BSD license, since it’s useful code and I couldn’t find a simple self-contained example like this anywhere.

Ok, you all know the rules, so I’m going to skip them. I’ve been tagged three times already (Dave, Benjamin, and Tomcat), so I guess it’s time to succumb.

7 things:

1. I have a slightly better than beginner-level knowledge of American Sign Language. My wife’s sister is deaf, and my wife is fluent, so I thought it would be rude to not be able to talk to her sister. It’s amazing how much appreciation just learning a little bit of someone’s native language will get you.
2. When I was a little over 2 years old, my family moved from New Jersey to a house that my parents built in Northeastern Pennsylvania. It was still under construction when we moved in, and the entry way was some planks crossing the 10 foot drop to what was at the time gravel in the basement. My older sister dared me to ride my big wheel over it, and of course I fell in. My mom says she made my dad go look because she was sure I was dead. As far as I’m aware I didn’t suffer any real injuries.
3. I played the trumpet in marching band for 9 years (4 years in high school, 5 years in college). Yeah, I’m a band geek.
4. My wife and I spent 2 months in Göteborg, Sweden after we got married in 2006. My former employer’s headquarters is there, and I managed to convince our US president that it was a good idea. Sweden in the summertime is an awesome place to be.
5. I’m only 5′3″ tall. But if you’ve met me in person you probably already figured that one out.
6. Mozilla is my third employer since I graduated college in 2002. (Hopefully I’ll stay here longer than my previous two!)
7. I’ve become addicted to mapping in OpenStreetMap. I mapped my entire town (Although not the surrounding area, that’s data that was imported from the US Cenus’ TIGER data.) and got another GPS for Christmas.

I’m not tagging anyone else. I think this meme has reached the saturation point.

Josh recently landed a test plugin, with the intent of finally getting some test coverage of our plugin-handling code via mochitests. This is awesome, as plugins are an area of code where we’ve caused lots of regressions in the past, and until then had zero automated test coverage. After it landed, I took a peek at the code and noticed that it would be pretty easy to extend it to make it usable in our layout tests (reftest) as well. I just landed some patches to add this functionality, so we can now test that our layout of plugins doesn’t regress. If you’d like to write some reftests yourself using this, you can check out the basic tests I added along with the patch. (Note: it’s mac-only at the moment, but there’s gtk2 code ready to land any minute now, and a win32 implementation should be forthcoming.)