Ted’s Mozilla Blog

Some time ago, Lukas Blakk implemented support for a source server on our Windows builds as a class project in Dave Humphrey’s class at Seneca College. Of course, soon after that we switched our main VCS from CVS to Mercurial, which broke all of her hard work. Thankfully, we got another one of Dave’s students, Jesse Valianes, to fix things to make it work with Mercurial. We landed his patch, but as it turns out we never enabled a setting on our build machines to make it actually work. However, when we finally tried to do so, I found out that another patch we had landed in the interim had broken things. I finally landed a fix for that, and we flipped it back on, and so today’s trunk build is source-enabled again.

If you have no idea what any of this means, it means you can download a Windows nightly build, attach a debugger, have it download the debug symbols automatically from our symbol server, and the debugger will download the matching source for you automatically.

I hope to get this backported to our 1.9.2 and 1.9.1 branches ASAP, so that our 3.5.x and 3.6 release builds will be similarly debuggable.

I recently landed some changes (on trunk and 1.9.2) to the way Firefox packaging works. There are two immediate consequences of this you should be aware of:

1. Mac builds now use a packaging manifest just like Windows and Linux. If you add a file that you intend to ship on Mac, it needs to wind up in a packaging manifest. (bug 463605)
2. All the  packaging manifest files have been combined into one single file: browser/installer/package-manifest.in. This should save everyone some time and annoyance. (bug 511642)

These changes had no effect on applications other than Firefox.

Without a lot of fanfare, a patch landed recently that enables the use of SSL with the test HTTP server we use in our Mochitest test harness.

About five months ago, I read an article about how Fedora wanted to standardize on NSS as the cryptography solution for their distro in order to be able to leverage a common certificate database, among other things. The article went into detail on how they wrote an OpenSSL wrapper around NSS so they could easily port applications that only supported OpenSSL to use NSS instead. As a concrete example, they showed a ported version of stunnel using NSS. This gave me pause, as one of the things we were lacking in our Mochitest harness was SSL support and stunnel would do exactly what we needed in this case. Considering we already build and ship NSS with every copy of Firefox, and it was clearly possible to implement the functionality we needed using NSS, I set out to figure out how to implement a bare-bones version of stunnel from scratch. After a bit of poking through the online NSPR and NSS documentation, I had a proof of concept application which I called “ssltunnel.” After some insightful review comments from NSS developers I committed it to CVS.

Unfortunately, that wasn’t the end. We still needed to hook this program up to the test harness, and I just didn’t have the motivation to do so. I filed the bug, and hoped someone else would do the work. (as I often do!) Thankfully, that someone appeared in the person of Honza Bambas, whom I can only describe as a “programming rockstar.” He not only integrated ssltunnel into Mochitest, but he rewrote large sections of it to make it work robustly and made it work as an HTTP proxy while he was at it. After some reviews, and a couple of landings and backouts due to unrelated test failures, and some time spent languishing in bugzilla, we finally made his patch stick.

Of course, now that we have this capability, we need tests to use it! Honza has written some great documentation on what is currently available via Mochitest, and how to add custom servers and certificates other things you might want. If you get motivated to write some tests and hit a rough spot, feel free as always to track me down on IRC and ask me about it.

Just something I threw together this morning: MochiTest Maker. It’s a pure HTML+JavaScript environment for writing MochiTests. It’s not as full-featured as the real MochiTest, as you can’t set HTTP headers or include external files, but it should serve for a lot of simple web content tests.

Ideally at some point I’d like to add a CGI backend to this so you could specify a directory, and have it generate a patch against current CVS to include your test in that directory. That would lower the bar even further for getting new tests into the tree. Another cool addition would be to integrate this with my regression search buildbot (currently offline), so that you could write a mochitest and then with one click submit it to find out when something regressed. That shouldn’t be hard to do, but my buildbot needs to find a more permanent home first.

I think there’s still a lot more we can (and must) do to lower the bar for writing tests. We need all the tests we can get!

Some time ago, we set up a symbol server for our Windows builds. This was sort of an afterthought, it just happened to be really easy to do in our new crash reporting architecture. It turns out that this is incredibly useful for people. This shouldn’t be surprising, given how difficult it is to build your own Firefox. Some time after we set this up, I found out that Microsoft’s debuggers also supported something called a source server (Note: this page did not contain this much information when this project started). This sounded interesting, but it wasn’t something I had time to work on, so I added some information to Seneca’s wiki, hoping an interested student would pick it up as a class project.

To say that I got more than I hoped for would be an understatement. Lukas Blakk took the project and ran with it, producing a working prototype and fleshing it out to the point where it now works perfectly on current nightly builds. She’s done an incredible job working with a practically undocumented feature of Microsoft’s debugging tools and having the perseverance to stick it out. As a result, you can now debug nightly Windows builds with full source available. We’ve got a handy MDC document available to tell you how. You’ll need a nightly from today (April 15th) or newer, and this will be available in the Firefox 3.0 release builds. Happy debugging!

So, after much wrangling, we have turned on profile-guided optimization on our Windows nightly build machine. The immediate impact is that we got faster, by about 10% on some of our benchmarks. We also exposed at least one tricky layout bug that relied on undefined order of evaluation, but dbaron fixed it. Big thanks to Rob Sayre and everyone else that made this possible!

Next up is probably going to be turning this on on our Linux nightly build machine. I think we’ve resolved the issues there, but we’re going to wait until after beta 4 for that. Apparently we shipped Firefox 1.0 nightlies with PGO, so it should be ok, although that was back with gcc 3.3 or so.

We’d like to do this on Mac, but that still needs some work. I’m hopeful that we’ll get there before Firefox 3 ships.

I always get a kick out of seeing Firefox being used on TV shows.  My wife was watching an episode of Nip/Tuck that she had taped (yes, taped, we don’t own a TiVo) and one of the characters was looking up some information on the web.  I did a double take and made her rewind and sure enough, they were using Firefox.  Even better, it was on a Mac!  In your face, Safari!  Of course this is Nip/Tuck so the characters have found out they’re related and are looking at a pro-incest website, but that’s one of the least edgy bits given the show content.

In Firefox 3, the bookmarks and history systems are getting an overhaul, collectively known as “Places.”  I was skeptical of this work when it first started, and I’m sure I wasn’t alone in that view.  I rarely used bookmarks, preferring to just search Google to recall a page I had previously visited.  Just how relevant was any overhaul of bookmarks going to be to my modern browsing?

Well, I was wrong.  Places is here, and it’s awesome.  It has improved my browsing workflow so much that I can’t even use Firefox 2 anymore.  In Firefox 3, the urlbar now autocompletes what you’ve typed from both URLs and page titles from your history and bookmarks.  Maybe that doesn’t sound impressive, but in practice it’s fantastic.  I tend to visit a lot of pages on the same site (mostly bugzilla), and with the old urlbar autocomplete, it was hard to find things from bugzilla.  With the new autocomplete, all you need to do is type in part of a unique word from the page title or URL, and it will be found.  In addition, if you bookmark a page (just by clicking the handy star there) it will get weighted into your autocomplete results.

I hear there’s still more work to be done, including improving the formatting of the results, but it’s already vastly improved my day-to-day browsing.  Hats off to the Places team!