Justin Dolske’s blog

Mega status update

Posted on December 4, 2009 by Justin Dolske

I’m terrible at remembering to send out regular status updates, so here’s a belated list of things I’ve been up to recently in Firefox-land…

* CrashKill. We’ve been making a big push to reduce Firefox crashes. I’ve largely been working with 3rd party crashes (eg bug 519340). Some of these 3rd parties are very responsive and quickly work to resolve the crashes their product is causing, others are… less so. It was surprising to me to find that other software is directly responsible for a huge number of Firefox crashes… Antivirus software, Flash, and malware seem to be the worst.

* Firefox on Tegra. Working on getting Firefox 3.6 running well on Tegra netbooks, and diving into localization issues so we can fully support Windows CE like other platforms.

* Firefox application updater. I mentioned in a previous update that my fix for bug 517102 made the updater 3x faster. I’m working on what should be another big performance and reliability boost in bug 529464.

* Password manager fixes and features. These have been in-flight for a while, but unfortunately didn’t make Firefox 3.6. But I’m pushing to wrap them up in coming weeks and get them into the next release. Notable changes are bug 499417 (refactoring the “master password” code, a first step to bigger changes), bug 499233 (finally killing multiple master password prompts on session restore), bug 465636 (add timestamps to login metadata), and bug 223636 (support automatic login for HTTP Auth).

* Rewriting the Weave’s crypto backend (bug 513798). Last year I rewrote Weave’s crypto code to use NSS (Firefox’s own crypto library) instead of OpenSSL. That’s been working swell, but it’s a binary component, and is a real pain to deal with when making Weave available on other platforms (like all the new mobile devices we’re working with). Dan Witte recently landed JS-Ctypes, which allows JavaScript code to call into native libraries. So now I’m porting the Weave C++ code to JS to take advantage of this.

* Using CSS Transitions in HTML5 video controls (bug 521890). David Baron has added support for the CSS Transitions spec to Firefox trunk, so I took a look at having our HTML5 video controls use this feature. When I implemented the controls for Firefox 3.5, I had to use a pile of JavaScript timers and callbacks to implement the fade and slide effects… CSS Transitions GREATLY simply doing such things. Currently blocked on one issue.

* Performance work. Noticed an issue with Necko not caching protocol handlers well (unknown what the perf impact is yet), looked how often we check perferences at runtime (a LOT, no bug yet), and am starting to look at the impact of all the timers we have firing and how they might be impacting power usage. Also investigating a problem with restoring maximized windows. We inefficiently restore the size, which slows down opening a window on Tegra devices (which generally use maximized windows).

* Got a speedy new Windows 7 desktop, and have been getting it configured and running.

My next big project is finishing up Doorhanger Notifications, which are a critical UI feature for the revamped Firefox 3.7/4.0 themes. Matt Noorenberghe started the implementation this summer as an intern, so I’ll be wrapping that up and making it fit updated designs.

(*phew*)

Posted in PlanetFirefox, PlanetMozilla | 2 Comments

Sneaky software installs

Posted on October 13, 2009 by Justin Dolske

The Mozilla Plugin Check page was released today, so I loaded it up to see the latest changes. “Looks good,” I thought, and skimmed the list of plugins it displayed for me. Quicktime, Silverlight, Flash, and… Woah, wait.

Silverlight?

On my OS X box? How the fuck did that get there? I sure don’t remember installing it. Grrrr!

A little web searching later, and I found my answer. It’s silently installed with Flip4Mac (a set of Quicktime components to allow playing Microsoft proprietary media formats on OS X), which I had installed a week or two ago to try something, and then promptly forgot about. The installer doesn’t have a word to say about it, unless you click a little “Customize” button on the 5th screen on the install:

That really annoys me. Silent, sneaky software installs are evil, evil, evil.

At least Flip4Mac includes an uninstaller, so I ran that. It’s actually a package, so you’re confusingly prompted to “Select the disk where you want to install the uninstaller software.”, but it was otherwise painless. Now to just restart my browser, check about:plugins, and…

GAHHHH! It’s still there. Their uninstaller uninstalls the Flip4Mac bits, but not the Silverlight plug. Solution:

rm -rf /Library/Internet\ Plug-Ins/Silverlight.plugin/

It’s finally dead, Jim.

Posted in Firefox, PlanetFirefox, PlanetMozilla | 8 Comments

Making progress

Posted on October 8, 2009 by Justin Dolske

If you’re running nightly builds of trunk or Firefox 3.6, you may notice a smoother progress bar while installing the nightly updates…

The old progress bar behavior had always seemed a bit odd to me — it would do nothing for a bit, move to about 15% point, and then suddenly finish. I didn’t get around to looking at why it did that until I used the updater on a Windows CE netbook. The device is a lot slower than a normal system, so the unusual progress bar movement could appear frozen for over a minute! That’s really poor UI feedback, so I set about fixing it.

You can read the gory details in bug 517102, but the end result is that the progress bar now tracks progress more accurately, and the updater runs 3 times faster too!

Posted in Firefox, PlanetFirefox, PlanetMozilla | 1 Comment

Is the tree REALLY green?

Posted on October 8, 2009 by Justin Dolske

While sheriffing yesterday, I was a bit confused when dbaron asked if I was looking into all the orange. The tree (http://tinderbox.mozilla.org/Firefox/) looked mostly green, except for a handful of earlier Talos oranges. So, yadda yadda, it turns out that most of the test boxes report to a separate tinderbox tree now — http://tinderbox.mozilla.org/Firefox-Unittest/. There are similar splits for the 3.6 and 3.5 trees.

This was probably announced somewhere, but 3 other developers also didn’t know this was the case. So, I figured a blog post was in order.

Tinderboxpushlog already includes both sets of data, and I’ve just updated isTheTreeGreen.com to use both sets as well.

Posted in Firefox, PlanetFirefox, PlanetMozilla | Comments Off

Experimenting with build times

Posted on October 4, 2009 by Justin Dolske

I got a new Windows desktop last week, and was curious if enabling disk compression would have any effect on build times… The theory being that source code compresses well (binaries too, to a lesser degree), so less disk IO should result. The price is more CPU usage, but this is a Core i7 box, so that’s basically free.

Result: Theory Busted.

Without disk compression, a full build took 23 minutes. With disk compression, 24 minutes. [Drive formatted between tests.] I’d guess some combination of not being IO bound plus disk seeks dominate, so compression doesn’t really help and just adds overhead. I also tried disabling “write-cache buffer flushing” to no effect, although I’m curious if splitting my build across two drives will help at all [src on one, objdir on the other; currently it's in a RAID-0 config].

One thing that would be nice is if parallel make worked correctly with Mozilla on Windows. If I build with anything higher than MOZ_MAKE_FLAGS=”-j1″, the build hangs at random (?) places. Seeing as this is a quad-core box (8-core, if you count HT), that makes me all sad and frowny.

Update: Mitch on IRC pointed out bsmedberg’s pymake work. With -j1 it’s about 30 seconds slower, but with -j10 it did a full build in 11 minutes (without hanging like a gmake build did!). Yay! I should also note that until now I’ve been doing Windows builds in a VM, which can take well over an hour to finish.

Posted in Firefox, PlanetMozilla | 4 Comments

Ahoy, mateys!

Posted on September 17, 2009 by Justin Dolske

Yarrr! Talk Like A Pirate Day be comin’ up. Ye sprogs that been usin’ isthetreegreen.com for tinderbox status should join the rest of us gentlemen ‘o fortune at me new site, bethetreegreen.com. It even be usin’ a pirate CSS @font-face! Shiver me timbers!

Posted in PlanetMozilla | Comments Off

Device orientation

Posted on August 24, 2009 by Justin Dolske

Doug just posted about the device orientation support he recently landed. Sitting here with my MacBook Pro, I just had to play with it — resulting in this little demo. Orientation + CSS Transforms FTW!

Posted in Firefox, PlanetMozilla | Comments Off

Simple instructions for building Firefox from source

Posted on June 4, 2009 by Justin Dolske

In the past, figuring out how to build Firefox from source tended to be really confusing. The documentation explained too much about how the build system worked, how multiple projects were built, presented too many choices, and generally had lots of details not relevant to most people.

A couple weeks ago, someone on IRC pointed out that jorendorff had created a simplified set of instructions: https://developer.mozilla.org/En/Simple_build I think this is great!

There are still multiple steps (especially if you’re on Windows), but it’s a lot more streamlined than other documentation and is geared towards doing cut’n'paste. In a nutshell, it’s a lot less intimidating.

Posted in Firefox, PlanetMozilla | 4 Comments

Happy Form History Expiration Day!

Posted on May 13, 2009 by Justin Dolske

Over the last year, I’ve slowly been poking at Firefox’s form history implementation (aka “satchel”). Triaging the backlog of bugs, fixing some issues, and generally laying the groundwork for other exciting changes to come in Firefox.next. One notable change will start taking effect today or tomorrow (for Firefox 3.5 users) — the expiration of old, unused form history. A little background…

Back in November I implemented bug 463154, which added some additional data to the form history database. It now stores timestamps for when data was first added, when it was last used, and how many times it has been used. The first consumer of this data was the Clear Recent History dialog — we needed the timestamp to allow clearing the last hour (or whatever) of form history.

The first time you run a build with this change, all existing form history entries are assigned timestamps with the current time. So, if you were using Firefox trunk nightlies back in November 2008, you’ll have a bunch of entries with a first-used date set to around November 14th (the first nightly with this fix). Beta users got this in Firefox 3.1 Beta 2 (released December 8th), and it was also backported to Firefox 3.0.9 (released April 21st).

The second piece of background is that I recently fixed bug 243136, which adds support for expiring old form history entries. You might be surprised to learn that until this bugfix, we basically only removed form history entries when explicitly requested from the Clear Private Data dialog, or when a specific value was manually removed from the autocomplete dropdown by the user (hilight one and press delete or shift-delete!). Now, once a day or so, form history entries that haven’t been used for a long time will be deleted. The maximum age is determined by the browser.history_expire_days preference, which defaults to 180 days.

Now, if you’re clever, you might notice that November 14th 2008 + 180 days = today. If you’re not clever, well, reread the last sentence. So, Happy Form History Expiration Day! Note that expiration can only happen with Firefox 3.5 Beta 4 (or newer). And the exact date expiration starts depends on when you first ran a Firefox build that included timestamp support… Other common Form History Expiration Days may begin on June 6th (Dec. 8th + 180 days), and October 18th (Apr. 21st + 180 days). The latter will be extra special, since that’s the case that will apply to the bulk of our 1/4 billion Firefox users (most of whom will hopefully have upgraded to Firefox 3.5 by then).

Why do form history expiration at all? Three main reasons: privacy, relevance, and performance. Privacy and relevance are fairly obvious — retaining data forever isn’t good practice, and data unused for 6 months is highly unlikely to be relevant in the future. Performance is interesting… In my personal profile, I currently have 27,600 form history entries, with 12,700 of those from Google searches alone! Trudging through all that data takes time; in some cases an autocomplete dropdown can take hundreds of milliseconds to appear on my laptop. If I expire the data older than 180 days, those numbers drop dramatically — 4,600 entries / 1,800 from Google. This will immediately help improve form autocomplete performance, and help allow us to do more complex queries in the future (to improve the relevancy of the autocomplete results that we offer — stay tuned!).

Posted in PlanetMozilla | 6 Comments