When I joined the Mozilla Corporation’s gfx group in February of 2008, I was tasked with what seemed like a simple job: create a hashtable-based cache for imagelib, so it no longer had to use necko’s memory cache. (The work to implement this new cache was tracked in bug 430061.) While this seemed like unnecessary reimplementation, I was assured by Stuart and Vlad that necko’s memory cache was meant for an entirely different class of object, and that the large images stored in it were crowding out those objects (such as pages loaded over SSL).

Initially, this seemed like a simple job, but it turned out to be a multi-month effort that involved a lot of rewriting, debugging, collaboration, and patience. The last two attributes were especially embodied by Boris Zbarsky, who went out of his way to help me debug problems I didn’t understand, reviewed far too many iterations of patches, and was generally helpful in a way that I think exemplifies Mozilla’s community spirit. Thank you, Boris.

The most important fruit of all this labour is the reduction in memory use it made possible: a clever eviction policy lets us halve the size of the cache while maintaining the same real-world performance.

The remainder of this post will be a detailed explanation of the cache’s design, how it is implemented, and how I came to the decisions I made. I plan on rolling this into into an MDC article at some point, so if you have questions, please ask them.

Image loading is managed by two classes: imgRequest and imgRequestProxy. imgRequest is the class that takes care of the network load and decoding images, but since the same image can be loaded multiple times, when you call imgLoader::LoadImage() you get a reference to an imgRequestProxy. imgRequestProxy is a lightweight handle that implements nsIRequest and imgIRequest; it gets events from, and forwards events to, its imgRequest.

Since imgRequest does the actual work, it’s the class that needs to be kept in the cache. To make doing this a bit easier, I created a data class, imgCacheEntry, that holds a strong reference to imgRequest (which also holds a strong reference to imgCacheEntry, creating a cycle that is broken only once the imgRequest has no more listeners), and also contains useful information, such as last touched time and size. It is these imgCacheEntries that the cache manipulates.

The cache itself consists of three data structures:

1. A hash table from URI to imgCacheEntry. Every image in memory is accessible from this hash table unless it has been explicitly removed from the cache.

2. An instance of my favourite data structure of all time, the heap. (Dijkstra invented the heap when he needed a faster way to implement his shortest-path graph algorithm, because that is the sort of thing that Real Computer Scientists do.)

   Once there are no more listeners for a given image – say, once a user has navigated away from a page – we add the image to this heap, and evaluate whether anything needs to be removed from the cache due to space restrictions. If so, the image at the top of the heap is evicted, and (since there are no other references to the image) it is deallocated. (I plan a blog post on precisely how I came to the eviction scheme used in imagelib in the near future.)

   (A brief aside about performance: Even though heaps are relatively quick, updating one on every change to the cache turned into a 2x performance hit. The main problem is that updating the heap implemented in the STL (I believe a binary heap) is an O(n) operation. I investigated a couple of different replacement data structures for solving this – one, described in a paper called “Implicit dictionaries with O(1) modifications per update and fast search” turned out to not be applicable to the problem; another, Fibonacci heaps, looked promising, but its attributes weren’t precisely right. In the end, it turned out that lazy evaluation – marking the heap as dirty, and re-heapifying only when necessary – solved the performance problem.)

3. Our memory tests showed that we weren’t freeing up as much data after all tabs had been closed, so I added an nsExpirationTracker to the mix.

   nsExpirationTracker runs a timer to forcibly expire elements from the cache after some length of inactive time. (This is currently set to 10 seconds, which feels a little quick. I suspect we’d be better off if the image cache expiry time was the same as the bfcache expiry time, 30 minutes.) Like the heap, images are only added to the expiration tracker once they have no listeners, since evicting a cache entry that has a reference count above 1 makes website authors depending on previously loaded images being available cry.

These three structures combine with imgRequest and imgRequestProxy in probably-overly-complex ways to implement our image caching strategies. I had lots of problems along the way – from many, many leaks that took weeks cumulatively to debug, to trying to understand necko’s twisty passages, to debugging topcrashes without ever having seen the crash myself – but it’s made me a better developer, and given me a chance to understand a lot more about this code which I have come to own.

As part of the changes for bug 466586, I’ve made it so that images that are still in use – for example, an image displayed in a webpage that you have open – are always accessible from the cache. Only once the image isn’t being used anymore do we start thinking about removing it from the cache (and freeing up its memory). Unfortunately, my current patch (available on the bug, and checked in to mozilla-central) has some latent bugs that I have never been able to reproduce. I do have proof that these bugs exist: in particular, several crashes have shown up when expiring images from the cache since I checked in the patch.

Currently, this code is only checked in to mozilla-central, i.e., the 1.9.2 or mozilla-central branch. I don’t have any test cases for these bugs (though I’m trying to get URLs for the crashes), but if you see a crash in your up-to-date Minefield build (2009-02-18 or newer) that contains “img” in the stack trace, or you get an abort in your debug build in imgCacheEntry, please file a bug with what you were doing, whether it’s reproducible, what URL you were at, and any other details you can find. (MDC has some good information about how to report and get information on crashes in Firefox.)

The bug this patch fixes is a Firefox 3.1 beta 3 blocker, so the more users this code has (and consequently the more eyes we have on it), the better.

UPDATE: Bug 480352 has been filed on this unknown crash, along with some bare-bones instructions on how you can help.

Here are the rules for this particular meme

1. Link to your original tagger(s) and list these rules in your post.
2. Share seven facts about yourself in the post.
3. Tag seven people at the end of your post by leaving their names and the links to their blogs.
4. Let them know they’ve been tagged.

My seven things:

1. I’ve had five non-retail jobs in my life:
   • In Grade 11, I had a summer job doing Microsoft Access and VB consulting at a small, now-defunct shop called Idea People. Surprisingly, this was my first job, even before any retail work I eventually did.
   • Three of my jobs came through the University of Waterloo’s co-op program:
     • Texas Instruments in Toronto, formerly GO DSP, working on the UNIX port of their IDE and on various infrastructural tasks;
     • SOMA Networks, doing Linux kernel device driver work and other custom development for their Linux-based wireless broadband systems.
     • A Houdini developer at Side Effects Software, where I worked on 3D animation software for films like Disney’s The Wild (which was created almost entirely in Toronto by CORE).
   • After graduating, I continued working at Side Effects Software for a couple of years, on all manner of 3D-related things. (I consider this the “same job” as my co-op terms at Side Effects.)
   • Finally, I found myself applying to and being hired by the Mozilla Corporation, my experiences at which is more or less what this entire blog will be about.
2. Without my wife, who at the time was my girlfriend, I might not have gone to Waterloo at all. It was between U of T and Waterloo; U of T had offered me a substantial scholarship, on-campus residence, and had a professor call me to sell me on the school. Waterloo gave me no money and told me to find my own place to live. Lisa convinced me that Waterloo was still the right choice, which turned out to be true: without my co-op job experience, I would be in a very different place, career-wise.
3. I’ve had precisely one significant other throughout my life: Lisa, my wife. We are literally highschool sweethearts who survived the separation of university to get married after we graduated.
4. Before I wised up and moved to the city, I owned a car – a 2005 Saturn Ion. I wrecked it in a hilariously poorly-thought-out attempted U-turn to avoid a long line for left turns.
5. I have never left North America, but Lisa was born in Malta.
6. In the past, I was a fairly active Debian developer; I even organized DebConf 2 at York University here in Toronto. (These days, I am pretty disgusted with Debian; it seems it’s more about forcing other people to do what you want through voting than working out the best technical solution for users.) I have not used a Linux system at home since 2004.
7. I wrote mpg321 as a Free alternative to mpg123 (back when mpg123 was still non-Free). I had all manner of plans to extend that software, but they fell by the wayside, as things do. I now use iTunes and my iPhone to listen to music.

And now, as is part of this meme, I get to tag folk.

1. Vladimir Vukićević, my manager, because he doesn’t blog enough.
2. Ryan North, because he is awesome and draws Dinosaur Comics.
3. Shawn Wilsher, who doesn’t eat shawarma for unknown reasons.
4. Dave Townsend, who now looks like a little boy.
5. Jenny Boriss, because she makes me laugh.
6. Katie Bonnar, who I beat out for Prime Minister of Student Council because of damn dirty tricks.

   And finally,

7. Jeff Muizelaar, because he told me not to tag him.