Graydon’s work on Mozilla

Error analysis

January 14th, 2009

I also have something interesting to mention: I’ve been working with Taras and Dave’s static analysis system, writing an error-code checking analysis.

The notion of my particular analysis is that you should be able to tell the compiler that some function like:

  bool tryToDoSomethingScary();

is the sort of function that uses its return value as a possible error code, where for example the return value of false should mean an error occurred, and someone ought to handle the error. If you do so — via a gcc attribute annotation like __attribute__((user("SETS_ERR"))), you can then run my analysis script on code that calls the error-code signaling function, and gcc will enforce a bunch of subtle-but-important rules:

1. If you call the the function you can’t ignore its return value. Someone has to check it.
2. If you call the function, and check the return code (as you must), all subsequent possible code paths after the call must satisfy a bunch of checks:
   1. On a path corresponding to a false return — indicating an error being signaled — you must either call a matching function annotated as “handling” the error in question or you must annotate the calling function to return the same type of error itself, and you must return false on the error path.
   2. On a path corresponding to a true return — indicating a lack of error — you must not call to clean up the error, since no error is pending.

This results in something similar to the model of checked exceptions in Java — the compiler won’t let you get away with ignoring, mis-handling or mis-propagating an error code — only using explicit error codes rather than try/throw/catch and exception specifiers.

I’m impressed that this flow sensitive feature can be implemented in 300 lines of javascript and bolted into a production C++ compiler with a week’s work (and a lot of hand-holding from the pros). Everyone who writes C++ should try.

Back from the grave

January 14th, 2009

Memetics

Oh fine. Seven facts of minimal significance, but probably less-well known:

1. I’ve broken both my hands and both my feet. More recently I’ve had a tooth knocked out and my face torn open. Been broken a lot.
2. I walked 1300km last year and biked 2500km. Including a single ride of 600km. Despite this, I look very low-key and not sporty at all. I just don’t know how to drive (no points about “cars I’ve owned”) so I generally transport myself “the long way”. I like it that way.
3. I haven’t eaten meat since I was 13 and I’ve done sitting meditation off and on since about 16. In some practical respects I’m Buddhist, but probably not enough to call it a religious affiliation.
4. I spent a solid portion of the 90s happily lost in rave culture and flirting with life on communes.
5. In my childhood I too acquired the skills of figure skating (what the hell?) and, in fact, curling. But not hockey. I don’t even like hockey. This means I cannot run for public office in Canada, but that’s ok.
6. I have owned and operated a 300bps acoustic-coupler modem and, later, a Fidonet BBS. Me and computers seem to have known each other a while. My first, like that of many here, was a cassette-tape-fed Z80 machine: though mine was a glorious Timex/Sinclair ZX-81. We’re back down to its form factor now with netbooks, only they have a million times as much memory and run a thousand times faster. And don’t come with a ROM BASIC interpreter.
7. I dislike television and really anything that moves around too fast. I like things that move slowly or, better yet, sit still. My first and possibly favorite job — incidentally where I learned the most about programming — was at a bookstore, where I got to quietly organize shelves and read things. I should probably be a gardener or a librarian, but I’m somewhat hooked on programming.

I don’t like propagating pyramid-scheme internet memes so I’ll stop with that. No further tagging. I think this blog subspace is fully saturated anyways.

Cycle collector landed

January 5th, 2007

The previous attempt at landing the XPCOM cycle collector failed due to some unacceptable performance regressions. Yesterday, after tidying up a few of the more obvious offenders, we appear to have accomplished a landing that only hurts Tp2 by 5-15%, depending on platform and noise. I know this sounds like a big number and I will endeavor to make it smaller, but it was a big change that causes a lot of new pointer operations and a big new cost center in the GC itself. So getting it that low is mildly satisfying. Thanks to jst, vlad, brendan and others for all the hand-holding.

I’ll repeat my previous qualifications of this work, though. Despite the claims in the literature, no real GC system is free — they all cost time for the scanning and space for the transient garbage — and this is possibly one of the least satisfying GC systems because you have to manually add every class you want to participate in it. In the short term, you can expect: some performance loss, some heap increase, and a good number of leak-analysis tools to complain that memory is being leaked due to assumptions they make about the lifecycle of objects.

It is also reasonably likely that the collector will trigger new crashes; gecko still carries a lot of assumptions about pointer lifetimes, and it’s easy to accidentally write a traversal method that violates one. As we bring more classes into the collection regime and make more pointers strong, such opportunities should decrease.

In coming weeks I’ll try to work through each reported problem like this that comes up. Please let me know if you have a specific result that’s worrying you, and make sure to CC me on any bug that has a cycle collector frame in its stack.

Cycle collector landing

November 21st, 2006

Tonight I’ve submitted a mostly-final version of my XPCOM cycle collector for general consumption on the development CVS trunk. It’s a big patch — 230k — and has been in review for several months. In addition to the patch itself, testing builds have been available for a couple weeks. Alas by sheer number-of-files-touched it is likely to still cause lots of regressions I haven’t noticed while testing. There’s also a fair bit of logic in there that I touched but only partly understood. I’ve done my best to integrate reviewer’s comments, but more review and feedback this week would be great. Mostly I’ve been trying to keep it from crashing; unfortunately it’s doing something very delicate that is very easy to get wrong, and crash.

This work removes all the existing “DOM GC” code and replaces it with a general-purpose, multi-language device that can find and disconnect a conservative subset of cyclic garbage in the browser, if the classes involved in the cycle have been modified to play along. This includes both pure XPCOM cycles, and cycles that cross between script language runtimes such as spidermonkey with independent heaps and pointers connected both ways to XPCOM objects. The modifications required to make an XPCOM class participate are reasonably easy to make, and I’ve included a dozen or so examples throughout the DOM and content classes, which should cover all the types previously involved in DOM GC.

There are several runtime knobs, controlled by environment variables.

• The variable XPCOM_CC_DO_NOTHING, if set, disables the collector entirely: the code is still present — so any regressions caused by reorganization of code may still exist — but it is inert, so it should be possible to isolate regressions due to new dynamic behavior using this variable.
• The variable XPCOM_CC_REPORT_STATS, if set, causes the collector to print a fair amount of runtime chatter to the standard error stream, describing its activities.
• The variable XPCOM_CC_FAULT_IS_FATAL, if set, causes runtime logic errors in the collector to trigger a process abort, rather than the standard behavior of disabling the cycle collector for the remainder of the program’s execution.
• The variable XPCOM_CC_DRAW_GRAPHS, if set, attempts to spawn the graphviz program “dotty” a subprocess
  and feed it a graphic display of any garbage cycle the collector finds, just prior to unlinking the nodes.
• The variable XPCOM_CC_HOOK_MALLOC, if set, inserts a small additional amount of sanity checking into the collector by ensuring that freed regions are never simultaneously present in the “purple” buffer that the collector ages pointers in.
• The variable XPCOM_CC_LOG_POINTERS, if set, writes an extremely verbose pointer log to the file “pointer_log” in your current directory. It should only be used in extreme cases such as malfunctioning debuggers.
• The variable XPCOM_CC_EVENT_DIVISOR, if set to an integer value, controls the resolution of the collector (which is driven from the top-level event loop). If you set this number smaller, more time is spent in the collector but collection increments will be smaller and thus cause shorter pauses. The default value is 64.
• The variable XPCOM_CC_SCAN_DELAY, if set to an integer value, controls the size of the “purple” buffer pointers are aged in. If you set this number smaller, more time is spent in the collector but there is a shorter delay between forming a garbage cycle and collecting it, so the heap overhead might go down a bit.

In general, unfortunately, you should expect this patch to cause the browser to use somewhat more memory and somewhat more CPU time than before the patch lands. It is not a panacea. The purpose of the patch is only to provide infrastructure for simpler ownership rules. Rather than try to reason about which pointer ought to own which other pointer inside gecko, and which pointers are safe to consider weak or raw, you now get a simple and universal rule. When in doubt you can now:

• Modify your class to participate in cycle collection
• Change all the pointer members of your class to owning nsCOMPtrs
• Ensure that the pointers you can reach transitively through those nsCOMPtrs are also cycle collection participants, or else obviously acyclic subgraphs.