Year: 2011

Categories
about:memory AdBlock Plus Firefox Memory consumption MemShrink

MemShrink progress, week 27

Adaptive Memory Behaviour

The biggest news this week was that Justin Lebar landed code to make Firefox’s memory consumption adaptive on Windows.  The code works by wrapping VirtualAlloc() and some similar OS-level allocation functions.  Each time memory is allocated, the amount of available virtual and physical memory is measured by the wrapper.  If either of those amounts are below a threshold, a memory pressure event is triggered, which causes numerous components within Firefox to take steps to reduce their memory consumption.

It might not sound like it, but this is a big deal, and it was a MemShrink:P1 bug.  Firefox runs on a huge variety of machines and devices, and trying to find one-size-fits-all heuristics for discarding regenerable data is a recipe for unhappiness somewhere.  Adaptive behaviour is much better.

The initial settings for this feature are conservative, and some tuning may be needed.

  • The threshold used for virtual memory is 128MB, i.e. if the amount of available virtual memory drops below 128MB a memory pressure event is triggered.  (This number can be changed in about:config via the “memory.low_virtual_memory_threshold_mb” setting.) Hopefully this will reduce the number of OOM crashes that occur on Windows.
  • The threshold used for physical memory is 0, which means that it is currently ignored.  (This number can be changed in about:config via the “memory.low_physical_mem_threshold_mb” setting.) The reason for ignoring it currently is that low physical memory may be caused by a program other than Firefox — we don’t want to start discarding data if Firefox is only using a small fraction of physical memory.  In the future we can probably do better here by triggering memory pressure events if the fraction of physical memory used by Firefox exceeds some threshold;  hopefully this will avoid paging when memory usage is high.
  • Memory pressure events are throttled so they aren’t sent more than once every 10 seconds.  (This number can be changed in about:config via the “memory.low_physical_memory_notification_interval_ms” setting.)  This prevents thrashing in cases where the memory pressure events don’t help reduce memory consumption.

Memory pressure events are also currently unevenly observed — plenty of data could be dropped that currently isn’t — so there is room for improvement there.

Finally, there’s another bug still open to add similar adaptive behaviour on Mac, Linux and Android.  Note also that memory pressure events are also triggered on Android when Firefox goes into the background.

Add-ons

There has been good news for add-ons:  the #1, #6 and #8 most popular add-ons on AMO have all recently seen some MemShrink-related improvements.

Adding memory reporters to add-ons is a really good thing to do, and I plan to write some documentation to encourage other add-on developers to follow suit.  However, if you are an add-on developer and are thinking of doing this, I need to stress one thing:  the reports must have KIND_OTHER so that they end up in the “Other Measurements” list in about:memory.  If they are put in the “explicit” tree, then some memory bytes may be counted twice (once by Firefox, once by the add-on) which will completely screw up about:memory’s results.

Memory Reporters

I landed several improvements to memory reporters.

Bug Counts

Here are the current bug counts.

  • P1: 26 (-2/+1)
  • P2: 139 (-6/+6)
  • P3: 64 (-2/+2)
  • Unprioritized: 0 (-1/+0)

Nice to see the total count drop, even if only by one.

Due to the Christmas break there won’t be a MemShrink meeting nor a MemShrink progress report next week.  I’ll be back in two weeks, enjoy the break!

Categories
about:memory Firefox Memory consumption MemShrink

MemShrink progress, week 26

It’s been six months since MemShrink started!  Here are this week’s events of note.

Memory Reporting

I integrated DMD support into Firefox.  This is a big deal because DMD is crucial for improving the coverage and correctness of memory reporters.  There are several consequences of this.

  • You no longer need to apply a patch to Firefox if you want to use DMD, you just have to configure with --enable-dmd.  (Full instructions are here.)
  • Any memory reporter written in the new style (i.e. using a nsMallocSizeOfFun function created with NS_MEMORY_REPORTER_MALLOC_SIZEOF_FUN) will automatically get coverage in DMD.
  • DMD finds bugs in memory reporters, but it can also find bugs elsewhere.  For example, this week it found that some BaseShapes were being shared incorrectly, which Brian Hackett fixed.

I updated the layout memory reporters to use the new style.  I also updated the documentation on writing memory reporters, to cover more complicated topics like how to measure classes that involve inheritance.  I also added a request:  if you write a memory reporter, please add me as a co-reviewer.

Johnny Stenback implemented per-window DOM memory reporters, which gives much more detail for DOM memory usage in about:memory.  Here’s an example:

per-window dom reporter output in about:memory

Memory Usage Improvements

Boris Zbarsky lazily initialized some rulehash tables, saving about 45KB per blank tab;  this is a nice win for those people who have on-demand tab loading combined with sessions with many tabs.  The same patch also made one of the rulehash tables smaller when it is initialized, which can cause multi-MB savings on workloads of only a few tabs.

Bobby Holley fixed a bad memory leak in xpconnect that could cause unbounded amounts of memory to be leaked, albeit in somewhat unusual circumstances.  The bug was only in Firefox 9 and Firefox 10;  it’s been fixed for 10, and the patches to fix for 9 are ready to land.

Fabrice Desré fixed a regression that caused a zombie compartment when visiting addons.mozilla.org.

Bug Counts

Here are the current bug counts.

  • P1: 27 (-1/+1)
  • P2: 139 (-5/+3)
  • P3: 62 (-0/+2)
  • Unprioritized: 1 (-0/+1)

That’s a net change of +1, and we only had to triage seven bugs in today’s meeting.  And if the trees hadn’t been closed for the past few days I could have fixed two more P2 bugs.

Just for fun, I also counted the number of RESOLVED FIXED (i.e. genuine, not duplicates or invalid) MemShrink bugs.

  • P1: 29
  • P2: 70
  • P3: 14
  • Unprioritized: 35

(If you’re wondering why the number of unprioritized fixed bugs is so high, it’s because MemShrink bugs are only prioritized in MemShrink meetings, which means that any bug that gets filed and fixed in less than 7 days doesn’t get prioritized.)

So there are more bugs still open than have been fixed, but the rate of bug growth is close to zero.

Categories
about:memory Firefox Memory consumption MemShrink

MemShrink progress, week 25

It was a good week for MemShrink.

ObjShrink

The biggest news this week is that Brian Hackett landed his objshrink work.  This has roughly halved the size of Firefox’s representation of JavaScript objects.  It’s also reduced the amount of memory Firefox uses for shapes, a related data structure.  If  you look at about:memory you can see that “gc-heap/objects” and “gc-heap/shapes” entries account for many MBs of memory, so this is a big win.  Read here (under the “Objects” and “Shape” headings) if you want more details.

Memory Reporting

I added some more JavaScript memory reporters and fixed some problems with the existing ones.  Specifically…

  • New: “runtime/threads/temporary” counts various pieces of short-lived data, mostly parse nodes.  It sometimes spikes up to multiple MBs.
  • New: “runtime/threads/normal” measures thread data on the heap that isn’t covered by “runtime/threads/temporary”.  It’s normally a few 100s of KBs.
  • New: “runtime/contexts” measures JSContexts and various related structures, which usually account for a few 10s or 100s of KB.
  • New/fixed: “runtime/threads/regexp-code” measures code generated by the regexp compiler.  This was previously measured under “mjit-code/regexp” but that reporter was broken recently when the location of this generated code was moved, so I fixed it.
  • Fixed: “runtime/atoms-table” and “runtime/runtime-object” are existing reporters that together often account for up to 4.5MB.  They were incorrectly marked as measuring non-heap memory instead of heap memory, meaning that the “explicit” and “heap-unclassified” totals were both incorrectly inflated by that amount.  While fixing this, I also confirmed that we weren’t making the same mistake with any of our other memory reporters.
  • Renamed: “runtime/threads/stack-committed” was previously called “stack-size”.  I renamed it to go with the other “runtime/threads” reports because that’s where it conceptually belongs.

I also added a memory reporter for XPConnect.  It typically accounts for 1MB or more.

I have a lot more work in the pipeline to improve the coverage and correctness of memory reporters.  More about this in the coming weeks as I land the relevant patches!

Other

Joel Maher got RSS memory measurements working for Android on Talos, Mozilla’s performance regression testing suite.  This was a MemShrink:P1 bug.  Graphs of the live data can be seen here.

Benoit Jacob made some improvements how Firefox manages and reports memory used by WebGL.

Bug Counts

Here’s the current bug count.

  • P1: 27 (-1/+1)
  • P2: 141 (-3/+5)
  • P3: 62 (-1/+2)
  • Unprioritized: 0 (-0/+0)

Only three more MemShrink bugs than this time last week!  Pretty good.

Categories
about:memory Memory consumption MemShrink

MemShrink progress report, week 24

Something that happened a while ago but I failed to notice was that Chris Leary added a regexp cache which, among other things, is flushed on every GC.  This fixed an existing problem where cold regexps were not being discarded, which was a MemShrink:P1 bug because 10s or even 100s of MBs of compiled regexp code could accumulate in some circumstances.  Nice work, Chris!

Andrew McCreight fixed two leaks (here and here) involving WeakMaps.  WeakMaps are an EcmaScript 5 feature and so are not used that much at the moment, but their popularity will increase over time.

I landed some memory reporter infrastructure changes.  These will make it much easier to integrate DMD with Firefox, which will help drive about:memory’s “heap-unclassified” number down.  They also do some sanity checking of memory reporters, and this checking has already found some bugs in existing reporters.

Terrence Cole made the JS engine to a “shrink” GC on memory pressure events, such as when about:memory’s “minimize memory usage” button is pressed.  A “shrink” GC is one that causes unused pages to be decommitted.

I wrote two pieces of documentation.

  • The first is a guide to zombie compartments, including instructions on how to test if an add-on causes them.  This guide is similar in spirit to an old blog post of mine, but explains things more carefully.  There is a QA test day planned for Friday, December 16, and hopefully some extensive add-on leak testing will happen on that day.
  • The second is a guide to implementing memory reporters.  Please read it if you ever have to implement one.

In publicity news, ZDNet published a story about how Google’s +1 buttons consume a lot of memory, particular in the newly redesigned Google Reader.  The author used Firefox’s about:memory page to determine this, which enabled him to point the finger at Google’s JS code instead of Firefox.  (Dietrich Ayala wrote about this topic previously, and wrote the Wallflower add-on in response;  the Antisocial subscription for AdBlock Plus apparently has the same effect.)

Here’s the current bug count.

  • P1: 27 (-5/+1)
  • P2: 139 (-4/+11)
  • P3: 61 (-0/+1)
  • Unprioritized: 0 (-4/+0)

The P1s went down because in today’s MemShrink meeting we reprioritized several that are now less important than they were.

Categories
JägerMonkey MemShrink Tracemonkey

MemShrink progress report, week 23

The only significant MemShrink-related change that landed this week was that David Anderson removed TraceMonkey, the tracing JIT compiler.  In fact, TraceMonkey was disabled a while ago, so the effects on code size and memory consumption of its removal have been felt since then.  But it feels more real now that the source code is gone (all 67,000 lines of it!), so I figure it’s worth mentioning.  (BTW, many thanks to Ryan VanderMeulen who has been going through Bugzilla, closing many old TraceMonkey-related bugs that are no longer relevant.)

People have asked why TraceMonkey isn’t needed any more.  In my opinion, tracing compilation can be a good strategy for certain kinds of code, such as very tight, non-branchy loops.  But it tends to do badly on other kinds of code.  Before JaegerMonkey, JS code in Firefox ran in one of two modes: interpreted (super slow), or trace-compiled (usually fast).  This kind of bimodal performance is bad, because you lose more when slow than you gain when fast.  Also, because tracing was the only way to make code fast, huge amounts of effort were put into tracing code that shouldn’t really be traced, which made TraceMonkey really complicated.

Once JaegerMonkey was added, the performance was still bimodal, but in a better way:  method-compiled (fairly fast) or trace-compiled (usually fast).  But the heuristics to switch between the two modes were quite hairy.  Then type inference was added to JaegerMonkey, which made it faster on average than JaegerMonkey+TraceMonkey.  Combine that with the fact that TraceMonkey was actively getting in the way of various additional JaegerMonkey and type inference improvements, and it was clear it was time for TraceMonkey to go.

It might sound like there’s been a lot of wasted effort with all these different JITs.  There’s some truth to that.  But JavaScript is a difficult language to compile well, and people have only been writing JITs for it for a few years, which isn’t long when it comes to compilers.  Each new JIT has taught the JS team about ideas that do and don’t work, and those lessons have been incorporated into the next, better JIT.  That’s why IonMonkey is now being developed — because JaegerMonkey with type inference still has a number of shortcomings that can’t be remedied incrementally.

In fact, it’s possible that IonMonkey might end up one day with a trace compiler for really hot, tight loops.  If it does, this trace compiler would be much simpler than TraceMonkey because it would only target code that trace-compiles easily;  trace compilation would be the icing on the cake, not the whole cake.

Enough about JITs.  Time for this week’s MemShrink bug counts.

  • P1: 31 (-0/+2)
  • P2: 132 (-3/+8)
  • P3: 60 (-0/+2)
  • Unprioritized: 4 (-0/+4)

Not a great deal of movement there.  The quietness is at least partly explained by the fact that Thanksgiving is happening in the US this week.  Next week will probably be quieter than usual for the same reason.

Categories
about:memory Firefox Memory consumption MemShrink

MemShrink progress report, week 22

This was a quieter week.

Andrew McCreight finished his static analysis to detect cycle collector leaks.  See the details in the bug (and talk to Andrew) if you are interested in using this analysis.

I shrunk the size of js::HashTable by 4 bytes on 32-bit platforms and 8 bytes on 64-bit platforms.  This saves a few 10s or 100s of KB on typical workloads.

Marco Bonardo decreased the default maximum page size of SQLite connections, which can reduce SQLite memory usage somewhat.

Olli Pettay avoided some wasted space in one of the cycle collector’s data structures.  The cycle collector uses lots of memory but for a short time when it runs;  this change will reduce the size of this memory spike.

Gian-Carlo Pascutto added a memory reporter for one of the data structures used by the url-classifier.  This shows up in about:memory under “explicit/storage/prefixset” and is often over 1MB.

Justin Lebar improved the measurement of nsTArray’s memory usage, which will reduce the size of “heap-unclassified” in about:memory by a small amount.

Justin also wrote a good blog post about the challenges of addressing leaks in add-ons.

We only had seven new MemShrink bugs to triage in today’s meeting;  I’m pretty sure that is the fewest we’ve ever had.  Here are the current bug counts.

  • P1: 29 (+1/-1)
  • P2: 127 (-2/+3)
  • P3: 58 (-3/+2)
  • Unprioritized: 0 (-0/+0)

These counts are notable because the total number (214) is the same as last week!  Maybe the number will start dropping soon.

One thing worth pointing out about the +/- numbers is that if a bug is opened and closed between my weekly reports, it does not get counted in the +/- numbers.  In a way this is good, because it means that duplicate bugs and invalid bugs don’t affect the numbers.  But it also fails to capture bugs that were reported and fixed quickly.  (I usually describe such bugs in my posts, however.)

Categories
about:memory Firefox Garbage Collection Memory consumption MemShrink SQLite

MemShrink progress, week 21

MemShrink:P1 Bugs fixed

Terrence Cole made a change that allows unused arenas in the JS heap to be decommitted, which means they more or less don’t cost anything.  This helps reduce the cost of JS heap fragmentation, which is a good short-term step while we are waiting for a compacting garbage collector to be implemented.  Terrence followed it up by making the JS garbage collector do more aggressive collections when many decommitted arenas are present.

Justin Lebar enabled jemalloc on MacOS 10.7.  This means that jemalloc is finally used on all supported versions of our major OSes: Windows, Mac, Linux and Android.  Having a common heap allocator across these platforms is great for consistency of testing and behaviour, and makes future improvements involving jemalloc easier.

Gabor Krizsanits created a new API in the add-on SDK that allows multiple sandboxes to be put into the same JS compartment.

Other Bugs Fixed

I registered jemalloc with SQLite’s pluggable allocator interface.  This had two benefits.  First, it means that SQLite no longer needs to store the size of each allocation next to the allocation itself, avoiding some clownshoes allocations that wasted space.  This reduces SQLite’s total memory usage by a few percent.  Second, it makes the SQLite numbers in about:memory 100% accurate;  previously SQLite was under-reporting its memory usage, sometimes significantly.

Relatedly, Marco Bonardo made three changes (here, here and here) that reduce the amount of memory used by the Places database.

Peter Van der Beken fixed a cycle collector leak.

I tweaked the JavaScript type inference memory reporters to provide better coverage.

Jiten increased the amount of stuff that is released on memory pressure events, which are triggered when Firefox on Android moves to the background.

Finally, I created a meta-bug for tracking add-ons that are known to have memory leaks.

Bug Counts

I accidentally deleted my record of the live bugs from last week, so I don’t have the +/- numbers for each priority this week.

  • P1: 29 (last week: 35)
  • P2: 126 (last week: 116)
  • P3: 59 (last week: 55)
  • Unprioritized: 0 (last week: 5)

The P1 result was great this week — six fewer than last week.  Three of those were fixed, and three of those I downgraded to P2 because they’d been partially  addressed.

For a longer view of things, here is a graph showing the MemShrink bug count since the project started in early June.

memshrink bug count

There was an early spike as many existing bugs were tagged with “MemShrink”, and a smaller spike in the middle when Marco Castellucio tagged a big pile of older bugs.  Other than that, the count has marched steadily upward at the rate of about six per week.  Many bugs are being fixed and definite improvements are being made, but this upward trend has been concerning me.

Future Directions

So in today’s MemShrink meeting we spent some time discussing future directions of MemShrink.  Should we continue as is?  Should we change our focus, e.g. by concentrating more on mobile, or setting some specific targets?

The discussion was long and wide-ranging and not easy to summarize.  One topic was “what is the purpose of MemShrink?”  The point being that memory usage is really a secondary measure.  By and large, people don’t really care how many MB of memory Firefox is using;  they care how responsive it is, and it’s just assumed that reducing memory usage will help with that.  With that in mind, I’ll attempt to paraphrase and extrapolate some goals (apologies if I’ve misrepresented people’s opinions).

  • On 64-bit desktop, the primary goal is that Firefox’s performance should not degrade after using it heavily (e.g. many tabs) for a long time.  This means it shouldn’t page excessively, and that operations like garbage collection and cycle collection shouldn’t get slower and slower.
  • On mobile, the primary goal probably is to reduce actual memory usage.  This is because usage on mobile tends to be lighter (e.g. not many tabs) so the longer term issues are less important.  However, Firefox will typically be killed by the OS if it takes up too much memory.
  • On 32-bit desktop, both goals are relevant.

As for how these goals would change our process, it’s not entirely clear.  For desktop, it would be great to have a benchmark that simulates a lot of browsing (opening and closing many sites and interacting with them in non-trivial ways).  At the end we could measure various things, such a memory usage, garbage and cycle collection time, and we could set targets to reduce those.  For mobile, the current MemShrink process probably doesn’t need to change that much, though more profiling on mobile devices would be good.

Personally, I’ve been spreading myself thinly over a lot of MemShrink bugs.  In particular, I try to push them along and not let them stall by doing things like trying to reproduce them, asking questions, etc.  I’ve been feeling lately like it would be a better use of my time to do less of that and instead dig deeply into a particular area.  I thought about working on making JS script compilation lazy, but now I’ve decided instead to focus primarily on improving the measurements in about:memory, in particular, reducing the size of “heap-unclassified” by improving existing memory reporters and adding new ones. I’ve decided this because it’s an area where I have expertise, clear ideas on how to make progress, and tools to help me.  Plus it’s important;  we can’t make improvements without measurements, and about:memory is the best memory measurement tool we have.  Hopefully other people agree that this is important to work on 🙂

Categories
AdBlock Plus Firefox Google Chrome

Converting a Chrome user to Firefox: a follow-up

I wrote yesterday about how I converted a relative of mine from a Chrome user to a Firefox user.  The post was picked up by the tech press including Tom’s Hardware and Conceivably Tech.  (Tom’s said I “described why getting users back from Chrome may nearly be impossible”, which I think is a laughable exaggeration of what I wrote.)

The good news is that the two major stumbling blocks I faced during the conversion are well on the way to being addressed.

  • The situation with third-party add-ons will be greatly improved in Firefox 8, which is scheduled for release in just a few days (2011-11-08).  The first time it runs, Firefox 8 will present the user with a window that lists all the installed add-ons, distinguishing between add-ons the user installed explicitly and third-party add-ons that they didn’t install explicitly.  The user will be asked to confirm which add-ons they want, and the third-party add-ons will be disabled by default.  The bug for this feature is here, and there is a follow-up bug here.
  • An “import history from Chrome” feature is being worked on right now.  Here is the feature page, a tracking bug, and  two dependent bugs.  The aim was for it to make Firefox 10, which is scheduled for release on 2012-01-31, but it looks like it might not make that release.  Hopefully it will make Firefox 11, which is scheduled for release on 2012-03-13.

Another point raised by commenters was that Chrome has a version of AdBlock Plus.  However, the Chrome version still has major shortcomings compared to the Firefox version.  This page states “We are currently working on providing the same experience for Google Chrome as what you are used to from Firefox. Please keep in mind that we are not there yet and much work still needs to be done. There are also known Google Chrome bugs and limitations that need to be resolved.”  This page lists the major shortcomings.  Maybe those shortcomings will be overcome in the future, but until they are, it’s not a reasonable comparison.

In conclusion, the point of my post yesterday was not to say “OMG Firefox is crap the sky is falling in”, but rather “here’s what happened when I tried this”.  I knew that the two obstacles I listed were being worked on, though I didn’t know the details.  (Many thanks to the commenters who filled me in.)  The fact that they are being worked on and/or have been fixed is rather encouraging.  It’s also worth noting that Firefox’s new rapid release calendar like these make it into the hands of ordinary users only 2 to 3 months after they are implemented.  With the old release calendar, these two improvements wouldn’t have made it into a release until mid-2012 or later.

Categories
AdBlock Plus Firefox Google Chrome Personal

Converting a Chrome user to Firefox

[Update: before commenting, you should probably read this follow-up post that clarifies certain things about this post.]

On my recent vacation I was staying with a family member, let’s call her Karen (not her real name).  She was a Google Chrome user, and I managed to convert her to a satisfied Firefox user.  Here’s what I learnt along the way.

tl;dr:

Bad things about the experience:

  • The third-party add-ons situation on Windows is awful.
  • We need a “import history from Chrome” feature.

Good things about the experience:

  • Mozilla’s non-profit nature is compelling, if you know about it.
  • AdBlock Plus is great.

The Initial Situation

Karen is a moderately sophisticated computer user. She knows what a browser is, but didn’t know how many there were, who made them, or any notable differences between them.

Her machine that is probably 2 or 3 years old, and runs Windows Vista.  She had used IE in the past (not sure which version) but didn’t like it, switched to Chrome at some point — she didn’t remember how or why — and found it to be much better.  She was running Chrome 14.0.835.202 (no, that’s not an IP address!) which was the latest stable version.

She also had Firefox 3.6.17 installed, but judging from the profile she hadn’t used it much — there was very little history.  She had the following Firefox add-ons installed:

  • Java Console 6.0.20 and 6.0.27.
  • The .NET Framework Assistant.
  • Some media player thing.
  • Some Norton “safe search” toolbar, and Symantec IPS, whatever that is.

(What is the Java Console?  What is the .NET Framework Assistant?  As far as I can tell they are (a) very common and (b) useless.)

I told her that I worked on Firefox and suggested that she try it and she was open to the idea.  I talked about the differences between Firefox and Chrome and some of my work on Firefox.  The thing that caught her attention most was that Mozilla is a non-profit organization.  She hadn’t known this and it appealed to her greatly — she said that browser speed and the non-profit nature were the two most important things to her.  She was also somewhat interested when I said that Firefox had an ad-blocking add-on.  At the end of the conversation, she agreed to let me install Firefox and make it the default browser.

Installing Firefox

I removed the existing Firefox profile manually — I wasn’t sure if this was necessary, but I definitely wanted a fresh profile — and then uninstalled Firefox through the Control Panel.  I then installed Firefox 7.0.1.  (BTW, I stayed with Karen for two weeks at this point, and I had deliberately waited until Firefox 7 was out before doing this because I knew it had much lower memory usage than Firefox 6.)

An unexpected thing was that the Firefox installer asked me to close all the other running programs;  it explained that this would mean that I wouldn’t have to reboot.  I’m used to running Firefox on Mac and Linux so I’m not used to this, but I’m familiar enough with Windows that I wasn’t totally surprised.  Still, it was annoying;  Karen had MS Word and some other programs open and I had to go ask her if I needed to save anything before closing them.  I realize this is Windows’ fault, not Firefox’s, but it was an obstacle.

Starting Firefox

When I started Firefox it asked me if I wanted to make it the default browser and I said yes.  (I explained to Karen how to switch the default browser back to Chrome if she was unhappy with Firefox.)

It also asked me if I wanted to import history/bookmarks/etc from IE.  But there was no equivalent for Chrome!  Karen had a ton of bookmarks in Chrome, but fortunately she said she only used a handful of them so I was able to copy them manually into Firefox’s bookmarks toolbar (which I had to make visible).  I’ve heard that someone is working on an “import from Chrome”  feature to Firefox but I don’t know what the status is.  We need it badly.

Once Firefox started, another unexpected thing was the state of the add-ons in the new profile.  The Symantec add-ons (including the ugly Norton toolbar) were present and enabled.  I had to disable them in about:addons;  I wanted to completely uninstall them but I couldn’t, the “uninstall” button just wasn’t present.  The Java Consoles and the media player were disabled because they were incompatible with 7.0.1, but I was also not able to  uninstall them.  This horrified me.  Is it a Windows-only behaviour?  Whatever the explanation, the default situation in a fresh install was that Firefox had several unnecessary, ugly additions, and it took some effort to remove them.  I’m really hoping that the add-on confirmation screen that has been added to Firefox 8 will help improve this situation, because this was the single worst part of the process.

I then tweaked the location of the home and reload buttons so they were in exactly the same position as in Chrome.  I probably didn’t need to do that, but  with those changes made Firefox’s UI looked very similar to Chrome’s, and I wanted things to be as comfortable for her as possible.

The best part of the process was when I installed AdBlock Plus.  With Karen watching, I visited nytimes.com in Chrome, and I had to skip past a video advertisement before even getting to the front page, and then the front page had heaps of ads.  Then I visited in Firefox — no video, no ads.  It was great!

Follow-up

A week or two later I sent Karen a follow-up email to check that everything was ok.  She said “All is well! The ad blocking is a great feature.”

So, Firefox has a new and happy user, but there were some obstacles along the way, and the outcome probably wouldn’t have been so good if Karen hadn’t had a Firefox expert to help her.

Categories
Firefox JägerMonkey Memory consumption MemShrink Tracemonkey Uncategorized

MemShrink progress, week 20

Surprise of the week

[Update: This analysis about livemarks may be wrong.  Talos results from early September show that the MaxHeaps number increased, and the reduction when the “Latest Headlines” livemark was removed has undone that increase.  So livemarks may not be responsible at all, it could just be a coincidence.  More investigation is needed!]

Jeff Muizelaar removed the “Latest Headlines” live bookmark from new profiles.  This was in the Bookmarks Toolbar, and so hasn’t been visible since Firefox 4.0, and most people don’t use it.  And it uses a non-zero amount of memory and CPU.  Just how non-zero was unclear until Marco Bonardo noticed some big performance improvements.  First, in the Talos “MaxHeap” results on WinNT5.2:

Talos MaxHeap graph

And secondly in the Talos “Allocs” results on WinNT5.2 and Mac10.5.2:

Talos Allocs graph

In the WinNT5.2 cases, it looks like we had a bi-modal distribution previously, and this patch changed things so that the higher of the two cases never occurred.  In the Mac10.5.2 case we just had a simple reduction in the number of allocations.  On Linux the results were less conclusive, but there may have been a similar if smaller effect.

This surprised me greatly.  I’ve done a lot of memory profiling of Firefox and never seen anything that pointed to the feed reader as using a lot of memory.  This may be because the feed reader’s usage is falling into a larger, innocuous bucket, such as JS or generic strings.  Or maybe I just missed the signs altogether.

Some conclusions and questions:

  • If you have live bookmarks in your bookmarks toolbar, remove them! [Update: I meant to say “unused live bookmarks”.]
  • We need to work out what is going on with the feed reader, and optimize its memory usage.
  • Can we disable unused live bookmarks for existing users?

Apparently nobody really owns the feed reader, because previous contributors to it have all moved on.  So I’m planning to investigate, but I don’t know the first thing about it.  Any help would be welcome!

 Other stuff

There was a huge memory leak in the Video DownloadHelper add-on v4.9.5, and possibly earlier versions.  This has been fixed in v4.9.6a3 and the fix will make it into the final version v4.9.6 when it is released.  That’s one more add-on leak down, I wonder how many more there are to go.

TraceMonkey, the trace JIT, is no longer built by default.  This means it’s no longer used, and this saves both code and data space.  The old combination of TraceMonkey and JaegerMonkey is slower than the new combination of JaegerMonkey with type inference, and TraceMonkey is also preventing various clean-ups and simplifications, so it’ll be removed entirely soon.

I refined the JS memory reporters in about:memory to give more information about objects, shapes and property tables.

I avoided creating small property tables, removed KidHashes when possible, and reduced the size of KidHashes with few entries.

I wrote about various upcoming memory optimizations in the JS engine.

Justin Lebar enabled jemalloc on MacOS 10.5 builds.  This was expected to be a space improvement, but it also reduced the “Tp5 MozAfterPaint” page loading benchmark by 8%.

Robert O’Callahan avoided excessive memory usage in certain DOM animations on Windows.

Drew Willcoxon avoided excessive memory usage in context menu items created by the add-on SDK.

Bug Counts

  • P1: 35 (-1, +1)
  • P2: 116 (-2, +5)
  • P3: 55 (-2, +3)
  • Unprioritized: 5 (-4, +5)

At this week’s MemShrink meeting we only had 9 bugs to triage, which is the lowest we’ve had in a long time.  It feels like the MemShrink bug list is growing slower than in the past.