Firefox 7 is currently in the Aurora channel. Its memory usage improvements have been getting a lot of attention this week, with many people reporting the 30% improvement claims from the official blog post. I was worried about the post claiming a specific percentage improvement, because there are various ways to measure memory usage, and it varies so greatly depending on the workload, but I haven’t seen anybody dispute it so far. In fact, the only non-Mozilla measurement I saw was one where a reviewer opened 117 bookmarks at once (one bookmark per tab) and saw a 40% reduction in private bytes on Windows! This was a pleasant surprise as we were expecting improvements mostly when users (a) closed lots of tabs, or (b) left the browser idle for a long time.
The most significant MemShrink-related landings this week were things that were subsequently backed out. Paul Biggar landed jemalloc support for Mac, then had to revert it due to some crashes and possible memory usage regressions. This support has ended up being an enormous hassle, and Paul’s been tirelessly battling various Mac OS X quirks and annoyances over a period of several months. An earlier appraisal that “It should be pretty trivial… I don’t think any of this would be a ton of work. Maybe a week of trying a few different things and running it through our unit tests?” turned out to be spectacularly wrong. Hopefully Paul will be able to fix the problems and re-land soon.
Brian Hackett merged his JavaScript type inference from the jaegermonkey repository to the tracemonkey repository. Type inference speeds up computationally-intensive JavaScript code significantly, but unfortunately it increased memory usage a lot, due to (a) the memory required for the analysis itself, and (b) some extra memory used by the executing JavaScript code. As a result, Brian had to back it out from the tracemonkey repository. He’s now making good progress on reducing the overhead. One reason that this happened is that the jaegermonkey repository gets very little use, so the problem wasn’t noticed until it landed on the tracemonkey repository which gets more use. So if you want to help Brian out, please try browser builds from the jaegermonkey repository.
On the tools front:
- Per-compartment memory reporters have found several cases of “zombie compartments”, which is when a tab is closed but one or more of its compartments stay around. Bug 668871 is tracking these leaks; if you see any like this please report them there.
- Benoit Jacob landed some memory reporters for WebGL, which will show up in about:memory, and I fixed a problem that was causing JavaScript typed arrays to erroneously fall into the “heap-unclassified” bucket in about:memory.
- Speaking of which, that “heap-unclassified” number is still higher than we’d like, often in the 35–45% range. If you see a particular site that causes “heap-unclassified” to go unusually high, please report it here.
Finally, here’s the MemShrink bug count, with the changes since last week:
- P1: 24 (+6)
- P2: 49 (+5)
- P3: 29 (+4)
- Unprioritized: 2 (+0)
The increases look bad, but I think that’s not because progress isn’t being made. Rather, it’s a reflection that MemShrink efforts are still ramping up, and people are coming up with lots of new ideas and filing bugs for them. The upwards trend will probably continue for several more weeks.
21 replies on “MemShrink progress, week 4”
Keep it up.
I have actually started using nightly day to day because it just feels better (I assume because of less paging)
I have a stupid question: there’s a “minimize memory usage” button in about:memory, why isn’t this done by default ? is there any caveat to click on it ?
Tarek: No caveats, other than it takes a bit of CPU. It mostly just runs the GC+CC multiple times, which is something that would happen eventually anyway. You don’t want to run GC+CC too often, as it can cause pauses.
It also triggers some “low memory events”, which will eventually cause the browser to do things like flush caches; this will reduce memory usage at the cost of requiring certain operations to be re-done, eg. decompressing images.
It’s mostly useful for testing purposes, though feel free to use it yourself if it helps you 🙂
Love what you guys are doing.
This is actively helping keep Firefox competitive in the ecosystem.
Fantastic work. Lately, Firefox has been more responsive for me than I can ever recall it being.
And it’s very pleasing to see such an emphasis on analysis. Excellent analysis provides the essential foundation for excellent synthesis, and Mozilla seems to be stepping up its analytical efforts in several areas.
Things are looking up all round — just a shame there’s no yearly LTS. 😉
I would not worry about the bug count increasing. The more attention that is paid to memory/performance management, the better. Arguably this has been neglected in favour of mobile and the impressive list of new stuff supported in Fx4.
Keep up the great work.
I only wish I could keep updating Aurora 7 but alas key Add-Ons are breaking.
Can I just confirm, you’re interested in about:memory details from any compartments that stay around well after the tab that created them has been closed?
They killed my bugzilla account. Is there any other way I can contribute such information?
pd: Yes, I’m interested in “zombie compartments”. Particularly if you can reproduce it reliably just after start-up — eg. open about:memory, open the site in question in a 2nd tab, close the 2nd tab, click “minimize memory usage” in about:memory (a couple of times just for good luck), wait a few minutes. If it’s still there after doing that, I definitely want to hear about it. It’s also important to know what add-ons you have installed, because add-ons can be at fault; if you can reproduce the zombie compartment in safe mode that’s important to know.
(Hmm, I should write those instructions up more clearly as a blog post.)
I don’t suppose you have another email address that you could start another Bugzilla account with? (Combined with avoiding whatever behaviour got you banned in the first place 🙂 If not, just email me directly.
pd if you disable addon compatibility almost all addons work. I don’t have a single addon which doesn’t work once compatibility is disabled and I have like 30.
Deathshot, not Firebug – even the beta release (1.8.5).
Yes others work ok but there’s no guarantees, it’s just that they’ve not changed anything substantial that breaks a major add-on lately.
I note this weblog has links for the comments lists but not for the individual comments. I am therefore unable to bookmark your comment “on 14 Jul 2011 at 10:38 am ” but must instead bookmark the whole lot “http://blog.mozilla.org/nnethercote/2011/07/13/memshrink-progress-week-4/” Unless I look at the page source to get http://blog.mozilla.org/nnethercote/2011/07/13/memshrink-progress-week-4/comment-page-1/#comment-1094
That is not problem as long as there are not many comments, but it would be handy to have the facility to bookmark individual comments themselves directly from a link.
Is this a restriction with this particular site.
John99: I think it’s a restriction of the site. I’m using Mozilla’s WordPress installation, I’m not an expert on it.
There is another problem of Firefox’s memory usage, that is, I think, important for the average users.
When you browse with few tabs opened (one, two or three), that is how most average people use to browse, Firefox uses a lot of memory compared to other browsers.
For example for my PC with only a tab opened (this page):
(“Private working set” from Windows’ Task Manager):
Firefox 7: 108 MB
IE9: 23,3 MB
Chromium 14: 29,7 MB
(“Private memory” from Chromium about:memory):
Firefox 7: 127,7 MB
IE9: 34,6 MB
Chromium 14: 60,4 MB
From the about:memory in Firefox (it says 85 MB of occupied memory) I could see that a lot of memory is allocated for JavaScript (62.38 MB (59.61%)) and for Heap Unclassified (29.63 MB (28.32%)).
Under JavaScript 36.48 MB (34.85%) for system principal compartment and 21.19 MB (20.25%) for GC Heap Chunk Unused.
Please correct me if I’m wrong.
I think that Firefox reserves memory in case it will be eventually needed in future instead of freeing it (that is Heap Unclassified and Heap Chunk Unused).
So it seems that it uses more memory, but in reality it’s using more or less the same than the others.
The problem is that many people see only the appearence, so if they see in the Task Manager that Firefox is using 100 MB, they think it’s actually really using 100 MB of physical memory. Couldn’t we do like the other browsers, that probably hide this virtual memory usage?
Another area where there is much to improve is addons.
I’ve noticed that with addons disabled, Firefox uses 45 MB of memory. With AdBlock Plus, that is a really spread addon, the memory consumption rise up to 75 MB.
This is too much. If I enable another addon, HTTPS-Everywhere, the memory consumption becomes 90 MB. Practically doubled!
Maybe there is some improvements that could be done to make the addons lighter, improving the handling of addons in Firefox or maybe improving directly the most used addons.
Thanks a lot Nicholas for your efforts. I really appreciate your efforts. I also like to help you. I did some testing on my behalf on this bug 291643. Should I upload (on mediafire or any which you desire) log txt file, which I compiled?
I described my methodology also in Log(kind of mediocre). I took several about:memory reading for testing.
I really like to help. Don`t think of it as a spam. Should I make bugzilla account?
Ahmad: yes, by all means, open a Bugzilla account and upload your files. Thanks!
@Nicholas
Thanks for reply Nicholas, I posted my comment. You can check if you need any help regarding testing. I will try my best. Also if procedure and method was wrong, please describe me correct way. I will surely follow.
Nicholas,
This is fantastic progress on the memory usage of Firefox. There is ongoing work to run each process in its own tab a la Chrome (and the next Safari). Transitioning to that architecture should help with memory management significantly and there is a possibility that the work being put into memshrink will no longer apply. So, is this a stop gap measure to reduce memory usage until Firefox is re-architected or will some of the work done here also apply in the process-per-tab world?
Thanks,
Manoj
Manoj: the memory usage reductions will still definitely be a good thing once process separation is implemented.
Hello theres some irc where we can find you?
Leonard: try the #memshrink or #developers channels on Mozilla’s IRC network. See https://wiki.mozilla.org/IRC for more details.