Snappy, Jan 5
January 5th, 2012 by tglek
I expected to a slow week, but there was a surprising amount of progress. I take this as further evidence that having managers go on vacation does wonders to engineer productivity 
Interactivity with lots of tabs
We spent a lot of time pondering how to approach browser sluggishness in light of having tons of tabs open. On one hand people should understand, that one can’t expect the browser perform the same whether 1 tab is active or infinity. On the other hand we should do more to a) make the browser punish background tab hogs and b) communicate hogs to the user.
For now we will look at throttling background setTimeouts better (bug 715376, 715378, 715380), XMLHttpRequest loops, etc more aggressively. We also plan to make more use of interactive state so Firefox can suspend non-critical tasks (bug 712478).
Occasionally the cycle collector misbehaves, Andrew will look into not running cycle collection frequently when it is slow: bug 710496. Olli has been fixing many of the cycle-collection extremes, I don’t have bug #s for that, but apparently the improvements are dramatic.
Super-Slow Startups
Thanks to telemetry we now know that some users experience tragic startup speeds ranging from 30seconds to 34hours (bug 701872). Our network cache is to blame for some of these (bug ). Another theory is that an unfortunate turn of events causes us to start loading webpages before the UI is shown (bug 715402).
Vlad will post some of his analysis and interested people can help us with telemetry forensics.
Profiling
Being able to profile interactivity bugs is an important key to making the browser snappier. Large parts of Benoit’s interactivity profiler have landed (bug 713227). Using this extension on nightly win/mac should give you an idea of what it will look like when completed.
We make heavy use of compiler optimizations. Unfortunately one of them is to omit the stack pointer. Ehsan has setup a developer-friendly profiling branch.
Vlad is making progress on non-destructive chromehang(bug . Traditionally we could not do this, but with a combination of telemetry + cycling Ehsan’s shiny new profiling branch on nightly channel… we’ll be in developer heaven.
Responsiveness testing
Peptest should be landing on try soon, Aki is wrapping stuff up. This should enable us to catch responsiveness regressions on our infrastructure.
Smooth Scrolling
Jared is almost done fixing tests to land smooth scrolling to gather feedback and move on to fancy physics (bug 710372).
Other ongoing projects with nothing specific to link to: Vlad’s slow-sql telemetry, Rafael’s quest to close sql connections so we can exit(0), QA browser-cache-effectiveness comparisons.
31 Responses to “Snappy, Jan 5”
https://bugzilla.mozilla.org/show_bug.cgi?id=703337 needs some love! =)
Bug 698298 (slow image decoding) has also been fixed with was a Snappy P1 on Mac
“On one hand people should understand, that one can’t expect the browser perform the same whether 1 tab is active or infinity.”
I don’t think the user should be able to feel whether they have 10, 100 or 1000 tabs open. The GUI takes very little processing power and when the page is loaded it doesn’t take much processing power to read/move. As a user I would expect slower loading and switching when I have a lot of tabs. But a loaded page should be lightning quick with a snappy GUI even when thousands of tabs are loading in the background.
@Ferdinand
basically the only way to allow large numbers of background tabs (hundreds+) is to have a sleep state where the tab is completely unloaded from memory and looses all ability to run anything, and push tabs into this sleep state when unused. You could even go so far as to call this sleep state “closing the tab” and suddenly, we already do that, Firefox can cope with 1000s of bookmarks/history entries, there’s not much more that could reasonably be achieved. I suppose an addition where tabs can exist in a state where they are just a glorified bookmark with associated history would be interesting to experiment with, but im not sure how it could be done without breaking peoples current expectations of a browser (if you don’t look at Facebook for too long, it stops receiving chat messages? people would not be happy with that).
TL;DR: history & bookmarks are the closest thing we will ever have to 1000s of tabs open at once.
Well while we have NO snappy bugfix, we have lots of new bug to look forward to.
Please take some time to allow auto migrate profile when new version install. It helps many users to actually have their profile cleaned and enjoy the new changes from Firefox. Instead of just the same because of profiles problems.
@Scratch
Why can’t you just prioritize? I can run thousands of processes that all use 100% cpu. As long as they have a low priority I still have a snappy fast computer.
What I’d expect from firefox is to set priorities for its threads. Right now my firefox has 22 threads spawned, but all of them have the exact same priority. I’m not quite sure how firefox manages its tabs since I have a lot more than 22 tabs open, so it is not 1 thread/tab.
Anyway I think the proper thing to do is to lower the priority of all threads except the one of the foreground tab. Reading the NSPR thread reference that would probably mean setting thread priorities to PR_PRIORITY_{NORMAL,LOW}, or even HIGH for foreground tab. Hopefully this would make a difference in all platforms.
I can file a bug if this can be applied to firefox’s thread model.
Keep up the good work with the snappy project
“On one hand people should understand, that one can’t expect the browser perform the same whether 1 tab is active or infinity.”
AFAIR – Opera works that way. I have 100+ active tabs, and I cycle through them very frequently, and the only slowdown in opera is the startup one. So it’s possible, so why not with firefox?
I’ll come back to firefox when it starts giving a shit about the platform I use.
I have donated to mozilla for years, submitted bugs, organized release parties, filled surveys, and word of mouth recommendations to everyone I know, yet I have recently switched to chromium. Mozilla does not give a rat’s ass about linux.
I’m not trying to say that mozilla does not hire a full person on working on this platform but this is how I feel it, which is sad given mozilla’s background and how much it gain from the free software movement.
If you submit a bug and no one knows you in an open source project, it may get ignored, @mozilla I think that besides ignoring there’s also prioritizing on platforms.
I’m no web developer but every web developer I know works either on Linux or Mac, yet even on this post I only see Mac/Windows mentioned, and this is a common practice @mozilla.
Needless to say how firefox runs on Linux compared to Windows, and needless to rant about hardware acceleration, yes yes the drivers are bad but haven’t heard any bug reports or public statement on mozilla’s part on this matter.
Regards
I have to limit my network cache to 100MB, otherwise shutdown times (and unclean startup times) explode beyond all belief.
I have my profile directory (including the cache) encrypted on Windows using EFS. Shutting down Firefox takes up to a minute, and after a process terminate a startup takes up to a minute.
It used to be beyond ridiculous before I reduced the network cache size. I mean, seriously ridiculous, I’d terminate Firefox while it was still starting up because I thought it was crashed.
Note: EFS is single-threaded on Windows. So even though I have 8 cores it could be using, it only uses one. Also running on SSD with 12GB ram.
A bug entry for the issue I describe in my previous comment, over half a year ago: https://bugzilla.mozilla.org/show_bug.cgi?id=647479
Yeah that startup bug has been there a long time. I do not know what it is really. But it made me switch to chrome.
Ah well I hope you guys get it worked out!
Yup, will fix the startup bug. Yes, we should be able to match opera responsiveness when lots of tabs are open. Some of the bugs mentioned in the post are specifically related to that.
Barry, that’s helpful.
Scratch, you are spot on. Dietrich has been exploding actively unloading tabs.
jimis, backseat coding is not helpful.
I feel like Firefox is focusing on the computer-science-y solution, rather than the practical one.
Sure, it would be cool if Firefox did throttling and deprioritization and cycle collector improvements and physics-based smooth scrolling.
But 99.9% of the time, what’s happening to me is that I have one tab that’s got some horrible Javascript that’s eating 95% of my CPU, and the other 100 tabs are doing nothing at all. Firefox doesn’t have any way for me to find out which one it is. My only recourse is to close tabs until my CPU meter returns to normal.
I think even a simplistic model (like, “if the JS runtime for a tab has to wake up more than 1/sec, turn the tab red”) would be far more useful in real life than almost any of these other things.
How about the idea of throttled startups for multiple tabs? If some kind of track can be kept of the age of the tabs relative to one another, then in case of a re-start the tabs could be re-opened from newest to oldest. This would place an emphasis on fast recovery and it would achieve the twin goals of letting the user get back to browsing and letting the user know work was going on. It’s also not unreasonable to assume that whatever the user was last looking at is probably most relevant to him.
Just a suggestions.
You should try to measure shutdowns time.
Mark another victim of the startup bug. Sometimes it’s unreal. Doesn’t happen with a fresh install of Firefox. You’re probably losing market share over it.
You havent known this???????
As developers?
Firefox 7 opens and closes with 40 open tabs slower than version 2.6 / 3.
The startup times are awful, but the shutdown times are even worse – they can take up to 3 minutes for me. Not to mention that something is terribly wrong with the tab history (although I admit IM using tab mix plus) – when I kill the firefox process to help it shutdown, I lose my history of open tabs…
What I have noticed is that when I close firefox, your “plugin container process” decreases by 10-50 mb every few seconds, starting from say 800mb.
I dont even know why 40 tabs need 800mb of ram, Im not a programmer, but I think it is shitty programming on your side.
How about Android version? It’s abysmal, do sth about it or change it’s name. The worst browser or Android.
tglek: perhaps you can help me figure out how firefox handles its threads. I have 100 tabs open, what are those 22 threads that I see? Is there a thread dedicated to the foreground tab? Some pointers where to look in the source?
I’ve contributed patches to firefox in the past, but when I don’t understand the basics all I can do is backseat coding.
jimis: Unfortunately, there is no relationship between tabs in firefox and threads. All web content (as well as the GUI) run on the main thread; the other ones you see are things like the cycle collector, the network thread, and various assorted background threads for things like database access (we use a bunch of different databases). Thread priority is unlikely to be a very productive line of investigation at this point, in my opinion.
@jimis I’m an armchair follower of mozilla, but I reckon you might want to look at mozilla electrolysis as a start, there are some nice wiki entries about it.
jdm: Thanks for the info, I didn’t know that all javascript runs on the same (main) thread. I should have known better since I’ve noticed many times the UI freezing because of a single page. Obviously my analysis doesn’t apply.
How about this for snappy!
GC mode: full, timestamp: 1326318174386000, duration: 21766 ms.
GC mode: full, timestamp: 1326316148589000, duration: 27422 ms.
GC mode: full, timestamp: 1326313038949000, duration: 27657 ms.
GC mode: full, timestamp: 1326309057495000, duration: 14609 ms.
I can’t make head nor tail of the timestamp format but that is four crazy-long garbage collection delays.
What was I doing to cause them? Merely attempting to make the Firefox window active again after a few hours of leaving the computer alone.
Why can’t GC be done when the browser is idle?
MemShrink is 6 months old and according to Nic Nethercote, Firefox’s memory handling will only get back to Firefox 3.6 levels when Firefox 10 is released. With all due respect, that’s pathetic! Now there’s Snappy. Boy do you guys have a lot of work to do and I really and truly hope you can do it very flipping fast.
I’ve been with Firefox since Firebird days and earlier but despite my dislikes I just think it might well be time to give in to the Chrome juggernaut like every else
Mozilla you’ve forced me to make this hugely disappointing move!
One thing I completely agree on though: managers are , more often than not, useless!
Of course firefox should be able to handle 1000+ tabs! This is 2012! Unused tabs should be unloaded, I have enough disk space to spare, so FF should store any needed state on disk. At some point there probably should be a javascript suspend/resume tab event, but that would require standardisation.
The major problem I have with firefox is that several times a day it just freezes and doesn’t update _anything_ for a significant amount of time (5-20 seconds commonly). this will happen while I’m typing a reply, and if I just keep typing blindly the text will eventually appear. If I change desktops, the firefox windows on the desktop do not get filled in until firefox unfreezes.
I’m currently runnuning Aurora and have stats enabled
this isn’t the machine running out of ram (I was having that problem around firefox 7), as other apps keep running without any problem when firefox freezes
@somejan why not 10000+ tabs? In a year’s time when people such as yourself will state the arbitrary and irrelevant “It’s 2013″ will you expect 100,000+ tabs?
Do us all a favour: be reasonable. There has to be a reasonable limit.
I’m actually with @somejan in the long run. I think it’s doable, but not realistic in immediate future.
David, that’s exactly what we are aiming to eliminate.
What is the point of having tabs that are ‘open’ when they are unloaded? All that is creating is a false impression of what tabbed browsing. Ok so the browser might unload 998 tabs from memory but surely that will just cause lots of paging and more complaints about a lack of snappiness (what is the opposite of snappy, unsnappy?) ?
I already use the “Don’t load tabs until selected” so I think I have some perspective. This doesn’t stop the browser from lagging and eating up heaps of RAM but it does slow down the effectiveness of tabbed browsing.
Really, unloaded tabs and accepting an infinite number of tabs is just creating a browsing session with 98% of content sitting there as semi-active bookmarks (disguised as tabs).
On another point, how on earth do you expect to have hundreds of tabs and a snappy browser without multi-process?
Why was e10s put on the back burner? Part of the reason the browser with the most ‘snappy’ reputation has gained that reputation is multi-process from years back. I was aghast when I found out e10s is delayed/dead/’deprioritized’.
How many bad decisions can Mozilla keep making before it pinches itself and realizes it is getting beaten in the market due to those decisions? I am reading a vibe of “well, we’re up against three mega corporations, we are doing pretty well considering” in Planet Mozilla articles lately. This is the exact opposite attitude to the attitude that created Firefox in the first place.