Measuring UI Responsiveness

June 27th, 2011

One of the goals for the Firefox team is to ensure that the user interface remains responsive to input at all times. Clearly a responsive interface is incredibly important to making the browser a useful application, but how do we measure “responsiveness”?

Dietrich has done some work on this, writing an add-on that measures the time that various UI actions take. This covers the direct case, where a user initiates an action and expects a response in a reasonable amount of time. Clearly we want to make sure that individual actions don’t take an extraordinary amount of time.

I took the opposite tack, with an eye on being able to detect when the application was not responsive to user input regardless of what actions the user was taking. Building on some work by Chris Jones and Alon Zakai, I wrote some code that instruments the main thread event loop to find out how long it takes to respond to events, which ought to be a reasonable proxy for measuring responsiveness. When the instrumentation detects that the event loop takes too long to respond (more than 50 milliseconds, currently) it writes a data point to a log giving the current timestamp and the amount of time the event loop was not responsive.

When I implemented this I had my eye on Talos integration, where we could run the browser through some automated UI tests with this instrumentation enabled, and then correlate “UI actions” with “unresponsive periods” and ensure that the browser did not become unresponsive during those actions. Talos integration has been shifted off as a longer-term goal, with the more immediate goal being “find UI actions that are the worst offenders of unresponsiveness”. To that end we’ve filed some other bugs about correlating this unresponsiveness data with JavaScript execution, and correlating the data with C++ execution. If you’ve got any ideas please feel free to contribute to those bugs!

If you’d like to try out the responsiveness instrumentation I implemented, it landed on mozilla-central a while ago, and there’s some reasonably complete documentation in the source code. There are implementations for Windows, Linux/GTK2 and OS X currently. (And a patch for an Android implementation in a bug.)

5 Responses to “Measuring UI Responsiveness”

  1. Brian Says:

    Great that you are making responsiveness a priority. One action that I always dread is opening a new browser window. Please reduce the time taken to open a new browser window. — Thanks!

  2. Alexander Limi Says:

    Nice!

    You and Taras should talk (if you haven’t already!), this would be great to have as part of the Telemetry stuff. 🙂

    https://blog.mozilla.org/tglek/2011/05/13/firefox-telemetry/

    I’m wary of measuring things like this in synthetic test suites, since it ends up ignoring whole classes of problems that exist in the wild. And since you make what you measure, it’s important to measure the things that make a difference for our users, not just our test setups.

  3. mucinch Says:

    Please test with atleast 5 tabs (Some with flash and images) and a bunch of popular addons. That will represent a more common use-case. It is awesome to see Mozilla finally concentrate on the important parts. Please do follow-up posts based on your findings. Eagerly Waiting!

  4. Gaba Says:

    Make sure you have some idea about responsiveness on a system like this!

    http://www.mozilla.com/en-US/firefox/5.0/system-requirements/

    It’s just ridiculous how long Fx claimed to have a minimu requirement of 233 MHz processor, when even on a faster machine one could watch a menu opening slowly …

  5. goals for multi-process firefox | Products Says:

    […] can start to see a preview of tools for measuring responsiveness in one of Ted’s posts. Our investment in tools is happening along side of the multi-process […]