Introducing Perf-O-Matic 2.0


Here’s a quick transcript of the video:

Hello, this is Rob Helmer from the Mozilla WebDev team.
I’m here to introduce a project we’ve been working on
for the past few months: Perf-O-Matic 2.0

Perf-O-Matic is the web application we use to display Firefox performance
data. Every time a developer “checks in” a change to Firefox, a fleet of
machines runs the browser through a battery of tests to measure performance.

These tests range from startup/shutdown performance, the time it takes to
load a web page, and browser benchmark suites like WebKit’s SunSpider,
Mozilla’s Kraken, and Google’s V8 Benchmark.

Perf-O-Matic 2.0 features a new, streamlined interface, designed by Chris Howse.

This is the dashboard, which allows for a quick overview of popular tests.

Clicking on any graph in the dashboard brings us to the
“Custom Chart” screen. The graph is interactive – hover over any point
to get a tooltip with more information about this test result. Every
test result will have an associate changeset id in mercurial,
which links to more information about what changes the developer made.

The toolbar of the “Custom Chart” screen allows us to select date range,
data type (running time, time delta, time delta %), zoom in and out,
and you can even show all changesets for a highlighted range.

The “Link” button gives a quick link to the chart you’re looking at.
“Embed” gives you some HTML code you can use if you want to display
interactive graphs on your own site. I’ll give a quick demo of this.
Here’s an embedded, interactive chart; this is inspired by arewefastyet.com.

On the left, we have information about the currently loaded
data series – SunSpider benchmark for Firefox on Windows 5.1. Each point on
the graph is actually an average of several test machine runs, so we can
“explode” the data series to see the raw, individual data. Clicking again
will “implode” the data, showing us the more human-readable average.

Finally, more test data can be added, for making direct visual comparisons
between multiple data series. You can multi-select here, to get multiple data series at once.

This project builds on the work of many others, including (but not limited to):
vlad
rdoherty
anode
ianbicking
many, many others on the graphserver side

We also received valuable early design review and feedback from jonath, rsayre, eshan, and many others.

Perf-O-Matic uses many existing open-source projects, such as python, flot, and node.js

Perf-O-Matic 2.0 is being staged for testing.
We’ve got a handful of known bugs, and we need your help to find the rest.

Please try it out at:
http://graphs-stage2.mozilla.org

You can find more information on the Perf-O-Matic UI wiki page, including links to mercurial and github:
https://wiki.mozilla.org/Perfomatic:UI

6 responses

  1. Nadav wrote on :

    Perf-o-matic 2.0 looks cool! I love the interface and the graphics. I am sure that this will become an invaluable tool in making Firefox faster.

    At work, I develop a high-performance application which I tune for performance. Can I use Perf-o-matic for tracking performance over time ? How difficult is it to setup and configure perf-o-matic for my needs ?

  2. Mark Finkle wrote on :

    Is support for Mobile data coming? Should I file a bug?

  3. rhelmer wrote on :

    @Nadav – For 2.0 (the version in this video), you can find the latest at http://github.com/rhelmer/graphs

    Be sure to see the README and INSTALL files, and please let me know if you have any problems installing.

    Instructions for the production version (1.0) are here:
    https://wiki.mozilla.org/Perfomatic/Installation

  4. rhelmer wrote on :

    @Mark Finkle – Sure, file a bug here for any requests for changes to the dashboard:
    https://bugzilla.mozilla.org/enter_bug.cgi?product=Webtools&component=Graph%20Server&version=2.0

    Let us know which of the available tests you’d like to see up there:
    http://graphs-stage2.mozilla.org/graph.html

  5. Robert Kaiser wrote on :

    Looks interesting, but buggy. If I go for custom graphs and try to get something for SeaMonkey, it shows me all those things that we never report as selections, but not those we do, like MaxHeap, malloc leaks, codesize, etc.

  6. rhelmer wrote on :

    @kairo – thanks for the feedback! Filed https://bugzilla.mozilla.org/show_bug.cgi?id=633038