Graph Server Re-Write

Over the past few months the Graph Server team and I have been hard at work re-writing the back end for the Graph Server and it’s finally come to fruition.

For those that don’t know, the Graph Server is used to display performance test data of Firefox builds reported by Talos.

Our work initially started as performance improvements and some new features, but the more we worked with the old architecture, it became quite apparent it would not scale (performance and feature-wise).

The old database schema duplicated test data in multiple tables and stored similar, but different data in the same tables. Tables had ballooned to millions (and billions) of rows that were queried for basic information such as all unique test names, resulting in queries that ran forever. And the queries that did finish were looped over in JavaScript to pull out test information, resulting in the browser locking up because it was looping over hundreds of thousands of rows.

If it’s not clear already, one of main issues was with the database schema; it needed to be normalized.

Here’s the old, non-normalized schema:

And here’s the new, normalized schema after the team was locked in a room for an afternoon:

Much cleaner, no duplicated data, easy to understand the various machines, branches and tests that are used for displaying test data. No need look at entire tables to find basic information such as test names.

With this new schema in place, it also required a re-write of our server-side scripts we use to fetch test information for the front end graphing component. Since Mozilla is as open as possible, instead of just changing what was needed, I decided to implement a JSON API that would allow anyone to easily retrieve test data.

Lastly, our Talos <--> Graph Server communication needed to be re-written. Lars rewrote the collector script that accepts values from Talos and Alice rewrote the pieces of Talos that send data to the Graph Server.

After all that work, we now have a working stage server (Firefox 3.1 or higher required due to native JSON requirement) with our new code. We have a bit more testing and some performance benchmarking to do before it goes live, but we’re happy that all the pieces are working.

Want to know more? We have a wiki page with more information at https://wiki.mozilla.org/Perfomatic#Rearchitecture.

Tags: Graph Server, javascript, mysql, performance

This entry was posted on Friday, February 6th, 2009 at 10:12 am and is filed under Graph Server. You can follow any responses to this entry through the RSS 2.0 feed.