That’s the amount of time we shaved off average page load times for addons.mozilla.org after last night’s work. The Amsterdam Reboot in effect!

It’s been nearly ten months since we served production traffic at these levels out of Amsterdam.

I remember coming back and was up late at night (fighting jetlag) setting up the Netscaler load balancers. By early January we had a CVS mirror up and running and a week later had staged www.mozilla.com and www.mozilla.org in Amsterdam. By the middle of February we had shifted European production traffic over to Amsterdam.

By May of 2007 (oh and here) we started serving addons.mozilla.org out of Amsterdam too.

Since then, the San Jose data center has grown from seven racks to twenty-four and nearly 500 servers (for the sake of this post I’m counting the 150 Mac Minis as “servers”).

Unfortunately, Amsterdam hasn’t seen the same sort of server growth and in the past half year or so we’ve had to pull sites back from Amsterdam and serve them from San Jose only. When the load balancers there could no longer handle the SSL traffic in January we stopped serving addons.mozilla.org out of Amsterdam too.

The Reboot

On September 2, we’ll reboot Amsterdam. Much like The Six Million Dollar Man, “… we can rebuild [it]. We have the technology… We can make [it] better than [it] was before. Better, stronger, faster.”

This morning we shipped a half loaded HP c7000 BladeSystems chassis out to Amsterdam. Next week both Arzhel and Derek will be in Amsterdam to deploy new servers and turn down some of the old legacy hardware.

One of the issues with the current Amsterdam deployment is that we can only really serve static sites – sites that don’t have a database behind them. It’s a bit more complicated to replicate databases to remote locations and keep them in sync with San Jose.

Based on our success with the Zeus ZXTM platform, we’re deploying a Nehalem based ZXTM cluster. Much like we originally did with addons.mozilla.org, we’ll have a platform where we can proxy/cache any Mozilla website and serve it out of Amsterdam.

We’ll also be deploying a new global load balancing system that Arzhel’s spent time staging and getting ready for production.

The Plan

During tomorrow night’s downtime window we’ll start the process of temporarily shutting down Amsterdam. We’ll pull back all websites to San Jose and stop serving web content out of Amsterdam for about two weeks.

We’re doing this in advance of the actual deployment to make sure we don’t hit any surprises.

What won’t be affected?

There are two VMware ESX servers in Amsterdam that we won’t be touching. A quick list of hosts that won’t be affected by this are:

• dn-vcs01
• gravel
• geodns02
• l10n-01
• rhino01
• sea-qm-centos5-01
• sea-qm-win2k3-01
• konigsberg

I’m excited about this. I’ve said before I’m still a network guy at heart and I really like getting content closer to consumers. I’m hoping this Reboot becomes an easily deployable platform for other parts of the globe.