Mozilla Webdev

Download Counts Halted

• By Mike Morgan

The download controller was modified on Thursday to prepare for the release of the 1.5.0.12 -> 2.0.0.4 major update. During release cycles, AMO takes abnormally high load which sometimes causes interruptions in service.

To avoid this situation we agreed to cache public download hits from the AMO install buttons. This does two things:

• Relieve application load by allowing the hardware load balancer to cache file requests — which are ultimately redirects to releases.mozilla.org
• Relieve the database by not having constant inserts on the download table — which causes extra load because of indexes that were put on the table to make count updates work correctly

The benefits can be seen below. First, traffic for a web node:

Second, a CPU usage graph from the same node:

You can see the positive effect on load near noon on Wed.

The status of download counts is tracked in bug 384084. The plan is to create a background maintenance script that parses AMO logs via a cron job in off-peak hours. Counts will be updated once a day.

In addition to this, we are taking the opportunity to do a couple of things. For our summer goals, we plan on improving statistics for developers by offering:

• Actual update ping counts in bug 384086
  
• An improved API for aggregating add-on statistics and integrating it into your blog or external sites

We plan on resuming download counts early next week and update pings should be available by mid July. Thanks for your patience as we make adjustments to accomodate server load, and have a great 4th of July!

{

About Mike Morgan

Mike Morgan is the Director of Web Development for the Mozilla Corporation. Mike joined the Mozilla community as a volunteer in 2003 and has worked on Mozilla's mirror management software, update system and addons.mozilla.org. These days he spends most of his time on strategy, recruiting and helping his team embrace their inner nerd.

This entry was posted on Saturday, June 30th, 2007 at 5:50 pm and is filed under AMO. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.