Mozilla Webdev

The AMO team is finalizing the latest update, 3.4.1. This is a bug-fix release addressing things like double escaping, improper L10n redirects, and category cleanup. A list of all the bugs targeted for this release is available.

We’ll be committing the last of the patches very shortly and the changes will be available on preview.addons.mozilla.org. Feel free to look at the changes and send us any feedback you have. We expect to push the changes live this Thursday evening (May 1st).

Our next release, 3.4.2, is currently scheduled for May 15th. 3.4.2 will also be a bug-fix release - there are currently 19 candidates.

Last Friday we pushed some important updates to Socorro:

• Bug 426940 - Reduce or eliminate delay in collector to monitor hand-off
• Bug 426940 - Fix processor handling of error conditions
• Bug 428300 - status page too slow

This means:

• When you submit a crash report you won’t have to wait longer than 30-60 seconds to view your report
• The processor now has better handling of minidump_stackwalk fatal errors
• There is an improved server status page where you can view stats on the current queue

Thanks to Lars and Aravind for getting this out the door.  The next couple of weeks will be spent improving reporter performance and closing out milestone 0.5 bugs.

History can tell you that companies don’t disclose crashes in their software. They keep a pretty close eye on what crashes and bugs are disclosed.

Mozilla doesn’t.

Rather than being the exception, openness is the rule, and that is one of the coolest things about being a part of this. My job, my everday tasks, they aren’t secret, and they are not to drive profits. They are to drive the web.

In that spirit, our crash reporting system (Socorro) is available to whoever wants to view it. Aside from user-bound statistics, crash information is available in full and anybody in the community can learn about where in the code their client crashed. They can also help provide hints or comments about what they were doing at the time they crashed.

This opens the door for the community to learn valuable things about their software and how they use it:

• What crashes the most? What crashes the most over time? What is the breakdown across branches, versions and products?
• Where did we crash? Crash signatures provide a head start for locating the cause for a crash. From there, full stack traces are available to analyze callback and find the source of the actual crash.
• What was installed? What modules were installed for a given crash? Soon we will also be able to understand what extensions were installed so we can understand the correlation between core client crashes and crashes caused by faulty extensions. The end result is a closer relationship with the extension developer community and better quality in our add-ons space.
• How are we doing? Overall the jackpot question is — are we crashing more or less? How are we doing with this beta, alpha or rc1? Are we regressing in real-life situations despite positive automated testing results??

All of this was possible because of a collaborative effort between quite a few parties:

• Mark Mentovai and the breakpad team, for writing a great client and processor under a flexible open source license that is easy to integrate
• Ted Mielczarek for his work on the client, processor and integrating the project into Firefox 3
• Benjamin Smedberg and Robert Sayre for their work in getting the initial versions of the breakpad server off the ground

Where do we go from here?

• Learn more about the project
• Send in your ideas

Of the many projects we have in 2008, this is one of the most exciting. It’s an opportunity to open up information that hasn’t historically been available to the masses, and hack on a great tool for improving the quality of all Mozilla projects

addons.mozilla.org was updated last week. AMO 3.2.1 was a maintenance release (26 bugs fixed) for any major issues with 3.2.

Our next release will be AMO 3.4.1, the first of three dot releases for AMO 3.4, which is our next milestone to be completed before Firefox 3.

We’ve pushed some important updates in the last couple of days:

• refactor of processor code, which is 1/3 of the breakpad server
  architecture
• update of reporter to allow for instant queuing of requested reports

This means:

• If you submit a crash, going to that crash page will:
  • Show you a “haven’t queued it yet” page instead of a 404
    page that will update in < 10 min
  • Once queued, you’ll see a “report pending” page that will
    redirect to the finished report in < 21 seconds
• Wait time for reports from testers is reduced to 10 min max,
  sometimes 21 seconds best-case
• We are working on eliminating the 10 min portion but there are
  reasons why we can’t spam the monitor that is responsible for
  queuing new reports that are on disk — more on that next week (I
  want this to get down to: load, wait 20 seconds, BAM! see your report)

Thanks for everyone’s patience with the crash report backlog during releases — we hope this helps many of you.

Let me know if you have any questions.  More to come in the next few weeks!  Thanks to Lars, Ted and Aravind for their help with developing/ testing and pushing these updates.