Blog of Metrics

Like last time, let’s start with the punch line:

• People love Firefox!
• People are also notably frustrated with two experiences – (1) Firefox crashing too often, and (2) Firefox not saving their settings (e.g., home page)

These insights were arrived at via our current user outreach efforts.  When people visit the Firefox “Whatsnew” page – which users automatically hit after each Firefox update – they have the option of providing feedback and comments.  More than 16,000 people have been kind enough to provide feedback over the past six weeks.  If you’re curious how this works from the user’s perspective, click here to the Fx3.5.4 whatsnew page and then click on the orange feedback button.

Here’s a breakdown of what has recently been on the mind of users:

We previously shared analysis of comments from people visiting the Firstrun (installation) page, and the chart above looks somewhat similar.  The fact that so many people made a point of saying something positive is amazing!

On the other hand, there is one negative item that we first took note of last time.  A significant number of people left a comment along the lines of, “why does this page open every time I open Firefox?  Please get rid of it.  I previously set my home page and Firefox won’t save the setting.” On the plus side, we have already taken this insight and turned it into action.  The Firefox team has prioritized a fix via bug 495735, and it should ship it in the next release of Firefox.

Lastly, there was one other group of people who were particularly frustrated.  For each of the groups in the pie chart above, we can also look at their average satisfaction score (scale of 1 to 5):

Fortunately, the crashiness of Firefox is a known issue, and making Firefox less crashy has been a top priority for the Mozilla community in recent months.  Given our current focus in this area, I would guess that if we rerun this same user outreach program a few months from now, we’d see that green pie slice either dramatically shrunk or completely eliminated.

At least in one large region of the world, the answer is “yes”.

The folks at Gemius have been kind enough to aggregate their individual country data (e.g., www.en.ranking.pl/) into a single view across their entire sample – a sample totaling more than 60 Billion page views each month.  For an overview of the various market share providers and their samples, please read here.

We’ll eventually look to expand the conversation around this data, but for now, we’ll highlight just one breathtaking view.  The chart below shows weekly browser market share data since the beginning of 2007 and it includes aggregated data from across nine countries – Czech Republic, Bulgaria, Estonia, Hungary, Lithuania, Latvia, Poland, Russia, and Ukraine.

Let’s start with the punch line:

• People love Firefox!
• There is a small segment of existing users running into a particular problem – Firefox is not saving their preferences

Taking a step back, let’s walk through how we arrived at these findings.  We previously described our efforts integrating user outreach into mozilla.com.  For this post, we’re going to focus just on comments left by users during their visit to the Firefox “First Run” page, which brand new users hit upon downloading/installing Firefox for the first time (analysis of the Whatsnew and Getting Started pages will soon follow).

About 1,200 people left feedback over the past month, and below is a breakdown of the feedback.  If you’re curious how this worked from the user’s perspective, click here to the firstrun page and then click on the orange feedback button.

So, overall, the feedback looks really amazing.  For such a high percentage of people to go out of their way to say something positive is incredible.

On the downside, there were two issues identified by users that we weren’t previously aware of:

1. Many people complained about the open video freezing.  I’m not sure exactly what the issue is, but we can note that the next version of this page (for Fx3.6) will not include the same video.
2. Perhaps more critically, many users talked about their frustration around Firefox not saving their settings.  The most common comment was along the lines of, “why does this page open every time I open Firefox?  Please get rid of it.  I previously set my home page and Firefox won’t save the setting.” In addition, some people noted other preferences/settings not saving – add-ons, personas, favorites and bookmarks.

How do we know that this latter group of users is frustrated?  In addition to reading their actual comments, we can also look at the average satisfaction score (scale of 1 to 5) left by users in each feedback group.  The findings are below… it’s pretty clear that this group, with a satisfaction score of 1.9, has been feeling extremely frustrated.

What’s the good news?

Thanks to this insight, we’ve been able to prioritize a fix via bug 495735, and we’re hoping to ship it in the next release of Firefox!  Thanks to Dietrich Ayala, Mats Palmgren, the Firefox team, and several others for making this happen.