The folks at Gemius have been kind enough to aggregate their individual country data (e.g., www.en.ranking.pl/) into a single view across their entire sample – a sample totaling more than 60 Billion page views each month.  For an overview of the various market share providers and their samples, please read here.

We’ll eventually look to expand the conversation around this data, but for now, we’ll highlight just one breathtaking view.  The chart below shows weekly browser market share data since the beginning of 2007 and it includes aggregated data from across nine countries – Czech Republic, Bulgaria, Estonia, Hungary, Lithuania, Latvia, Poland, Russia, and Ukraine.

• People love Firefox!
• There is a small segment of existing users running into a particular problem – Firefox is not saving their preferences

Taking a step back, let’s walk through how we arrived at these findings.  We previously described our efforts integrating user outreach into mozilla.com.  For this post, we’re going to focus just on comments left by users during their visit to the Firefox “First Run” page, which brand new users hit upon downloading/installing Firefox for the first time (analysis of the Whatsnew and Getting Started pages will soon follow).

About 1,200 people left feedback over the past month, and below is a breakdown of the feedback.  If you’re curious how this worked from the user’s perspective, click here to the firstrun page and then click on the orange feedback button.

So, overall, the feedback looks really amazing.  For such a high percentage of people to go out of their way to say something positive is incredible.

On the downside, there were two issues identified by users that we weren’t previously aware of:

1. Many people complained about the open video freezing.  I’m not sure exactly what the issue is, but we can note that the next version of this page (for Fx3.6) will not include the same video.
2. Perhaps more critically, many users talked about their frustration around Firefox not saving their settings.  The most common comment was along the lines of, “why does this page open every time I open Firefox?  Please get rid of it.  I previously set my home page and Firefox won’t save the setting.” In addition, some people noted other preferences/settings not saving – add-ons, personas, favorites and bookmarks.

How do we know that this latter group of users is frustrated?  In addition to reading their actual comments, we can also look at the average satisfaction score (scale of 1 to 5) left by users in each feedback group.  The findings are below… it’s pretty clear that this group, with a satisfaction score of 1.9, has been feeling extremely frustrated.

What’s the good news?

Thanks to this insight, we’ve been able to prioritize a fix via bug 495735, and we’re hoping to ship it in the next release of Firefox!  Thanks to Dietrich Ayala, Mats Palmgren, the Firefox team, and several others for making this happen.

Congrats everyone!

For those interested in “before and after” pictures, below is one way to visualize the browser market share landscape (based on Net Applications’ data).

Now that this new program has been live for about a week, we thought we’d check the numbers (cumulative since last week):

Considering that this page hasn’t been directly pushed to users, these numbers are significant.  There are a couple spots within mozilla.com where users can click to find this page, but otherwise, our web analytics reporting tells us that about 90% of the traffic has been coming from news and technology related web sites.

Next, you may be wondering where people ventured upon clicking to “Update” or “Research”.  Here’s a breakdown:

We’ll continue to keep an eye on these numbers, and perhaps more excitedly, there are plans to soon integrate this feature into the browser itself.

Two months later, Eric’s model does a remarkable job predicting future usage.  Below, I have plotted actual against predicted usage.  The correlation between these two lines is remarkable.  Since September, the average prediction was off by just 1.63%!

To better understand daily growth variations, I created a heat map that visualizes month over month usage growth. Each cell contains one day’s growth rate over the same day of the week, 28 days earlier.

Some highlights:

• Strongest growth occurs during Spring and Fall
• Periods of growth and decline are clustered
• Growth is outperforming predictions by 2.58%

We must be careful not to overreach with our conclusions.  We are working with a limited range of data and have likely missed significant confounding variables.  That said–thanks to Eric’s work–we’re off to a great start.  Please leave your thoughts and any suggestions on how to improve our projections in the comments.

Why is this important?

• It provides users with an opportunity to share their praise/criticism or any other thoughts
• It strongly complements our web analytics efforts (i.e., we’re able to answer the “why” questions in terms of user experiences)
• It ultimately allows the Fx team, marketing team, and many other groups across Mozilla to gain valuable insights and improve the Fx user experience

To accomplish this, we’ve implemented a feedback button/form on three critical touch points at mozilla.com:

Getting Started page

First Run page

Whatsnew page

The feedback form on each page is slightly different (looking at the “categories” and “sub-categories”), depending on a user’s expectation of that page and the typical point in a user’s experience when he/she hits that page.  As an example, below is the form you will see upon hitting the feedback button on the whatsnew/update page.  And like our previous implementations, we’re not 100% sure what feedback to expect, so we’re going to closely monitor user comments and tweak the feedback forms accordingly.

Stay tuned!

Emerging from the summer months, both reports show Firefox gaining about 1% (over the past couple months).  And looking at September specifically, Net Applications shows Firefox gaining more than 0.75%.

Net Applications:

StatCounter:

Beyond just the past couple months, you’ll notice that overall trending also looks quite good.  Over the past year, Net Applications shows Firefox picking up more than 4 percentage points, and StatCounter shows more than 6 percentage points.

You may have noticed that Mozilla Labs implemented its very first experiment for Test Pilot earlier this month.  The goal is to answer the following two questions (or at least put some quantitative framework around the questions):

1. What would be the best default tab behavior after users close a tab?
2. What should be default placement when users open a new tab?

As someone who downloaded Test Pilot and participated in this study, I was impressed by a couple things.  First, the introductory page did a nice job of summarizing what was going on (first image below).  And second, at any time I could view what data was being collected about me (latter image).  At the end of the experiment, it was then my decision whether or not to submit my data.

Moving forward, the metrics team looks forward to partnering the Test Pilot team in making an impact with the resulting data.  Stay tuned!

• 99% of internet users (desktop) have Flash.
• The vast majority of people have an out of date version.  One study claims 80% and mozilla.com’s own traffic stats show about 75% of visitors on a non current version.

So, what has transpired since last Wednesday?

In one week, 10,000,000 people have clicked on the “flash update” link below.

Taking this analysis one step further, we wanted to gain a better sense for users’ interaction with this page.  Breaking down the data by day, we looked more carefully at the en-US version of the 3.5.3 whatsnew (or update) page and pulled the following numbers:

1. How many total people hit the whatsnew page?
2. Of this cohort, how many had an out of date Flash version, and hence, saw the message above?
3. And of this smaller cohort, how many people actually clicked on the flash update link?

Beyond the total impact of 10,000,000 clicks, the most impressive pattern that stands out is the click through rate.  While the Firefox whatsnew page generally sees a click through rate below 5%, the flash update link alone has generated a click through rate north of 30%.  Phenomenal!

• Which search terms lead to which articles?
• Are visitors ultimately satisfied with the articles they choose to view?

Let’s take a look at the data:

Highlighted in green and red are paths our support site seems to do particularly well and poorly with, respectively. A positive take-away from just glancing over the data is that in four cases (under searches private browsing, clear cache, cache, and export bookmarks) we  have top articles performing really well to visitors’ standards.

Focusing on some of our poorer marks, we see the cookies, enable cookies, bookmarks, and clear history searches are not directing users to the content they want. With help from David Tenser, we were able to come up with some reasoning behind these numbers.

Starting with the clear history search to the “Clear Recent History” article, there seems to be a solid explanation for the poor grade. A continuing problem we have had (documented concretely here) with the Awesome Bar is that users do not realize their bookmarks show up while typing in an address. When this is the case, users that believe they are clearing their history will still see any undesirable bookmarks in the Awesome Bar and hence think the “Clear Recent History” article has not helped their problem. Users need to change the content their Awesome Bar remembers in the Privacy section of Firefox preferences to accomplish this task.

Visitors hitting the “Lost Bookmarks” page from the  bookmarks keyword have given the article a low approval rating. An issue with the “Lost Bookmarks” page is that it does not link to an article explaining how to import bookmarks. Users who have upgraded their browser or have just come from another browser may not know how to import their bookmarks, and they navigate to “Lost Bookmarks” in order to find a solution.

We are looking deeper into users’ experiences with two other articles highlighted above in red — “Websites say cookies are blocked” and “Cannot log into websites” — to see if any content is missing and needs to be added. The problems are straightforward enough — visitors making their way to these pages are probably not clicking on them by accident or confused about their problem. The SUMO team is currently implementing changes across the site to increase user satisfaction in areas like those discussed above. This should provide for a much smoother help site experience across the board for our users.

]]> http://blog.mozilla.com/metrics/2009/09/09/getting-help-when-using-firefox-part-ii/feed/ 4