November, 2007

plans — 10 Comments
26
Nov 07

Mozilla Online Advertising – an Experiment

Perhaps one of the business world’s easiest arenas for running experiments is online advertising (e.g., pay-per-click search advertising). With the click of a button, you can easily vary one of many variables – messaging, keywords, budget, etc. – to instantly understand its effect.

Over the past couple of months, I’ve been collaborating with David on some different ways to optimize our online advertising efforts. The goal includes improving the downloading path/experience of new Firefox users, as well as understanding and optimizing the investment Mozilla is making in this channel.

One upcoming experiment relates back to funnelcake. Some of you may remember that the source of downloaders looked this:

Thanks to the popularity of our download page and some recent search optimization efforts, the Firefox download page occupies the #1 organic search result in most major search engines (when searching for “Firefox”). With that in mind, one natural question that arises is: if we did zero pay-per-click (ppc) advertising, would our organic traffic (i.e., unpaid search) rise by a corresponding amount? In other words, given the following scenario:

  • I am prospective Fx user
  • I visit a major search engine searching for Firefox
  • I have a propensity to click on paid ads

If I am not presented with an ad in the search results (an official Mozilla ad directing me to www.mozilla.com/firefox), will I instead click on the first organic search result (which also directs me to www.mozilla.com/firefox)?

So, from a results perspective, if we turn off our ppc advertising, we’ll be looking to see if the pie chart above morphs into the chart below (note the 38% organic picks up the 10% ppc from above). While the answer will likely be a strong “no,” we wouldn’t be responsible marketers without at least giving this question some thought and understanding how this dynamic impacts our true cost of acquiring a Fx user through online advertising.

We look forward to soon sharing our methodology and findings.

Uncategorized — 1 Comment
13
Nov 07

Mozilla Comes to the Defense of Radiohead

Polvi and I have been following the release of Radiohead’s new album, In Rainbows, for a couple reasons. First, it’s a fresh and great approach to open source marketing – open pricing! Second, their process has some flavors of our funnelcake project. For In Rainbows, they’re considering the path of fans as: visit -> download -> pay $0 or any amount -> listen.

The link above references a comScore study attempting to roughly measure the success of In Rainbows (via the number of downloads). The entrance of their funnel relies on an estimate of 1.2 million visitors to Radiohead’s web site, InRainbows.com, during the month of October. From the study’s numbers, one can then see what percent of visitors paid for their download and what average price those fans paid.

Radiohead has come out contesting comScore’s analysis. Radiohead and comScore can have their own disagreement over the middle of the funnel, i.e., download percentages and prices paid. Where Mozilla can perhaps shed some light on the situation is the entrance of the funnel, i.e., how many fans visited Radiohead’s site in October? All other analysis is completely dependent on this piece of the puzzle, so it’s critical that any discussion be certain that the answer to this question be 100% accurate.

So, how can Mozilla help?

For the month of September 2007, Mozilla saw 42.7 million unique visitors (this includes just our www.mozilla.com domain, which hosts about 50% of all Mozilla traffic).

What about comScore and Compete.com? How close did they come to estimating our traffic for September 2007?

  • comScore’s study says 16.7 million unique visitors
  • Compete shows our “people count” as 3.4 million

Perhaps Radiohead has a legitimate concern…

results — 5 Comments
12
Nov 07

Vista Users Enjoy the Web on Weekends

For funnelcake, we’ve looked at various analyses by OS. Firefox users primarily use three operating systems: XP (first place by a wide margin), followed by Vista and then Mac. When looking at Firefox usage by OS, one pattern that stands out is usage by Vista users does not seem to drop significantly during weekends.

In a previous post, we considered the effect of day of the week on Fx usage (and internet usage more generally). The swing from a typical weekday to a Saturday or Sunday amounted to about a 20% drop-off in usage. For funnelcake users, we’ve seen that most of the Fx weekend usage decline in driven by XP users. However, this is explained by the fact that they represent the greatest percent of total users by OS; on a percentage basis, XP users show about a 20% weekend decline. Of the three biggest OS cohorts, Mac users display the greatest weekend drop-off on a percentage basis – about 25%.

And Vista users of Fx?

Their Fx usage decline on weekends amounts to a tiny 8%.

Furthermore, when we look across all Fx users (not just funnelcake users), the numbers are fairly similar. We’re not entirely sure what to make of this analysis… If you’re aware of any reasons or causal factors, we’re all ears!

results — 30 Comments
2
Nov 07

Firefox’s Funnel Factor

As Alex pointed out in our previous funnelcake post, we shipped funnelcake01 on October 4th. While there are a lot of interesting findings that can be extracted from the data, we want to turn our initial focus to something we call a “funnel factor.” In the business world, there is a very basic process by which a customer (which we’ll also refer to as a consumer or user) engages with a provider of goods or services. This process is applicable across industries and across enterprise sizes, i.e., it applies to virtually all companies and their customers.

Here’s a rough outline of the process:

  1. A consumer makes a decision to purchase (or download) a product or service
  2. The consumer visits the seller’s store (online or offline)
  3. The consumer makes a purchase (or download)
  4. Thereafter, the consumer may or may not use (or install) the product
  5. Once the consumer uses the product for the first time, he/she then decides each day whether to continue using the product

These are the steps that we set-out to understand.

So, why is funnelcake so unusual, i.e., why is Mozilla’s methodology and openness remarkable in this case?

  • Within the steps above, you’ll notice that businesses have historically understood the flow of their customers through the first three or four steps (or between two certain steps), but not across all five steps simultaneously.
  • If I walk into a Gap store today and buy a shirt, they won’t know if/when I eventually wear that shirt for the first time, nor will they know how often I eventually wear it over time. Fortunately, the software industry is one arena (among many other industries and examples) providing ripe opportunities for understanding the full spectrum of interactions between a product and its customers.
  • Even if other software providers have figured out a way to optimally measure the flow and experience of their customers, I’m not aware of any marketers who have made such findings publicly available.

Thus, for an initial presentation of our findings, we’ve focused on the part of the funnel that has traditionally been most unknown: purchase –> initial use –> long-term use. Translating this path to Firefox creates what we’ve defined as our “funnel factor”: download –> install –> daily active use.

Here’s what we found:

funnelfactor

Of those visitors who downloaded Fx, 57% installed the browser. Then of this smaller cohort who installed Fx, 49% continued using it on a daily basis. We’re generating this latter number when looking at usage approx. 30 days after the download and install processes. For those curious, that 49% usage number was 59% after one day and 58% after one week had passed.

Taking these two numbers together and tying the “download –> install –> daily active use” steps together, we then arrive at our funnel factor. In other words, what is the percentage of downloaders who eventually become regular active users? We previously estimated each step as a 50% drop-off, translating to a 25% funnel factor (50%*50%). However, the results of funnelcake show an actual funnel factor of 28% (57%*49%). This means that, of users downloading Firefox, 28% become active daily users.

That sounds like a pretty good retention rate to me. The Firefox community deserves a great deal of congratulations for this type of success.

What are other funnel factors? Perhaps we’ll benefit more in the future as other software providers (or businesses in other industries) come around to being this open about the experiences of their users.

We look forward to continue sharing much more info related to our funnel… please stay tuned.