Advertising Firefox on Wednesdays

We’ve previously talked about search engine marketing as one of the key marketing efforts here at Mozilla.  Those previous analyses have pointed to experiments we’ve performed that helped increase our efficiency by approximately 50% earlier this year (that’s pairing a reduction in total spend with an increase in total clicks).  While those are some great gains, we’re always considering additional ways to improve our efficiency, and ultimately the new user’s experience, through this marketing channel.

An example of a potentially untapped area for improvement is day/time parting, i.e., better targeting by day of week or hour of day.  Intuitively, it’s easy to imagine that a particular day (e.g., Fridays) or a particular hour (11pm PT) might see a lower price (cost per click) or differing interest among new users (click through rate).

Fortunately, this is a fairly easy hypothesis to test thanks to regression analysis.  With our AdWords account, we can grab hourly data and see what variables (e.g., particular hours of the day) cause the price/interest changes mentioned above.  Continuing our example, perhaps most advertisers run out of their daily ad budget by the end of day, causing late night hours to see a lessened competitive bidding environment, and consequently, a cheaper cost per click.  Regression analysis would be able to tell us exactly that – and tell us that perhaps we should be diverting more resources to such market inefficiencies.

Below is the regression equation and full regression output.  We used a recent data set from our primary campaign containing more than 4,300 hours, which equates to about six months worth of data.  Clearly our equation could be strengthened in different ways (more instances, more control variables).  For example, the results did improve when we substituted the day of week and month variables with a dayofweek*month interaction variable.  For any stats fans reading this, we’re open to other suggestions for improving our work (comments encouraged).

Cost per Click = α + β1hour_of_day + β2day_of_week + β3month + ε

Note: hour of day results are relative to the midnight to 1am hour (pacific time), day of week results are relative to Fridays, and month results are relative to April.

So, how do we interpret these results?

  • For hour of day, it looks like there’s little inefficiency to be found.
  • However, the day of week findings are generally very robust.  Tuesdays, Wednesdays and Thursdays are relatively less expensive, and Saturdays and Sundays are relatively more expensive.
  • At a macro level, these findings tend to point to a generally efficient market, meaning we likely won’t be changing our marketing decision making based on this information alone.

6 responses

  1. Ivan Ičin wrote on :

    Why the hell cost per click would scale linear with hour digit?

    This is case for ANOVA, not regression

  2. kkovash wrote on :

    Ivan — the hour variable is a dummy variable, i.e., the equation sees it as a string and not numeric character. I probably could have explained that better. Ken

  3. Ivan Ičin wrote on :

    It seemed like that from equation, I guess it is more clear in the table below, which I just scanned.

  4. Ivan Ičin wrote on :

    Hi,

    I would have use of applying this model on some data, but I haven’t learned exactly this model on my university course of statistics (only plain regression).

    So I was wondering if you could give me some info like some link, or the name of the test, or program that can output this results, or anything.

    Thanks very much.
    Ivan

  5. kkovash wrote on :

    Ivan — thanks for asking. There are many statistical programs that allow you to run a regression this way, e.g., SAS, SPSS, R, Stata, etc. Something as basic as Excel might even do the trick.

    The challenging part can simply be knowing the right command. For example, using Stata, I can run the regression above by simply entering the command “regress costperclick i.hourofday”. costperclick is my left-hand side (dependent)variable and hourofday is a dummy on the right hand side (independent variable).

    I think I’ve seen other programs where you would have to, in the example above, manually code each of the 24 hours of the day as a unique dummy, but once you’ve learned how your program handles this, you should be able to treat this as a plain regression.
    Ken

  6. Ivan Ičin wrote on :

    Thanks Ken,

    I was searching round for exact model name (as it is not plain regression) so I guess it should be this: http://en.wikipedia.org/wiki/Multinomial_logit