Comments on: Improving a User’s Experience with Firefox Support (part II)

By: Donnie Berkholz

Donnie Berkholz — Tue, 30 Jun 2009 04:24:38 +0000

You might want to try some real statistics to prove whether the two distributions are actually different. Try Student’s t-test.

By: Blake

Blake — Fri, 26 Jun 2009 16:10:21 +0000

Nice job with the experiment Ken! I’ve created a rough mockup of another SUMO homepage that we may want to test. I think Google does a pretty good job with their support pages and have borrowed a few elements from them, including the prominently featured “Top Articles” section and the link organization. It may also be helpful to use an auto-suggest drop down in the search box.

Mockup: http://farm4.static.flickr.com/3572/3663094710_57b89c1833_o.png

By: Majken "Lucy" Connor

Majken "Lucy" Connor — Fri, 26 Jun 2009 03:32:50 +0000

Can you tell how many people made use of the search suggestions?

By: JH

JH — Fri, 26 Jun 2009 02:33:36 +0000

Because an A/B test was run instead of an MVT, you won’t understand WHY recipe B won. Maybe it was the cloud, or the “new to firefox” section. I’m surprised Mozilla isn’t doing something more sophisticated than a simple A/B.

By: Pino

Pino — Thu, 25 Jun 2009 18:02:07 +0000

I think these figures really interesting and think it is a great thing you people test it in such a way what method works best.

The way I interpret the first table however is that the new page worked better overall, because the search box performed a lot better. The rest of the articles became less interesting on the other hand. This could be due to people normally doing “other” now doing a search, but it could also be new people searching and people who would click on the other pages on the old version, are no longer interested in the new version. (if you understand what I mean, lol) Have you experimented combining both pages, so that you keep the higher ‘other action’ from the old page and get the new search box?