SUMO Blog

As most readers of this blog are already aware, Mozilla’s community-powered user support project SUMO was launched to address the need for an official and centralized support channel for Firefox that could scale to match the rapidly growing user base. While the ground work started back in 2007, the project really took off in 2008.

Looking back at the achievements we’ve made over the year makes me humble. Here are just some of them:

• We got first-hand insights about our Firefox users. With a combination of article polls and traditional website metrics like page hits and search & navigation patterns, we now have a deeper understanding of what our users are saying about Firefox. Every week, we provide a list of the most commonly reported Firefox problems, which is very useful information for the Firefox development, QA, and marketing teams.
• We grew our community. In 2008, the SUMO community doubled in size to over 50 active contributors per week. We still need to grow in order to keep up with the 200+ million user base, but it’s exciting to see that our Mozilla community is starting to understand the importance of user support.
• SUMO development took off. The SUMO development “team” went from what was essentially a one man show — Nelson Ko from the TikiWiki community — to an actual team of people. We also got a formal development lead: Laura Thomson from Mozilla’s web development team. We got high class QA attention from Stephen Donner. This all enabled us to move forward in a much quicker pace, which was something we were in great need of. In fact, we will need to grow even more to keep up with the ambitious plans for 2009. Interested in hacking on SUMO? Let us know!
• We launched the Support Forum. To create a support experience where the most common problems can be solved in the Knowledge Base and everything else in user to user support solutions, we launched the Support Forum as the first step to complete what later became known as the support funnel. Today, around 1,500 support questions are answered by our SUMO community every week.
• We launched Live Chat. Actually, Live Chat had its virgin session exactly one year ago, New Year’s Eve 2007, and it was immediately being picked up by lots of users and helpers, much thanks to some press attention that caught us by (pleasant) surprise.
• The Knowledge Base became localized. Initially, the Knowledge Base articles were only available in English. This changed in 2008, as we enabled support for multiple languages and got help from our incredible localization communities around the world. Some of the more active locales include French, Italian, Japanese, Portuguese, and German.
• We moved Firefox’s product help to SUMO. Starting with Firefox 3, the product help is now hosted online. As a result of the effort of moving the content over to SUMO, we engaged with Mozilla’s large localization community. With their help, we were able to translate the updated product help articles into 18 languages before the Firefox 3 launch.
• Last but not least, we became helpful. The Firefox Support website currently helps roughly 30,000 users every month solving their Firefox problems. Although that is a pretty impressive number in itself, it’s actually very conservative since we’re only counting the users that took the time to let us know that their problem was solved.

I’d like to thank everyone who contributed to the SUMO project in 2008. Without your help, we wouldn’t have been where we are today. Thank you!

Of course, the fun doesn’t end on this last day of 2008. We have lots of things planned for 2009, so please stay with us! I’ll blog about our plans for 2009 shortly. Until then: Happy New Year!!

Photos by Eskilstuna photographer & journalist David Naylor.

A couple of weeks ago, the bug fixes for SUMO 0.8 were pushed to
support.mozilla.com. This update has quite a few goodies!

New search engine

Over the summer, student Alexandre Michelis and mentor Nelson Ko worked on a Google Summer of Code project to create a search engine specifically for SUMO. In SUMO 0.8, the back-end code for the new search engine was implemented. After many months of hard work, we just need to do some final testing, then we are going to switch over to the new search engine. When that happens, we’ll post in detail of all the great advantages of the new search engine.

Support for Firefox 3.1 specific content

In the Firefox Support Knowledge Base, we use a great feature called “SHOWFOR” that allows articles to display different content, depending on the users’ operating system and version of Firefox. When it comes to version numbers, we have only been able to differentiate between Firefox 2 and Firefox 3 content. Now I am pleased to announce that we can now differentiate between Firefox 3.0 and Firefox 3.1! For more information on how it works, and updating articles for Firefox 3.1, you can read the previous blog post.

Template for new articles

So you’ve decided to create a new article. Great! But you open the article editor to start writing the article, and don’t really know how to format the article. We now have pre-filled markup and text in the article editor when creating a new article, that includes where to put the problem and solution, but how to format the problem and solution. This should make it easier for people to get involved, as well as improve the continuity of Knowledge Base articles. And….it’s localizable! I’ll be in contact with Firefox Support locale leaders with instructions, and if you don’t want to wait, email me or comment on this blog post.

Secure log in

When you log in or register, it is done through a secure (https) connection.

Forum performance

There has been quite a bit of work done to improve the performance of the forums, which has been a major issue.

There are more highlights that we post about soon. In all, SUMO 0.8 is the without a doubt the most significant SUMO release; and the SUMOdev team deserves a big applause for working so hard. You can see a full list of SUMO 0.8 bug fixes in Bugzilla. Remember that if you see any bugs on support.mozilla.com or you think of a great new feature for the web site, please let us know about it in the Contributors Forum or file a bug in Bugzilla.

Thanks to a tremendous effort by Wil Clouser, it is now possible to localize the SUMO (support.mozilla.com) site UI using Verbatim. For those unfamiliar with Verbatim, it’s an online translator tool for various Mozilla projects, including addons.mozilla.org. It’s designed to be easy to use, encourage help from casual contributors, and provide a clear overview of the translation progress.

The addition of SUMO support means that localizers can use Verbatim to translate the site’s UI rather than using the somewhat clunky built-in interface on SUMO itself. Note that this is strictly for UI localization, not for translating Knowledge Base articles. That’s still done by clicking on the “Translate this page” link on actual articles.

Here’s how the dashboard looks like for SUMO:

Verbatim is still in alpha, but if you want to test this out, please drop by at #verbatim on irc.mozilla.org right now and ping clouserw to get permissions (in Verbatim) to modify SUMO.  The actual translation process is still being worked on, but it’s already possible to start translating strings that will show up on the SUMO website once we import the changes (which will probably happen monthly).

Attendees: djst, cilias, zzxc

Sumo

• Weekly metrics
  • Lots of KB updates in part because of our friend Bo!
  • Would be interesting to see how many locales are being edited per week
  • djst to add a wiki page with the metrics we’d like to collect in the automatic dashboard
  • Bug with the bottom understandable articles list — different top articles depending on how many articles you display in the list. cilias to file bug.
  • Significantly lower bounce rate on both start pages. No good explanation; normal start page got an updated top issues lust, but what about the inproduct start page?
• Last week’s weekly support issues
  • Got crash logs from Live Chat for Google Desktop. zzxc to attach to relevant bugs.
  • Got lots of feedback from new users of Firefox 3. Top confusions:
    • After clearing history, things still showing up in Awesome Bar. The question is how much impact the addition of the Privacy preferences in 3.1 will have if people don’t understand what they’re seeing.
      • Ideas on how we can reduce the confusion users are experiencing; djst proposed a “notification bar” inside the Awesome Bar that would appear when there is no history, notifying the user that the list still list bookmarks, with info on how to change that behavior. That could hopefully make it more obvious to the user that the Clear Private Data feature isn’t broken.
    • People are confused by the UI for adding a bookmark to a specific folder. Needs to click two small arrow buttons to select sub-folders (one to show folder selection, and another to expand a parent folder).
      • zzxc to summarize the feedback
• Start page layout looks crappy [1]

2008 review

• We’ve come a long way since the beginning of 2008:
  • From en-US only Knowledge Base to over 20 active locales
  • Moved in-product help to web based help on SUMO
  • From launch of Live Chat to over 20 active helpers per month
  • 50 active SUMO contributors per week
  • Helping at least 30,000 users of Firefox every month (these are people that actively said “yes, this solved my problem”)
  • Robust metrics about common issues in Firefox reported to QA/dev teams every week
• Room for lots of improvements in 2009:
  • Performance is a top complaint among contributors — top priority in Q1 2009
  • Better communication with members of the SUMO community. Personal outreach, e-mail, Skype meetings, etc. The SUMO community is part of something really important that ties into everything else that we do at Mozilla; we need to communicate that. MozillaCMS is interesting for SUMO.
  • Dig deeper into our data — need an automated metrics dashboard for the weekly metrics so we can focus on digging deeper
  • Preliminary 2009 roadmap is up on the wiki [2]

Knowledge Base

• Private Browsing article is up
• Lots of updates for 3.1 content
  • Big thanks to our friend Bo, who has been doing most of the updating!
• In Firefox meeting I asked about what is being about Smart Location Bar
  • Drop-down setting in Privacy panel to set suggestions to “History and Bookmarks”, “History”, “Bookmarks”, or “Nothing”.
  • UI was recently put in nightlies, but pref |browser.urlbar.search.sources| has been in about:config for a while. It should be in Beta 2.

Live Chat

• Contribution way up with the 3.0.5 release, total chat sessions almost doubled
  • 10 new accounts last week, out of which 5 helped with at least one chat
  • New account approvals: mindstormskid, tom06, listerofsmeg, mcgiwer, dan
  • New room monitor. Welcome to the team, Tanner!
• Live Chat CSAT is now collecting data. Several users have replied to us in the forum using the forum link on the CSAT page.
• Limited hours this week due to holiday

Roundtable

• Thanks for a fantastic year and Merry Christmas to the SUMO community!
• Meeting as usual next Monday

In the Firefox Support Knowledge Base, we use a great feature called  “SHOWFOR” that allows articles to display different content, depending on the users’ operating system and version of Firefox. When it comes to
version numbers, we have only been able to differentiate between Firefox 2 and Firefox 3 content. Now I am pleased to announce that we can now differentiate between Firefox 3.0 and Firefox 3.1!

What this means is that now we can start updating current articles with Firefox 3.1 content!

I’ve updated our contributor reference on how to use SHOWFOR. Content currently tagged with {SHOWFOR(browser=firefox3)} is now displayed for both Firefox 3.0 and 3.1 users. To tag content as specific to 3.1, use {SHOWFOR(browser=firefox3.1)}. To tag content as specific to 3.0, use {SHOWFOR(browser=firefox3.0)}.

For example:

{SHOWFOR(browser=firefox3)}

This content would display for 3.1 and 3.0 users, but not Firefox 2 users.

{SHOWFOR}

{SHOWFOR(browser=firefox3.1)}

This content would display for Firefox 3.1 users only.

{SHOWFOR}

{SHOWFOR(browser=firefox3.0)}

This content would display for Firefox 3.0.x users only.

{SHOWFOR}

{SHOWFOR(browser=firefox2)}

This content would display for Firefox 2 users only.

{SHOWFOR}

So what content needs updating?

We’ve been keeping track of user-end changes in Firefox 3.1 using a page on wiki.mozilla.org, then tracking which articles need updating using a page on support.mozilla.com. Remember that Firefox 3.1 is still in development, so the list of changes is still likely to grow; and if you notice a user-end change in Firefox 3.1 that is not in those pages, you can update them.

What about screenshots?

The Firefox 3.1 user-interface is not final either; and it is too easy to include a screenshot that contains UI that will be different when Firefox 3.1 is released. Do not add Firefox 3.1 screenshots for now.
I do not expect to have clearance for this until there is a release candidate.

Let the updating begin!

On Tuesday December 2nd, the bug fixes for SUMO 0.7.3 were applied to the Firefox Support web site.

One new feature that should help us get a better handle on forum responses is the ability to classify response as actual attempts to help the user, rather than something like a “me too” post or someone clarifying the problem.  Specifically, you can classify your response as a proposed solution to the problem or a request for more information. We can then have more accurate numbers on how many threads are actually getting help; and contributors and filter threads based on the whether or not a solution has already been proposed, using the “Solved Status” drop-down menu mentioned in the SUMO 0.7.2 post.

To classify your response when you compose a forum response, click on the drop-down menu just above the text area, called “Select purpose of your response:” and choose the most fitting item.

In addition, there was a major bug which caused new articles not to be assigned to any language, and prevented contributors from removing the “Content may be out of date” warning on some articles. That bug has now been fixed. You can test this out by creating a new article. As a result, localizers should now be receiving notifications of new translations again.

You can see a full list of SUMO 0.7.3 bug fixes in Bugzilla. Remember that if you see any bugs on support.mozilla.com or you think of a great new feature for the web site, please let us know about it in the Contributors Forum or file a bug in Bugzilla.

Attendees: djst, cilias, zzxc, cww, lucy

Sumo

• Weekly metrics
  • Bounce rate on start pages consistently high. chofmann, djst, and jslater are working on improvements.
• Last week’s weekly support issues
  • New Bugzilla keywords are now added for the SUMO/QA common issue tracking teamwork. Will evaluate for a brief period then announce.
  • common-issues bugs
• About this site page in review bug 413592

Knowledge Base

• Almost 200 article comments (140 since Wednesday night)
  • Caused by the major update
  • cilias will post useful comments in Contributors forum
• Filed bug 467814 for a separate article about places being locked.
  • For clarification: unchecking the “Read-only” attribute not the solution. Solution is to rename (or delete) places.sqlite and import a backup
• 3 new articles to review this week.
• “Images or animations do not show” overhaul waiting for review [1]
• cilias will post summary of l10n feedback shortly
• How to handle Firefox 2 content discussion [2]

Forum

• Users who are answering without logging in or registering. Probably a good thing, will have better stats when we start tracking statuses.

Live Chat

• Openfire server problems should be resolved now
  • Sometimes Foxkeh wasn’t showing
• Archiver broken by update, not possible to get last week’s weekly metrics
• Stats from last week: 4 new accounts, 1 account approval, 1 new user ready to be approved
• Top issues (cww to merge this with Weekly Issues doc)
  • Bookmarks in the Smart Location bar after history is cleared
  • Firefox crashes while browsing, downloading, or printing: 9
    • Plugins and extensions seem to be the most common causes
  • Passwords not saved: 4
  • Corrupt cookies, logins not saved

Roundtable

• We need to get a tools server set up so we can do automated weekly metrics and other stuff on a dedicated server: bug 467829
• bug 467018 causing trouble. zzxc to test and provide more info, and djst to bring this up during the sumodev meeting.
• Blog schedule for this week is empty/open.
  • Publish an official blog schedule for the SUMO blog?
    • Doesn’t seem like it’s needed considering the team size and blogging frequency. We just want to make sure everyone doesn’t blog the same day.

I’ve talked before about the so-called Support Funnel and how the Knowledge Base is the heart of SUMO, ensuring that people find solutions to the most common problems without necessarily having to interact directly with our support community. The reasons why this is important are many:

• It ensures that the solutions to the most common problems are written in a clear, concise, and straightforward language that is easy for our users to understand.
• It reduces the pressure of our community of volunteers since most users are self-served.
• It gives us a powerful way of tracking which problems are the most common with the help of metrics.

The Support Funnel. For more information, see The vision for SUMO – Part 2: Understanding the bigger picture.

So, how do we ensure that people find the solution to their problem in the Knowledge Base? Well, one way of finding the solution is by searching for it — something we try to make very obvious on the start page. The reason why we think searching is the best way of finding the solution is because the Knowledge Base is big. Really big. While we have a list of the most popular support articles right there on the start page, it’s hard to make it obvious that there is a lot more content in the Knowledge Base than what is shown on the start page.

The start page of Firefox Support, clearly emphasizing on the search function.

So, is everyone really comfortable searching? Actually, the almighty and ever so wise chofmann and I have started to see evidence that some people prefer to browse for the solution rather than searching for it. Among the people that visits the Firefox Support start page and doesn’t instantly leaves the page, only roughly half of them actually searches. The other half either clicks on one of the hand-picked popular support articles, or clicks on some other link on the page.

We’re not sure what the reason for that is, or if there are several reasons. It could be that people are unable to accurately describe the problem they’re seeing — considering how many people that are using Firefox today, this is not surprising. Even my older brother, who has been using computers for at least ten years, has problems describing some of the problems he has with his computer, and usually I have to pay him a visit, have a nice cup of coffee, and fix the problem myself.

Another reason could be that people simply prefer to browse a categorized list of articles instead of searching — essentially just clicking on a few links instead of actually typing. I talked to our creative genius John Slater a few weeks ago and he said that he’s usually a little skeptic about internal search engines and that he prefers to just browse.

Francisco Picolinni from the Mozilla Hispano community provided a third possible reason why people are unwilling to search — because they might not think anyone else has the same problem.

Regardless of why not everyone searches, it seems like we should work on providing a good way of browsing the Knowledge Base. We currently have a link at the bottom of the list of popular support articles saying “Browse all Knowledge Base topics.” However, the page that link takes you to is just a long list of all articles ordered by hit count — not exactly easy to navigate.

chofmann, John Slater and I recently brainstormed around how we could ensure that as many people as possible find the solution to their problem in the Knowledge Base with minimum effort. Since the Knowledge Base articles are loosely connected with tags like “bookmarks,” “location bar,” etc., one interesting possibility would be to show these tags in a tag cloud that would provide a better sense of the diversity of the content while still pointing to specific, popular topics. Clicking on a tag would filter the list to only show the articles with that particular tag.

A tag cloud around the topic Web 2.0.

Another problem to solve is making sure that people really understand that they can browse for solutions as an alternative to searching. We want people to understand that Firefox Support has the answer to their problems and that they should stay on the site until the problem is solved. This probably means we have to take a closer look at how the start page is designed to see how we can better communicate this. If you have ideas on how we can achieve this, we are interested in hearing them!

What we really want a user with a problem with Firefox to feel when visiting Firefox Support is: “These people are here for me, and they won’t give up until my problem is solved.”

We just started to work on this, so stay tuned for more…

Attendees: djst, cilias, cww, zzxc

Sumo

• Weekly metrics
  • Not as much KB traffic as before — localization saturation?
  • Number of contributing locales would be interesting to monitor. cww to test if it’s possible.
  • New metric: Active forum contributors — contributors who have posted in at least 5 threads last week
• Last week’s weekly support issues
• cww has meeting with timr about QA/SUMO common support issue system

Knowledge Base

• For IE users + ActiveX articles updated
• cilias to update the Running Firefox on mobile devices article and also add info about iPhone. djst to ask PR for an official reason why we can’t/won’t/aren’t allowed release Fennec on iPhone.

Forum

• Would be cool if the #sumo bot could track how many people click on its links to the Forum — could write a script for it (but would depend on better hosting of stuff like this on Mozilla’s own servers)

Live Chat

• Traffic was up last week despite Thanksgiving.
• 5 active contributors last week
• 6 new accounts, 2 actually helping

Roundtable

• Need SUMO tools server for stuff like foxkehbot, stats, queries. djst to talk to Laura about it.
• We should have a welcome e-mail that’s sent out to people who register on SUMO (or Live Chat, which is still technically a separate registration). zzxc to write email template that we can use to send manual e-mails for now, until we have this set up automatically.
• SUMO logo is going through the legal process until we can take it to the next step.

A couple of weeks ago, the Firefox Support web site was updated to SUMO 0.7.2, which includes a number of bug fixes and a couple of new features. You may have already noticed some improvements.

If you have started a forum discussion, and get the answer you were looking for, you can mark the discussion as solved and rate the experience with solving your problem. It’s a great way to find out how satisfied users are with the forum.

Contributors looking to help people in the forum will then see that the discussion is marked as solved, and know to skip over it.

You can even set the forum to only display discussions with “Solved” or “Unsolved” status! There’s a drop-down menu called “Solved Status” at the bottom of the forum.

There is now a link at the top of forum pages (i.e. breadcrumb) to take you back to the forum list.

For a full list, you can see our list of SUMO 0.7.2 bug fixes. Remember that if you see any bugs on support.mozilla.com, or you’ve thought of a great new feature for the web site, you can let us know about it in the Contributor Forum or file a bug in Bugzilla.