SUMO Blog

One of our goals over the past few months was to get the knowledge base average CSAT score up to 85%.

CSAT is short for Customer Satisfaction, and the way we measure it is with the “Please rate your experience with solving your problem on support.mozilla.com from 1 (very unsatisfied) to 5 (very satisfied).” poll that appears on each article. This is the primary metric that we use to measure the success of the Knowledge Base.

One of the ways we tackled it was to take some of the larger articles that cover generic subjects like bookmarks or cookies, and split the content up into individual articles addressing specific questions seen in the list of top searches.

Recently, “cookies” became our top search term, so we decided to split up that article first.

Over the month before we split up the Cookies article, it had a CSAT score of 4.36 (87.2%).

Our top cookie related search terms cover 3 tasks:

• Enabling and disabling cookies – which covers cookie settings and how to make sure cookies are enabled when a website tells you that you need to have cookies enabled.
• Deleting cookies – which covers how to clear cookies for a single site and how to clear all cookies.
• Blocking cookies – which how to block individual websites from storing cookies.

We created an article for each of those tasks. As a result, each article was shorter and to the point. The average CSAT score of all three articles plus the original since the split is 4.5675 (91.35%).
If we take the original article out of the equation, the CSAT score rises to 4.71 (94.2%).

This action was one out of many that we’ve taken, which has risen the average score, and we’re not done yet. The Bookmarks article has been split, and we’re looking into the need to split up the Pop-up blocker article and maybe Tabbed browsing.

The lesson here is to let the search terms dictate how we spread out our documentation, so the article to question ratio is 1:1.

A list of top 15 search terms can be found in the SUMO Weekly Metrics, and if anyone wants to see a full list of our top search terms, just post in the Contributors forum, and any SUMO team member will be happy to provide the information for you.

For the past few weeks we have been gathering a list of changes from Firefox 3.5 to 3.6. The next step is to go through the list of articles and determine which articles need to be updated for each of those changes, and everyone can help. If you want to help updating the knowledge base for Firefox 3.6, here’s what to do:

1. Go through the list of articles.
2. Look for content that needs to be updated according to the changes we have gathered. Some notable changes to look for:
   • The Main preferences panel has been renamed to the General preferences panel, so any references to the Main panel will need to be updated.
   • Thanks to the great work by Curtis Bartley and other Firefox developers, any documentation pointing users to their profile can be updated to make use of new about:support page. (e.g. “Open the Help menu and select Troubleshooting Information…. In the resulting page, click Open Containing Folder.)
3. For each change in Firefox, list the article that need to be updated on this wiki page.

In a couple of weeks, once we have got a good idea of the affect each change has on our documentation, we can then decide which updates can be automated. If you have any questions, just ask in the Contributors forum thread. Thank you!

Facebook has become an extremely popular social network the last couple of years — and we all know how quickly Firefox is growing its market share. Therefore, a little deductive reasoning tells us that many people must use Facebook with Firefox.

Last month we started to see a large increase in the amount of Firefox users having problems with Facebook. To help those users, we created a knowledge base article specifically for problems with Facebook. However it is a very generic article that suggests general troubleshooting solutions. Facebook users are having specific problems, like not being able to enter text in their status, or Facebook chat not working.

We would like to improve the article by listing each problem and its corresponding solution, but we’ve had some difficulty pinpointing the specific solution for each problem. Maybe one or two Facebook applications are causing a certain issue?

If you have experienced any problems in Firefox when using Facebook, we’d like to hear from you! Post whatever information you can provide in this Contributors forum thread. You need to log in to post, so if you don’t already have an account, you can register here. Thank you for any help you can provide.

(And if you’re on Facebook and contribute to SUMO, remember to join the Mozilla Firefox page and the SUMO Contributors Facebook group.)

Last week, Microsoft released Windows 7, the latest version of the Windows operating system. The majority of our visitors to Firefox Support are Windows users today, and “Windows 7” is one of our most popular search terms because users want to know if Firefox works on Windows 7.

We need to make sure all the information in the knowledge base provides them with correct information, and is user-friendly to Windows 7 users. This will be especially important when the EU browser ballot is deployed in Europe, because we expect to see a large increase in Firefox users running Windows 7. Many of those users will probably be computer novices that have never heard of the term web browser before.

If you are a Windows 7 user, we could really use your help going through the knowledge base articles and testing the instructions on each article. If you see anything that needs updating, we can show you how to edit an article. Even for instructions that are correct, we would still love to get help with updating screenshots and screencasts so they are for Windows 7 instead of XP/Vista.

To get started, we have instructions on how to edit articles, and a list of articles to go through. If you have any questions, just ask in the Contributors forum. Thanks for any help you can provide.

Firefox 3.6 is the next significant Firefox milestone release, and the first beta release is coming soon. We need to start thinking about what updates we need to make to the knowledge base, and plan how to deploy those updates.

The plan itself is not set yet. First, we would like to gather a list of changes from 3.5 to 3.6 that affect user support. Once we have a good idea of what has changed in Firefox, we can:

2. Assess what changes need to be made to the knowledge base.
3. Establish which changes can be done using a script that will automatically update all articles.
4. Draft an update plan.

I’ve started a wiki page on wiki.mozilla.org to gather a list of changes in Firefox 3.6. If you know of any changes not on the list that affect user support, please add them to the page with a link to relevant information (bug number or blog post).

If you have any questions, just ask in the Contributors forum thread. Thank you!

The primary method of navigating the Firefox Support website is through search. For all the documentation we have on the website, it isn’t worth much if users cannot find what they are looking for. Earlier this year, we switched to a new search engine primarily because it gave us greater control of which items appeared in search results for any given search term.

Terms like “bookmarks” and “cookies”, which are too vague for us to know what is being asked, are especially important because they are among our top search terms.  If we know the most frequently asked questions, we can make sure the top search results for generic terms are representative of the most frequently asked questions associated with those terms (e.g. if the most common question about bookmarks is “How do I back up my bookmarks”, we should make sure the ‘Backing up and restoring bookmarks‘ article is the first result when searching for “bookmarks”).

Sometimes users use bad terminology as well. A common issue since Firefox 3 was released is that users want to clear items in the Location bar. After clearing browser history, the Location bar still contains bookmarks, which causes users to think that clearing history does not work.

Figuring out the most frequently asked questions among Firefox users is harder than you would expect. Data from the support forum and live chat is gathered on a weekly basis, and documented in our weekly common issues page. However we funnel users through the knowledge base before they reach the forum/live chat. The issues people are having there are good for finding new issues not covered in the knowledge base, but if the most frequently asked questions are already covered in the knowledge base, the most frequently asked questions are theoretically not being posted in the forum/live chat.

A couple months ago, Cheng, Matthew, and I went through our metrics and page view data to try get a good grasp on the most frequently asked questions (keeping all of the above in mind). Most of it was based on the “Top articles searched for” metric and “Top articles by score”, then we used the top searches and data from article feedback comments to fill out the rest of the list and clarify inconsistencies. We came up with this list. We can’t be sure that it is 100% accurate, but we think it is reliable enough to use for manipulating search results.

1. How do I use Private Browsing?
2. How do I enable cookies?
3. How do I set the homepage?
4. How do I clear my private data?
5. How do I clear the location bar?
6. How do I make Firefox the default browser?
7. Pop-ups are not being blocked.
8. Username and password is not remembered.
9. Firefox takes too long to start up.
10. How do I export bookmarks to an HTML file?

Using that list, we can list common search terms for each question, and make sure the articles that directly address those questions are tagged with those terms and have the right keywords.

Anyone who wants to help can try testing search terms to see which article appear first in the search results. If you don’t think the search results are optimal for your search terms and you know which articles should be more prominent, try editing the article to add the appropriate tags.

Last Thursday, the bug fixes for SUMO 1.3 were pushed to support.mozilla.com. The big focus was on productization, and we also managed to add some other great changes.

So what is productization? From the beginning of the project, the plan was to create a support solution for Firefox, but do it in such a way that makers of other software could use SUMO for their own support sites. In the future, Fennec (the mobile version of Firefox) will have its own support site, using its own implementation of SUMO. In SUMO 1.3, major work was done on separating the code that is specific to Firefox Support from code that can be used for other products. In addition, work was done on turning the code into something you can install yourself. If you would like to try it yourself, we have instructions on wiki.mozilla.org.

Somehow, in all of that work, we managed to add a significant new feature: Article review email notifications. We’ve said in many previous blog posts that our knowledge base works differently from other wikis. All edits go through a review process before being public. For new contributors, who have not visited the Contributors forum, this is a major point of disconnect. When reviewing edits, it is important to state why you are approving or rejecting an edit. Even a simple “Thank you” message can go a long way.

In SUMO 1.3, we take an approach similar to posting to a moderated mailing list. When a contributor edits an article, an email is sent to the contributor, saying that their edit is waiting for review. The reviewer can approve or reject the edit, and a text box is provided for the review to give their feedback to the contributor. When the edit is reviewed, an email is sent to the contributor, telling them the result of the review, including the message from the reviewer. Just as important is the fact that all of these messages include an invitation to the Contributors forum.

This new review notification feature should make a huge difference in growing the SUMO community. Instead of newcomers making one edit and leaving (oblivious to the fact that their edit is not public yet), this gets them thinking about how SUMO works, connects them to the rest of the community, and gives us a chance to personally thank each contributor.

The full list of SUMO 1.3 bugs is on Bugzilla. This release has over 90 bugs, which is the most ever in a SUMO release (around three times more than most SUMO releases). A big thank you goes to Paul Craciunoiu, James Socol, Eric Cooper, Stephen Donner, Vishal Kamdar, Krupa Raj, and SUMOdev lead Laura Thomson for an extraordinary amount of work. Not to mention IT for all their work with the release push. If you have any questions, please feel free to post them in the Contributors forum, or join the #sumodev IRC channel to thank them yourself.

One of the most frustrating thing for users is to have Firefox crash on them. As we’ve discussed before, we’ve been writing a series of articles to document the top crashes and get them searchable by crash signature.

Just documenting the top 50 crash signatures will greatly improve the experience of users coming to SUMO. It will also help give us somewhere to point users to from the forums or live chat. Towards that end, we’ll be having a one-day sprint to write these articles and get them into the knowledge base and we could use your help!

On September 3rd, we’re going to get together to write as many of these crash articles as we can. We’ll have members from Mozilla’s QA team as well as platform and branch development teams on hand over IRC to help with understanding crash bugs or stacks and answer any questions you may have about the crashes. We’ll be tracking our progress on this page.

We look forward to seeing you there!

Helping users troubleshoot crashes has always been a hard thing to document in the knowledge base. We try to help the user better define the circumstances of their crash, then list possible causes and solutions for those circumstances. An (obviously) unintended consequence is that there is so much information to digest, it confuses users.

The problem is that if Firefox crashes, it could be for any number of reasons. The range of causes and volume of troubleshooting is so great that we end up doing more to try help the user navigate crash documentation, than offer a solution. In most cases the solution is vague and not very helpful, which confuses users even more.

Each Firefox crash reported to Mozilla using the Mozilla Crash Reporter has a crash ID and lists the type of crash, called the crash signature. Usually, each crash signature has a much more specific cause/solution. Instead of asking users to define each circumstance of their crash, they can get to the solution more quickly if we ask for the crash signature, then provide a document for each crash sig.

What we’ve done is turned our Firefox crashes article into a tutorial on accessing your crash report via about:crashes. At the top right corner of each crash report on crash-stats.mozilla.com, you’ll notice that there is a [Get Help] button. What that does is search support.mozilla.com for the crash signature from the report.

By creating an article for each crash signature, and putting the crash signature in the article content, the Get Help button on crash-stats.mozilla.com will provide the user a link to the article that addresses their specific type of crash.

Naturally, there are a lot of crash signatures. We can’t provide an article for every crash. However, we can get a list of the most common crash signatures, and try to make sure the 10 most common crashes have articles in the knowledge base. Ever since Firefox 3.5 was released, we have been keeping an eye on the top Firefox 3.5.x crashes, and adding them to a list here.

What we need now is people to draft an article for each crash sig. There is a bug link for each crash sig on the list, that contains details about known causes/solutions for each crash. If you need help creating articles, we have a contributor page about creating articles. For any further help, just ask in the Contributors forum.

Back in May, we came together and worked on a user manual for Firefox as part of the FLOSS Manuals Project, an open-source book publishing project. It was a huge success and we now have a great Firefox Manual you can download, read online or get as a nicely bound paper copy.

Now that Firefox 3.5 is out, it’s time to upgrade the manual as well. As a community, we’ll be getting together and making sure that the new manual reflects all the changes and improvements that went into this new browser and that all the content is current. It’d be great to have you join us and help make sure the manual is up to date.

When?

Thursday, July 16 at 10 AM Pacific time, 1 PM Eastern time, 19:00 Central European time. If you’re in Europe or Asia and want to start earlier, you are of course more than welcome to!

Where and how?

1. Register to get an account at the FLOSS Manual site.
2. Go to the Write section of the Firefox Manual.
3. Pick a chapter that you want to read, click the “edit” link, and start updating it!

On the right side of the website is a chat window where you will be able to chat with other participants.

What needs to be done?

Many pages need just small updates, new screenshots or changes to be updated for Firefox 3.5. Here are just some of the things that you can do:

• Update number of users and number of locales on the Introduction page
• Update graphics (number of total users) on the Open Source page
• Mention this sprint in the page about the manual
• Update screenshots in all chapters
• Discuss new Clearing Recent History features (A new chapter replacing part of this one on the URL)
• Discuss new Private Browsing features (may integrate with above)
• Discuss new features (tab tearing) in the Tabbed browsing page
• Read through full manual to correct minor changes

Just 20 minutes of your time will help keep our documentation relevant, current and useful for all Firefox users, and by making edits, your name will forever in the Credits section of both the online and paperback version.

See you Thursday!