For those of you who didn’t go to the Mozilla Summit (which was amazing BTW) or didn’t see my presentation, I wanted to recap it for you because it’s the basis for some of the things I’d like to do with the Knowledge Base moving forward.

Last quarter, we worked on finding ways to increase the helpfulness of our articles by 2%. We started to run some multivariate tests but found that they’d take too long to give us results that we could use. So about 2 weeks before the end of the quarter, we decided to try rewriting some of our most popular articles. Instead of running this test through the metrics team’s tools (and only sending a fraction of the SUMO traffic to each article) we just made the new articles live for everyone. This allowed us to get enough results from the survey at the bottom of each article to get some meaningful results.

What we found was that the rewrite increased the helpfulness of these articles by over 8% translating into helping about 800,000 more people each year. This is really important for us because helping more people with knowledge base articles is the only way we can keep up with our 400 million (and growing) users in dozens of languages.

So here are some examples of the techniques I used in the rewrite of the How to set the home page article. You can see the slides from the presentation here. The main idea I focused on was to use techniques that keep your brain engaged. These mostly involve trying to keep things sounding like an actual human conversation which is more difficult than you might think.

Conversational writing style
Before:

Firefox shows the home page when you start it. You can change the home page to any web site, several web sites, or even a blank page.

That’s accurate information.

After:

Setting your home page in Firefox is easy. Can’t decide on just one page? No problem. Firefox lets you set a group of websites as your home page. This article will give you some examples and step-by-step instructions for customizing your home page settings to best fit how you work.

This version sounds a little more friendly. It also doesn’t try to list everything the article explains but instead gives you an overview of the types of information included. This way a reader can quickly say, “Yes, I want examples and step-by-step instructions.”

Humor, emotion! and motivation
Using humor is great if you can make it work. In our case, it’s a little difficult because we translate everything into dozens and dozens of languages. I cut many of the funny lines I tried to write because they would be difficult or impossible to make work in another language. In one draft of this article I had a nice pun (if you like puns) about keeping “tabs” on things. David – the native Swedish speaker – pointed out that by definition, English puns don’t work in other languages. But there are other emotions you can touch. “I didn’t know that!” or “I Rule!” are a couple.

Example:

This is the way to have one click access to all of your favorite websites. For example, you can set the Firefox home button to open your email, favorite news site and Facebook all at once.

When you can take something complex and break it down into simple chunks or show someone how they can make their life easier, you give them the experience of kicking ass. People don’t care so much that you can set 52 pages as your home page in Firefox. What people care about is that you can open your email, news and Facebook by clicking one button.

Multiple learning styles and repetition

I want the big picture
Give me an example – how will this help me?
Just give me the steps to get it done.

Here we get to kill two birds with one stone. We address multiple learning styles while repeating things. If you can repeat information in different ways and use different media if possible it has a better chance of sticking in somebody’s brain. In this section of the home page article we quickly give the big picture, an example and some step-by-step instructions.

Set more than one website as your home page

This is the way to have one click access to all of your favorite websites. For example, you can set the Firefox home button to open your email, favorite news site and Facebook all at once.

1. Open a new window and load up the first website you want to be your home page.
2. Click the new tab button and open the next website you want to be part of your home page group. Repeat this step until you’re done opening all the pages that you want in new tabs.
3. At the top of the Firefox window, click on the Tools menu, and select Options….
4. Select the General panel (Main panel in Firefox 3.5).
5. Click on Use Current Pages.
6. Click OK to close the Options window.

Assignments, Challenges, Questions
Another natural part of conversations is a little back and forth. You don’t usually just talk at people, you ask questions, they respond – it’s a dialog and that’s something that keeps your brain working. Also, just because you gave someone step by step instructions doesn’t mean they actually did them. If they’re like me, they’d have just read over them at least once to make sure you weren’t going to tell them to do anything crazy. This is an opportunity to give them a little challenge. Reading about how to do something is different than actually doing it.

Try it out: Go ahead and close all of your tabs and click the home button. Watch all of your chosen pages open up in tabs. Do it again; you know you want to.

But people don’t read webpages!
They scan them for keywords. They look for the pieces that are important to them (and if they don’t see them right away they hit the back button).

That’s all true but it doesn’t mean we can’t make those little bits that they do read engaging and effective. With practice you can do a lot with just a single sentence. The weekend before my first day at Mozilla I was reading a wiki page for new hires and there, under the Don’t Panic section, was written, “People here are busy, but they’re also really friendly, and they have all gone through it before. Ask. Learn. We already like you, don’t worry.” That last little sentence did it for me. It made me laugh and I called my friend to read the paragraph back to her. It let me know that I was in the right place.

Wrapping up
After the presentation we had a good discussion about the difficulties of localizing language like this. It was suggested that some good guidelines for localizers are needed and I agree. In fact my plan for this quarter is to work on the contributor documentation – especially guides for writing and localizing. This presentation is that first part of that. So what do you think? Let’s talk about it in the contributor forum.

To help get the goal discussion started, it’s obviously helpful to know why SUMO exists. In my opinion, there are three main reasons:

1. To help people have a great Firefox (and by extension web) experience
2. To provide key user and product insights to the Mozilla community
3. To strengthen and grow Mozilla’s community

Based on this list, we can create three focus areas, or “buckets” for our 2010 goals:

• Improve the support experience for users
• Provide better/more accurate/more detailed metrics and insights for other Mozilla teams and the entire Mozilla community
• Make the SUMO experience more enjoyable for contributors

In today’s SUMO meeting, we’ll kick off the discussion by spending 15-30 minutes brainstorming ideas. You’re very welcome to call in! That said, if you don’t have time to call in, or would rather share your ideas in writing, there is an active thread in the SUMO contributor forum dedicated for this. Please feel free to post there with thoughts, ideas, or, if you’re feeling particuarly creative, screenshots, mockups or screencasts of what you have in mind.

Of course, if you don’t have a SUMO account and for some reason don’t want to create it, you are welcome to participate by commenting on this blog post too.

The problem is that if Firefox crashes, it could be for any number of reasons. The range of causes and volume of troubleshooting is so great that we end up doing more to try help the user navigate crash documentation, than offer a solution. In most cases the solution is vague and not very helpful, which confuses users even more.

Each Firefox crash reported to Mozilla using the Mozilla Crash Reporter has a crash ID and lists the type of crash, called the crash signature. Usually, each crash signature has a much more specific cause/solution. Instead of asking users to define each circumstance of their crash, they can get to the solution more quickly if we ask for the crash signature, then provide a document for each crash sig.

What we’ve done is turned our Firefox crashes article into a tutorial on accessing your crash report via about:crashes. At the top right corner of each crash report on crash-stats.mozilla.com, you’ll notice that there is a [Get Help] button. What that does is search support.mozilla.com for the crash signature from the report.

By creating an article for each crash signature, and putting the crash signature in the article content, the Get Help button on crash-stats.mozilla.com will provide the user a link to the article that addresses their specific type of crash.

Naturally, there are a lot of crash signatures. We can’t provide an article for every crash. However, we can get a list of the most common crash signatures, and try to make sure the 10 most common crashes have articles in the knowledge base. Ever since Firefox 3.5 was released, we have been keeping an eye on the top Firefox 3.5.x crashes, and adding them to a list here.

What we need now is people to draft an article for each crash sig. There is a bug link for each crash sig on the list, that contains details about known causes/solutions for each crash. If you need help creating articles, we have a contributor page about creating articles. For any further help, just ask in the Contributors forum.

There are few sources of information to we draw from:

• Top searches: The most common search terms in the SUMO Weekly metrics document.
• Weekly common issues: Our Weekly Common Issues page tracks the most common support issues each week.
• Article polls: At the bottom of each article, there are poll questions: “Did this article solve a problem you had with Firefox?“, “Was this article easy to understand?“, and “Please rate your experience with solving your problem on support.mozilla.com from 1 to 5” (For more precise data there’s the PageView Data.)
• And of course, Article comments: There is a text field on each article for users to provide feedback about the article. When logged in as a contributor, that feedback is displayed at the bottom of the article.

Here’s how that data is utilized to measure the quality of the knowledge base, and make it better:

The top search terms are tested to find out if the first search results contain the article the user is most likely searching for.
If they don’t:

• The correct article may need to be renamed to match the search term.
• The top article in search results may be mistaken for a different issue; so a link to the correct article is added in the intro of the first search result. If users are being redirected to the correct article, the poll data should improve.
• Keywords that match the search terms are added to the correct article.

For generic search terms the article comments for each result may clarify what users are asking about.

The weekly common issues page is checked for any items that need documentation in the knowledge base. If enough information is available to create documentation, the relevant articles are updated or a new article is created.

The comments in articles with the lowest understandability score are checked to get details on what is not understandable in the article, so we can assess what can be done to eliminate that confusion. Sometimes that means rewording or reformatting the article. In some cases it is a matter of adding screenshots. In other cases, it’s a matter of streamlining or purging the article to simplify it for users.

In the end, it’s about taking the data, analyzing why the data is what it is, and what we can do to improve each issue. As a result, the article poll scores should go up, and users will get answers to their questions about using Firefox. We’ve outlined these tests in a contributor page, so everyone as a community can be most affective in making the knowledge base better each week. You can post any suggestions for improvement in the Contributors forum.

There can be many different reasons why people leave a website without interacting with it. When it comes to a support website, one of those reasons can be that the website isn’t helpful enough, or doesn’t provide sufficient instructions on how it should be used. This is something we are trying to minimize on SUMO so the support platform becomes as easy to understand as possible.

In order to improve the current start page, our first step was to figure out how people are using it today and identify areas where we could improve it. chofmann dug up a lot of the initial research about common web design mistakes which our start page was suffering from, and proposed some ideas on how we could use those insights when redesigning the page. chofmann and I then sat down and brainstormed about how we could improve the page, after which I created a simple mockup of our ideas.

Before we could actually test our ideas, we needed to turn the mockup into a polished web page that we would feel comfortable showing to our users, so we turned to Mozilla’s master of design and creativity, John Slater, who connected us with web designer Naz Hamid. The result of our collaboration can be seen below:

The new start page. Click on the image to see a version of it with notes explaining the differences between the current start page.

The test turned out to be successful. With the new start page:

• More people used the search box (+1.3%), which is the best way to use SUMO to find the solution to your problem.
• Fewer people left the site immediately without interacting with it (-0.5%), which means that more people are able to get their problem solved.

For the full report and many interesting insights about how people interact with this new page versus the current page, read the original blog post based on the full analysis of the A/B test by Ken Kovash and Mozilla intern Eric Hergenrader: Improving a User’s Experience with Firefox Support (part II).

Our effort to improve the support experience for our users will of course not end with this test. It’s an ongoing process and a continued focus of the SUMO team to make our support platform as easy to understand and use for as many users as possible.

When looking at the results of our test, it should be noted that the bounce rate is still very high (86%). As I mentioned earlier, there could be many other reasons why people quickly leave the website. One reason, that I suspect plays an important role here, is the fact that you can reach Firefox Support simply by pressing F1 on your keyboard. My theory is that many people accidentally do this when typing on a web page, leading to many unwanted visits to Firefox Support.

The most common SUMO bookmark?

That is one of our next things to test on SUMO: among the people that visit Firefox Support by pressing F1 on the keyboard, how many people close the website right away? Are the people that visit the site by selecting the Help option in the menu more interactive?

We will have the answers to these questions soon.

Our great friend Ken Kovash helped us with analyzing the results of this first test, so please visit the Blog of Metrics for all the details.

We are preparing to launch the second A/B test with a brand new start page design; more on that on this blog soon!

• The Localization Dashboard helps localizers track how much of the site is localized in their language, and view which pages are the most important to translate.
• It is now possible to get notifications when a article edit in their locale is need of review.
• We fixed many website software bugs that were making it difficult to localize.

There is still more we can do, which we plan on addressing soon, such as:

• Listing the differences between Contributors, Approvers/Reviewers, and Locale Leaders, and publishing what permissions each group has, as well as listing who is in each group. This helps contributors identify who the leaders of each community are, and who to contact if they have questions or requests from the leaders of their locale.
• Making it clear what is different in SUMO as opposed to other wikis. Some communities have their own sites, and many contributors are already familiar with other systems. This creates an expectation of how SUMO works, and confusion when SUMO does not work they way they expect. Examples of common causes of confusion from community members include tikiwiki markup, the staging and review system, how article translations work, and how to create/remove the “Content may be out of date” warning.

We’ve been keeping a summary of all l10n feedback on wiki.mozilla.org, so you can take a look if you’re interested. If you are a SUMO localizer, and would like to meet with us, just contact us on this blog or post in the Contributors forum. We’re always willing to meet with you!

We haven’t forgotten! In the next three months, we’d like to test out
the idea, and see how it goes. The purpose of the newsletter would be to
provide a digest of news from the SUMO world for those who are not
actively involved, but still interested in knowing what we’re working on.

Over the next three months, we’ll publish a newsletter each month, and
depending on the feedback from the community, we’ll know whether or not
to make the newsletter a regular thing. Stay tuned for more information.

Monday Jan 12th 2 PM PST/5 PM EST/10 PM GMT:

• California: 650-903-0800 then extension 92, conference number 280#
• Toronto: 416-848-3114 then extension 92, conference number 280#
• Toll-free (US): 800-707-2533 then password 369, conference number 280#
• Skype (free worldwide): +18007072533 then password 369, conference number 280#

The backchannel (where we post links and share text) is #sumo on irc.mozilla.org and our agenda and notes is on the Mozilla wiki.

If you can’t make the time, never fear, just tell us your thoughts in IRC at any time or come to one of our weekly phone meetings Mondays at 10 AM PST/1 PM EST/6 PM GMT.

• It ensures that the solutions to the most common problems are written in a clear, concise, and straightforward language that is easy for our users to understand.
• It reduces the pressure of our community of volunteers since most users are self-served.
• It gives us a powerful way of tracking which problems are the most common with the help of metrics.

The Support Funnel. For more information, see The vision for SUMO – Part 2: Understanding the bigger picture.

So, how do we ensure that people find the solution to their problem in the Knowledge Base? Well, one way of finding the solution is by searching for it — something we try to make very obvious on the start page. The reason why we think searching is the best way of finding the solution is because the Knowledge Base is big. Really big. While we have a list of the most popular support articles right there on the start page, it’s hard to make it obvious that there is a lot more content in the Knowledge Base than what is shown on the start page.

The start page of Firefox Support, clearly emphasizing on the search function.

So, is everyone really comfortable searching? Actually, the almighty and ever so wise chofmann and I have started to see evidence that some people prefer to browse for the solution rather than searching for it. Among the people that visits the Firefox Support start page and doesn’t instantly leaves the page, only roughly half of them actually searches. The other half either clicks on one of the hand-picked popular support articles, or clicks on some other link on the page.

We’re not sure what the reason for that is, or if there are several reasons. It could be that people are unable to accurately describe the problem they’re seeing — considering how many people that are using Firefox today, this is not surprising. Even my older brother, who has been using computers for at least ten years, has problems describing some of the problems he has with his computer, and usually I have to pay him a visit, have a nice cup of coffee, and fix the problem myself.

Another reason could be that people simply prefer to browse a categorized list of articles instead of searching — essentially just clicking on a few links instead of actually typing. I talked to our creative genius John Slater a few weeks ago and he said that he’s usually a little skeptic about internal search engines and that he prefers to just browse.

Francisco Picolinni from the Mozilla Hispano community provided a third possible reason why people are unwilling to search — because they might not think anyone else has the same problem.

Regardless of why not everyone searches, it seems like we should work on providing a good way of browsing the Knowledge Base. We currently have a link at the bottom of the list of popular support articles saying “Browse all Knowledge Base topics.” However, the page that link takes you to is just a long list of all articles ordered by hit count — not exactly easy to navigate.

chofmann, John Slater and I recently brainstormed around how we could ensure that as many people as possible find the solution to their problem in the Knowledge Base with minimum effort. Since the Knowledge Base articles are loosely connected with tags like “bookmarks,” “location bar,” etc., one interesting possibility would be to show these tags in a tag cloud that would provide a better sense of the diversity of the content while still pointing to specific, popular topics. Clicking on a tag would filter the list to only show the articles with that particular tag.

A tag cloud around the topic Web 2.0.

Another problem to solve is making sure that people really understand that they can browse for solutions as an alternative to searching. We want people to understand that Firefox Support has the answer to their problems and that they should stay on the site until the problem is solved. This probably means we have to take a closer look at how the start page is designed to see how we can better communicate this. If you have ideas on how we can achieve this, we are interested in hearing them!

What we really want a user with a problem with Firefox to feel when visiting Firefox Support is: “These people are here for me, and they won’t give up until my problem is solved.”

We just started to work on this, so stay tuned for more…