SUMO Blog

Sumo

• Weekly metrics
  • “Norton” is a popular search term and so is “fast dial”. Cww to investigate “fast dial”.
• Last week’s weekly support issues
  • New issue: Cannot right-click. Looks like an add-on issue (Yahoo Toolbar?) Will get a KB article created for it.
• Is there a bug to make Omniture tell the difference between 3.5 and 3.5.1?
  • Cww to ask metrics team if they about it.

Knowledge Base

• Any help on crash docs would be appreciated.[1][2][3][4][5], and new this week “_PR_MD_SEND”[6]
• We’re looking at improving articles with a csat score lower than 4. Check out the discussion in the Contributors forum.
• Locale Leader feedback about using HTC to let non-en-US contributors find local communities and their locale leader has been good. Next step is to draft a localized version.
  • This is also a good opportunity to get feedback/ideas from localizers on how to make the en-US page easier to use.
• We want to try simplify the process of translating articles. Discussion.
• Plan for centralizing KB communication is set: create a KB forum with a discussion thread per-article. I’ll start drafting a PRD this week.
• New articles: Cannot scroll with mouse wheel or touchpad and How to clear the cache.
• Sumodev KB stuff:
  • We’re working on making all translations inherit the *additional* category info (Firefox 3.0, Troubleshooting, etc.) of en-US pages.
  • “2.0″ will be removed from the “Show content customized for” selector.

Forum

• Tracking contributor retention, what new metrics can we look at? Try tracking individual users who post as “proposed solution” but don’t register. — zzxc
• Mentoring on forums? How?
  • Cww needs to think about it some more before proposing any ideas.

Live Chat

• Top chat tags (crashes and firewall issues most common last week)
• Welcome to new contributors: itsme95, athaba, moses, robomaster
• Working on documentation for web-client.

Roundtable

• Some praise from a blogger: How can I prevent thought fatigue?
• Users are getting notifications that someone tried to reset their password.
  • bug 506547
  • It doesn’t happen to everyone, but the people that do get them tend to have common usernames.
  • There is an “I forgot my password” link in the login form, which asks for a username or email address, and is accessible to everyone. Misuse of that page is likely the cause.

If you’re at OSCON, be sure to not miss senior software engineer, PHP guru, and SUMO lead developer Laura’s presentation titled Scaling Firefox Support with PHP. From the presentation summary:

SUMO – support.mozilla.com – was launched as the official in-product help for Firefox 3. SUMO is built on top of TikiWiki (an open source wiki built with PHP and MySQL) which had never been used on such a large scale website before. In this case study we’ll cover the steps used to get SUMO up for Download Day and keep it up since. We’ll cover:

• Systems approach and diagnosis
• Profiling
• Database architecture, replication, and full text search
• Making caching work (in so many ways)
• Tricks, hacks, and workarounds
• Load testing: the architecture of our test cluster and tools and how we use them

The presentation starts at 1:45 pm in meeting room J2. Be there, or be sqrt()!

Sumo

• Weekly metrics
  • Private Browsing the most commonly searched
  • How many of those people are looking at the Private Browsing article in 3.5 vs 3.0? In other words, is it existing 3.0 users looking at the prospect of updating, or is it 3.5 users wanting to learn more about the feature?
    • 89.5% come from Firefox 3.5, 7.2% use Firefox 3.0.x
• Last week’s weekly support issues

Knowledge Base

• Adding Screencasts now includes guideline to only provide screencasts for step-by-step instructions
  • cilias to lay out the steps to create both Open Video and Flash version of screencasts, explain why open video matters, and put this info up on the Adding Screencasts article
• New articles: Add-ons are disabled after updating FirefoxImporting bookmarks from Google ChromeFirefox launches automatically when your computer starts
• Article bugs for top 5 crashes. Please draft them! [1][2][3][4][5]

Forum

• Still high traffic compared to before 3.5 was released
• Cww to present full data of 1000 thread sample this week
• How/where should we write guidelines for asking questions?
  • On the form asking question
  • Make mockup, then file bug and change it
• Tags part of ForumUX?
  • Cww to talk to paulc about the feasibility, but the main problem is getting people to use it. Explain why it matters.

Live Chat

• Top live chat tags since the release (last week)
  • Cww and zzxc to work together on most common Live Chat and forum issues to see if they are similar
• New blog post about launch day and new chat hours
• Work on the Web Client is progressing; slightly behind schedule.
  • Target: initial framework and demo by this Wednesday, fully functional version by mid-August
  • Bug list

Roundtable

• Alix and chofmann working on plugin detection page — want to know which plugins are the most common and problematic
  • Cww to build list of used plugins in forum and create wiki page for people to fill with more info. Work with zzxc on this.

During the Firefox 3.5 launch, the Mozilla Support community opened Live Chat extra hours to help people experiencing problems after the release. In all, we were able to answer over 1000 users with over 40 community members involved. Chatting with users right after a release lets us share feedback with the Quality Assurance and Launch teams, helping identify any new bugs that are affecting users. A number from the Mozilla Quality Assurance team were also available to assist with more difficult chats during launch week, and several were able to respond to chats when Live Chat was busy.

Overall, support during launch week was a huge success, and we’d like to thank everyone who helped make 3.5 the best Firefox release yet! Almost 50% fewer chat sessions were requested after Firefox 3.5 than after Firefox 3.0, which let us chat with over 90% of people requesting help. We were only able to chat with about half of users requesting help after Firefox 3.0, so this was a huge improvement! The top issues we saw following the release involved how to use new functionality in Firefox 3.5, add-on compatibility with 3.5, and a bug fixed in Firefox 3.5.1 that caused slow start-up times for some users. We regularly keep track of which issues are being seen often in Live Chat, and the full data for the 3.5 release is available in the SUMO Contributors Forum.

Since we have received positive feedback about the revised Live Chat schedule from the release, we have decided to keep some of the changes permanent. The changes have been announced in the Contributors Forum, and we would be glad to get more feedback about which times are best for the community. Since Live Chat requires users and helpers to be online at the same time, spreading out the times that Live Chat is open lets us reach as many people as possible.

Back in May, we came together and worked on a user manual for Firefox as part of the FLOSS Manuals Project, an open-source book publishing project. It was a huge success and we now have a great Firefox Manual you can download, read online or get as a nicely bound paper copy.

Now that Firefox 3.5 is out, it’s time to upgrade the manual as well. As a community, we’ll be getting together and making sure that the new manual reflects all the changes and improvements that went into this new browser and that all the content is current. It’d be great to have you join us and help make sure the manual is up to date.

When?

Thursday, July 16 at 10 AM Pacific time, 1 PM Eastern time, 19:00 Central European time. If you’re in Europe or Asia and want to start earlier, you are of course more than welcome to!

Where and how?

1. Register to get an account at the FLOSS Manual site.
2. Go to the Write section of the Firefox Manual.
3. Pick a chapter that you want to read, click the “edit” link, and start updating it!

On the right side of the website is a chat window where you will be able to chat with other participants.

What needs to be done?

Many pages need just small updates, new screenshots or changes to be updated for Firefox 3.5. Here are just some of the things that you can do:

• Update number of users and number of locales on the Introduction page
• Update graphics (number of total users) on the Open Source page
• Mention this sprint in the page about the manual
• Update screenshots in all chapters
• Discuss new Clearing Recent History features (A new chapter replacing part of this one on the URL)
• Discuss new Private Browsing features (may integrate with above)
• Discuss new features (tab tearing) in the Tabbed browsing page
• Read through full manual to correct minor changes

Just 20 minutes of your time will help keep our documentation relevant, current and useful for all Firefox users, and by making edits, your name will forever in the Credits section of both the online and paperback version.

See you Thursday!

Sumo

• Weekly metrics
  • We’re still dealing with the problem of only getting about 20-25% of the proposed solutions to forum questions actually confirmed as solved threads. Should we send a reminder e-mail for threads with status “solution proposed” so the user gets another chance to verify it?
    • Sounds like a good idea, but we currently have a bug that makes it impossible to ask a question and NOT provide an e-mail address. This should be fixed before we start sending out reminders.
  • Vietnamese locale has shown lots of activity lately. The main localizer (Hung) is also an active participant in the Contributor forum. Yay! Welcome Hung!
• Last week’s weekly support issues
  • Score is a bit weird because new search hasn’t been working the last week [1]
  • The new article [http://support.mozilla.com/en-US/kb/Firefox+takes+a+long+time+to+start+up

Firefox takes a long time to start up] needs to be updated to clarify that the problem with reading temporary Internet files is a new problem in 3.5 that will be addressed in 3.5.1 (see bug). Cww to update article.

Knowledge Base

• How do we document top crashes in which the cause/solution is unknown? [2]
  • It’s better to say that we’re aware of the problem but don’t yet know the cause, than to not say anything at all
  • Make these articles contain as much info as we currently have, e.g. links to relevant forum thread(s)
  • Also, make sure the QA team reviews the content and collaborates with SUMO community to find the cause/solution asap
  • Austin in the webdev team has already helped us add links from crash reports to SUMO, so it’s possible for users to get solutions for the most common crashes. Austin++
• Articles with csat score below 4: [3]. Help us improve them!
• So far only one proposed solution for centralizing communication [4]. We should get more ideas.

Forum

• Traffic was obviously up during launch week, but contributor participation increased with the same speed, resulting in the same ratio of responded threads
• Some people seem to register and post a few (3-5) posts, then never come back. Cww to investigate whether these people are posting separate threads, or responding to their own single thread, and also figure out what the threads are about (are they asking for help, or are they trying to help others?)

Live Chat

• The percent of chats marked as “unsolved” was higher last week than most previous weeks; zzxc is investigating to see which reason(s) were selected
  • One reason could be that Live Chat contributors can now post a link to the CSAT poll before the chat ends
  • zzxc to post results of investigation in Contributor forum shortly
• Top chat tags

Roundtable

• Discussion about new Contributor Home Page design
  • Looks a bit busy, but functionality is good. Everyone to provide better feedback in the forum.

Sumo

• Firefox 3.5 post-mortem
  • Very good job, very prepared.
  • One thing that stood out: problem with add-ons not being compatible. We needed to get a list of unsupported add-ons. Next release would be nice to have automated list of add-ons not supported.
    • Main issue was that many unsupported add-ons were not hosted on AMO (e.g. Norton)
    • There is a list for all AMO extensions, but you have to parse it to find the uncompatible ones
  • Live Chat went really well last week; 44 people helping out, some also helping out in the forum
  • More Live Chat hours, meant that forum traffic wasn’t insane. Keeping pace pretty well with the forum.
  • Getting new KB articles up quickly. Writing quick drafts, etc. Example: Closing the only tab closes the window
  • Projecting some SUMO stats in the MV office. Increased visibility of the support effort and made some developers walk into the room and ask if they should look out for any new common issues.
  • Carlo, a Firefox users that helped out on SUMO during the 3.5 launch called in to share his feedback: “Amazed by how much the developers and people care about this browser. This is why I’m so willing to help out. One of the best things I’ve noticed is the level of support you get from people in the actual chat room, and how [Live Chat contributors] are so willing to help each other.”
• Last week’s weekly support issues
  • Slow startup (possibly bug 501605) or extensions. (Needs article, Cww working on it): (25)
  • Extension related issues: .NET Framework (older versions), Norton IPS, Google toolbar are the big ones (60+)
  • Closing single tab closes browser: Closing the only tab closes the window (12)
  • Lost boomarks: 16
• Weekly metrics
  • Are we going to change the day a week starts for weekly metrics from Sunday to Monday?
    • This question comes from the new automated metrics dashboard using Monday as the first day of the week. Cww to file a bug to change to Sunday and make it a pref.
  • We did really well this release. Traffic went up, but the ratio of solved problems remained.

Knowledge Base

• Goal for Q3: Get CSAT (all locales) above 85% (currently 81% last month)
  • Will publish plan shortly

Forum

• Lots of unregistered helpers this week (maybe, still figuring out query)
  • Cww to work on the query today and report back tomorrow

Live Chat

• High participation last week with the release. (44 active helpers, including 6 from Mozilla QA and 1 from engineering team)
• We are now tracking the number of forum responses to live chat. Out of ~1100 chat sessions last week, about 55 continued to the forum.
  • How many of those 55 threads are solved now? zzxc to look up and post in Contributors forum
• Top tags from the 3.5 release. (Crashes, extension compatibility, slow starting, firewall issues, and broken websites were the most common issues)

In March, I posted about using article feedback to improve knowledge base articles and the importance of making knowledge base articles easy to read; but those are specific areas that are part of a greater knowledge base goal, which is to make the process of Firefox self-help as easy as possible.

There are few sources of information to we draw from:

• Top searches: The most common search terms in the SUMO Weekly metrics document.
• Weekly common issues: Our Weekly Common Issues page tracks the most common support issues each week.
• Article polls: At the bottom of each article, there are poll questions: “Did this article solve a problem you had with Firefox?“, “Was this article easy to understand?“, and “Please rate your experience with solving your problem on support.mozilla.com from 1 to 5” (For more precise data there’s the PageView Data.)
• And of course, Article comments: There is a text field on each article for users to provide feedback about the article. When logged in as a contributor, that feedback is displayed at the bottom of the article.

Here’s how that data is utilized to measure the quality of the knowledge base, and make it better:

The top search terms are tested to find out if the first search results contain the article the user is most likely searching for.
If they don’t:

• The correct article may need to be renamed to match the search term.
• The top article in search results may be mistaken for a different issue; so a link to the correct article is added in the intro of the first search result. If users are being redirected to the correct article, the poll data should improve.
• Keywords that match the search terms are added to the correct article.

For generic search terms the article comments for each result may clarify what users are asking about.

The weekly common issues page is checked for any items that need documentation in the knowledge base. If enough information is available to create documentation, the relevant articles are updated or a new article is created.

The comments in articles with the lowest understandability score are checked to get details on what is not understandable in the article, so we can assess what can be done to eliminate that confusion. Sometimes that means rewording or reformatting the article. In some cases it is a matter of adding screenshots. In other cases, it’s a matter of streamlining or purging the article to simplify it for users.

In the end, it’s about taking the data, analyzing why the data is what it is, and what we can do to improve each issue. As a result, the article poll scores should go up, and users will get answers to their questions about using Firefox. We’ve outlined these tests in a contributor page, so everyone as a community can be most affective in making the knowledge base better each week. You can post any suggestions for improvement in the Contributors forum.

To say that the SUMO knowledge base is just a wiki really doesn’t do it justice. While it’s very easy to start contributing to the knowledge base and simply assume that it works just like many other wikis (e.g. Wikipedia), there are a number of characteristics of SUMO tailored toward user support which make it different. This can be both a blessing and a curse. We need to understand how SUMO is different; but once we are familiar with it, the community has better tools to provide better user support.

If you’re a new knowledge base contributor, please read our new How we are different page, which lists the differences between SUMO and other wikis and gives an overview of why we are different. New knowledge base contributors will also be able to find a link to that page in our introduction to contributing to the knowledge base.

This is also part of our response to localizer feedback, which we have been continually gathering from active SUMO localizers. Remember that if you are a SUMO localizer and would like to meet with us, just contact us on this blog or post in the Contributors forum.