Community Survey Blog

For the second community survey we focused our efforts on learning more about the community helping with support.mozilla.com (SUMO).

Where did we get this idea?

The topic for this survey came from Jesper Kristensen, a Danish localizer, who participated in a discussion on the Mozilla Europe’s mailing list about new survey ideas.  He wrote:

“SUMO (support.mozilla.com) is a great project, and it is probably going to support localization. But the project relies on volunteers from the local communities. I think it would be interesting to know if the local communities are interested in participating in that project. [...] David and Axel speculates if support of Firefox only vs. all Mozilla will influence how much volunteers will contribute to the project. Maybe a survey could be used to find the answer.”

We took this idea and expanded it to include Mozilla support in general.  It made most sense to then contact David Tenser, the project manager of SUMO, and invite him to help us with the objectives and questions.  Kindly, he agreed to help.

Preparing the questions

Just as we did in the first survey, we kept the questions short to help increase responses rates while still providing valuable information to us.  After a couple of long calls and e-mails, we agreed on a set of objectives listed below:

1. To understand and measure how Mozilla is doing at support in the different locales;
2. To get a better understanding on people’s perception of the quality of support;
3. To find out how can we make Mozilla support more useful.

We asked four questions in total, three of which directly related to our objectives.

In the first question,we asked the survey takers to gauge overall support on different Mozilla products.  In order to identify support-related issues in local communities, we provided a list of possible experiences for people to rate on a scale from one to five.

Question 1: How would you rate the support that community provides for Mozilla products in your language? (1- lowest quality, 5- highest quality)

• Mozilla overall
• Firefox
• Thunderbird
• Calendar
• SeaMonkey
• Camino
• Add-ons

Question 2: How accurate are the following statements for your community? (1 - least accurate, 5 - most accurate)

In my community…

• most of the relevant information can be found in my locale
• the support sites have a structure that is easy to read and navigate
• users ask questions in the right channels
• users rarely ask the same question more than once
• the support sites are up-to-date
• the support sites cover well the most common issues
• the support sites load quickly in my browser
• the support sites make use of screencasts and tutorial videos to help our users
• new issues/problems with Mozilla software are quickly added to support sites

Unlike last time, we made sure to include the “Don’t Know” option.  When we closed the survey, we turned up with a few missing values in our results, but it was still interesting to see how they were distributed, and how they could be interpreted.

For the third question, we asked about how support content might be delivered.

Question 3: What resources make or would make the user-to-user support in your language more effective?

• Tutorials/How-to’s
• Troubleshooting articles
• Support available in my language
• Support available from a single site
• Screenshots
• Screencasts/videos
• Specific versions of support depending on operating system (Windows/Mac/Linux)
• Forums
• Live Chat

And, you’ll remember the lesson learned from last survey:  Next time, ask a demographic question!  We did this time.  Without a specific target for this survey or control of who would take it, we could discover a bit more about who took the survey by asking the following:

Question 4: What’s your involvement in the Mozilla project:

[  ]  I’m a user of a Mozilla product
[  ]  I follow the news from the project
[  ]  I help other users in the discussion boards
[  ]  I write documentation
[  ]  I localize documentation
[  ]  I localize Mozilla products
[  ]  I develop Mozilla code

We then pushed these questions live in Stas’s survey tool and started to collect answers.  What do you think?  Do you think the questions are composed in a way that will elicit information that will meet our objectives?  Your feedback is appreciated.

By the way, a couple of weeks ago, Seth described on his blog the process of creating the second community survey. If you’re interested in a peek behind the scenes of the Community Surveys project, you might find his post interesting.

In closing, we realize this is coming a bit late, but with all the things we learned about survey analysis and presenting the results, time was not lost. Now that we know a few more of the challenges we may encounter, we can publish much more quickly.  A big thank you to everyone for supporting our efforts and keeping us motivated as you have done so far.  We hope it’s been enjoyable and informative.

Trackback URI | Comments RSS