Are you up for the challenge?

This week Mozilla launched the security quiz on Facebook. We encourage you to take the quiz and see how much you know about web security!

Similar to our plugin checker, the security quiz is a part of our larger effort to raise awareness about web security.

Help us spread the word and make the web safer for everyone. And don’t forget to check your plugins!

Notable changes are:

• Migrating from CakePHP to Django
• Moving from SVN to Git
• Continuous integration
• Faster deployment
• Processing data offline
• Improved documentation

Take a look, it’s a good overview of the technical challenges of managing a large and complex website at an enormous scale.

Why is this important to you?

• Crashes are the number one concern for Firefox users, and we are listening.
• At least 30% of all Firefox crashes are caused by third-party plugins.
• Many major security vulnerabilities exploit out of date plugins.

Why is this important to Mozilla?

Increasing awareness about plugins makes the web better, and that’s our mission.

• We want the web to be safer.
• We want the web to be less crashy.
• We want to help everyone — not just Firefox users — to address the plugin problem. (though admittedly it doesn’t fully work with all browsers yet, it will)

What did we do?

The plugin checker has three components:

• The Server: Plugin Finder Service (PFS2)
• The Javascript: Perfides
• The Web Page: mozilla.com

The end result is actually pretty simple — and that’s how it needs to be. Here’s your plugins, and here’s their statuses:

Putting it all together, we reach a workflow similar to the graph below. Our goal is to query a central database that contains plugin information and inform users about the status of their plugins. This was built so it could be used to support Firefox directly in the future.

What will happen next?

The three components above are a good start, but together we can do more.

• Integrate this experience with the Firefox client. Firefox will identify vulnerable plugins and help you update them.
• Create a self-service panel for vendors to update their plugin info as new releases come out.
• Create an open directory for all plugin information (sort of like Plugindoc but dynamic)
• Evangelize plugin detection via an embeddable widget — get it out on Wordpress, etc.
• Integrate with our crash reporting system so we have a report card/dashboard for which plugins are most crashy

How can you help?

This entire project is open source. You can work on any of these components to help contribute to the effort:

• View the server code for PFS2
• View the client code for Perfides
• File a bug if you find one
• Tell us about plugins we don’t know about

We could use your help! Please hit our testing server’s Plugin Check. We will be able to capture information about plugins and help fill-out the PFS2 database. See an issue? Look through current bugs and leave feedback in Bugzilla.

Screenshot of plugin detection

• Adobe Acrobat
• Windows Media Player Plug-in
• RealPlayer (on Mac only, Windows exposes version information)

Some plugins don’t expose a good version number in the description, but can be detected by instantiating the plugin. We’re using Eric Gerds’ PluginDetect for this type of plugin.

On the other hand, kudos go to Microsoft’s Silverlight team for the following information: name=”Silverlight Plug-In” description=”3.0.40818.0″. That’s exactly what we need to identify when a Plugin has fallen out of date.  If a vulnerability is discovered and published against 3.0.40818.0, we can alert the user to pick up the newest version.

It’s very fast and easy for us to detect your release version, when the proper information is provided by the plugin. Doing so is a win for you and your users. We’ll be encouraging Firefox users to keep their plugins updated to the latest and greatest. This means better distribution and lower support costs for you. We’re contacting many vendors right now to make this happen.

Firefox 3.6 is going to be adding enhancements to the way Plugin information is exposed to JavaScript. We’re looking forward to how this will simplify this task.

Interested in the code under development? Check out PFS2 server, PFS2 client and of course Mozilla.com where it will eventually live.

Update 10/3 @12:50 PDT: Thanks to everyone who has filed bugs! Additionally, here is the list of Plugin states, copy, and links. This is going to change, based on your feedback, but I think it will help the discussion.

I think the consensus is that the copy for Current and Old send the wrong message.

Historically, we’ve had different ways to track new releases and their downloads:

• 2004-2007 (Firefox 1.0, 1.5, 2.0) – Bouncer database statistics tracked redirects on the download redirector in a MySQL database.
• 2008 (Firefox 3.0) – Jeremy Orem’s download stats page pulled stats from Daniel Einspanjer’s metrics feed.
• 2009 (Firefox 3.5) – Daniel and Dave created the 3.5 download stats page, which closely resembles what we have now.

Blast from the past – Jeremy Orem’s Firefox 3.0 stats page:

As a part of our renewed efforts to innovate more rapidly on mozilla.com, our first sprint brings download information to mozilla.com. From release to release we hope this page changes in exciting new ways. In the meantime, it’ll have a nicer place to live.

The idea was pretty simple:

• Creating real-time JSON feeds was Daniel’s work with SQLStream to manage and constantly query incoming download activity. It was no small task, but the feed seems to work fairly well.
• Using JavaScript to integrate feeds into existing pages was Dave Dash’s work. He had to do it in both YUI and jQuery since mozilla.com uses YUI but the stats code uses jQuery. Yes, fun, we know.
• Creating widgets and a landing page was possible because of Silver Orange. Steven Garrity and Stephen DesRoches styled the stats landing page and worked with John Slater from Mozilla to figure out points of entry on other mozilla.com pages.

The end result is a nicely sized update to mozilla.com that we were able to finish in two weeks. Publishing our totals gives new users a feel of activity and freshness, and funnels them to a statistics page that we can always expand later.

We’re also excited that this page uses some neat tools we’ve grown to love:

• The world map is done in SVG/Canvas thanks to Alistair MacDonald
• The dynamic table uses jQuery and some jQuery plugins (tablesorter, sparklines). It updates every minute.
• All feed data is available in JSON format which makes it easy for pages to update themselves via JavaScript

Here’s a widget on Firefox product pages:

Here’s a widget on the personalize Firefox page:

Here’s the main sections from the main stats page:

We hope you like the small changes we made, and if you have ideas on how to make this better, please let us know! If you’d like to play with these feeds, you can find them here:

• Country report JSON
• Total downloads JSON

A lot of hard work was done by our de, es, fr, and it teams to make it into the first release.

What exactly has to be done for each locale? The following assets are translated:

• 230+ pieces of content (in the messages.po file)
• 20 question faq.php which is an HTML file
• Site header image
• Several affiliate buttons
• Service Week Flyer

In addition to the 4 locales listed above, several other communities are busy working on Mozilla Service Week localizations. We’ll be shipping them as they are completed.

This tracking bug has links to all of the L10n work being done. There are many ways to help us build out Mozilla Service Week, so get involved and Be the Difference.

In part 2 of the series, I explained some organizational challenges when growing a web localization team healthily. In this post, I want to outline some of issues localization tools are facing on the web.

Note: In this article, the term localization tools is referring to developer tools that human beings can use to localize web applications. This is unrelated to automatic translation tools like Google Translate or Yahoo Babel Fish.

The thin line between developer and localizer

Most translatable Web content at Mozilla is represented as GNU gettext .po files as part of the regular the source code. Localizing web applications is therefore still very much a technical task: Checking out a source tree, editing a .po file, checking it back in, (optionally waiting for the staging server to pick up the changes and checking out one’s changes online), that sounds very much like a developer’s work, not as much like the core concern of a localizer.

A lot of tools do not try to change that. They are mere frontends to translating .po files, but they do not help you translate the website. One way to change this would be building a “PO live edit” tool, as my colleague Austin King describes in a recent blog post.

Please read his post for more details, but the idea is to build a translation tool that requires zero set-up, has a small learning curve, and ties web site translation together “end-to-end”, i.e., from development to deployment in one go.

.po or not .po?

Another problem we have is that not all of our pages are completely localizable in .po files only. Most notably, the web site mozilla.com mainly consists of straight HTML files, tied together with some PHP. But also other pages, like AMO, have some content in localizable HTML files. The rationale is that these texts would add a huge number of strings to the .po files if they were split up in that fashion, and it would be significantly harder for localizers to translate these out of context. Therefore, having localizers translate straight HTML from a file is considered the “lesser evil”.

The reason why these are so different is that these pages sometimes do not only contain translatable content, but they may also require the localizer to add or remove bullet points, or replace links to a US-centered site with another locally applicable option.

Currently, we are not aware of a tool that makes these localizations easier. Some tools, like html2po, try to extract strings from an HTML file and drop them into a .po file (one string per block-level element, for example), with varying success. While this allows using .po-capable tools on the resulting files, it removes the flexibility to perform “content-changing localization” as described above. Finally, it also makes the .po file authoritative, i.e., if one localizer edits the generated .po file but another one prefers editing the HTML template, we are going to run into non-trivial merging problems.

Making HTML translation easier is definitely a “slippery slope” towards building a full-blown HTML editor, which by all means we want to avoid. However, I can imagine integrating Bespin with a web site instance and allowing localizers to collaboratively edit the HTML file right where it will show up eventually.

Comments?

This is where you come in: Please leave comments. If you are a localizer yourself, please tell us what does or what does not work well for you. What is your workflow like? Are you happy with it or would you like to change it? If you have used or witnessed a tool before that you found provided an innovative solution to the problems outlined above, please tell us.

Acknowledgments

This series of blog posts was a result of many discussions with sethb, gandalf, stas, ozten, pascal, pike and others interested in making Web localization better. Thanks for your input, guys!

Photo Credit: “Old Tools”, CC by-sa licensed by Svadilfari on flickr.

In yesterday’s post, I described the problem we are facing with an increasing amount of localizable websites under the roof of the Mozilla project. In short, I concluded that getting more possible localizers involved is a way to solve this — but scaling the localizer community has to happen in a healthy way.

Crowd-Sourcing Localization?

Engaging an arbitrary amount of people in solving a problem even has a fancy buzz-word nowadays: “crowd-sourcing”. But is this what we want for the L10n community? Increasing the sheer amount of translations is not going to help anyone, unless they are good translations. In other words, we want more signal, not more noise.

In addition, when improving Web L10n, one rule is the most important: Don’t break what works. We have an amazing group of localizers who are indispensable and it would be foolish to demote them to some sort of “translation monkeys”. Likewise, merely shifting their work from actual localization to the tedious task of approving “crowd-sourced” strings one by one is not going to make their lives any easier or more fun.

Scaling Locale Ownership: Hierarchies

One possible way to tackle this dilemma is what we could call “hierarchical community management”. One or more locale owners should be able to delegate localization authority for one or more projects to a set of other localizers. These contributors can, in turn, either localize a particular project themselves or decide to pass on localization rights in a similar way, with or without the need to sign off on the changes their respective delegates perform. By making the whole process opt-in and as fine or coarse-grained as the localizers desire, it is ensured that for one locale, a small group of very active people can cover it all, while for another locale, the work can be spread over a substantial number of shoulders.

With a hierarchical permission system, delegating work to known localizers is not the only thing getting easier. It gives newcomers a very structured way of entering the L10n world in the limits of a single project or two, and to “climb up the tree” by building reputation in the community.

But what are good translations? — Building a Reputation Model

You might have noticed, for approving translations from other people, the per-project localizers will still have to read and accept a possibly huge amount of translations. This is not only tedious, it may also take longer than just translating things oneself. Therefore, when building a foundation for next-generation Web L10n, reputation should not only be assignable by “admins”, but also earnable from within the system.

A good example of such a system is Yahoo Answers, an online Q+A/advice community. They do not only give you points for contributing (i.e., adding to either “noise” or “signal”), but users are also able to “thumbs up” or “thumbs down” individual contributions, therefore dividing noise from signal. If we give localizers the opportunity to approve contributions in bulk that were added by a trusted individual or based on a specific thumbs up/down level determined by the community, it’ll be a powerful tool for involving more contributors without decreasing the translation quality. If we, in turn, give the contributors a measurement of reputation for submitting successfully approved translations, they are able to build a tangible reputation level that’ll facilitate their advancement in the L10n community.

Summary

On an organizational level, next-generation Web L10n needs to be based on a flexible, hierarchical community structure that retains strong locale ownership in order to ensure a high level of quality, but at the same time enables and encourages “newcomers” to build a tangible reputation in the community.

Photo Credit: “pronouncing dictionary”, CC by licensed by Muffet on flickr.

Please note that none of this is authoritative — it’s a complicated topic and making mistakes is easy. If you feel something is misrepresented, please leave a comment and speak your mind.

The status quo

Here at Mozilla, we are very proud and honored to have an awesomely active localization community that works hard to bring mozilla products and web sites to you in your language. Without them, it would be impossible to publish, for example, Firefox in 75 languages and AMO in 33.

What’s wrong?

Product localization at Mozilla works very well: The localizers show unmatched dedication to, and pride in, getting every version of Firefox to their fellow speakers. And a lot of these localizers don’t stop there: They also translate websites like AMO, SUMO, mozilla.com, or Mozilla Service Week. But the more web sites are being released, the higher the work load gets for the localizers. Our teams in some locales can handle that pretty well, both because they are mature, well-oiled teams and because their size allows them to delegate different projects to different people without overworking individual localizers. Others however, be they a newly-founded team or a locale just naturally smaller in size, can find it overwhelming to translate not only the flagship application Firefox but also many other projects that are equally as important to the end users.

I want to make clear that these localizers are not slackers: Their day has 24 hours like everyone else’s, and they have a day job and families to take care of as well. Therefore, they have to prioritize and some projects just don’t make the cut.

In addition, starting to be a localizer can be hard. Some of our existing localizers are developers with a non-English mother tongue (like myself, for example) who can check out an SVN repository, edit a .po file, submit a patch, etc. before they’ve even had breakfast. Other possible contributors might not be that technical and still want to help. Currently, when somebody asks “how can I translate project X?” the answer is often neither obvious nor trivial.

The fact that some localizers and locales cannot keep up with the amount of localization needed for various projects under the roof of Mozilla is not a bad thing. It’s rather a sign of success: All over the Web, Mozilla included, localized websites are becoming the rule rather than the exception. After users have become accustomed to a browser in their own language, websites in their language are the logical next step. But no-one can expect single localizers to bear that task all by themselves.

What can we do?

What we would like to do is lower the entry-level to localizing Mozilla websites by making it quick and easy, while not impairing the established workflows current localizers have. This will not only reduce the individual work load on existing localizers and their projects, it can also make it easier for interested community members to become a localizer, by taking on projects that are less overwhelming in size and therefore easier to digest.

Why is this not as easy as installing a production instance of the numerous “translation tools” out there and pointing it at our projects? This will be covered in the next two blog posts, addressing some organizational issues first, followed by some technical challenges.

Photo credit: “Touching Rosetta”, CC by-sa licensed by namlhots on flickr