Mozilla Security Blog

Refresh of the Mozilla Security Bug Bounty Program

07.15.10 - 05:09pm

Mozilla launched its security bounty program in 2004 and while the original mission of protecting users by supporting security research has not changed, the security environment has changed tremendously. In recognition of these changes we are updating our security bounty program to better support constructive security research.

For new bugs reported starting July 1st, 2010 UTC we are changing the bounty payment to $3,000 US per eligible security bug. A lot has changed in the 6 years since the Mozilla program was announced, and we believe that one of the best way to keep our users safe is to make it economically sustainable for security researchers to do the right thing when disclosing information.

We have also clarified the products covered under the bounty to better reflect the threats we are focused upon. We still include Firefox and Thunderbird obviously, but we also added Firefox Mobile and any Mozilla services that those products rely upon for safe operation. These are products we have traditionally paid bounties for in a discretionary basis anyway, but we wanted to make that explicit. Release and beta versions of those products are eligible. Mozilla Suite bugs however is no longer eligible, as it is not an officially released nor supported Mozilla product.

In concert with those changes, we are also updating the eligibility language to make it clear that Mozilla reserves the right to disqualify bugs from the bounty payment if the reporter has been deemed to have acted against the best interests of our users. To be very clear, we are not modifying our position regarding payment for publicly disclosed bugs; Mozilla bounty payments are not contingent upon confidential disclosure. While Mozilla strongly encourages researchers to disclose bugs to us privately (and most researchers have), we also believe that researchers should ultimately retain control over when and how the details of their research are disclosed.

We hope other organizations will match our program and actively support constructive security research.

Full text of the security bounty program: http://www.mozilla.org/security/bug-bounty.html

Security bounty FAQ: http://www.mozilla.org/security/bug-bounty-faq.html

Lucas Adamski
Director of Security Engineering

Category: Uncategorized | | 3 Comments »

Responding to the Adobe advisory: Plugin Checker in action

06.15.10 - 05:13pm

Adobe recently released a security advisory for Flash Player,  Adobe Reader and Acrobat. The advisory stated a critical vulnerability existed in all versions of Flash prior to and including 10.0.45.2.

Late last week, Adobe released an updated version of Flash that does not contain the security vulnerability; version 10.1.53.64. After considering the importance of updating our users as fast as possible Mozilla has taken the following steps:

• Updated our Plugin Checker to notify users with vulnerable versions of Flash to update to the latest version
• Added Flash version detection to our What’s New pages when users update Firefox. Users with out-of-date versions of Flash will receive a prominent message to update.
• Added messaging to our First Run pages prompting users to check that their plugins are up-to-date, linking them to the Plugin Checker.

Keeping your software up to date is one of the most important things you can do to stay safe online, and Mozilla will continue to look for ways to make that process as easy as possible for our users.

Brandon Sterne
Man-in-the-middle

Category: Uncategorized | | 11 Comments »

Plugin Check for Everyone

05.11.10 - 12:02pm

It’s been a few months since I wrote about the work our plugin check team has been doing, but there are a couple of pretty excellent pieces of news I’d like to share, most notably: the Mozilla plugin check now works for users of other browsers as well.

Plugin Check: A Refresher

Last fall, we started a program to help our users keep their plugins up to date. Outdated plugins are a major source of security and stability risk for web users, and some studies have put the proportion of users with older versions as high as 80%.

In the months since we’ve deployed the page, we’ve seen some great success. These days, over 60% of the users we see on the plugin check page with Adobe’s Macromedia Flash plugin installed are running the most recent version, and the number grows to more than 75% if we include the second most recent. That’s much higher than the web as a whole, and there is still a lot of work to do to get that number up, but we’re confident that the integrated checks for outdated plugins in Firefox 3.6 will improve things even further.

What’s New

We believe that plugin safety is an issue for the web as a whole, so while our initial efforts focused on building a page that would work for Firefox users, the team has since expanded plugin check coverage to work with Safari 4, Chrome 4, and Opera 10.5. We have added support for Internet Explorer 7 and 8 for the most popular plugins, as well, but since IE requires specific code to be written for each plugin it will take us a little longer to get to full coverage. You can see the updated page for yourself here.

This has been a phenomenal amount of work to develop and test, and the matrix of browser, plugin and OS grows very quickly. Our web team is remarkable, but they couldn’t have done it without the continuing support of Mozilla community members like Lloyd Hilaiel who helped write some of the plugin-specific logic.

Plugin Check Badges

Finally, now that we have delivered a more universal plugin check page, I wanted to call your attention to the plugin check site badges our team developed a little while ago. Adding these banners to your site will help your readers stay current regardless of which browser they use, and make the internet a safer place for everyone.

One More Request

Our Plugin Directory will eventually become the main way we keep our data about plugins up-to-date. If you’re a plugin vendor, we need your help! The directory is currently in alpha stages, and we need vendors to let us know as new versions come out, and old versions become dangerous. Please email plugindir @ mozilla . com for information about how you can get involved.

Johnathan Nightingale
Director of Firefox Development

Category: Announcements, Security | | 12 Comments »

Removing the RSA Security 1024 V3 Root

04.06.10 - 02:08pm

There’s been confusion today about the work we’re doing on our root store, the set of trusted certificate authorities shipped with Mozilla products. The short story is this: we’re removing the “RSA Security 1024 V3″ root from that list. Its owners have confirmed that it is not in use, and not covered by current audits. We regularly check for roots whose audits have lapsed or for whom we don’t have an up to date point of contact – it’s part of keeping our root program healthy.

The confusion stems from a comment made in the newsgroup threads discussing the removal which suggested that the root didn’t have a current owner. We know where the root came from, it was added at RSA’s request several years ago and vetted according to our inclusion guidelines. When we contacted RSA to confirm current contact and audit information for it, though, we didn’t get a clear answer as to whether or not it was in use, covered by recent audits, or decommissioned. We expect every root in our program to have a clear and active owner and, failing to get that clarity from RSA, we moved to pull this root from the product.

RSA has since confirmed that this root is no longer needed and can be removed from the product. That clarity, while late, is welcome and confirms our original decision.

This legitimate but inactive certificate will be present in all consumers of Mozilla’s NSS security library until the removal takes effect. Questions about Apple’s inclusion of this root in their keychain system, and their plans for removal, are best directed to Apple.

Johnathan Nightingale
Director of Firefox Development

Category: Security | | 12 Comments »

Plugging the CSS History Leak

03.31.10 - 06:41am

Privacy isn’t always easy.

We’re close to landing some changes in the Firefox development tree that will fix a privacy leak that browsers have been struggling with for some time. We’re really excited about this fix, we hope other browsers will follow suit. It’s a tough problem to fix, though, so I’d like to describe how we ended up with this approach.

History Sniffing

Links can look different on web sites based on whether or not you’ve visited the page they reference. You’ve probably seen this before: in some cases, visited links are purple instead of blue. This is just one of the many features web designers use to make the web the best it can be, and for the most part that’s a good thing.

The problem is that appearance can be detected by the page showing you links, cluing the page into which of the presented pages you’ve been to. The result: not only can you see where you’ve been, but so can the web site!

Originally specified as a useful feature for the Web, visited link styling has been part of the web for… well, forever. So this is a pretty old problem, and resurfaces every once in a while to generate more paranoid netizens.

The most obvious fix is to disable different styles for visited versus unvisted links, but this would be employed at the expense of utility: while sites can no longer figure out which links you’ve clicked, neither can you. David Baron has implemented a way to help keep users’ data private while minimizing the effect on the web, and we are deploying it to protect our users. We think this represents the best solution to the problem, and we’ll be delighted if other browsers approach this the same way.

Technical Details.

The biggest threats here are the high-bandwidth techniques, or those that extract lots of information from users’ browsers quickly. These are particularly worrisome since they enable not only very focused attacks, but also the widespread brute-force attacks that are, in general, more useful to a variety of attackers (potentially including fingerprinting).

The JavaScript function getComputedStyle() and its related functions are fast and can be used to guess visitedness at hundreds of thousands of links per minute. To make it harder for web sites to figure out where you’ve been without radically changing the web, we’re approaching the way we style links in three fairly subtle ways:

Change 1: Layout-Based Attacks
First of all, we’re limiting what types of styling can be done to visited links to differentiate them from unvisited links. Visited links can only be different in color: foreground, background, outline, border, SVG stroke and fill colors. All other style changes either leak the visitedness of the link by loading a resource or changing position or size of the styled content in the document, which can be detected and used to identify visited links.

While we are changing what is allowed in CSS, the CSS 2.1 specification takes into consideration how visited links can be abused:

“UAs may therefore treat all links as unvisited links, or implement other measures to preserve the user’s privacy while rendering visited and unvisited links differently.” [CSS 2 Specification]

Change 2: Some Timing Attacks
Next, we are changing some of the guts of our layout engine to provide a fairly uniform flow of execution to minimize differences in layout time for visited and unvisited links. The changes cause all styles to be resolved on all links for both visited and unvisited states, and it is stored; then, when the link is styled, the appropriate set of styles is chosen making the code paths for visited and unvisited links essentially the same length. This should eliminate some of the easy-to-mount timing attacks.

Change 3: Computed Style Attacks
JavaScript is not going to have access to the same style data it used to. When a web page tries to get the computed style of a link (or any of its sub-elements), Firefox will give it unvisited style values.

What does this mean for users?

For the most part, users shouldn’t notice a change in how the web works. A few web sites may look a little different, but visited links will still show up differently colored. A few sites that use more than color to differentiate visited links may look slightly broken at first while they adjust to these changes, but we think it’s the right trade-off to be sure we protect our users’ privacy. This is a troubling and well-understood attack; as much as we hate to break any portion of the web, we need to shut the attack down to the extent we can.

We have to be realistic, though: there are many ways all browsers leak information about you, and fixing CSS history sniffing will not block all of these leaks. But we believe it’s important to stop the scariest, most effective history attacks any way we can since it will be a big win for users’ privacy.

If the remaining attacks worry you, or you can’t wait for us to ship this fix, version 3.5 and newer versions of Firefox already allow you to disable all visited styling (immediately stops this attack) by setting the layout.css.visited_links_enabled option in about:config to false. While this will plug the history leak, you’ll no longer see any visited styling anywhere.

Enhancing Privacy on the Web.

We want to bridge the gap between our users’ expectations of privacy and what actually happens on the web. Sometimes users have an expectation that we preserve their privacy a certain way, and if we can, we want to live up to it. Privacy isn’t a feature that can simply be added to a browser, though; it often comes at the expense of utility. We think we’ve found a fix that will balance flexibility for web developers while providing a safer experience for our users on the web.

Sid Stamm, Mozilla Security

Category: Announcements, Privacy | Tags: CSS, Firefox, history, Privacy | 67 Comments »

Firefox 3.6.2 Released

03.22.10 - 09:22pm

Mozilla has accelerated its timetable and released Firefox 3.6.2 ahead of schedule. This release contains a number of security fixes, including a fix to Secunia Advisory SA38608 which was previously discussed on this blog when we were first made aware of and were then able to confirm the issue.

For additional information please see Mozilla Foundation’s Security Advisory MFSA-10-08 as well as the Firefox 3.6.2 Release Notes. We urge users to promptly update to this release by selecting “Check for Updates…” from the “Help” menu, or by visiting https://www.mozilla.com/ for a free download.

Category: Announcements, Firefox, Security, Security Updates, Vulnerabilities | | 46 Comments »

Update on Secunia Advisory SA38608

03.18.10 - 08:20pm

Mozilla was contacted by Evgeny Legerov, the security researcher who discovered the bug referenced in the Secunia report, with sufficient details to reproduce and analyze the issue.  The vulnerability was determined to be critical and could result in remote code execution by an attacker.  The vulnerability has been patched by developers and we are currently undergoing quality assurance testing for the fix.  Firefox 3.6.2 is scheduled to be released March 30th and will contain the fix for this issue.  As always, we encourage users to apply this update as soon as it is available to ensure a safe browsing experience.  Alternatively, users can download Release Candidate builds of Firefox 3.6.2 which contains the fix from here:  https://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.2-candidates/build3/

Update: To clarify, as originally claimed this issue affects Firefox 3.6 only and not any earlier versions. Thunderbird and SeaMonkey are based on earlier versions of the browser engine and are not affected. People testing “3.7″ development builds should upgrade to 3.7 alpha 3 or the latest nightly build to ensure they have this fix.

Category: Uncategorized | | 40 Comments »

Secunia Advisory SA38608

02.22.10 - 05:30pm

Category: Firefox, Security, Vulnerabilities | | 14 Comments »

Fixing security holes without introducing new bugs

02.10.10 - 04:43pm

When fixing any bug, there is a risk of introducing new bugs, which we call regressions. Regressions caused by security fixes can be especially problematic because shipping a buggy security update can erode user trust for future updates.

Fortunately, we discover most regressions before we ship, thanks in large part to security researchers whose patience gives us time to review and test each patch well. But sometimes security releases are delayed slightly because we notice a regression as we are getting ready to release. Worse, we sometimes discover regressions after shipping a release.

This post explores patterns among regressions and suggests changes that could help us catch them earlier.

Motivation

In six cases since Firefox 2, we decided regressions in minor updates were serious enough to rush a followup regression-fix update. For comparison, we shipped 38 other minor updates, mostly to fix security holes, since releasing Firefox 2 in October 2006.

It’s hard to see trends and patterns in a list of six bugs, so I looked at a larger set of bug reports: all bugs reports marked as regressions caused by security bug fixes. I focused on the 176 such bugs filed between December 2007 and January 2010.

Most of these regressions were fixed before release and many of them were minor. But understanding why they weren’t caught immediately can help prevent major incidents in the future.

Overall regression frequency

Regressions seem to be declining in frequency. This calls for cake!

I believe regressions are declining in frequency mostly because of improvements in automated testing. The number of automated tests increased fourfold since Firefox 3.0: every trunk checkin now results in 220,000 tests running on each platform. Regressions caught by automated tests tend to be fixed immediately, without even being filed in Bugzilla.

Further improvements to automating testing could reduce the number of regressions even more. I noticed four patterns in the bugs that suggest areas where automated testing could be improved: (1) web site breakage; (2) firefox extensions; (3) printing; and (4) document navigation.

Web site breakage

About 48 of the 176 bug reports involved specific web sites. The sites ranged from private intranet sites to the most popular sites on the web. When simply loading a site is enough to reproduce the bug, it may be easy to detect it through automated testing.

Crashes, hangs, and leaks are easy to identify in automated testing. Carsten Book and Bob Clary have set up automated loading of top sites to find these bugs.

Visual problems are trickier to identify in automated testing, because no algorithm can reliably determine when a web site “looks wrong”. But determining whether a site’s appearance has changed can be done automatically. Since security fixes do not intentionally change the appearance of web sites, a tool to detect web site appearance changes could catch these regressions.

In addition to improving automated testing, we should remove support for some regression-prone, Mozilla-specific web features: signed scripts, enablePrivilege, XUL, and XBL. (Eight of the regressions affected sites using these features). These features have proven to be not especially useful to web developers, hard to keep stable, and hard to make secure. XUL and XBL, in particular, have together been responsible for over a hundred vulnerabilities.

Firefox extensions

About 21 of the regressions involved Firefox extensions. One way we could detect these bugs is to run automated tests with extensions installed. We should run Firefox’s entire test suite with popular extensions, and we should run at least basic tests with every extension.

Some regressions affect a feature of an extension rather than a feature built into Firefox. To catch these, we would need to let extension developers provide tests for their extension’s features.

We can also let Firefox developers search the source code of Firefox extensions, so when developers change APIs, they can tell which extensions the changes are likely to affect. We’ve already started to do this.

Printing

Five of the regressions involved printing. Printing bugs frequently make it to release users in part because most nightly users do not print web pages frequently.

I suggested adding printing to topsite testing and running existing reftests in print mode.

Document navigation

Five of the regressions involve document navigation. I suspect document navigation is prone to security bugs, and prone to regressions when those security bugs are fixed, for three reasons:

First, it is a place where many web features interact. Loads can be triggered by users (back, reload, link clicks, bookmarks clicks, submitting a form) or scripts (setting location, location.replace, history.go). Loads trigger many events (unload, beforeunload, pagehide), and scripts running from those events can trigger additional navigation. Additional layers of complexity arise from restoration of form contents, frames, and in-page navigation between hashes.

Second, expectations are complicated. To prevent accidental or deliberate “traps”, the “Back” button has to skip over sequences of purely-script-triggered navigations, which are often difficult to distinguish from user-triggered navigations. At the same time, AJAX web applications have good reasons to want the “Back” button to only change the hash part of the URL rather than leave the page.

Third, document navigation semantics have evolved along with the web, rather than being designed in a holistic way. Browser developers tried to fix security problems and web compatibility problems as they were discovered. This resulted in messy expectations and even messier implementations.

I suggested to our resident exhaustive-testing expert that we try to create a specification and a corresponding set of tests for document navigation and related features. In addition, I wonder whether rewriting document navigation code could simplify it and make it less fragile.

Nightly and beta testing

Firefox nightly build testers found more regressions than beta and release users combined. This is a testament to the power of the Mozilla community, which includes over ten thousand Firefox nightly testers and an incredible bug triage team.

But it’s also worrying: it suggests that in cases when a security patch can only have a few days of nightly testing, rather than several weeks, a major regression could easily slip through unnoticed. A patch might get only a few days of nightly testing if a developer feels that landing a fix effectively discloses a vulnerability, or when we’re rushing to fix a security hole that has already been disclosed.

Automated tests will never be able to catch every regression, so we want to help nightly and beta testers identify bugs as effectively as they can, whether a patch undergoes testing for weeks or days. David Boswell has been improving the nightly start page to highlight the best ways to look for bugs. Developers can now post temporary notes on this page to highlight features that need special testing.

We’re also making it easier to be a nightly tester. Automated tests now keep the worst regressions out of nightlies, making nightlies less prone to breaking. Automatic nightly updates now work for all localizations, and we’re planning to make nightly updates smoother.

To expand beta testing, we’re thinking of adding a checkbox to update preferences to let users choose to become beta testers. Beta testers are not always as active at reporting bugs as nightly users, but their numbers allow clear trends to appear in crash statistics.

Raw data

Maybe you can spot patterns that I didn’t, or suggest other methods of automated testing that could catch these kinds of bugs earlier?

I focused on the symptoms and testcases of the regressions. Someone reading the patches might notice different patterns. Would more bugs have been caught by writing new custom static analyses, paying attention to new compiler warnings, or getting some testers to report assertion failures from running debug builds? Could some architectural change have prevented a class of regressions (or the security bugs whose fixes caused them)?

These links show the 176 regressions caused by security bug fixes between December 2007 and January 2010. The spreadsheet contains my guesses as to how we found each bug and how we could have found it earlier.

List of 176 bugs | Spreadsheet on Google Docs | Spreadsheet as CSV

Thanks to Murali Nandigama for helping with Bugzilla queries. Dan Veditz, Melissa Shapiro, and Drew Ruderman provided valuable feedback on drafts of this post.

Jesse Ruderman
Security bug hunter

Category: Regressions | | 11 Comments »

Security Issues With Two Experimental Add-Ons

02.05.10 - 12:18pm

Important Note: One of the malware results has been verified to be a false positive.  Further details are available here: http://blog.mozilla.com/addons/2010/02/09/update-on-the-amo-security-issue/

Original blog entry follows below.

Two add-ons in the experimental section of addons.mozilla.org were found to be containing malware.  These were not originally detected with the anti-malware scanning tools that we have been using.  We have since increased the number of scanning tools, and will be taking additional steps to minimize the risk of further incidents.  Full details of the issue and recommended mitigation steps are here on the AMO blog:

http://blog.mozilla.com/addons/2010/02/04/please-read-security-issue-on-amo/

Category: Security, Vulnerabilities | | 3 Comments »