The pwn2own bug that Nils discovered at CanSecWest 2009 and the XSLT vulnerability recently made public by Guido Landi (http://www.securityfocus.com/bid/34235) are both critical issues that can result in malicious code execution.

Impact

These issues can be exploited by tricking a user into visiting a malicious web page hosting the exploit code. The pwn2own bug can be mitigated by disabling JavaScript.

Status

Both issues have been investigated and fixes have been developed which are now undergoing quality assurance testing. These fixes will be included in the upcoming Firefox 3.0.8 release, due to be released by April 1. You can follow our work in bugzilla.

Credit

The pwn2own bug was reported to Mozilla by Nils via the Zero Day Initiative (ZDI). The XSLT issue was discovered on http://www.milw0rm.com/exploits/8285, credited to Guido Landi.

Bit9 says it drew up this list by identifying popular applications that have had a critical vulnerability reported in 2008. This is an ineffective test, as it rewards software companies that conceal their security vulnerabilities. Mozilla focuses a great deal of energy on building world class code, and we stand by our reputation on security; we don’t play games with it.

Mozilla security process involves regularly identifying, fixing, testing, and releasing security updates to keep our users safe, and we do that in a public way so that others can scrutinize our processes and help make them better. To suggest that this openness is a weakness because it means that we have “reported vulnerabilities” is to miss the reality: that software has bugs. A product’s responsiveness to those bugs and its ability to contain them quickly and effectively is a much more meaningful metric than counting them.

Bit9 seems to understand this in its focus on application support for updates, but again it fails to account for the real world experience.  Firefox does not deliver WSUS updates, but our built-in update mechanism requires no user intervention, and we consistently see 90% adoption within six days of a new update being released.

The Firefox vulnerabilities Bit9 discusses are long-since fixed, with the majority of these fixes coming within days of it being announced.  That is the real measure of application security: are known vulnerabilities fixed promptly, tested carefully, and deployed thoroughly? When people have asked that question, Firefox and Mozilla have consistently come out ahead.

Bug counting is unfortunately common because it’s easy, but it should not be a substitute for real security measurement. This is why we’ve continued to work on things like the Mozilla security metrics project, to help people make informed decisions about the security of their software. We invite people who are interested to be a part of that process.

Johnathan Nightingale
Human Shield

This is great news for Mozilla, since it demonstrates that the work that has gone into the auto update mechanism and the restore session feature has really paid off.  In order to reduce the window of risk for users and minimize the time to deploy, we have put a lot of effort into making sure that it is as easy to install security updates as possible.  This is not the first time we have heard this, but it is great to get more numbers behind what we already know:  Firefox is safer because Mozilla continually works on security improvements, ships updates quickly, and makes it easier to stay up-to-date.

You will be hearing more about our effort to collect meaningful security metrics like these soon.

Asa has a few words to say about this on his blog.

Counting security vulnerabilities to compare the security of different software projects is flawed.  It is only a useful metric if you are comparing a project to itself over time.  I’ve discussed this topic here and here.  It’s even more ridiculous to try and compare an open source bug count to a closed source project because you can see all the bugs in an open source project.  You can only see the publicly found security issues for a closed source product, like Internet Explorer.

So what is interesting in the Techworld article is the measures of real risk to users:

“‘[Z]ero-day’ security bugs in Firefox were patched more quickly than in Microsoft Internet Explorer…”

“[I]n an examination of zero-day flaws – reported by third parties before a patch was available – Secunia found that Firefox tended to get more patches, sooner, compared to IE.”

“Out of eight zero-day bugs reported for Firefox in 2007, five have been patched, three of those in just over a week. Out of 10 zero-day IE bugs, only three were patched and the shortest patch time was 85 days.”

At Mozilla we work as hard as we can to ship fixes as soon as possible to minimize the exposure to our users.  It is great to see that the efforts we are making to minimize risk to users are paying off.

Since all software has bugs, it’s more important to consider how long it takes to get a fix out when a security issue is discovered than it is to count bugs. Number of vulnerabilities identified is a function of how many bugs are present, but is probably more influenced by things like who is looking, and how good they are at finding security issues. That makes it a misleading metric.

When you compare how long it takes Microsoft to fix Internet Explorer vulnerabilities versus how long it takes Mozilla to fix vulnerabilities in Firefox it becomes clear why he chose to count vulnerabilities in this report instead. Earlier this year Brian Krebs of the Washington Post came up with this when comparing IE to Firefox:

For a total 284 days in 2006 (or more than nine months out of the year), exploit code for known, unpatched critical flaws in pre-IE7 versions of the browser was publicly available on the Internet…

In contrast, Internet Explorer’s closest competitor in terms of market share — Mozilla’s Firefox browser — experienced a single period lasting just nine days last year in which exploit code for a serious security hole was posted online before Mozilla shipped a patch to remedy the problem.

Mike Schroepfer goes into this in more detail in his post today.

One of the goals of the bug counting report is to demonstrate that Microsoft fixed fewer bugs for IE than Mozilla did for Firefox. Unfortunately for Microsoft (and for anyone trying to use this report as analysis of useful metrics) he does not count all the security issues. If he were able to count them all, Microsoft could get credit for all the bugs they fixed. He counts only the public issues, because that is all Microsoft will tell us about. Microsoft is worried that if it ever says it has fixed X security issues, the world will focus on that it had X vulnerabilities in the first place, not that they are now fixed and no longer a risk for users. So the set of issues that are available for public comparison is limited to the set of vulnerabilities that are reported externally AND fixed in security updates.

This is a small subset of all the vulnerabilities, because the vulnerabilities that are found through the QA process and the vulnerabilities that are found by the security folks they engage as contractors to perform penetration testing are fixed in service packs and major updates. For Microsoft this makes sense because these fixes get the benefit of a full test pass which is much more robust for a service pack or major release than it is for a security update.

Unfortunately for Microsoft’s users this means they have to wait sometimes a year or more to get the benefit of this work. That’s a lot of time for an attacker to identify the same issue and exploit it to hurt users. Sometimes it just takes time to put in a complicated fix. Anyone that has shipped a major piece of software can relate to that. But this is not the case for every internally found security issue. Extending this process to include fixes that are ready and just sitting on the tree waiting for the preferred vehicle to ship increases risk for users. But it sure keeps those bug count numbers down.

If we as an industry would just acknowledge that counting bugs is useless then vendors could feel safe talking about what they are doing to protect users. At Mozilla we fix our bugs openly. When you count Mozilla security bugs you are seeing not just those that are reported externally, but also the ones that would be considered internal if we acted like most other software vendors. Since all software has security vulnerabilities, we consider a vulnerability identified and fixed a win. It speaks to the strength of our community based security efforts to actively identify and quickly fix security issues. We don’t let fixes languish on the tree waiting for a major release while users are vulnerable. We ship fixes regularly because securing our users is more important than protecting our PR team from having to respond to articles about counting bugs instead of looking at the metrics that actually indicate whether a vendor is doing reasonable things to keep users secure.

We’re not building fixes for our PR team, we’re building them for our users. Go ahead and count.

We spend a lot of time thinking about how we can get fixes out faster to users. But the window of risk is actually determined by two factors. The first is the time it takes us to create a patch, let call this Time to Fix. This includes the time to investigate a security issue, develop and test a fix, and finally ship the update. This is a better measure for understanding how safe a user is going to be than simply counting bugs.

But there’s another aspect to getting the fix to the user that often goes overlooked. That is the Time to Deploy. Time to Deploy is how long it takes for users to get a patch installed once the fix is available from the vendor. Auto-update has gone a long way toward minimizing Time to Deploy for Firefox, but there are still areas on which we can improve.

This chart shows how long it took for users to move from 1.5.0.5 to 1.5.0.6 last year:

This shows that it took eight days for about 90 percent of Firefox users to get updated. When I saw this last year I thought it was pretty fantastic. Firefox has millions and millions of users. Getting almost everyone updated in just eight days seemed pretty incredible to me.

I ran the numbers again this year after we shipped 2.0.0.4.

This chart shows how long it took for users to move from 2.0.0.3 to 2.0.04 last month:

This time it only took six days to update 90 percent of users. That’s a 25 percent decrease in Time to Deploy and a significant improvement in reducing the window of opportunity for attackers to take advantage of security vulnerabilities. It appears that some of the improvements in infrastructure have contributed to these numbers so a big thank you to everyone working in IT and to our partners that host mirrors. You’re helping to keep Firefox users safe.