Comments on: Bugzilla: What analytic questions should be answered?

By: Does Mozilla Champion the Voice of Firefox Users? < Blog of Metrics

Does Mozilla Champion the Voice of Firefox Users? < Blog of Metrics — Mon, 13 Jul 2009 16:59:36 +0000

[...] – there’s a current project underway aimed at answering some key analytical questions related to Bugzilla and providing folks with a [...]

By: Alan Bonnemaison

Alan Bonnemaison — Tue, 16 Jun 2009 18:40:03 +0000

The “velocity” metric Dan proposes seem to overlook bugs that never make it to production.

If a developer fixes a bug with the wrong branch of code, a QA person will quickly catch the regression and re-open the bug.

Consequently, the velocity of this bug would be extremely low and fit outside of the “data cloud”.

My 2 cents.

By: Dan

Dan — Mon, 15 Jun 2009 10:07:08 +0000

How quickly bugs go from being filed to the VERIFIED state. Then a breakdown as follows, to identify which areas take the longest:

How quickly bugs go from UNCONFIRMED to NEW, NEW to ASSIGNED, ASSIGNED to RESOLVED and RESOLVED to VERIFIED. Perhaps on a per product basis as well as overall. This is the best way to assess the general workflow I think.

By: Bob Dunn

Bob Dunn — Fri, 12 Jun 2009 20:14:13 +0000

@deinspanjer – I view them as follows:
Re-fix would be a bug moving from Resolved-Fixed to Reopened
(failed the QA verification)
Regression would be a bug moving from Closed to Reopened
(passed QA but was reopened later)
This could mean several things that the organization would need to identify internally.
QA improperly passed the fix when it should have been rejected.
Maybe it was fixed, verified, closed, and then got screwed up again later.

It simply may not be possible with Bugzilla as it currently exists.

By: deinspanjer

deinspanjer — Fri, 12 Jun 2009 16:24:36 +0000

@Bob
Could you tell us a little about how we might be able to detect the difference between a re-fix, a regression, and any other sort of re-opening?

By: Bob Dunn

Bob Dunn — Fri, 12 Jun 2009 16:10:00 +0000

I’d like to see tracking of bugs that get closed and then re-opened at a later date. Not re-fixes (failed to pass QA after being marked resolved), but regression on fixed bugs (passed QA, but someone used old code in a later build or broke it while fixing another bug).

By: Alan Bonnemaison

Alan Bonnemaison — Thu, 11 Jun 2009 20:56:43 +0000

(Component) Reliability would be an interesting metric.

Counting the number of bugzillas (where STATUS=’NEW’) for a given component, one can use a time factor to determine the number days elapsed in between to determine reliability.

The lower the number of days, the lower the reliability. The higher the number of days, the component is judged more reliable…

By: Majken "Lucy" Connor

Majken "Lucy" Connor — Thu, 11 Jun 2009 04:03:01 +0000

Bugzilla is a very interesting beast, it’ll be very cool to see what sort of questions do get asked as well as what the answers are.

There’s always a lot of debate about the UI, this sounds like it’ll really help define the bugzilla workflow and make it easier to improve it.

I think something important to measure would be time from a community submitted patch to some sort of response or review. Or even what areas are we getting outside interest in.

By: Max Kanat-Alexander

Max Kanat-Alexander — Wed, 10 Jun 2009 23:43:28 +0000

I’m interested in this information, too! :-) The Bugzilla Project would be very interested in finding out what you come up with as results, here.

-Max