Where we are in January 2012

The purpose of this post is to present a very high level picture of the current Firefox build & release process as a set of requirements. Some of these services are provided or supported by groups outside of releng (particularly it & webdev). This diagram will be useful in understanding the impact of changes.

System Scope

Here is how I’m defining the boundaries of the current system:

• Applies to anything & everything involved in producing artifacts that are directly installed on an end user’s device.
  • In: Firefox, Thunderbird, Seamonkey
  • Not in: services such as browser id, sync, the addons.m.o., apps, and other code that executes on Mozilla’s servers.
  • Not in: extensions themselves (they are installed into FF/TB, not directly onto the end user’s device). Yes, that is a hair I want to split. [1]
• But only if the entire process happens "inside the Mozilla firewall". (For example, the download mirrors are not in scope.)

[1]	There may be some extensions, such as the Thunderbird Lightening, that cross this line. For now, I’ll ignore those.

The Developer Services

N.B. these are only repository related services that a devloper commonly accesses.

With this setup, there is official support for:

• pulling from Mercurial repositories hosted at mozilla.org (hg.m.o)
• commit access from developer’s own clone of the Mercurial repository (with committer policy enforcement)
• "try server" access from developer’s own clone of the Mercurial repository
• continuous integration based on commits to hg.m.o
• tracking of results of commit compilation & testing, via tbpl.m.o.
• web viewing of repositories
• web cross reference of many repositories on mxr.m.o

The Picture

The Ideal Future

The Ideal: In the future, each contributor will be as free to choose their DVCS of choice as they currently are to choose a text editor.

Since that’s a pretty general statement, let me limit it a bit with some caveats that I expect will apply.

• This may only be true at a technical level, as various teams might have social conventions or alternate tooling that will limit DVCS choices.
• Initial planning will be for products [1] that are released via the Release Engineering team. [2]
• The current set of supported DVCS would be hg and git. By supported, I think folks roughly mean:
  • an officially supported way get the latest revision of any official branch/repository, without any extra work on the part of the developer.
  • an officially supported way to interact with existing Mozilla hosted tools, such as the try server, tbpl, etc.
  • an officially supported way to commit from the DVCS, that supports the current commit workflow (and policy).
• There will be support of social coding sites. I think support means providing an officially supported mechanism to keep "current" instances of each official branch in the social coding sites. The goal would be to allow developers to utilize the enhanced services offered by those sites to view, clone, and share work prior to doing an official commit.

Next Steps

1. This post will help validate that the above vision has a wider consensus. Please pass this link along and add your comments below.
2. Keep an eye on the main page for more material.

---

[1]	(1, 2) I’m using "products" to distinquish binaries Mozilla provides to the end users to install on their computers from "services" Mozilla hosts. Product examples include Firefox and Thunderbird. Service examples include Sync and BrowserID.

[2]

The rationale for starting with existing products is that is where the majority of our developers work. Additionally, the existing continuous build & test infrastructure will place some constraints on DVCS workflows. Once those constraints are documented, other projects can make informed decisions on their usage.

Posts in the Series

Overview & context:

• Where we want to end up
• Where we are today

Glossary

The project goal (above) has a few terms that need defining, so they mean the same thing to all of us. And the discussion will introduce some terms as well. As these come up, they will be added here.

Mozilla Products

Software delivered to the end user for installation directly on their computer.

Releng Infrastructure

The processes & machines that are used to produce Mozilla Products, from the source code repository through continuous integration and testing to the product binaries ready for the end user.

Related & Reference Material

• Master tracking is occurring in bug 713782
• John O’Duinn’s post "is it git or is it github"

What folks have already done:

• doublec’s mozilla-central on github https://github.com/doublec/mozilla-central
• bholley & jlebar’s tools to easy committing to hg from git https://github.com/jlebar/moz-git-tools

Summary

So things are different at Mozilla, but isn’t that always the case when changing jobs? Well, yes, but Mozilla appears to be more different:

• We’re guided by our Mission Statement.
• We produce open source products.
• We’re owned by a not for profit foundation.
• Much that most companies would consider “trade secrets” or “proprietary information” about process or plans, we discuss in public.

But, how different does this make the inner workings of Mozilla? As a release engineer, how would my day-to-day work change? How much of what-I-thought-I-knew need to be thrown out? Would I be putting myself into an “everything you know is wrong” situation? [2]

I’m sure I’ll be dealing with this cultural shift for at least my first year. I’m looking forward to this, as it will be a wonderful opportunity to examine a number of assumptions I’ve mad for years, that may no longer be useful. (And, even if one assumption is discarded, I don’t know if it will be replaced by one that contradicts the original, or subsumes the original with a more general understanding of the process.)

For example, one such question I’m currently pondering is: what is the role of the VCS (version control system) in an environment like Mozilla? Does it differ from private enterprise? If so, how? This question is one I’m keenly interested in, both philosophically, and practically. As is often the case, writing about it clarified some things for me. I hope folks who comment will help polish it further.

What Services are provided by a VCS

One might (and I suspect many readers will) think this is a silly question. A VCS exists to support developers. Period. Historically, that’s a very valid point. If one is just looking at software development, we can put the development team in the center of everything:

Most development operations now have a number of automated systems which interact with the VCS, including:

• defect tracking
• continuous integration
• metrics (code churn, bug density analysis, etc)

Toughest of all to spot are the non-automated policies that rely on certain behavior being stable over a long period of time. These implicit contracts impact operations such as:

• legal audit trails [3]
• disaster recovery plans
• the big boss’s custom metrics script that assumes things don’t change.

I won’t attempt to draw it, but now imagine that every data store and automated process in those diagrams need to be replicated or clustered for scaling and HA. All of a sudden, our nice neat little VCS system has gotten out of hand. Any change has to meet the needs of the entire user community, not just the developers.

---

[1]	Here I’m using “product” to mean something that is produced my Mozilla and installed on end user devices. (Firefox, Thunderbird, Seamonkey are all examples.) Mozilla also provides services, which may have different needs.

[2]	This is what programmers were told when they started to program the first Macintosh, as it required a change from procedural flow to event driven flow. Of course, that was a huge lie, as many programming fields already used event driven programming. (Even RPG II, but that’s another story.)

[3]	Mozilla has these – the commiter’s agreement combined with commit level authorization