Wowza! That’s a Feature!

Posted on 2012-02-21 by Hal Wine| Leave a comment

 

tl;dr

Wowza! I found the killer feature in git – you can have your cake and eat it, too!

Every time I’ve had to move to a new VCS, there’s never been enough time available to move the complete history correctly. Linux had this problem in spades when they moved off BitKeeper onto git in a very-short-time.

The solution? Take your time to convert the history correctly (or not, you can correct later), then allow developers who want it to prepend it on their machines, without making their repo “different” from the latest one.

I only found a reference to this feature last Friday – it used to require breaking the warranty, but is not supported as a first class command. More later, but to see how (and create you’re own prepended history), see: https://github.com/hwine/git_playground/tree/master/prepend_history

Much thanks to Scott Chacon and his writeup on “git replace” at http://progit.org/2010/03/17/replace.html

As my README shows, I have some more details to learn, but for now, just WOWZA!!!!

→ Leave a comment

Posted in Mozilla

Tagged ,

Release “As-Is” at high level

Posted on 2012-02-01 by Hal Wine| Leave a comment

[Refer to the main page for additional context.]

Where we are in January 2012

The purpose of this post is to present a very high level picture of the current Firefox build & release process as a set of requirements. Some of these services are provided or supported by groups outside of releng (particularly it & webdev). This diagram will be useful in understanding the impact of changes.

System Scope

Here is how I’m defining the boundaries of the current system:

  • Applies to anything & everything involved in producing artifacts that are directly installed on an end user’s device.
    • In: Firefox, Thunderbird, Seamonkey
    • Not in: services such as browser id, sync, the addons.m.o., apps, and other code that executes on Mozilla’s servers.
    • Not in: extensions themselves (they are installed into FF/TB, not directly onto the end user’s device). Yes, that is a hair I want to split. [1]
  • But only if the entire process happens "inside the Mozilla firewall". (For example, the download mirrors are not in scope.)
[1] There may be some extensions, such as the Thunderbird Lightening, that cross this line. For now, I’ll ignore those.

The Developer Services

N.B. these are only repository related services that a devloper commonly accesses.

With this setup, there is official support for:

  • pulling from Mercurial repositories hosted at mozilla.org (hg.m.o)
  • commit access from developer’s own clone of the Mercurial repository (with committer policy enforcement)
  • "try server" access from developer’s own clone of the Mercurial repository
  • continuous integration based on commits to hg.m.o
  • tracking of results of commit compilation & testing, via tbpl.m.o.
  • web viewing of repositories
  • web cross reference of many repositories on mxr.m.o

The Picture

Block diagram

→ Leave a comment

Posted in Mozilla

Tagged

The Ideal DVCS Goal

Posted on 2012-01-26 by Hal Wine| 2 Comments

[Refer to the main page for additional context.]

Based on discussions to date, everyone seems to have similar ideas about what "supporting git for releng" means. Later posts will highlight the work needed to ensure the ideal can be achieved, and how to arrive there.

For this post, I intend to limit the viewpoint and scope to that of the developer impact. Release notions (such as "system of record") and scaling issues won’t be mentioned here. (N.B. Those concerns will be a key part of the path to verifying feasibility, but do not change the goal.)

As a reminder, I’m just talking about repositories that are used to produce products. [1]

The Ideal Future

The Ideal: In the future, each contributor will be as free to choose their DVCS of choice as they currently are to choose a text editor.

Since that’s a pretty general statement, let me limit it a bit with some caveats that I expect will apply.

  • This may only be true at a technical level, as various teams might have social conventions or alternate tooling that will limit DVCS choices.
  • Initial planning will be for products [1] that are released via the Release Engineering team. [2]
  • The current set of supported DVCS would be hg and git. By supported, I think folks roughly mean:
    • an officially supported way get the latest revision of any official branch/repository, without any extra work on the part of the developer.
    • an officially supported way to interact with existing Mozilla hosted tools, such as the try server, tbpl, etc.
    • an officially supported way to commit from the DVCS, that supports the current commit workflow (and policy).
  • There will be support of social coding sites. I think support means providing an officially supported mechanism to keep "current" instances of each official branch in the social coding sites. The goal would be to allow developers to utilize the enhanced services offered by those sites to view, clone, and share work prior to doing an official commit.

Next Steps

  1. This post will help validate that the above vision has a wider consensus. Please pass this link along and add your comments below.
  2. Keep an eye on the main page for more material.
[1] (1, 2) I’m using "products" to distinquish binaries Mozilla provides to the end users to install on their computers from "services" Mozilla hosts. Product examples include Firefox and Thunderbird. Service examples include Sync and BrowserID.
[2] The rationale for starting with existing products is that is where the majority of our developers work. Additionally, the existing continuous build & test infrastructure will place some constraints on DVCS workflows. Once those constraints are documented, other projects can make informed decisions on their usage.

→ 2 Comments

Posted in Mozilla

Tagged

Releng & Git Project Overview

Posted on 2012-01-26 by Hal Wine| Leave a comment

This is the first in a series of posts about the "support git in releng" project. The goal of this project, as stated in bug 713782, is:

… The idea here is to see if we can support git in Mozilla’s RelEng infrastructure, to at least the same standard (or better) as we already currently support hg.

My hope is that blog posts will be a better forum for discussion than the tracking bug 713782, or a wiki page, at this stage.

These posts will highlight the various issues, so that the vague definitions above become clear, as do the intermediate steps needed to achieve completion.

Glossary

The project goal (above) has a few terms that need defining, so they mean the same thing to all of us. And the discussion will introduce some terms as well. As these come up, they will be added here.

Mozilla Products
Software delivered to the end user for installation directly on their computer.
Releng Infrastructure
The processes & machines that are used to produce Mozilla Products, from the source code repository through continuous integration and testing to the product binaries ready for the end user.

→ Leave a comment

Posted in Mozilla

Tagged

… a view from Outside

Posted on 2012-01-22 by Hal Wine| 2 Comments

tl;dr

One of the things that excited me about the opportunity to work at Mozilla was the chance to change perspectives. After working in many closed environments, I knew the open source world of Mozilla would be different. And that would lead to a re-examination of basic questions, such as:

Q: Are there any significant differences in the role a VCS plays at Mozilla than at j-random-private-enterprise?

A: At the scale of Mozilla Products [1], I don’t believe there are.

But the question is important to ask! (And I hope to ask more of them.)

Summary

So things are different at Mozilla, but isn’t that always the case when changing jobs? Well, yes, but Mozilla appears to be more different:

  • We’re guided by our Mission Statement.
  • We produce open source products.
  • We’re owned by a not for profit foundation.
  • Much that most companies would consider “trade secrets” or “proprietary information” about process or plans, we discuss in public.

But, how different does this make the inner workings of Mozilla? As a release engineer, how would my day-to-day work change? How much of what-I-thought-I-knew need to be thrown out? Would I be putting myself into an “everything you know is wrong” situation? [2]

I’m sure I’ll be dealing with this cultural shift for at least my first year. I’m looking forward to this, as it will be a wonderful opportunity to examine a number of assumptions I’ve mad for years, that may no longer be useful. (And, even if one assumption is discarded, I don’t know if it will be replaced by one that contradicts the original, or subsumes the original with a more general understanding of the process.)

For example, one such question I’m currently pondering is: what is the role of the VCS (version control system) in an environment like Mozilla? Does it differ from private enterprise? If so, how? This question is one I’m keenly interested in, both philosophically, and practically. As is often the case, writing about it clarified some things for me. I hope folks who comment will help polish it further.

What Services are provided by a VCS

One might (and I suspect many readers will) think this is a silly question. A VCS exists to support developers. Period. Historically, that’s a very valid point. If one is just looking at software development, we can put the development team in the center of everything:
Figure 1
Most development operations now have a number of automated systems which interact with the VCS, including:

  • defect tracking
  • continuous integration
  • metrics (code churn, bug density analysis, etc)

Figure 2
Toughest of all to spot are the non-automated policies that rely on certain behavior being stable over a long period of time. These implicit contracts impact operations such as:

  • legal audit trails [3]
  • disaster recovery plans
  • the big boss’s custom metrics script that assumes things don’t change.

Figure 3
I won’t attempt to draw it, but now imagine that every data store and automated process in those diagrams need to be replicated or clustered for scaling and HA. All of a sudden, our nice neat little VCS system has gotten out of hand. Any change has to meet the needs of the entire user community, not just the developers.

[1] Here I’m using “product” to mean something that is produced my Mozilla and installed on end user devices. (Firefox, Thunderbird, Seamonkey are all examples.) Mozilla also provides services, which may have different needs.
[2] This is what programmers were told when they started to program the first Macintosh, as it required a change from procedural flow to event driven flow. Of course, that was a huge lie, as many programming fields already used event driven programming. (Even RPG II, but that’s another story.)
[3] Mozilla has these – the commiter’s agreement combined with commit level authorization

→ 2 Comments

Posted in Mozilla

… irssi->growl on Lion

Posted on 2012-01-04 by Hal Wine| Leave a comment

Okay, it’s a lie – I’m only bring you a link. Lukas pointed me to this neat hack to hook a remote irssi instance into local growl.

My only tweak was I had to find where growlnotify is kept for the (paid) version 1.3. It’s here.

Just got a growl message that my boss mentioned me in the team channel, so I know it works!

→ Leave a comment

Posted in Mozilla

Tagged

Survey Surprise

Posted on 2012-01-04 by Hal Wine| Leave a comment

[Just a quick note, because I love being surprised. More details on the survey after I scrub it some more.]

As some of you know, I put out a survey to learn a bit about how Mozillians are using hg & git & github. (Don’t worry if you missed it, as further incarnations of it are likely to be coming as I work through the support-git project.) First, a disclaimer: this survey was not rigorous (I’m not a psychometrician) and the sample size was quite small (42 respondents), from a skewed selection of the community (mostly employees).

I expected to find quite a few folks answering John’s question with "it’s not git, it’s github", and I did. But what did surprise me is why they liked github. It had little to do with defect tracking, code review mechanisms, or other proprietary workflow enhancements. (These were cited by some, and will be in further analysis.)

It had everything to do with "social coding", and the positive impact it has on Community!

Two thirds of respondents who expressed an opinion on github cited "social coding" as a major reason. WOW! Some of the more exciting answers to "What I like best about github is" included:

How you can easily see what people are doing with forks of your project.

Discoverability. Put your [stuff] on GitHub and people will not only find it but also fix it. Simple as that.

incredibly low barrier to hosting code in a highly visible way, and collaborating with other projects.

Ease of collaboration, I get pull requests from people I don’t even know.

I had previously heard from project leaders that they perceived that social coding sites (like github & bitbucket) increased community involvement. Some of these comments suggest we may be able to collect data on the effect.

I do enjoy surprises like this – they help clear away the cobwebs of "stuff that ain’t so" from my brain. This point may well help me focus the abstract "bring git to parity" into "remove impediments to using social coding sites".

→ Leave a comment

Posted in Mozilla

Tagged

… news of yet another new blog!

Posted on 2012-01-03 by Hal Wine| 1 Comment

I’m Hal Wine – a newish Mozillian (since 2011-11-14), still adjusting to life in public (aka no longer working for a major corporation).

Hopefully, I’ll manage to keep these posts both useful and easy-to-read.

Basic background – I’m a member of John O’Duinn’s release team at Mozilla. He was nice enough to welcome me with only a wee hint of blarney showing through. (Folks who know me will know the blarney is NOT about my bad puns.)

I’m excited to be at Mozilla, and helping out with some of the really neat things going on here. Some of those things I plan to start blogging about, “Real Soon Now™”!

→ 1 Comment

Posted in Uncategorized