Ben's Blog

I’m hoping to tackle bug 505512 (Make infrastructure related problems turn the tree a color other than red) in the next few weeks. Most of the ground work for it is laid, which means that most of what I’ll be doing is parsing logs for infrastructure errors.

So, what errors do you see most from our build infrastructure? Are there other things that you would classify as infrastructure issues? Please add any suggestions you have to this Etherpad: http://etherpad.mozilla.com:9000/build-infra-errors

This morning I landed bug 486567 – which cleaned up the try server code significantly. There’s still more to be done there, particularly running unittests on packaged builds once it’s production counterpart lands (bug 383136). Both of these things help us keep the Try Server in sync with the rest of the world – which has always been a problem.

Looking forward a little bit, I’m looking to land a patch that enables e-mail notification for try server builds and unit tests on Tuesday. With this patch, every try submission would result in 6 e-mails to the submitter: (1 per platform/build type combination). Here’s what they’ll look like:
Build:
Your Try Server build (try-1c170baeac1) was successfully completed on linux. It should be available for download at http://build.mozilla.org/tryserver-builds/bhearsum@mozilla.com-try-1c170baeac1

Visit http://hg.mozilla.org/MozillaTry to view the full logs.

Unit test:
Your Try Server unit test (try-1c170baeac1) completed with warnings on linux. It should be available for download at http://build.mozilla.org/tryserver-builds/bhearsum@mozilla.com-try-1c170baeac1

Summary of unittest results:
check: 2/0

Visit http://hg.mozilla.org/MozillaTry to view the full logs.

(The unittest e-mails will have the full results listed, of course).

E-mail notification has been an oft requested feature so I’m really excited that this will be landing soon.

Both mozilla/tools/buildbotcustom and mozilla/tools/buildbot have now official moved to their new homes in Mercurial, http://hg.mozilla.org/build/buildbotcustom/ and http://hg.mozilla.org/build/buildbot/ respectively. The buildbotcustom CVS module will be deleted at the end of this week. We will be keeping the buildbot one for the time being because we have a Buildbot still using it, but all future work should be done against the Mercurial repository.

The time has finally come for these two CVS modules to get a better home. Here’s what I’m proposing:

1. Early next week I will import these modules (with history) into http://hg.mozilla.org/build/buildbotcustom and http://hg.mozilla.org/build/buildbot.
2. On Friday, January 30th, 2009 I will delete mozilla/tools/buildbot and mozilla/tools/buildbotcustom from CVS and put in READMEs pointing to their new homes. I don’t believe there is any reason to keep them around once the new repositories are live. They will only serve to cause confusion.

Moving these will be a change for the better, I believe. It will let us import new versions of Buildbot without the hassle of CVS and in the case of buildbotcustom, let others work on it more easily.

If anyone has objections to this plan please respond to this mozilla.dev.builds post. If I don’t hear anything by Tuesday, January 27th I will proceed with this.

During this time we will be upgrading both the Firefox 3 and Firefox 3.1 Unit test Buildbots to a newer version (0.7.9). In order to avoid interrupting running builds we will be closing the tree at 4am PDT and stopping any new builds from being scheduled. Once all builds have finished we will perform the upgrade and open the tree again. Depending on the timing this could take anywhere from 20 minutes to a few hours. The tree should be open again no later than 7am PDT.

If there is any reason why we shouldn’t go ahead with this please e-mail release@mozilla.com

Buildbot 0.7.9 was released a couple weeks ago. Today I will be importing it into our CVS tree, upgrading the aging 0.7.7 code. Before doing an import I will branch the current tip of trunk as BUILDBOT_0_7_7_BRANCH. This will let us easily checkout 0.7.7 code and if necessary, land 0.7.7 specific fixes.

Any checkouts of mozilla/tools/buildbot on HEAD will update to the new code with ‘cvs up’. If you don’t want this to happen you should delete your checkout and get the BUILDBOT_0_7_7_BRANCH (‘cvs co -r BUILDBOT_0_7_7_BRANCH’).

Here’s some of the more exciting fixes and enhancements:

• The /buildslaves page now highlights disconnected slaves, making it easier to see that information at-a-glance
• Build properties can now be passed via a Scheduler
• Build properties can be set with ShellCommand’s through the new ‘SetProperty’ BuildStep

Not much else to say – bug 437143 (try servers need to support building from a mercurial repository with a patch) has been fixed. You can now submit patches against HG repositories with a patch.

Many thanks to Vlad, who did most of the work on this.

Not much else to say – bug 437143 (try servers need to support building from a mercurial repository with a patch) has been fixed. You can now submit patches against HG repositories with a patch.

Many thanks to Vlad, who did most of the work on this.

During this time we will be moving around tinderboxes (as discussed in the “Moving Tinderboxes” thread). After this maintenance is done, the following changes will have been made:

* All Firefox 3.0.x/Gecko 1.9.0.x/CVS trunk tinderboxes will report to the soon-to-be-created “Firefox3.0″ tree. (http://tinderbox.mozilla.org/Firefox3.0/)

* All Firefox 3.1/Gecko 1.9.1/mozilla-central tinderboxes will report to the “Firefox” tree.
(http://tinderbox.mozilla.org/Firefox/)

* The ‘Mozilla2′ tree will be deleted.

Sometime today (May 12, 2008) the Mozilla2 Buildbot will be down to enable debug+leaktest slaves.. For the curious, details of the work are in bug 422296.

If there’s any reason we can’t go ahead with this please e-mail release@mozilla.org.