It turns out that the main problem with the older Mac Minis is caused by poor air circulation. Sometimes turning them on the side works but it wasn’t helping. Tony’s solution is up on the QA wiki.

On a plus note, the drill was actually won as a prize at the QA outing a couple of weeks ago so I’m glad that we finally found a use for it.

A certain amount of silliness seems to come into play at times though.

This is the man who recruited me to work at Mozilla with his shiny new Zoob hat:

In contrast, here is a chunk of the QA organization going over automation work:

Quite a contrast!

P.S. We love ya, Bret!

With getting 2.0.0.5 and then 2.0.0.6 out for Firefox and Thunderbird things have been a little… busy around here.

On the virtualization front, QA did decide to go with the solution based on VMware’s work. There were a number of reasons for this but the primary two are:

1. VMware virtual machines can run on any of the platforms we use in QA or Development (OS X, Windows, and Linux).
2. VMware is already supported for other systems in the company fore infrastructure.

It made sense to standardize around one set of tools.  The first reason is probably the most immediate since the different people in QA do their work in wildly different environments. We don’t mandate any single operating system, for example. Having virtual machines that everyone could use reduces a lot of redundancy (and redundancy means more work for yours truly).

We do still want to get the the point where we can have a pool of machines available that community members could connect to from remote and on which a QA supported virtual machine could be loaded. We don’t have an ETA for this but it is something that we are actively working on.

I still have no reliable solution for virtualizing OS X. This is one that could really bite us at some point as we can have a mix of “clean” and “dirty” environments for all of our other supported operating systems but not anything from Apple.

When we were doing 2.0.0.5 and 2.0.0.6 testing, have virtual machines available made working on update scenarios and localized installations a lot easier since we could switch operating systems very quickly in the same environment and restore them to a good state for the next set of tests.

Anyway, that’s all for now here. I’m hoping to post a bit more this month though.

Parallels has gotten a lot of good press lately and most of us are working on Apple hardware so it has been enticing to us. Since VMware Fusion is still in beta, we have not necessarily wanted to bet the farm on it. That being said, as most people in the community know if it is something they watch, we do use VMware on a lot of our servers. It would make sense for us to standardize on a VMware-based solution across the board.

If anyone has any thoughts on the pros and cons around that, I would like to hear them.

As I mentioned in my last post, we’re currently working on putting together profiles for test environments. There is a wiki page up about this effort. I’ll be importing a lot of this profile data into virtual machines that I’ve created, using the clean ones as a base. This will allow us to do quick QA on builds in environments that are a lot closer to what Firefox users actually have. Future plans beyond this are pretty open at this point.

One gaping hole in our virtualization strategy is what to do about OS X. Currently, none of the virtualization solutions allow you to work with OS X and Apple is pretty hostile to the whole idea. This makes things occasionally difficult as we sometimes have to either hunt down an appropriate Mac or run test builds on our work machines and hope something really bad doesn’t happen. There is no way to abstract things out. As it is, I’m probably safer testing on Windows XP inside of a Parallels instance if I hit a really bad build because I can always restore back to a snapshot from before the installation. I do wish that there was a supported (heck, unsupported even) virtualization solution for Macs.

Back when I first began doing QA work in the mid-90’s, most people involved would have a stack of machines next to or under their desk and a switch box. Tools would be used to wipe machines to a known good state before putting new builds of software on them. If people were really lucky (and moreso as time went on), they would have access to some nice tools, which as client-server imaging software, to automate a lot of this process of getting machines ready.

One of the nice changes over the last five years is the growing amount of virtualization technology being used. Companies like VMware and Parallels are making good businesses at it and there is a reason for it. At my last job, I had to test our software on many different configurations of operating systems and browser types in order to test that our web application was going to work well across the board. Instead of having the bank of machines that I used to have, I was able to do it all on a fairly beefy laptop by using VMware for my work.

Here in the QA group, we’ve been investigating what we want to do with virtualization. I know that we’ve had some discussion about having servers running virtual machines for the community to log into when they have time so that people have access to varied environments. We’re not quite up to that level yet but one of the things I’ve been doing is making a reference set of virtual machines available internally (unfortunately internally because of licensing issues with Windows) so that when a build comes out, we have stable and clean environments to quickly test on. From there, we are working on creating a variety of not-so-clean environments, because most people don’t run Firefox and Thunderbird in some kind of pure clean environment (my home machine is a mess) so we need to be able to test the builds in something more real. Our thoughts around this are being documented in an ongoing basis on the wiki.

If you have thoughts on any of this and the pros and cons of virtualization in general, I’d love to hear about it. Please take a look at the wiki page as well as the thoughts there are pretty preliminary also.

It looks like I’ll be working with Juan on the next 2.0.0.x release, when it occurs, along with some of the Firefox 3 QA efforts. I’ll probably post a bit more when I have something of substance to add…