Justin's Blog » Uncategorized

DNS cache poisioning – is your name server patched?

justin — Fri, 25 Jul 2008 03:15:40 +0000

As many of you have probably read, there is a lot of buzz about the recent multi-vendor DNS vulnerability. The details have to do with weak transaction IDs used by caching name servers and the ability to modify those cached DNS records if you can predict the transaction ID. Patches to all the major DNS systems are out and Mozilla DNS servers have been patched for some time, even though our publicly accessible name servers are not recursive or caching name servers.

While we have done all we can by patching our systems, you should check/yell/complain to your upstream DNS provider and apply pressure to get their servers patched as they mostly likely cache name records for you. There are a lot of tools out there to check if your favorite caching name server is vulnerable – http://www.doxpara.com/ and http://entropy.dns-oarc.net/test/ are two that I have seen used.

Virtual utility computing, finally a reality (mostly)

justin — Thu, 22 May 2008 03:24:23 +0000

For some time vendors of all kinds have been pitching the promise of utility computing for heterogeneous systems/infrastructure. “Just throw more servers in the pool, no need to think! It’s easy!” – yea, right. It’s an easier task if all your systems are running the same application – great load balancers have been around a long time to help with this, but not when they are running different applications and have different load profiles. The promise as pitched is quite compelling – if your compute pool is running hot, just add more resources to a large pool of cpu/memory/disk, allowing the “system” to load balance, handle failures, and alert of issues. The issue has been that the tool sets to implement, manage and maintain have been lacking – weak partitioning tools, HA schemes that didn’t work in all situations, difficult migrations or downtime events to add more resources.

Recently mrz took on the task of upgrading our VMWare infrastructure to the latest update – 3.5. As part of that upgrade, he investigated and subsequently implemented dynamic resource scheduler or DRS. Once up, he moved over all the ops/IT virtual machines (VMs) to the resource pool.

After seeing how it works, I think truly utility computing may finally have arrived. Once migrated to the resource pool, the system took inventory of all the VMs, inspected the load (both cpu and memory) of all the esx hosts, then made recommendations on how to load balance them – but thats not all. With *no* downtime, it automatically performed these migrations, with no user interaction via vmotion (which in itself is pretty amazing). Super slick. It keeps tabs on how loaded the esx systems are, will perform VM migrations automatically if cpu load or memory usage on servers change, allows for reservation of cpu/memory and gives partitioning functionality in case you want to make sure 2 VM’s aren’t ever on the same esx host.

After working with the system a bit – it became quite apparent that the idea of utility computing for heterogeneous machines is finally here, center stage. No longer do we need to manually balance pools of VMs, worry about downtime for upgrades, etc – it’s all handled. Quite literally, as we need more capacity we simply add more hosts to the resource pool.

As far as it’s come, I still think there are some areas for improvement (which may have already been solved, I just haven’t heard about them):
* Storage – still have to allocate a VM to an array of some kind. While there are ways to move the VM storage around while live, it’s not tied into the DRS system and is a manual process. I’d love to treat storage (both space and iops) as a resource pool and have dynamic allocation of storage resources.
* Management tools – I know of some third party tools that help with this, but I think this should be a focus point for VMWare. They have the most information on the system and really should make metrics a top priority.

Regardless, it’s come a long way, and using virtualization for the right areas of your infrastructure can really pay off. It’s really made a real difference to our users and admins in the areas of management, power usage and overall server utilization. Now, if someone could *please* compete with VMWare – some competition (no, Xen is not there yet, but it up and coming) would truly be welcomed.

Bugzilla improvement project update

justin — Wed, 13 Feb 2008 03:33:01 +0000

As you may know from my last post on Bugzilla, a lot of improvements/fixes are in the works. Wanted to give everyone an update on what has gone live on bugzilla.mozilla.org so far:

– Send mail in the background after confirming to the user instead of waiting for the mail to be sent while the user waits (related to bug 284184 – local backport)
– Fix for a regression from our last upgrade involving mid-air collision detection (bug 413258/415490)
– Fix for a problem with the OpenSearch plugin (bug 411844)
– Allow searching for ‘—’ in versions and milestones (bug 362436)
– Fix for Subject lines in emails being improperly line-broken and erratically spaced (bug 411544)
– Add a References header to notification mails to assist with threading in mail clients (bug 376453)

All in all, a very good beginning, but we have much more in store. Work has already begun on phase 1 of our project, scheduled to be complete in Q1 of 2008. We are very excited about the Bugzilla improvements, hoping it really helps improve productivity for the project!

Related, we are always looking for people to help out. If you are interested in working on making Bugzilla better for Mozilla and the rest of the OSS community, please get in contact me. The more people we have working on this, the faster the improvements come

Umbrellas make people happy

justin — Tue, 15 May 2007 21:14:11 +0000

Recently there have been a lot of cube moves at MoCo headquarters – IT included. We are back upstairs in building K, but the sun was creating an issue for a few of my guys. Enter Cosco to the rescue:

Kinda like their own IT oasis. Umbrellas make people happy – I mean, what else could get Aravind smiling like this