mrz's noise

noise from a mozilla IT/Operations wrangler

Archive for Mozilla

Level3, post BGP turn up

Derek turned up BGP peering with Level3 this morning out of San Jose (this was supposed to be part of last night’s maintenance but he ran into turn up problems on Level3′s side). We’ve grown (bandwidth-wise) to the extent that having two transit providers wasn’t sufficient. Should either fail, I wouldn’t have felt comfortable pushing… Read more

Categories: Mozilla, Networking

Switch IOS Upgrade Post-Mortem

Last night Derek attempted to upgrade IOS on core1 & core2 to pickup software support for Cisco’s ACE module. We ran into several issues and postponed the upgrade on core2 until those issues are resolved. The switch’s Compact Flash cards weren’t formatted in a format that rommon (the low level boot loader) could read and… Read more

Categories: Mozilla, Networking

Uptime

I’m fairly conservative when it comes to upgrading switches. I generally only upgrade to pick up security fixes. It’s rare that I’ll upgrade just to stay current. Tonight’s one of those rare exceptions. Derek is upgrading both core1 and core2 to pickup software support for Cisco’s ACE module. We’re planning on using this in conjunction… Read more

Categories: Mozilla, Networking

My face is cold

NSID 2008 had to come to an early close for me today. I leave Monday for my sister’s wedding in Chicagoland and while I would have loved to hold out till the end of the month or even a few more days, that just wasn’t realistic (something about looking presentable). You could argue I could… Read more

Tags:

Categories: Mozilla, Non Work

Load Balancer performance issues, fxfeeds.mozilla.org & versioncheck

I mentioned briefly in Monday’s meeting the performance issues we’re having with our load balancers.  Since then, we’ve been hustling to turn up something in the short term to handle Thursday’s Major Update and Firefox 3.0.5/2.0.0.19 release (see here). For a number of months we’ve been looking at Zeus and their ZXTM product.  It has… Read more

Tags: , ,

Categories: load balancing, Mozilla

border2 upgrade done, one more to go

Completed one of the router upgrades I mentioned the other day lastnight.  One real issue I ran into was the built in sup-bootflash: was too small to hold the IOS image I wanted and I wasted some amount of time deleting/squeezing sup-bootflash: and remembering the boot system syntax to boot off disk0:. Thought I’d share… Read more

Categories: Mozilla, Networking

Router upgrades, San Jose

A couple months ago I mentioned how things have grown in the past two years at Mozilla.  Back then we barely pushed any traffic to the Internet and survived on less than a dozen app servers. Things have changed.  I’ll highlight just a couple of them: Active Firefox users grew from roughly 20 million users… Read more

Tags:

Categories: Mozilla, Networking