Taras’ Blog

With much pain I managed to convince a Vmware WinXP virtual machine to run in VirtualBox. At first it ran very sluggishly(>3 hours to do a build?), but after I turned off IO Apic stuff in Windows, it’s become disturbingly fast. It now takes 40minutes to build a wince fennec vs almost 80minutes it took on VMware server.

VirtualBox’s disk throughput is phenomenal, in fact, this is the first time I’ve seen almost-native speed disk io in virtual machine. A benchmark I tried reported 45mb/s 4K read/writes (these were cached on the linux side).

Unlike Vmware server, VirtualBox’s usb works without troubles, shared folders were easy to setup etc. It’s an awesome app, hope Sun/Oracle keeps it up.

Python GDB Rocks!

I wanted a non-painful way to figure out what’s causing bonus file IO. I’ve noticed that gtk likes to open files, but I didn’t have the exact details. So I grabbed python gdb, and with some tips on syscalls from gdb old-timers managed to produce a report to assign blame for open()ing files to relevant Mozilla functions.

Other than the gdb-hating syscalls issue, achieving this was simple

1. Compile python-enabled gdb(Next set of distribution releases should have it..I hope)
2. Define a new gdb command in a python file. I called mine “taras” for lack of a better name.
3. Set a breakpoint, attach your command to it. :
   break open
source -p /path/to/your/script.py
command 1
taras
end
4. Have the script walk the backtrace to figure out the filename and the last Mozilla function. Log the info, issue gdb continue command.
5. Print out a report and profit:
   python report()

Here is my script. The only nasty part here is that I had to read the filename out of a register (i’m on amd64, on 32 it’d be $esi instead of $rdi) because gdb doesn’t deal well with system calls.

I’ve never throught it would be this fun to use gdb. I always thought debuggers should be scriptable, thanks to Tom Tromey (lots of gdb tutorials on Tom’s blog) and any others who finally made this a reality.

Every once in a while people want to be analyze C code. If you are one of those people, checkout bug 494960. For GCC 4.3, you’ll need to update your patch queue. If you you followed the Dehydra install instructions do something like:


cd gcc-4.3.0/.hg/patches
hg pull -u
cd ../..
hg qpush -a


and rebuild gcc to get C support in the plugin framework for GCC 4.5 (patch for trunk is in the bug).

Everything other than process_function should work in both treehydra and dehydra. Note that the objects provided by GCC will be slightly different in structure due to the underlying difference between gcc/g++, so scripts that were written for C++ may need to be updated.

I finally landed the remaining GCC patches needed to get a useful Dehydra without any of that gcc-patching pain!

This week I also got some nice contributions from a contributor with a masterplan.

Overall I expect cool things to happen once GCC 4.5 ships with plugin support. It’ll open up a jar of whoopass like C++ has not seen before.

Dehydra Docs

Benjamin redid the Dehydra documentation, it should be easier than ever to get started.

Static Analysis in Mozilla

Slowly but surely we are gathering static analysis momentum. More and more developers are pausing and thinking “I need to make sure this code has an analysis to go with it because there is no way I can prove it correct, time to write a Treehydra script…or ask someone else to make one for me”. This mindset is important for being able to introduce complex changes in a controlled manner.

On the refactoring side, things are held up a little by API compatibility. Perhaps multi-process stuff will shake things up. However, nothing can hold Chris up from letting his piglet/porky gremlins chew through locking code.

I wrote Fennecmark to automate some of the tasks that I did manually while doing performance debugging.

I tried to capture some of the “perceived performance” in numbers. My goal is to focus on user-visible areas of performance. Ideally it will enable us to track performance better to ensure that key features do not regress in performance and enable us to compare Fennec speed on various platforms. I need to spend some quality time with QA people to figure out how to achieve that.

Currently Fennecmark loads a slow-to-load webpage, zooms around it and then pans from the top to the bottom. This measures: responsiveness during pageload, zoom speed and panning lag.

See code at http://hg.mozilla.org/users/tglek_mozilla.com/fennecmark.

JSD Instrumentation

Spidermonkey provides an API that allows one to get a notification on every method entry/exit. I was able to do most of my Fennec performance analysis via a component in bug 470116. My stopwatch component times the execution of every js function call and spits out a log that has been very useful in figuring out what is taking up time in Fennec chrome.

Porkstain

I am itching to write a tool that can instrument large portions of Mozilla code such that it can be profiled across C++/JS boundaries and without any external tool support. I am guessing this would be most useful on platforms with crappy sampling tools, but it would be cool if it made finding slow codepaths easier in general. If you know any lightweight instrumentation techniques, please share.

I wrote a little prototype to insert stopwatch stuff into code deemed interesting by oprofile (stuff in the bug above). The code patching part works well, but it’s a big runtime hit and outputs too much data.

#static on irc.mozilla.org is now the correct irc channel for anything to do with static analysis.

Codecon
On Sunday, I will be presenting on Pork at codecon. I have been meaning to attend codecon since the days when P2P was considered cool, was not able to make it until this year. It is a historical milestone for me. Codecon was how people at Mozilla first heard of Elsa, which is now the foundation of all our refactoring tools (it is also inherited baggage I get to maintain).

Piglet

I adopted Piglet, Dave Mandelin’s de-oinkification project, and imported it into hg. It feels really good to finally be able to do a make -j without disturbing people nearby with surprise explosions of foul language. I plan to move all relevant static analysis tools into piglet. After that I shall finally merge a dozen or so elsa repositories and end up with Pork consisting of elsa/ + piglet/.

Pork*

Chris Jones is quietly working on making Pork magnitudes more useful to average developers. It’s exciting stuff and I’ll let him announce it when he’s ready. Between his work and David Humphrey’s DXR. I think we are finally going to make it easier to hack on Mozilla for a much wider audience than before.

After I wrote prcheck, I was surprised by the errors it found. I expected to find lots of cases of prbool variables having integers assigned into them. Indeed there were some of those, but the most frequent offenders were things like

NS_ENSURE_SUCCESS(rv,rv);

in methods with a PRBool return value. In this case (and many similar  return values within macros) the function will likely do the opposite of what was intended if there is an error condition. Here is a less hypothetical example in bugzilla.

So I’m thinking that instead of porting the prbool analysis to Treehydra (such that it’d based on a less buggy backend and can be integrated into the build) it might be more interesting to ensure that nsresults do not mix with other integer types. That would catch all of the worst prbool offenders and possibly other nsresult misfortunes.

Has anyone run into bugs like this that do not involve prbools?

I suppose a general solution would be to define a lattice of typedefs with rules specifying which typedefs can be assigned to each other. This would make GCC distinguish certain typedefs as discrete and incompatible types. Thoughts?

A couple of months ago Stuart casually asked me to investigate Fennec performance for moving about a page, zooming and loading pages in general. Beta 1 contains the result of that:

• There is little to no hardware graphics acceleration on mobile arm device. That combined with low memory bandwidth results in painfully slow screen updates (10x slower than crappy gfx on the desktop?). The painting engine now works hard to skip redundant draws of the page.
• During loading pages or zooming Fennec now only draws the minimum required. In my testing complicated pages load 2-5 times faster. Zooming is now 5 times faster.
• There is less DOM querying now. Things like checking an element’s size can cause pages to reflow resulting in a less responsive UI

Other performance highlights:

• Fennec now features a redesigned firstrun page which not only looks better, but also contributed to a 0.5second startup speedup. Overall Beta 1 should startup is almost a second quicker than the previous release
• The JavaScript JIT is now on by default providing a noticeable performance boost throughout Fennec.

For more info on Fennec Beta 1 and where to get it see Stuart’s blog.

I love new programming toys and this week is a good one for those. Ever since I laid the foundations for Dehydra I’ve been dreaming of a world where I can quickly lookup a piece of code(say something that someone complains about on IRC), fix it, get it reviewed and pushed in the most efficient manner possible. Seems that the pieces are finally falling into place.

1. I want quick semantically aware code lookup via DXR. And guess what, there is progress in that direction.
2. I want to DXR to provide a link to edit the code. Bespin looks like the most promising candidate for editing.
   As an aside, using canvas to do text editing is badass. I salute devs who are crazy enough to prove their point by reimplementing something (hopefully better) from scratch using an approach that hasn’t been tried before.
3. I want my changes to be saved as a diff into bugzilla. I want that to be two way so I can edit existing patches and save them as new bugzilla attachments.
4. From there I’d like a commit feature in bugzilla so the patch would go through try-then-push cycle that Jesse described.
5. Having all this inplace would make it trivial to integrate random features such as crash stack trace navigation or Pork automagic refactoring.

Now I’m sure that most of us would still run Emacs and other desktop editors for longer development tasks. But just imagine being bored with a computer at a webcafe, boring friend, etc and having the ability to quickly jump into in the development process as easily as logging into webmail.

When I wrote the initial prototype of Dehydra I pondered how long it would take before it’s adopted by security guys. Unfortunately, until now take-up has been non-existent. Grep and Perl still seem to rule in that community even though the plain text approach restricts the range of possible security scans.

Normally I would be tempted to rant on how grep is convenient yet limiting. However Ben Kurtz discovered Dehydra for security scans and did a great job explaining the issues involved. Thanks to Georgi for linking me to Ben’s post.