Taras’ Blog

Dear lazyweb,

Please explain to me why the following code works the way it does. From looking at the following code and stringstream::str(), stringstream::str(string) docs the behavior of the following code does not make sense to me.

#include <sstream>
#include <iostream>

using namespace std;

int main(int argc, char**) {
stringstream ss(”foo”);
cout << ss.str() << endl;
ss << “bar”;
cout << ss.str() << endl;
ss << “more”;
cout << ss.str() << endl;
}

Read the rest of this entry »

I work on static analysis by lamp-light and proudly cycle wearing Firefox imagery by daylight. Today I was pleased to discover that the flyer for the Portland Century ride now prominently features the Firefox logo in one of the 4 photos. It’s also in the scrolling thing on the website =D

This picture was taken on one ride with the most firefox fans ever. On parts of the course, it felt like every 5 minutes someone was volunteering info on how awesome firefox is.

When started on my static analysis quest just over a year ago. I imagined a perfect world in which I make tools and people use them to do awesome analyses. Since I did not want to be disappointed, I imagined this, but did not think it would come true.

Now we are at a point where static analysis use is growing rapidly, most analyses are done by people other than me as lately I barely have time to work on actual analyses. Treehydra and dehydra now have users and are well on their way to being released, which is taking up most of my time. Some of the most notable happenings:

• Vlad Sukhoy appeared out of nowhere and ported Dehydra to GCC 4.2 on OSX. This is exciting because it showed that the plugin system is portable between GCC release, and it’s the biggest patch from a non-core dev.
• We finished the paper on our static analysis work to be presented at the GCC summit. I am looking forward to meeting developers that built the GCC features that made the *hydras practical.
• There is a lot finishing touches being done such that we can release Dehydra 0.9 and eventually 1.0. Bug.
• Dave Mandelin implemented a proper testsuite for the *hydras. This is a massive step up from what we had before.
• There is a massive amount of Treehydra work going on. Looks like it is boldly going where no static analysis has gone before, even faster than Dehydra did. So far it looks like Treehydra is going to be a bigger deal than Dehydra could ever be. It is turning out to be a very potent combination of GCC for features and JavaScript for ease of use.

I have elaborate plans on how to take over the world with static analysis, more on that later. In the meantime I’ve started compiling a tracking bug of ongoing analyses for Moz 2.

I will be presenting on the work we are doing on Mozilla 2 at FISL08. Chris posted an excellent breakdown of Mozilla plans for the conference.

If you are at all interested in ground-breaking development happening in Mozilla 2, the challenges of static analysis then look me up!

This is all very exciting as it’ll be a trip of many personal firsts: Brazil, the southern hemisphere and a conference of this scale.

Joshua, pointed me at a fabulous article over at the Economist. The brave souls went to great deal of effort to compare, contrast static and dynamic analyses in an easy to understand fashion. My favourite part of the article:

Seth Hallem, the co-founder of Coverity, which makes a static-analysis tool, expects greater integration between programming and testing tools in future.

I suspect in the future, there will be awesome tools that even integrate into one’s compiler. Egads!

Treehydra

I spent a few days chasing my own tail looking for bugs in the C->JS conversion code. Turns it out it wasn’t bug, but a manifestation of GCC having slightly more AST mutation than I expected*. Bugs in the mental model hurt most The upside is that this forced me to switched the conversion process from eager to lazy which also gave a big performance boost. I hope to finally have something capable of doing initial analyses by the end of the week.

* In a perfect world compilers are written in functional languages where AST are transformed instead of mutated.

I have not blogged recently because I have been busy working on a rather fancy new mode for Dehydra. Turned out it is indeed possible to use JavaScript to walk and generate code to automagically convert thousands of recursive C structures into corresponding JS Objects. Now dehydra will have two modes: a simple pattern matcher that is easy to get started with and a hardcore mode for compiler geeks capable of advanced analyses. More on this later, for now subscribe to static analysis mailing list for more information.

Turns out that I’m not the first person to embed SpiderMonkey into GCC.

I have gotten in touch with two different GCC plugin projects. Seems that other projects are more academic in nature and still in the early design/development stages.

In constrast, Dehydra efforts are driven by existing unmet needs. In two months we went from having a crazy idea about using GCC for static analysis to having Benjamin integrate support for Dehydra checks into the moz 2 development repository to be run in a tinderbox.

I am also excited to see that people are discovering that Dehydra can be used to explore the codebase. We are not quite at the stage where one can interactively query the codebase from an ajax UI, but we are making steps in the right direction. As part of this trend, dehydra documentation is starting to migrate to MDC.

David Mandelin has been going through Pork fixing bugs and now it is almost working with GCC4. Prior to Dave’s involvement people would run into Elsa’s scary template bugs and switch back to GCC3.4 while trembling with fear. I asked Dave to fix a few easily work-aroundable (in Moz, not in Elsa) crashes in Elsa caused by Mozilla source and he did. Recently, I accidentally ran Elsa on a Mozilla .ii file produced by GCC4 and magically it processed! Turns out that the 3-4 bugs that Dave fixed also fixed 90% of the failures caused by GCC4 headers. The good news is that soon people won’t have to worry about that pesky GCC3.4 dependancy and run rewriting tools easier.

GCC Dehydra

I feel like I’m nearing the feature-complete milestone. I plan to have Dehydra generate conversion code from GCC tree structures to JavaScript objects so we’ll be able to easily hook into any GCC middle-end pass in pure JS.  Dynamic, GCed, introspective nature of JavaScript should allow for very rapid development of code police/extraction/grep tools. This should result in a consistent codebase for Mozilla 2.

I added an “ongoing work” and “tools” sections to the Mozilla 2 page.

The pork suite now has a wiki page.

#mercurial regulars kindly educated me about hg branches. Turned out hgimportsvn tries to map some subversion concepts onto hg branches which causes problems for people checking out pork using never versions of hg. For people googling for this issue: do “hg up trunk ; hg branch -f default; hg commit” to produce a magic empty revision to fix this problem.

If you have tried to compile pork before and gave up in despair, please try again using the pork wiki page for instructions.

I’m having a very painful time rewriting Mozilla source code to switch it to garbage collection. So I took a little break to think about Myk’s JavaScript rewriting idea.Then I found excellent info on parsing JS with SpiderMonkey. The downside to SpiderMonkey is that it is in C and thus hard to reuse, the upside is that will be able to parse all valid JavaScript.

I think I shall find some free time to prototype a little tool to take a .js file and produce a JSON representation of it such that one could write transformation passes in JavaScript. This will be useful as it will complete DeHydra by enabling it to process JS in addition to C++ and it will save a lot of time for various JavaScript refactorings.

I expect developing a JS refactoring tool to be relatively trivial compared to refactoring C++ as there is no source-code mangling going on due to lack of preprocessing (but JS is dynamic and can be embedded in various document types, so that could complicate things). Perhaps it would even be of use to mozpad people.

Looking forward to playing more with this idea.

I’m going to be on vacation until September 25th. I started the day by planning out how to do the rewriting tool to implement Benjamin’s spec. I’ll have to add more position information to elsa to do things properly and that might take a week or two. However, I’m into instant gratification so I wrote up a hacky prototype that results in a patch like this.

Say hi to garburator, a tool that will assist with the reference counting -> MMgc transition. The tool is less than 100 lines long, so someone could finish the hacky version while I am away. Either that or one could teach elsa to keep detailed position information in declarations and identifier names and clean up the postvisitS_decl() method in garburator.