Comments on: Rewriting JavaScript

By: fredrik

fredrik — Fri, 26 Oct 2007 08:55:09 +0000

Yeah, it sounds like a lot of fun.

By: tglek

tglek — Thu, 25 Oct 2007 18:29:39 +0000

fredrik,
Yes dehydra tries to simplify the ast as much as possible before passing it to js. It just so happens that it starts from an Elsa AST.

In reality to be compatible with DeHydra, one can just provide the standard API and call the same callbacks that DeHydra scripts expect from any other JS program(including a webpage?). Thus JSON is a natural choice since one can write the simplification step in JS to keep life really simple

Any chance of you wanting to play with the easier approach?

By: fredrik

fredrik — Thu, 25 Oct 2007 17:57:24 +0000

But DeHydra does consume the AST that Elsa/Oink spits out on some level, no? Through a syntax tree visitor (JSVisitor IIRC). That’s what I gathered from browsing the code at least.

I just figured that if that was the common ground, having DeHydra just needing to understand the AST format produced by astgen, it would make things easier (barring AST node name problems). You could still use SpiderMonkey, have it output the JSON and then build the AST from that (or just build it directly). It’s entirely possible I’m wrong.

You’re correct re: the catch-up problems with SpiderMonkey, and that was one of my major concerns. It’s probably thankless to use Elkhound/astgen for JavaScript, both short- and long-term now that I think about it. Saves me wasting time on it .

By: tglek

tglek — Thu, 25 Oct 2007 16:01:03 +0000

fredrik,

DeHydra actually doesn’t use the astgen stuff because it’s way too low level. I think writing a new JS parser in C++ doesn’t really make sense for a few reasons:
a) SpiderMonkey is the “standard” parser that you’d have to play catchup with every time someone adds a feature(or a bug) to the language. I’m in that boat with elsa and C++ right now, it sucks.
b) There is Esc a new ecmascript compiler written in JS. Using JSON as an intermediate format should make it fairly simple to switch to a pure-JS solution as soon as that becomes available.
We should talk about your work some more, sounds cool.

Justin & Axel, thanks for the heads up. As long as it’s just simple conditionals, we should be able to pretend that there is no preprocessor. Please tell me that there aren’t any function-like macros in this. In the worst case I can reuse my MCPP work on this.

Gijs, as I said to fredrik, it’s nice to use the “standard” even if it is a little painful.

By: Gijs

Gijs — Thu, 25 Oct 2007 15:55:04 +0000

I hope you know that there is a JS JS parser (narcissus) in the tree? And there’s Rhino, of course, and I was told ages ago that the Parrot people were “working on it” as far as JS support was concerned. So maybe those would be easier to use than Spidermonkey?

By: Axel Hecht

Axel Hecht — Thu, 25 Oct 2007 08:13:49 +0000

I think we actually feed most of our js through the xul preprocessor now. I guess most of it just passes through unchanged, but some of it is actually modified. Not as bad as C++ is, of course.

http://mxr.mozilla.org/mozilla/search?string=%23if&find=%5C.js%24&findi=%5C.xul%24&filter=&tree=mozilla has examples.

By: fredrik

fredrik — Thu, 25 Oct 2007 08:12:08 +0000

To learn a little about the Elsa backend stuff, I wrote a small Lua parser using Flex/Elkhound/astgen. (Well, it doesn’t build an AST yet because I fell ill with the cold from hell, but that shouldn’t be too hard to knock out once my head isn’t filled with cotton. It spits out parse trees like there’s no tomorrow, though.)

My goal with this is to ultimately build a JavaScript parser for the very reason you mentioned. To be able to run DeHydra on it seamlessly.

You using SpiderMonkey is one of those “I wish I’d thought of that” moments. You could even bridge the JSON output through astgen so you needn’t do too much to DeHydra in terms of how it interfaces to the representation of the underlying program you’re rewriting (essentially replacing Flex/Elkhound with SpiderMonkey itself, astgen and some glue).

I’m personally excited about the possibilities of extending all this to a toolkit for rewriting any language you can write a “plug-in” for. Possibly, source-to-source transformations between languages (many languages are, on some level, isomorphic). C++ to JS2 would be cool.

By: Justin Dolske

Justin Dolske — Thu, 25 Oct 2007 06:27:24 +0000

No JS preprocessing?

http://mxr.mozilla.org/seamonkey/search?string=ifdef&find=\.js&tree=seamonkey

Not anyway as bad as our C++, of course.