An idea for Dojo from GWT land

Now that I'm at Google I've been getting a bit more exposed to GWT (the Google Web Toolkit). I used to dismiss it because I'm a JavaScript hacker and GWT is in Java-land, but I've been learning some interesting things about it that I think could help in Dojo-land if we adopted them.

Ray Cromwell rolled something called GQuery, which is basically JQuery inside of GWT. What is unique is you can do JQuery style selectors inside of Java. Dojo has this as well, with dojo.query, but what Ray is doing is crazy. Because GWT gets compiled, he can turn something like this:


into this:


in the source. Here's Ray describing it:

"GQuery is progressing nicely. I implemented all of CSS3 selectors by following ExtJS's DomQuery implementation, only I added the ability to parse at runtime as well as via a generator at compile time. The compile time generator turns a CSS selector into 100% inlined code. That is, a selector like "#foo", will turn into "return document.getElementById("foo");", no parsing step involved. I've still got a bunch of optimizations to make, add support for XPath and native getElementsByClassName, but even now working with the library in GWT is very cool. I just started looking at DomAssistant as well, to incorporate (i.e. steal) the best algorithms from each."

One of the comments sums up the benefits of this nicely:

"You have an unfair advantage; since your selectors are compiled you can do any number of optimizations to them, and basically beat any other selector engine out there."

GWT itself does some interesting optimizations like this around what code is sent to specific browsers. I'm seeing numbers that blow me away, that because you can only send down the code needed for a specific browser you can really slim things down. For example, something like Dojo GFX has code paths for IE (VML), Firefox (SVG), and Safari (Canvas). If we could just send which one you need then the code would be 1/3 the size.

How can we get these kinds of benefits in Dojo? We already have a build time that runs through a JavaScript interpreter; can we start to do magic like this? The performance and size benefits I'm seeing in the GWT-world seem to indicate that it is useful.


Anonymous said…
qooxdoo has a concept called variants which are in-code declared variants of a specific implementation. Our generator/compiler system makes it possible to select a specific variant at compile-time. This allows it each qooxdoo application developer to generate separate output files for every browser. I think this not not comparable to every optimization GWT makes possible, but it still helps a lot to send code to the client which is never executed at all. You can read more about the concept in our wiki.
Russ said…
You know, GQuery seemed kind of silly to me, but that is a really, really good point. Good luck getting it into dojo.
Ray Cromwell said…
Nice write up. There's an additional benefit from GWT and GQuery. Since the selector engine is broken down into bunch of small methods, and a compile time generator translates a Selector into a sequence of calls to these methods, then any calls not used are pruned.

Thus, if you never use any of the fancier CSS3 pseudo selectors like not/nth-child/etc, those get pruned by GWT.

For example, if you only use say, ID and descent selectors, then GWT will essentially inline just those two methods (getById, and getDescendentNodes()), thus the overall size of your GQuery output could be small.

If on the other hand, you use a single dynamic runtime query, GWT will be forced to include the whole selector engine because the decision as to which methods are neccessary has been deferred until runtime.

So, you can potentially realize both size and performance gains no usually obtainable in Javascript, unless you use a really really fancy JS build system with uber preprocessors to slice and dice and concatenate your JS files.
Ray Cromwell said…
Variants look similar to GWT's Deferred Binding mechanism. In GWT, you can intercept a class being constructed and substitute a new type. There are two mechanisms available: You can either swap in a hand written implementation (DOMImplStandard vs DOMImplSafari), or you can invoke a Generator class, which generates an on-the-fly implementation. The latter is what GQuery uses. You try to construct a Selector, GWT intercepts it and delegates it to a Generator which compiles the selector down to a Java class.

GWT goes a step further. It calculates all possible permutations of compile outputs (say, cartesian product of the set {IE,FF,Saf,Opera} vs locales {English,French,Japanese} vs say, XPath support {yes,no}. It then compiles every unique permutation into a cryptographically strong hashed filename, and uses a small startup script that picks which version to run.

There's a more detailed explanation in my YouTube presentation here:

@Russ, GQuery is not intended to be incorporated into libraries like Dojo or ExtJS, but rather used by GWT developers with the GWT libaries available. While it is in theory, possible to export GQuery for use as a regular JS library, I'd say much advantage would be lost doing so, and sticking with jQuery would be better.
Anonymous said…
@Ray: Sounds interesting. Somewhat comparable was possible with the generator in qooxdoo for some time now. You can combine variants to different sets. For example there is a debug variant, or a variant which includes the profiling code, etc. Even locales are selectable. You can combine them in any way you like. The generator system is implemented in Python.

What we not have implemented yet is some sort of loader script which automatically handles the detection of which compiled file to load. In the forthcoming 0.8 release the whole generator system was reworked to be easier to customize. It is also leaps faster than the old version and has many new features. I think a loader script will be part of a minor release in the 0.8.x cycle. Shouldn't be to hard.
Russ said…
@Ray - I meant incorporating those same advantages, not the library itself.