Let us define our terms. We already know that many of the really interesting native applications in the mobile space (written, for example, in Objective C) make use of web technologies, either using established web APIs from third-party providers (making use of HTTP and JSON or markup payloads) or using HTML and CSS for presentation. This is also true for applications on the desktop. For example, the iTunes Music Store uses a browser-based presentation layer. We are not talking about that kind of use of web technologies here. We are also not talking about widgets of any kind here. Those are web-like veneers, but not quite The Web as we have in browsers, with the same security considerations of an HTML page served up from a URL. What we are talking about here is web pages in a web browser. Are we safely evolving capabilities that these need, so that future applications can safely take advantage of these capabilities? Let’s take a look at a few of these capabilities, and how they fit in with the web page.

Before the advent of HTML5 File API, integration with the underlying file system was pretty primitive: you’d use the <input type="file" /> element in forms, and couldn’t really do much with the selected file. Now that Firefox 3.6 implements the File API, you can do stuff like asynchronously examine the EXIF content of an image, or the ID3 tags within an MP3 file. The asynchronous part is important — we don’t want a page to block on read operations, especially on the main thread. While there is the introduction of a new top level object (FileReader), the actual file is selected only by user interaction with the standard <input type="file"> element. DOM methods are used to access the selected file data from the file element. The resulting API integrates with the web page’s intrinsic element-based model, using the DOM to obtain a handle to the data, and asynchronous read operations on top of it. The new device capability is a natural extension of the web page model.

In addition to being able to access the device’s native file system, hardware acceleration for graphics is another important area for the web to take advantage of, especially with Graphical Processing Units (GPUs) becoming more sophisticated, and with driver support for OpenGL ES 2.0 become more prevalent. We’ve already seen really creative uses of the HTML5 Canvas2D drawing API on the web. The next task was to take the same HTML5 Canvas element, and provide interfaces to access hardware-accelerated 3D graphics. This is what WebGL does: it takes the same notion of providing an “immediate mode” drawing context made available to 2D graphics in JavaScript, and adds support for a 3D API (which closely resembles OpenGL ES 2.0, but with judicious concessions made for how the API would look like in a memory-managed language like JavaScript). Again, the goal is integration with the web. WebGL relies on the Canvas element (a part of the DOM), and provides interfaces to programmatically draw into the Canvas.

Both WebGL and the File API provide examples of integrating device capabilities with an existing HTML element. Other device capabilities also integrate with DOM Events, the event model for web pages. For instance, the HTML5 Drag and Drop API, which allows you to drag resources from your device to a web page, generates events that a web page can listen for, a model that web developers have been familiar with ever since the introduction of onclick back in JavaScript’s infancy. And then, there’s Orientation Events, supported in Firefox 3.6 (with a really compelling demo — try it on the MacBook Pro or the Nokia N900 and Fx 3.6). Every time you tilt or “orient” your device, the web page can listen to events and do something interesting accordingly. While the Orientation API isn’t yet a web standard, we’d welcome further discussion with other browser vendors about it. These device capabilities integrate with the web page’s event model, just as other device capabilities integrate with the HTML element structure.

The W3C Geolocation API is one device capability that isn’t integrated with the DOM of a page, or its event model. Rather, it is an extension to the navigator object, and provides an asynchronous API to get access to the device’s location, and (after user consent is obtained), provides that information back to the web page. It is now used by Twitter, Google Maps, and Flickr, to name a few (run these web applications on Firefox 3.6). This is an asynchronous API; user-consent is also obtained asynchronously, blocking the feature until the user grants consent. We posited some of these principles as general design guidelines for Device APIs just before the formation of the nascent W3C Device APIs WG, which we’re a tentative member of.

Web pages are becoming more and more capable, and some of the need for proprietary plugins can be obviated. When we consider further device capabilities, we’re really looking for:

• An asynchronous design
• Whether the new capability can integrate with a known HTML element as a starting point. We’re watching the evolution of the HTML5 <device> stuff pretty closely.
• Whether the new capability can integrate with the event model for web pages.
• And finally, whether we need a new top level object

We want the web of web pages and hyperlinked applications to be the platform of choice for developers, based on open standards and a well-understood security model. Whether this will eventually supplant native platforms is an open question, but talk of a web operating system is popular in some circles.

Even without a draft specification of WebGL in circulation, we’ve seen some promising 3D content using WebGL appear on the web, put together mainly through developer ingenuity and the fact that Firefox, Chromium, and WebKit are open source projects with early support for the technology. Today, the WebGL Working Group at Khronos released a provisional public draft of the WebGL specification, and we are very excited for what this means for the web.

For one thing, it means more developers can get involved in the evolution of WebGL. There’s a public mailing list set up, so that you can engage directly with members of the WebGL Working Group, as well as a web forum. It’s important to note that the specification is not yet finalized. Participation from the web community is essential towards finalizing the specification, which we hope to do in the first quarter of 2010.

It also means that there are implementations of the draft specification that you can begin to test. You can obtain a Firefox nightly that implements the WebGL draft specification, and can turn on WebGL support in that build by following these steps:

• Type “about:config” in your browser’s URL bar
• Do a search for “webgl” in the Filter field
• Double-click the “enabled_for_all_sites” preference to change it to “true”

Other browsers that support the draft WebGL specification are listed on the WebGL Wiki.

Now of course, this is hardware-accelerated 3D graphics in an early beta, and for now requires an OpenGL 2.0-capable GPU and drivers. In particular, most flavors of Intel’s integrated GPUs will not work straight out of the box (such as the GMA900/GMA950 often found in laptops). Developers who build nightly builds of the browser can induce software rendering using Mesa, and should check out Vlad’s blog post for doing this on Windows. Caveat emptor: building software rendering using Mesa into a Firefox nightly is likely to yield slower performance, and is only for the intrepid.

WebGL is a royalty-free, cross-platform API that brings OpenGL ES 2.0 to the web as a 3D drawing context within HTML5′s Canvas element, exposed as low-level interfaces within the Document Object Model.

Developers familiar with the Canvas 2D context will recognize that WebGL as another context for Canvas:

// get the canvas element in the DOM
var canvas = document.getElementById("canvas3D");
var gl = canvas.getContext("experimental-webgl");

Note that till the specification is finalized, the context is called experimental-webgl.

WebGL uses the OpenGL shading language, GLSL ES, and can be cleanly combined with other web content that is layered on top or underneath the 3D content. It is an emerging web standard, and is designed to be used with other parts of the web platform. The release of the draft specification is one step in bringing about a plugin free 3D API to the web, usable straight out of the box. People have already begun using it to create compelling libraries. Check out X3DOM, which is a JavaScript library using WebGL that allows users to author content in X3D. We expect developer ingenuity to surprise and awe us, as it always has.

References

• Review the WebGL draft specification
• Get involved! WebGL Homepage on Khronos
• Read articles and contribute on the WebGL Wiki
• A blog about learning WebGL
• Find 3D recipes in the learning WebGL Cookbook

JavaScript performance in all major browsers has shown marked improvements over time, paving the way for it to be used for more sophisticated classes of applications. Many graphics applications, including popular games, leverage hardware acceleration, and subsets of OpenGL (such as OpenGL ES1.1) are already available on smartphones. The initial activity proposal is to consider a simple binding of OpenGL ES2.0 to JavaScript, and exposing that to an HTML5 Canvas context. This is how a 2D drawing context is exposed to the web for use with a JavaScript API; the proposed mechanism is to do this for a 3D context as well. As exciting as this is for the web and for us at Mozilla, it’s worth explaining our choices, and raising some questions.

We chose Khronos to start with, since that’s the home of OpenGL, and the proposal is to start with an OpenGL subset such as OpenGL ES2.0. This is a very low-level API with a lot of C language structures, and perhaps an early task for the working group would be to figure out how to expose that to JavaScript. This isn’t a fait accompli, and we look forward to help and collaboration from other major players. Naturally, security is a paramount consideration. Google announced support for the activity, and we look forward to others doing so soon as well.

Would web developers actually use something this low level? While we anticipate the influx of a new type of web developer (who may also be a games developer, or interested in 3D visualization), it’s worth pointing out that a standardization principle that’s served the web well is to expose pretty low level primitives in browsers, and allow third party JavaScript libraries to take on the task of syntactic sugar and higher level abstractions. Applications toolkits such as dojo, jQuery, and Prototype simplify developers lives; the same can occur in the 3D arena.

It’s also worth pointing out that this endeavor with Khronos to bring a subset of OpenGL to the web is just the first step. Higher level APIs with retained mode graphics are likely to follow suit. The discussion officially started yesterday, so we’re glowing with fresh possibility in uncharted terrain here.

Also read Vlad’s thoughts on all this — he wrote the extension that was the initial experimental implementation, and made this happen. And Chris Blizzard’s thoughts.