Brad's Blog

Camera input tag

09.23.08 - 02:15pm

Earlier this month I spent some time working on exposing camera functionality to content in fennec and firefox. First, I built upon Chris Double’s implementation of a GStreamer back end for the video tag and extended it to support a “camera://” protocol. When the video tag encounters this, it creates a GStreamer pipeline starting from a video for linux 2 source (v4l2).

This currently works on desktop linux that have a v4l2 supported camera (such as iSight) and the n800/n810. It will need to be implemented on mac and windows and configurations will need to be stored in preferences (in case you have multiple cameras or your camera uses a difference interface).

Next I wrapped a video tag, image and a few buttons in xbl and bound it to <input type=”camera” />. When a user hits a website using this tag, he or she currently will see a live video preview and a “take photo” button. When the user clicks the button, the photo is grabbed from the camera and shown to the user in the image element. The image element and video element are in a deck element so only one is shown at a time. After the photo is taken, there is a button that reads “Take another photo,” in case the user doesn’t like the one they just took. Once the user is satisfied, this element works like any other form element and the file can be uploaded to a web service.

After talking to a few people this should acutally be bound to <input type=”file” accept=”image/png”/>. Also, it needs some UI design both for what it looks like in content and for a configuration dialog. Finally, we’ll need to think about security and make sure no one can snap a photo of you when you’re not looking your best. Speaking of which, I might as well share:

I’ve notice quite a few iPhone apps that allow users to take a photo and upload it with out leaving the application. Absent the ability to do this from the web, these otherwise web centric services have been forced to write a native application to accomplish their goal. Hopefully this will eventually help web services stay on the web.

Category: mobile, Mozilla | | 17 Comments »

Gates and Seinfeld

09.05.08 - 06:06pm

So, I’m sure I’m not the first to say….. huh?

Category: mobile | | Comments Off

Kinetic scrolling

08.13.08 - 05:08pm

I’ve been working on optimizing scrolling/panning in general and kinetic scrolling specifically lately and just wanted to share a video of what I’ve got so far.

Category: mobile, Mozilla | | 3 Comments »

Windows Mobile Status

08.08.08 - 03:02pm

We continue to make significant progress towards Windows Mobile being a viable platform for Mobile Firefox. In the move from the 1.8 code base (Minimo) to the 1.9 code base (Fennec) we’re making substantial changes to how we run on Windows Mobile.

The first such major change is to reduce our shunt layer to the most minimal possible. This section of code (located in build/wince/shunt) is intended to fill in the gaps where Windows CE does not provide the same functionality as Windows NT. Some of this has been resolved by moving our target platform from Windows CE 4.2 (the basis for Pocket PC 2003 and Smartphone 2003) to Windows CE 5.0 (the basis for Windows Mobile 5 Pocket PC/Smartphone and Windows Mobile 6 Professional/Standard).

The second major reduction in the size and scope of the shunt can be attributed to removing narrow character Windows API calls from our code base. For a little background, most Windows APIs exist as two versions (wide and narrow). In this scheme “ExampleFunction” would actually be implemented as “ExampleFunctionA” which takes narrow char arguments and “ExampleFunctionW” which takes wide char arguments. At compile time “ExampleFunction” would be defined as one or the other based on whether or not the macro UNICODE was defined.

In the Mozilla code base, there are many instances where we would call “ExampleFunction” with narrow char arguments, necessitating UNICODE not be defined. Windows Mobile (and Windows XP/Vista) are wide char systems. On Windows XP and Vista, these narrow char API calls exist for backwards compatibility, but on Windows Mobile they do not. For Minimo, the shunt defined many of these narrow char function calls and redefined the ambiguous function (“ExampleFunction”) to the shunt’s version (“ExampleFunctionA”). For Fennec, we are finding the instances of these incorrect calls to the ambiguous functions and calling either the wide version (preferred) or the narrow version explicitly. This in turn has allowed us to reduce the size of the shunt considerably. It should also result in improved performance on both Windows Mobile and Windows XP/Vista due to the reduction in string conversions we perform.

With that work mostly complete, John Wolfe has been diligently working through some issues surrounding window management and file io. Currently, xulrunner can launch various xul apps without crashing.

Looking forward, there are some known issues on the horizon such as font shaping that much be fixed. We also need to turn an eye towards further optimizations. However, our progress to this point makes us quite optimistic about the future of Mozilla on Windows Mobile. If you have specific knowledge in fonting (especially experience with harfbuzz), an interest in tuning performance or just a general interest in making Windows Mobile go for Mozilla, please hop on the #mobile irc channel; we could use your help.

Finally, we would like to announce that Brian Crowder has bought a Windows Mobile phone. His frustration with Pocket Internet Explorer quickly lead to him expressing interest in helping out with Fennec; help which was gladly accepted. Brian brings with him a deep knowledge of Mozilla’s internals which I’m sure will prove to be invaluable in the quest to bring Windows Mobile to the mobile web rock party.

Category: mobile, Mozilla | | 16 Comments »

Canvas 3d fun

05.20.08 - 11:31pm

It occurred to me the other day that I haven’t blogged in a while. As it turns out, its been two months. Wow…..time flies when you’ve got your head down. Anyway, I wanted to share a bit of what I’ve been working on.

I have been playing with Vlads canvas 3d extension to implement fish eye zoom (Its was good way to kill some time on airplanes).

As far as I know, the original idea for the Fish Eye Zoom came from Asa, and it seems like a really good way to allow the user to inspect the content of a page when fully zoomed out. Or, if you think of it another way, it can give the user context they don’t usually enjoy in the traditional “key hole” view a mobile browser provides.

Big thanks to Vlad for his work in the 3d Canvas, I can see this really opening up new doors.

Category: me, Mozilla | | 1 Comment »

maemo buildbot

03.17.08 - 03:19pm

Several people have been asking for an updated build of xulrunner for maemo.  I know the tinderbox I set up in December has been broken since mid January.  The good news is the build/release team is taking over these responsibilities and will be doing an infinitely better job than I have in maintaining it.  The first output from this bot is now available here:

ftp://ftp.mozilla.org/pub/mobile/xulrunner-1.9b5pre.en-US.linux-arm.tar.bz2

The build bot is getting very close to being able to push these builds automatically, so soon you’ll be able to find the latest and greatest here every day.  Also, once I get off my but and make the changes to the build scripts to produce debian packages from this, you’ll be able to install it from the application catalog on your n800/n810.

You’ll notice that this is xulrunner and not firefox.  There are several reasons for this, not the least of which is we hope this will encourage people to roll their own browser (and other apps).  There are already several available to try out, hear is a sampling:

Browsers:

http://wiki.mozilla.org/Image:Touchscreenbrowser-atotic.zip (from atotic)

http://wiki.mozilla.org/images/b/b1/Touchscreenbrowser.zip (from venky & co.), discussion can be found here: http://wiki.mozilla.org/Mobile/UI/Designs/TouchScreen

http://people.mozilla.org/~blassey/SpBrowser.tgz (from me, 2 months old) discussion can be found here: http://wiki.mozilla.org/Mobile/UI/Designs/NonTouchScreen

http://benjamin.smedbergs.us/xulrunner/mybrowser-0.2.2.xulapp (from bsmedberg, very minimal)

Other:

http://people.mozilla.org/~blassey/tetris-example.tgz (tetris game I took shamelessly from croczilla)

http://benjamin.smedbergs.us/xulrunner/xulmine-0.9.xulapp (mine sweep originally by Neil Rashbrook, packaged as xul app by bsmedberg)

And there’s already a collection of apps running on xulrunner for the desktop.  Most of them are not packaged to be easily used on maemo with xulrunner.  If you’re feeling adventurous, give them a try.  If any of them work, leave a comment (maybe even a screen shot).

http://developer.mozilla.org/en/docs/XULRunner_Hall_of_Fame

and

http://developer.mozilla.org/en/docs/Category:XULRunner:Examples

Installation
This is the procedure I used to install xulrunner on my n810 (as root)

# cd /opt
# bunzip2 /home/user/MyDocs/.documents/xulrunner-1.9b5pre.en-US.linux-arm.tar.bz2
# tar -xf /home/user/MyDocs/.documents/xulrunner-1.9b5pre.en-US.linux-arm.tar
# cd xulrunner
# ./xulrunner --register-global

and then to install and run an app, I did the following (again, as root)

# cd /opt/xulrunner
# ./xulrunner --install-app /home/user/MyDocs/.documents/<application name>.xulapp
# /usr/lib/<vendor name>/<application name>/<application name>

A couple notes, I downloaded everything with microb, which puts downloads in the Documents folder and hence the path above. The vendor name and application name are specified in the application.ini file of the xul app. Finally, the –install-app flag assumes a zip archive (which is what the xulapp extension is). You may need zip the app up yourself to use it. I

I installed and ran xulmine, mybrowser and chatzilla.  They all ran (woot!).  To prove I’m not completely full off it, here’s a screen shot:

If you have your own ideas for a good xul app or what the browser should look like, put them into javascript and xul and post.  Leave a comment here so people can try them.

Note: some of these xul apps were written before mozilla 1.9 was released.  If you get a warning such as “Could not find compatible GRE between 1.8 and 1.8.0.*”, you’ll have to edit the application.ini file.  Just change MaxVersion from 1.8.0.* to 1.9.0.*

Category: mobile, Mozilla | | 18 Comments »

Xulrunner on Windows mobile smartphone

02.18.08 - 03:53am

Doug blogged friday about getting winEmbed and xulrunner going in the windows mobile emulators.  We were both using the windows mobile 6 professional sdk as the target (aka Pocket PC), because that’s the configuration that was checked into svn.  Unfortunitely, those binaries did not run on the HTC s730 I have to test with, which is windows mobile 6 standard (aka smartphone).

Well with some ever so slight massaging (there functions defined in the headers of the standard sdk that don’t seem to exist in the libraries, tisk tisk m$), I finally got a build going on my phone.

First I tried a very simple xul app I wrote that contained a toolbar, button, label and browser element (src=about:mozilla):

nice, very nice.  Next I pressed my luck and tried bsmedberg’s MyBrowser. Lo and behold… it ran too. Complete with the network connection:

One last note about this.  This is a full debug build for the entire tree, with all the binary bloat that implies.  This makes me fairly optimistic that we will be able to run reliably on these types of devices.

Category: mobile, Mozilla | | 4 Comments »

freakin Visual Studio

02.14.08 - 01:59am

no, really….its freakin out.

http://www.youtube.com/watch?v=FzmOpb7aRxE

Not surprisingly, it crashed after a couple minutes

Category: mobile, Mozilla | | 1 Comment »

My Debugger is lying to me

02.11.08 - 04:31am

I’ve been trying to figure out why the string being passed to XPCOMGlueStartup (and subsequently XPCOMGlueLoad) was “disappearing.”  Turns out it wasn’t.   My debugger just couldn’t see it.  As you can see from this screen shot, when I decided to print the content of the variable to stdout (mostly to prove to myself that I’m not going insane), it turns out that the string is there.  Visual Studio just decided to waste 3 hours of my life (thank you M$).

Category: mobile, Mozilla | | 1 Comment »

Hg as a collaboration tool

02.05.08 - 06:47pm

I want to clarify the point of this post. I am not attempting to make the point that hg is a bad tool. I am sure it is very helpful to some users. My point is that, IMHO, it does not solve an existing problem with Mozilla that justifies the inherent pain of switching from cvs to hg.

Over the last few weeks the mobile team has been using Mercurial to manage the code we’re using to build xulrunner for Windows Mobile. Up to the middle of last week it worked fine as dougt was the only one checking in patches while I was off in my own personal unicode hell.Then wednesday I finally got the code to compile with UNICODE defined. At that point I all I wanted to do was update, merge and create a diff. I’m not sure where thing went wrong, but wrong they did go.

Its almost a week later, and I still don’t have my patch. I’ve given up on that eureka moment when I suddenly discover that elusive string of commands. Instead its 3am 4am and I’m hand applying my changes to a freshly checked out working directly from our new svn repository.

All joking aside, I’d like to do a bit of a post-mortem on this. From talking to a few people at work week it seems everyone that is using hg for their work at mozilla is using it for changes they’re making to one or two modules that for the most part no one else is working on. In fact, I didn’t hear of any instances of two or more people collaborating to work on the same set of files at the same time. Granted in the version control world, this is the sort of scenario that makes us all recall late nights of hand merging conflicts, but we’ve all done it. Its boring, its tedious but we know how to do it.

With hg, I don’t know how to do it. I have received a fair amount of advice and and several suggestions from various people on the subject. In fact I want to sincerely thank jorendorff for spending upwards of an hour with dougt and I trying to sort this out. But it seems once you dig down enough the conclusion becomes, “hg wasn’t designed to do that.” That being produce patches that peers can review and can eventually be checked into a central repository that everyone can pull. What it was designed for is combining the efforts of several people who are working on very distinct modules and are publishing their changes either through patch queues or their own repository (hg serve).

Please don’t take this blog post as a bleary eyed developer slinging mud. I hope this will spark some discussion around which tools are good for which jobs. From what I’ve seen, hg is not very good for collaboration on the peer reviewed central repository model that mozilla has traditionally worked on. It does seem good for experimental work like that being done by the moz2 folks. Eventually we need to make a coherent product out of those experimentations, and I don’t see how that will happen with the current set up. Maybe I’m wrong. In fact I’m sure I’m wrong on several points (I just don’t know which ones), please point them out.

There are a few things I’m sure of. First, hg works on a model completely different to that of cvs and svn. Not only does every one using it need to get their head around that model from the beginning (admittedly, my first failing). They also need to follow some new and unique procedures. In fact a whole new set of “social” norms need to be developed (or at least published) for this tool.

Finally (and probably most importantly), in its current state hg introduces a barrier to entry for new contributers that seems entirely unnecessary.

Category: mobile, Mozilla | | 27 Comments »