My patch for bug 506814 landed. Thanks, Josh and Dao.
Created a new project, async Places containers, which is the topic of the remainder of this post.
Spent time thinking about a Storage “live view” interface, mocked it up, talked to sdwilsh about it. The idea was a live view, where modifications to the view’s underlying data would automatically trigger a view refresh. The motivation was the fact that Places query results now (well, will—bug 499985) use the same collations in two locations: in the database on initialization (which makes possible APIs that return data in async batches), and in code when changes occur to the results. That means duplicate collation code in Storage and Places, and the collations used by each must be equivalent. But if there were such a thing as a generalized live view, it could all be handled by Storage.
However, I realized it’s not possible to in general and in constant time determine whether an arbitrary change to the database occurred within a given view. In the current code it’s O(1) to do this check against a list of query results exactly because a list of query results is not generalized. So I dropped this idea.
The problem of duplicate collations remains, and now I’m thinking of creating temp tables in Places code to store query results instead of the arrays of nodes it currently uses. That way Storage and its collations do the heavy lifting, handling both initialization of and updates to results. We’d turn the data into nodes only when it’s leaving the API. There are big roadblocks to doing that, though. There’s lots of post-Storage processing and filtering of data that would need to be moved to Storage. I think the reasons we don’t do this processing in Storage have more to do with an inadequate schema and layers of modifications to the code over a long period of time than any substantive problems with using SQL or Storage. In general I really wish we took better advantage of SQLite.
Rather than trying to solve big problems right off the bat, it might be better to start small: forget about the duplicate collations for now (making sure they’re equivalent) and just update the initialization path of the bookmarks query results, the simplest kind, to use the Storage collations. Even that isn’t so straightforward. We have a single, immutable mozIStorageStatement—among many others—that gets a container’s children. It’s created from a long string of raw SQL. If I merely want to change the sort order of that query, or any other for that matter, it’s not so clean or easy.