IndexedDB API clean up / promises

jonathank · 2015-06-10

From an outside perspective the IndexedDB API could be much simpler which:

Reduces implementation errors
Could be shimmed on top of the current API
With a simpler API browsers might be able to do some perf magic

As I said to @annevk and @jonthanfielding it would be nice to clear up a nicer API for IndexedDB: https://twitter.com/KingstonTime/status/606236654743994368

Joshua Bell worked on a simplified API using IndexedDB and promises:

gist.github.com

https://gist.github.com/inexorabletash/c8069c042b734519680c

@ SimpleDB.md

SimpleDB - Like Indexed DB, but Simple
--------------------------------------

A simple asynchronous data store.

> STATUS: This is a thought experiment, not a serious proposal. Would basic async storage like this be useful? With this plus some locking primitive, could you build Indexed DB?

Like Indexed DB:
* rich value types - store anything you can structured clone
* rich key types - Number, String, Date, Array (of other key types)

This file has been truncated. show original

simpledb.webidl

[NoInterfaceObject] interface GlobalSimpleDB {
  readonly attribute SimpleDBFactory simpleDB;
};
Window implements GlobalSimpleDB;
Worker implements GlobalSimpleDB;

interface SimpleDBFactory {
  Promise<SimpleDB> open(DOMString name);
  Promise<void> delete(DOMString name);
  // TODO: Enumerate?

This file has been truncated. show original

simpledb_polyfill.js

(function(global) {
  var SECRET = Object.create(null);
  var DB_PREFIX = '$SimpleDB$';
  var STORE = 'store';

  function SimpleDBFactory(secret) {
    if (secret !== SECRET) throw TypeError('Invalid constructor');
  }
  SimpleDBFactory.prototype = {
    open: function(name) {

This file has been truncated. show original

There are more than three files. show original

This suffers from race conditions because of the code mitigating the locking which is an issue. I have not had much time to look at this however it strikes me this can be resolved by transactions being queued before being resolved in the promise. So long as all transactions come from the same instance then the issue should not be an issue here. By implementing a stack of transactions that are called FIFO ordering the API of IndexedDB removes the need to manage state from the outside.

As Joshua has stated this was just a thought exercise implementing schemas and indexing would also be important I feel.

jonthanfielding · 2015-06-11

Here is the link to the Edge Moderator which shows quite a lot of interest in this https://www.google.com/moderator/#15/e=214e4f&t=214e4f.44

inexorabletash · 2015-06-11

Ugh, let’s not dwell on that silly “SimpleDB” thing :(. Truly just a thought experiment about minimal async storage so that other things could be built upon it. Given that the first commentary above is that it needs transactions, schemas, and indexes, it shows that IndexedDB’s feature set is not, in fact, too much. It’s the UX of the API that needs help. And unfortunately the current API ties the transaction model to the event loop very tightly - see IDB+Promises for the gruesome details.

More realistic thoughts on evolving IDB include:

Having requests and transactions expose promises to observe lifetimes - in IDB+Promises
Allowing transactions to “wait” on promises to compose larger operations - also in: IDB+Promises
Explicit commit control for transactions: https://gist.github.com/inexorabletash/d55a6669a040e92e47c6
“transactionless” operations - I need to write this up/prollyfill it, but basically allow read/write requests that get an implicit transaction that commits immediately. These could just yield promises.

I think those would provide the basis to experimentally evolve the API to be easier to use and compose better with other Promise-based code flows. They wouldn’t necessarily make it easier on their own, but without more explicit control over transaction lifetimes you can’t do much to improve the API in JS and create Promise-based cowpaths to pave.

inexorabletash · 2015-06-11

Crazy “transactionless IDB” thought: https://gist.github.com/inexorabletash/280016e79188b6a28247

Pros:

One less thing to grok about when transitioning to IDB

Cons:

bifurcates the API - sometimes get() returns one thing, sometimes it returns another
if we had an external transaction primitive (e.g. https://gist.github.com/inexorabletash/a53c6add9fbc8b9b1191) you can order things, but it doesn’t let you create a batch and commit/abort as a unit

nolanlawson · 2015-06-28

I think the explicit commit control is a great idea. I also wonder if we could kill two birds with one stone and solve the issue of promises/callbacks within transactions at the same time.

If you could open an IDB transaction while explicitly saying “hey, I want to control the commit,” then there’s no need for fancy Promise fixes because the transaction can just patiently wait until you commit(). An added benefit is that this would also work for non-promise async APIs that you might want to access inside of a transaction, such as FileReader or XHR.

Example: https://gist.github.com/nolanlawson/c2580896749cd475fb30

As for SimpleDB/transactionless, that sounds quite similar to the “async local storage” idea. It seems nice, but might not be such a big win as promises or transaction control.