AsyncLocalStorage

(moving from https://github.com/slightlyoff/ServiceWorker/issues/906)

localStorage is handy for much use cases, but unfortunately it’s api based on Sync. so in service worker we need to use Indexed DB for every kind of data, only if you need to store 1byte flag.

I think if we don’t need index for large data search, and don’t wanna preparing ceremony for storing data, it’s better to have a localStorage which has Async API.

or other sight of view, Cache API is more handy than IDB, but it’s specialised for Request/Response class. AsyncLocalStorage will looks something like CacheAPI which is generalised for Plain Old JavaScript Object. (we may see it as parent class of Cache API)

this will solve may use case I think, for example…

• every use case for localStorage at window context
• saving request data for sync event
• saving simple/single/small data like flag, counter etc
• no need to prepare scheme before using (at install/activate)

will seems…

// class AsyncStorage
asyncStorage.open('v1').then((store) => {
  store.put('key', {data:'value'}).then();
  store.match('key').then(console.log.bind(cosnole)); // { data: 'value' }
  // etc but, no add/addAll
});

// class Caches extends AsyncStorage
// with adding method like add/addAll
// and optimised for Request/Response

Take a look at the localForage library, which implements an async but localStorage-like API that uses IndexedDB (amongst others) behind the scenes: https://github.com/mozilla/localForage

I agree that it might be best to just keep building on IDB for now. The community is also looking at making IDB less crap to use (via promises), which I think will really help.

I agree that LocalForage and similar libraries are the solution to this. Extensible web and all that.

Hm actually, is it possible that AsyncLocalStorage would have performance benefits over IDB because of a lack of transaction overhead? Important question if most folks are using IDB as a glorified key-value store (which I think they are).

Wouldn’t an AsyncLocalStorage just do the same thing internally?

If AsyncLocalStorage shared storage with (non-async) localStorage for the same origin this could allow ServiceWorker to be added to existing web apps already using localStorage without forcing the rest of the app to be rewritten or the data to be migrated

AsyncLocalStorage is not only for ServiceWorker. this will help Window Context without blocking main thread, and handling friendly with other new Promise based DOM API.

AsyncLocalStorage cannot share with non-async localStorage; it would negate the benefits because browsers have already made tradeoffs for the synchronous case (e.g. Firefox loads all data in-memory on page navigation). It would have to share with IDB or (more likely) be its own store.

The question is whether there actually is a potential perf benefit. According to my colleague Ali Alabbas, there isn’t any benefit for Edge.

I note this has been a request I’ve often heard from the Node community. (“Just give me LevelDB in the browser.”) I admit it is more extensible web-y.

I was asked to submit my proposal here.

I have 3 ideas for lower-level APIs for persistence.

Level DB Style

This is highly influenced by https://github.com/google/leveldb#features

• Keys and values are arbitrary byte arrays.
• Data is stored sorted by key.
• Callers can provide a custom comparison function to override the sort order.
• The basic operations are Put(key,value), Get(key), Delete(key).
• Multiple changes can be made in one atomic batch.
• Users can create a transient snapshot to get a consistent view of data.
• Forward and backward iteration is supported over the data.

// Basic operations
db.put(key, value) -> Promise
db.get(key) -> Promise<value>
db.delete(key) -> Promise

// Atomic Batch Writes
db.batch() -> Batch
batch.put(key, value) -> Batch
batch.delete(key) -> Batch
batch.write() -> Promise

// Snapshot reads
db.snapshot() -> Promise<Snapshot>
snapshot.get(key) -> Promise<value>
snapshot.iterate() -> Iterator

// Iterator
db.iterate() -> Iterator
it.seekToFirst() -> Promise
it.seekToEnd() -> Promise
it.valid() -> Promise<isValid>
it.prev() -> Promise
it.next() -> Promise
it.key() -> key
it.value() -> Promise<value>

This is less well designed, but we’ll need some globals for bootstrapping this process:

window.openStorage(name, options) -> Promise<DB> // create if not found
// options.compare(a, b) -> Number - custom comparator 
window.delStorage(name) -> Promise
window.listStorage() -> Promise<names>

MMAP Style

Some applications may prefer a different approach to storage. Consider if we wanted to compile sqlite3 using emscripten and wanted to persist the data somehow.

window.mmap(name) -> Promise<MMap>

mm.lock(start, end) -> Promise<ByteArray> // Ensure a range of bytes is in ram and get access as byte array
mm.unlock(start, end) // Free a range to be cleared from ram cache.
mm.msync() -> Promise // Flush all changes in memory to disk.
// If you try to access a bytearray that's not in ram cache, an exception will be thrown.

Append-only Store

Many databases can be implemented with simply an append-only stream.

window.logStore(name) -> Promise<LogStore>
log.write(value) -> Promise // Write to file, promise resolves when data in kernel space
log.fsync() -> Promise // Do a full sync (flush kernel buffers)
log.close() -> Promise // sync and close.

This would need more APIs for reading the logs for later playback obviously.

I may be off on some places which functions need to be async (return a promise), but this is the general idea of the core interfaces.

Iterators are synchronous, it should probably use AsyncIterators.

Sounds great. I’m not super up to date on the latest iterator specs. Also I’m not sure which calls in levelDB would need to be async because they depend on I/O that may miss caches.

But the basic idea is this interface with async where you might need to wait for I/O.

I’m still not sure what the benefit of any of this over using a library backed by IndexedDB is?

I’ve written up a spec for async local storage at https://domenic.github.io/async-local-storage/; we’re working on prototyping it Chrome.

Interestingly, it’s actually specified directly on top of IndexedDB, almost like a library. This is part of a larger project for allowing shipping high-level features (like async local storage) on top of lower-level things primitives already in the browser (like IndexedDB), called layered APIs.

You can learn more about the motivations behind layered APIs at that link (e.g., why we think they’re a better choice than having every web developer that wants to use a feature pull in a library). My one-sentence summary is that they’re fulfilling the second half of the extensible web manifesto, where once web developers have taken the lead on showing how a high-level feature should work, it’s time to bring that in from the cold and into the platform so everyone can benefit.

Hey folks! The new “async local storage” API is fairly far along. The spec lives at https://wicg.github.io/kv-storage/ (transferred from domenic/async-local-storage), with the explainer at https://github.com/WICG/kv-storage/blob/master/README.md.

Why the name change, you might ask? Well, some fine folks at Mozilla pointed out that “async local storage” implies something that would access the local storage database, not a separate IndexedDB-backed storage area. (And, as @nolanlawson pointed out upthread, this separation is crucial.)

Things are starting to look pretty settled for the API, with the biggest remaining piece of work on the issue tracker being the conversion of keys()/values()/entries() to return async iterators. Feel free to take a look!