Share your biggest challenges with the broader community

Awesome! Thanks!

That is not entirely correct. AudioWorkletNode can transfer data using MessagePort outside of Worklet global scope.

A MediaStream can be recorded resulting Blob converted to ArrayBuffer then transferred to a Worker.

MediaSource is now exposed in Worker https://github.com/w3c/media-source/issues/175.

What is lacking in standard browser APIs is the capability to read video frames (images) from a media container (file) without using an HTML <video> element, potentially faster than “real-time”.

This is a similar concept https://webwewant.fyi/wants/56/, OfflineMediaContext where the context can be Window, Worker, Worklet, etc.

I meant support for Web Audio and the audio/video elements. My point is: being able to do these things without having to co-operate with the main thread or other contexts.

1 Like

Yes, gather your point Can we use HTMLMediaElement or equivalent object without a document? #2981, Proposal: Implement OfflineMediaContext.

From perspective here the underlying encoders, decoders, container writers used by the browser should be exposed irrespective of context.

Taking the point a step further it should be possible to parse a media file or fragment without a document, HTML <video> or <audio> elements. And yes, Web Audio API should be defined in Worker, SharedWorker, ServiceWorker, Worklet, et al. contexts. The functionality and capability should be portable; e.g., https://bugzilla.mozilla.org/show_bug.cgi?id=1416663, https://rustwasm.github.io/wasm-bindgen/api/web_sys/struct.HtmlMediaElement.html in a Worker.

The question from perspective here is how to create such an API began with the consideration for all of the attendant and closely related capabilities for media creation, editing, post-production, and both codec and container creation from scratch and implementation of existing codecs and containers.

The browser already contains most if not all of this capability already, merging the disparate portions into a single API instead of specialized units that may or may not be interoperable, extensible and updated corresponding to all parts thereof, is the task.

Starting from scratch with a clear comprehensive goal would make the requirement achievable. Asking an entity might take just as much effort and yet not yield the complete API.

Am willing to help where am able.