Share your biggest challenges with the broader community

Hey all! I wanted to let you know about a new collaboration between Microsoft, Google, and Mozilla (and hopefully others soon): The Web We Want. We using it as a listening tool to identify issues worthy of exploration. We’re not looking for solutions, but rather problem areas. If there’s something you’re throwing a lot of code at to work around a missing API or implement some design idea, please share it. If there’s a missing feature in DevTools or a missing piece in Web Driver, we’d like to know.

We’re also going to be running pitch sessions at web design & development conferences where folks can present their wants to the community in order to get direct feedback and support for their cause. The first one is in Washington DC at An Event Apart, but we’ll also be at Smashing Conference, Beyond Tellerrand, View Source, and others TBA.

Please share your wants, we’re listening!

PS - If you’re running an event and would like to host a Web We Want session, please reach out.

4 Likes

Memory-efficient APIs. Garbage-collection is great when you don’t care about when the memory is freed, and an intractable problem when you do.

In particular, getting ImageData from a MediaStream (local webcam, remote feed via WebRTC, captured from a video using HTMLMediaElement.captureStream(), etc) can generate a lot of garbage if you’re doing processing on each frame and don’t care about it once you’ve processed it. Trying to process 1080p@30 FPS will generate 230 MB/sec of discarded ImageData objects, which is a lot. 4K@30 FPS would be nearly 1 GB/sec.

Leaving it up to garbage collection leaves web developers at the mercy of the implementation. With low-spec devices this often isn’t up to snuff.

1 Like

You probably know that Edge has that UserVoice page where developers can vote on features that they want to see in Edge. (Of course, now that Edge is Chromium based, that page is no longer relevant unless it’s updated.)

Well, how about that UserVoice (or similar) voting system but for the web platform instead. That way we could see which missing features are really most requested.

For example, I would vote for a standard API for event throttling, among other things.

2 Likes

Yep.

We are looking at rolling a feature like that into webwewant.fyi :slight_smile:

Can you submit the event throttling idea?

1 Like

This is an interesting idea. Can you submit it?

This is an interesting idea. Can you submit it?

Ok, I submitted it to The Web We Want. There’s also a topic on this forum (Efficiently Get ImageData from a Video MediaStreamTrack) and a (mostly) fully-formed proposal on GitHub with a link to patch implementations for both Chromium and Firefox, and performance test results showing the impact, but from what I’ve seen in the past, it’s most likely it will continue to be met with silence.

2 Likes
  1. Similar to the concept of @dsanders11, an OfflineMediaContext (https://github.com/guest271314/OfflineMediaContext; https://github.com/whatwg/html/issues/2824; https://github.com/whatwg/html/issues/2981) or videoWorklet (https://github.com/w3c/css-houdini-drafts/issues/905) which, using the language of Web Audio API, " OfflineMediaContext is a particular type of method for rendering/mixing-down (potentially) faster than real-time. It does not render to the video hardware, but instead renders as quickly as possible, fulfilling the returned promise with the rendered result as an ImageBitmaps ."

  2. Add replaceTrack() https://github.com/w3c/mediacapture-record/issues/167 and/or ability addTrack() and removeTrack() at MediaRecorder without the recorder instance stopping recording https://github.com/w3c/mediacapture-record/issues/147, or otherwise the ability to add multiple audio and video tracks (potentially having variable width and height) to a single recording https://github.com/w3c/mediacapture-record/issues/166 that is capable of playback at <video> element at Chromium (https://bugs.chromium.org/p/chromium/issues/detail?id=983777) and Firefox.

  3. Fully implemented SSML parsing for Web Speech API, without any external services involved (https://lists.w3.org/Archives/Public/www-voice/2017OctDec/0000.html). A start https://github.com/guest271314/SpeechSynthesisSSMLParser.

  4. Full implemented SpeechRecognition for Web Speech API, without any external services involved (https://github.com/w3c/speech-api/issues/56). At the very minimum the mandate that users of Web APIs should be asked permission, conspicuously, before the browser records their PII biometric data (voice) and sends the recording of their voice to an undisclosed, unknown remote web service, potentionally FOREVER, to do with what they will.

  5. Since WebM is used by MediaRecorder, and though the author/maintainer is not interested in working on such a project themselves (https://gitlab.com/mbunkus/mkvtoolnix/issues/2576#note_185904126), a complete port of mkvmerge (https://mkvtoolnix.download/doc/mkvmerge.html) to JavaScript. In lieu of using Emscripten (https://github.com/emscripten-core/emsdk/issues/260) to port C++ (mkvmerge) to JavaScript, a workaround using Native Messaging and Native File System (“mkvmerge needs the whole source file to be present on disk.” https://gitlab.com/mbunkus/mkvtoolnix/issues/2576#note_185902903) https://github.com/guest271314/native-messaging-mkvmerge.

  6. Not necessary, a browser API though challenging nonetheless (both to describe and achieve): Determine the exact multiple of 9 (OEIS A217626 https://oeis.org/A217626) which derives N permutations specifically without calculating all of the permutations (“I am offering a bounty of 500 for a solution that works out the elements directly without calculating the actual permutations.” https://codegolf.stackexchange.com/questions/175693/9erilous-9ermutations).

  1. For the OS to not freeze when visiting this site. The OS (*nix, 32-bit) has now frozen twice when visiting this site and attempting to post a comment.

@AaronGustafson Should the list at Share your biggest challenges with the broader community be posted at the linked site as-is, or a post for each list item?

They should be posted individually to the webwewant.fyi site.

Submitted five proposals.

Is there a mechanism to publicly reference submitted proposals? Does The Web We Want send an email acknowledging receipt of the proposal?