As someone who’s not experienced in audio, I have always felt very stupid when trying to use the WebAudio API in a practical way. Turns out it wasn’t my fault.
Jasper St. Pierre wrote a great piece about the state of audio on the web: http://blog.mecheye.net/2017/09/i-dont-know-who-the-web-audio-api-is-designed-for/
I think we should work towards a better Audio API.
So, this is a call out for everyone. Please speak up, and lets bet on the web.
The Web Audio API works great in my experience. We use it for game engine audio, and it’s well suited for that purpose. I think critical posts tend to get more attention than positive ones, which can distort the perception of a technology.
Having said that, it does sound like if you want low-level sample-level control, things are tricky. Perhaps there could be some extensions to the API to help cover that. As ever, concrete proposals are the most useful way forward.
Hi, I’m the author of the blog post in question. I’m definitely curious if you are deep-graph or shallow-graph.
I do tend to care more about shallow-graph usage because I require accuracy to the sounds that I’m reproducing, so that’s what I wrote about in my blog post. From my experience quite a few years ago, deep-graph had its issues but it’s possible those have been resolved. See: http://funny.computer/cloud/Peggle/ (warning: large download)
I’ve left a proposal for simple sample-based playback here: https://github.com/WebAudio/web-audio-api/issues/1322
Other important issues to watch:
I doubt you will get any traction with a feature that processes real-time audio on the main thread. It’s not just that heavy JS can block the page, it’s also that the page can block the JS: some heavy CSS layout or non-trivial DOM changes can cause 50ms+ of layout, which will hitch the audio stream. So it’s not a sensible approach for processing real-time audio. It has to be lifted off the main thread and in to a worker for any half-way reasonable guarantees about responsiveness, but AFAIK they want to use a new “worklet” primitive for this, which is still in development. I get that it’s frustrating that this all literally takes years, but it is in the works and I think will result in a better API than a quick “expose audio output to the main thread” API, which will never work well in practice.
I think you’re also too quick to jump to conclusions about the wide variety of use cases for the Web Audio API. For example you suggest completely removing decodeAudioData(), but this is an important part of the API for us. Games often use a set of very small sound effects, the memory overhead of which is negligible. In this case fully decompressing them all allows for low-latency playback, and it’s super simple. For long tracks you can - and we already do - pipe a HTMLAudioElement in to the Web Audio API for streaming playback and low memory overhead for things like background music. This all works great in our engine. There is no need to remove decodeAudioData completely, this is a perfectly valid use case and it works well for that. Sure, you can ask for it to do more, but why jump to the extreme right away?
And this is before I even start to consider the compatibility problems. If you suggest removing decodeAudioData completely, you are proposing to break audio playback in all content produced in our game engine over the past 5 years, let alone the rest of the content on the web, and it will guarantee a tide of angry customers coming our way. Again, maybe not all existing APIs are perfect, but removing them is very, very costly to many people in many ways.