Acoustic Echo Cancellation is a necessary component of WebRTC. However, the the only control we have over it from JavaScript is a boolean to turn it on or off: the signal played by the device’s speakers is always cancelled from the signal recorded by its mic.
It’s a shame we can’t access this at a lower level: providing two arbitrary audio streams and canceling one from the other. For instance:
var recording = await navigator.mediaDevices.getUserMedia({ audio: true });
var impossible = recording.removeEcho(someOtherStream);
This would be useful in situations where the device knows about other sound in a room, but isn’t the device playing the sound.
A major use case I’m thinking of is for watching a stream with other people (e.g. via Netflix Party, Disney+ GroupWatch, Twitch) while talking over a video call. This has become increasingly popular due to the pandemic.
In this scenario, you have a video call going on between everyone’s phones while the stream is being cast from each phone to each person’s TV. If you try this today, the video call picks up the sound from every TV and you get lots of echo.
If WebRTC included an echo cancellation API, each phone could cancel out not only the audio that it’s playing from its own speakers (the video call) but also the audio from the TV.