What about if you want to change the debouncing options, such as to disable it, or alter the timeouts in response to user settings or activity? With the event listener API is it intended that the event has to be entirely removed and re-added with new settings? That will have other subtle side-effects like possibly re-ordering its firing sequence relative to other listeners, as well as requiring re-specifying all other event listener options at the same time (passive, capture etc).
Performant by default is of course a worthy goal, but it seems to solve a lot of problems, apply to more cases (e.g. pure JS events), and make it more flexible and future-proof if you move this in to JS land. We could look to standardise a JS class to do this, but then by that point, why not just write a library? Specs are great, but once v1 ships they become difficult to change, and in the long term can end up as a backwards-compatibility headache. Libraries avoid all of that. So in my opinion, generally anything that can be a library, should be.