Ah, sorry, I hadn’t remembered to revisit the explainer.
We should get feedback from @fserb who added AnimationFrameProvider to the spec. Not all users of AnimationFrameProvider need the VideoFrameMetadata parameter, so maybe it makes sense for HTMLVideoElement’s requestAnimationFrame to be different from AnimationFrameProvider’s.
Intent makes perfect sense and would be very valuable! Why is this API different from ontimeupdate? Is there a reason why the DOM event system is not sufficient? I’d expect this to be named something like onframeupdate which fires on every frame painted. Is this a question of timing? Can this question be addressed in the explainer?
Sorry for the delay, I’ve been out of office. This doesn’t use an event that would always be fired since it’s expensive to preserve the video frame for the callback. E.g., holding onto a frame from a hardware decoder will slow down decoding since it can’t always issue more frames until one is returned. So the API is designed to only function when the page is actively interested in this capability.
What we could do is provide an event that does not provide the WebGL frame guarantees. I.e., you would not be able to upload the exact frame to canvas or WebGL based on the event. You would only be able to get the frame properties. This may be enough for most users.
The video.rAF callback would be called once per composited frame. I.e., every unique composited frame will receive a callback. It is intentional that this can happen before play() if the first frame is composited before playback (For Chromium this is always for preload=auto elements, and upon visibility for preload=metadata elements).