[Proposal] navigator.scheduling.isFramePending

The goal is to provide web developers with more complete information about the state of the browser. With this information, javascript code can choose to defer work and yield control back to the browser in time to meet video refresh deadlines. The net effect should be fewer missed video frames (less jank).

I think this should be made a little bit clearer. From my understanding the API can not say wether the browser workload is already close to exceeding the frame budget and I probably would not skip most tasks.

One thing where this API obviously helps is layout thrashing:

window.addEventListener('scroll', (e) => {
   if (navigator.scheduling.isFramePending) {
     requestPostAnimationFrame(readLayout);
   } else {
     readLayout();
   }
});

With other words I really look forward to both API.

To be clear: requestPostAnimationFrame will force the scheduling of an animation frame, just as requestAnimationFrame already does. So if your goal is to avoid forced layout, then you can just use requestPostAnimationFrame unconditionally; there’s no need to check isFramePending.

I imagine isFramePending being used like this:

let mainLoopCallbackId;
function scheduleMainLoop() {
  if (mainLoopCallbackId)
    return;
  mainLoopCallbackId = setTimeout(mainLoop);
}
function mainLoop() {
  mainLoopCallbackId = 0;
  while (thereIsWorkToDo() && !navigator.scheduling.isFramePending()) {
    doOneUnitOfWork();
  }
  if (thereIsWorkToDo()) {
    scheduleMainLoop();
  }
};
button.addEventListener('click', e => {
  enqueueWork(createWorkFromEvent(e));
  scheduleMainLoop();
});

I tried an experiment and you’re right: chromium will schedule a frame when the href attribute changes, regardless of whether the link color changed.

This makes me nervous, though. It means that every line of code that triggers a pending frame (for style changes, or paints or whatever) has to be vetted to make sure that link visited state doesn’t affect the behavior.

One example case appears to be unmute event of a MediaStreamTrack. AFAICT there is no way to programmatically estimate or determine exactly when a MediaStreamTrack will dispatch unmute event which indicates media is available https://www.w3.org/TR/mediacapture-streams/

The muted/unmuted state of a track reflects whether the source provides any media at this moment. The enabled/disabled state is under application control and determines whether the track outputs media (to its consumers). Hence, media from the source only flows when a MediaStreamTrack object is both unmuted and enabled.

A MediaStreamTrack is muted when the source is temporarily unable to provide the track with data. A track can be muted by a user. Often this action is outside the control of the application. This could be as a result of the user hitting a hardware switch or toggling a control in the operating system / browser chrome. A track can also be muted by the User Agent.

Even given the case of a MediaStreamTrack of kind "video" constantly having video frames (images) input to the track and the track muted property is false it might still be necessary to await mute event before unmute dispatches when the MediaStreamTrack is replaced using RTCRtpSender.replaceTrack(), which through experimentation have observed could take anywhere from less than 10 seconds to 30 seconds to 3 minutes or more, that is, an indeterminate time span (see https://github.com/w3c/mediacapture-main/issues/614, https://plnkr.co/edit/Axkb8s?p=preview).

Does this API intend to set navigator.scheduling.isFramePending to true when a MediaStreamTrack muted property is set to false following unmute event being dispatched given the MediaStream where the MediaStreamTrack providing media is set as srcObject to a visible (or not visible) HTML <video> element?

If navigator.scheduling.isFramePending is intended to observe a MediaStreamTrack, would navigator.scheduling.isFramePending evaluate to true before unmute event is dispatched at the MediaStreamTrack instance?

What is the scope of APIs which navigator.scheduling.isFramePending is intended to handle and not handle?