[Proposal] navigator.scheduling.isFramePending

isFramePending is similar to the proposed isInputPending API. Whereas isInputPending signals that an input event has been received but not yet dispatched, isFramePending signals that the browser intends to update the visual display – including updating animations and running requestAnimationFrame callbacks – at the next opportunity.

The goal is to provide web developers with more complete information about the state of the browser. With this information, javascript code can choose to defer work and yield control back to the browser in time to meet video refresh deadlines. The net effect should be fewer missed video frames (less jank).

Explainer

It feels like the intent here is to allow some potentially long-running JS to yield back to the event loop, allowing the render steps to run.

Given that, would isFramePending be true for pending frames that aren’t dependant on the main thread? Eg, compositor-driven animations & scrolling.

I don’t get it. Wouldn’t the isFramePending API return the estimated duration till next display refresh? Not just a bool? What would that mean exactly?

@szager, did you mean to link the explainer to isFramePending instead of isInputPending?

@briankardell

According to this Twitter reply, seems it was intentional, although I don’t see how they are similar enough to warrant simply linking to the other?

Seems the developer would want to know the duration remaining until the next display refresh? If it was just a bool, what would that even mean? Display refresh is guaranteed to happen, input isn’t. I must be misunderstanding though.

No, isFramePending would not be true for compositor-only animations. Long-running javascript won’t cause those animations to jank, so there’s no need to yield the main thread.

Whoops, fixed; thanks for the catch.

We initially investigated returning the estimated time until the next rendering deadline, as you suggest. Unfortunately, it’s not always possible to determine that number accurately; and an inaccurate time budget is in many ways worse than no time budget at all.

The return value of isFramePending indicates that the browser intends to run the update the rendering steps without delay at the next opportunity (after the currently-running javascript task finishes). Developers would presumably use it like this:

button.onclick = evt => {
  doSomeWork();
  if (isFramePending()) {
    // Schedule the rest of the work to run after the rendering update
    setTimeout(doMoreWork);
  } else {
    doMoreWork();
  }
};

Unfortunately, it’s not always possible to determine that number accurately

Anywhere I can read as to why that’s the case?

Not that I know of. The way rendering opportunities are identified varies significantly between browsers and also between operating systems. Some use software timers to match the cadence of the display hardware’s refresh rate, some rely on a signal from the OS.

In general though, it’s possible to extrapolate the time of the next rendering opportunity fairly accurately if the browser has been generating frames steadily. But if the page has been idle and hasn’t produced an animation frame recently, and then the page’s DOM is modified in a way that requires a display update, it’s impossible to predict accurately when the next rendering opportunity will happen.

To me this feels like a scheduling API and even mentions isInputPending as a counterpart. Is there a reason this isn’t in the navigator.scheduling namespace, alongside isInputPending, rather than directly on window?

To know “time to next frame” at any arbitrary point when JS is running requires that the browser be continually running a requestAnimationFrame-like refresh loop, since it has to know the last frame time to estimate the next frame time. That would mean that all idle pages have to be constantly running an update loop, which is a huge power drain.

A concern I have here is that this API exposes things that trigger rendering updates, which I don’t think is currently detectable. Consider:

let hasFrame = window.isFramePending(); // hasFrame is false
foo.style.color = 'red';
let hasFrameNow = window.isFramePending(); // hasFrameNow is true

Maybe that’s OK?

You are quite right, this was my mistake; it is meant to be navigator.scheduling.isFramePending.

That’s a good point; for example, we wouldn’t want this to be a vector for visited-link exploits:

<a id=probe href="http://not-a-real-site.com">foo</a>
<script>
probe.href = "http://www.amazon.com";
if (navigator.scheduling.isFramePending()) {
  // Frame was scheduled to paint the link with the "visited" color
  browser_history_contains_amazon = true;
}
</script>

Another chromium-specific concern comes to mind, which is that isFramePending may enable a process-isolated iframe to detect the fact that it is process-isolated. When an iframe is same-process, and the parent document invalidates style, then an animation frame will be scheduled for the entire in-process frame tree. But for a process-isolated iframe, an invalidation in the parent document will not cause the iframe process to schedule an animation frame.

Yeah, agreed… Depending on different optimizations or what not it may be different between engines. For example if the above happened on a disconnected subtree or on a display: none subtree Gecko has a fast way to skip the whole rendering update. Which means it’s going to be pretty hard to implement interoperably… But I guess the whole navigator.scheduling proposal is sorta like that.

As I understand your suggested isFramePending method this is not a concern or problem at all. If something was changed in DOM or CSSOM the browser does know that style/layout is dirty and the browser has to recalculate it. But the browser does not know at this time how those changes affect style/layout - if at all, because this information is only available after the browser has calculated it which did not happen at this time.

This means even a change from http://not-a-real-site.com to another http://not-a-real-but-different-site.com will make your isFramePending() return true. This is by design and you can’t change this.

If you want isFramePending() to really know that a DOM change also results in a style and/or layout change the browser would need to synchronously calculate it and this is basically what you want to avoid.

So my main question about this API is wether your API means something like: The browser has to recalculate style/layout because of something. (DOM/CSSOM changed, the user moved his mouse into a hitbox area or whatever).

I tried an experiment and you’re right: chromium will schedule a frame when the href attribute changes, regardless of whether the link color changed.

It’s not true, though, that the browser can’t detect the style change without computing style. Chromium actually handles the href attribute as a special case and always schedules a frame when it changes, for the very purpose of defeating these kinds of exploits.

I don’t know your test setup but saying it would be a special case that is done solely because of this exploit possibility is quite far fetched. (But of course I don’t know how you are coming to this conclusion).

Of course browsers might have some performance optimizations built in which make sure that some DOM changes do not invalidate layout. But these have to be very simple and fast because they have to happen sync otherwise calculation of these would degrade performance instead.

In case of the link I would assume that it is not because of the exploit but simply because there is a selector :visited which might match or not and calculating wether it matches is already a performance degradation.

If your test setup looked something like this:

<div></div>
<script>
	document.body.onclick = () => {
		document.querySelector('div').setAttribute('data-foo', '1');
		console.log(document.querySelector('div').getBoundingClientRect());
	}
</script>

You can trick chrome by simply adding an empty selector matching the attribute:

<style>
input[data-foo] {

}
</style>
<div></div>
<script>
	document.body.onclick = () => {
		document.querySelector('div').setAttribute('data-foo', '1');
		console.log(document.querySelector('div').getBoundingClientRect());
	}
</script>

At the end my main point was that this can’t be used as an exploit.