Manual priority control of resource fetching

I’m aware of the priority being set based on screen positions. You’re solution is what I was trying (and failed, sorry) to quickly mention in the quote below:

This is not a good enough solution in my opinion as it includes DOM manipulation which impacts FPS, especially during scrolling with hundreds of images. In any case, having to reverse-engineering browser rules for priority does not seem like the best way to handle this compared to an opt-in option.

If you avoid reflows and use requestAnimationFrame, I don’t think DOM manipulations should be a big problem. The biggest problem I see is image decoding, but even with two simple img tags, you can clearly see that the problem exists.

https://www.zikinf.com/lab/priority/decoding.php Load it twice : the animated gif is cacheable, not the landscape photo. On the second loading, set CPU: 20x slowdown. You can see that the problem is decoding. Now load this page again in Fast 3G: the browser is decoding while the file is downloading. Smaller chunks to decode, no obvious freeze.

If you try to load the image file into an Image object, the browser will download the entire file, then decode it. So it might seem to be worse to use Javascript, because the main thread will process a lot of informations at once and will noticeably freeze (try width 20x slowdown, Fast 3G): https://www.zikinf.com/lab/priority/decoding2.php

But if you change the src of the img tag, Chrome is decoding the image while it’s downloading, and you end up having the same behavior that the one you had with two simple img tags (try with 20x slowdown, Fast 3G):. https://www.zikinf.com/lab/priority/decoding3.php

Note that the second technic is the worst for now, but it could become the best soon (except that you won’t see the progressive loading): https://www.chromestatus.com/feature/5637156160667648

Last but not least: here’s my example of prioritized loadings with a limit of 6 parallel requests, with 600 pictures (1000x1000px resized to 100x100px). https://www.zikinf.com/lab/priority/limit-parallel-requests-600pics.php

Just the 600 pics in simple img tags without priority: https://www.zikinf.com/lab/priority/600pics.php

I can’t spot any difference between the two on my LG G3 (very slow redraws on scrolling in both cases) and my iPhone SE (smooth slow scrolling, noticeable redraws on fast scrolling). Except for the loading order, of course.

This has come up in discussions a few times since H2 shipped (and maybe even a bit before).

A few questions that are going to need to be answered:

  • Is it declarative markup, an argument to fetch(), or a serviceworker-like JS that manages network connections? Or all of the above?
  • What abstraction is exposed? Full h2 dependency trees are complex, but expressive; if something simpler is desired, then a mapping into them will be necessary, and it won’t be as capable.
  • Is re-prioritisation an important use case?
  • Is it going to try to paper over the differences between H1 and H2, or expose them? (see especially: re-prioritisation)

I am very supportive of this proposal.

I believe it would encourage web application authors to break up their resources more thoughtfully if they could tell the browser when certain pieces are critical, not only on load, but in considering the user’s tuple of next actions. I feel this will become essential for interactivity involving animations as monitor refresh rates begin to eclipse the frame rates we are capable of achieving within milliseconds.

I think that even more important than high priority may be low priority. If I want to load critical data and show my user what they want quickly and smoothly, using the cache during the downtime is key.

Very good questions that no doubt we’ll need to answer eventually, but I think the initial discussion we need to have here is mostly around the use-cases, rather than the solutions. And it seems like there’s overwhelming support for the need here.

There are certainly a few hurdles on the way to supporting those use cases:

  • Resource priorities are, as you say, unspecified and there’s very little chance we’ll be able to make them so (and if we will, it’ll cripple future innovation by browser engines, so should be a non-goal).
    • I think we may be able to overcome that issue by defining keywords that upgrade/downgrade priorities rather than defining absolute ones. e.g. <img src=foo priority="+1"> (or somesuch) to indicate that the image is of high importance to the content.
  • Defining full dependency trees in markup can become complex fairly quickly, but maybe we can define some form of dependency, which can help the browser define smarter H2 dependency trees.
    • The basic assumption must be that the developer would not markup everything to express a full tree. I think the chances of that ever happening are slim.
    • We need to address the “B depends on A and browser never discovers A” case.
  • There’s currently no standard mapping between browser priorities and H2/QUIC dependencies/weights. I think we don’t need to define one for the purpose of tackling these use cases.

Can you elaborate on why is would be necessary? Currently there’s no standard mapping between how browsers perceive resource priority and H2 dependency trees. (e.g. Firefox and Chrome mark critical JS and async JS in very different ways, both in dependencies and weights)

I believe we can tackle the stated use cases by providing something simpler, such as “upgrade/downgrade” semantics as well as optional dependencies or deference indications (“critical” vs. “important” vs. “other”).

I agree that defining a full mapping of resources to H2 priorities would be potentially more powerful, but:

  • Servers will rarely have knowledge of all the resources a certain page will trigger, and won’t necessarily know the ideal dependency tree.
  • If they do, and all said resources come from a single server (again, rare), the server could apply those ideal dependency trees while ignoring the browser’s sent priorities.

In terms of a fetch API, this could go onto a FetchController, which extends AbortController.

const controller = new FetchController();
const signal = controller.signal;

fetch(url, { signal });

// …

controller.setPriority(…);

I’m not so sure about this, as it kinda ties you to what a particular browser does today. Eg, if a browser later decides to make images higher priority, your +1 may make their priority too high.

I’m coming round to something like:

<img fetch-class="shop-item first-ten" higher-priority-than="fetch-classes" lower-priority-than="fetch-classes">

(I haven’t given thought to the naming)

We’d need dynamic priorities to deal with the “B depends on A and browser never discovers A” case. As in, if A is never discovered, it has no priority impact, but if A later turns up while B is in-progress, priority can change to A.

Do we need a way to say “A should have a slightly higher priority than B” vs “Any bandwidth spent on B at the expense of A is bad”? Or is the latter enough?

I think there also needs to be a way to limit priorities per domain. I could see situations were third party widgets would mark all their downloads as highest priority, an embedding site considering its own priorities would get overridden or end up in a situation were everything is marked as priority 1 in order to combat it.

1 Like

A way to express priority on an iframe (and therefore all fetches it performs) would do this.

I don’t think we can add anything to police 3rd party scripts running on the same origin. Any switches we add could just be switched back.

If we decide that the priority of an iframe affects the fetches it makes, would we say the same about dedicated workers & css?

Fair.

The would require us to define default “fetch-class” values for current resource types, so that you e.g. use <img higher-priority-than="important-resources" lower-priority-than="render-critical-resources"> to say “this image should be fetched in high priority, after any critical resources”.

I guess that’s possible, but I’m struggling to figure out when you’d need that.

I suppose you might want to say “this image is higher priority than font downloads”, and I’m not sure how you’d express that.

You’d either need a way to tag something with a fetch class within CSS (but that doesn’t help with 3rd party CSS), or as you say have some reserved classes (like all of preload’s “as” values, which includes “font”). I suppose user defined values could require a hyphen, like web components :smile:.

You’d mostly need that when you’d want to impact the download priority of a single resource without adding “fetch-class” attributes on all other resources. If our solution would require people to perform a lot of manual work in order to cover the basic cases (which are most of the cases I heard from people), they won’t. Providing default fetch-classes would enable people to express what they need without marking-up all the resources (including the ones loaded by scripts which are out of their control and, as you mentioned, CSS based resources).

Hmm yeah. Summarising the different ways a fetch can be identified in terms of relative prioritisation:

URL

  • :white_check_mark: You can target it no matter how the fetch was triggered.
  • :negative_squared_cross_mark: Could get verbose & violate DRY if you’re trying to prioritise something higher than “all images”.
  • :negative_squared_cross_mark: In some cases you don’t know the URL ahead of time, eg Google fonts.

CSS selectors

  • :white_check_mark: Allows you to target all elements of a type, whether they’ve been specifically tagged or not.
  • :white_check_mark: Nth-child lets you do some positional stuff
  • :negative_squared_cross_mark: CSS background images & fonts can’t be selected.
  • :negative_squared_cross_mark: fetch() can’t be selected.

Destination

  • :white_check_mark: You can target types of requests, eg “font”.
  • :negative_squared_cross_mark: Not granular enough. eg “image” means both img and CSS backgrounds.

“Fetch class”

  • :white_check_mark: Allow you to tag elements and also programatic fetches.
  • :negative_squared_cross_mark: Doesn’t work if the fetch is initiated by a third party, and they haven’t given it a fetch class, eg font CSS.
  • :negative_squared_cross_mark: No way to assign a fetch class for CSS fetches. We’d have to invent new syntax.

Maybe we need some combination of the above, or a better idea. It may also be a sign that we’re trying to go high-level too early.

It sounds like there’s strong support for the need for this capability.

Is it declarative markup, an argument to fetch(), or a serviceworker-like JS that manages network connections? Or all of the above?

My take is that there’s value in exploring a declarative markup solution to this (an attribute-based approach that indicates a resource is higher or lower priority to others) in addition to a lower-level JS-based one (Jake’s FetchController suggestion lower down).

How do these priorities map to “default” (and unspecified :)) browser priorities?

I’m a fan of expressing to a UA that some resources are more or less important than others (rather than relying on integers) and allowing the browser to then map out what this should mean when when factoring in the priorities of other mechanisms like H2 and QUIC.

Defining full dependency trees in markup can become complex fairly quickly, but maybe we can define some form of dependency, which can help the browser define smarter H2 dependency trees.

I concur. If we can devise a design for expressing dependencies without it requiring the full-blown tree to be specified in markup, that sounds more usable. The dependency resolution note you raised (“B depends on A and browser never discovers A”) is fair and definitely something we’ll want to work out how to handle.

If our solution would require people to perform a lot of manual work in order to cover the basic cases (which are most of the cases I heard from people), they won’t.

Excessive manual annotation would limit the usability of this feature (or heavily defer it to build tools to help provide better abstraction) so trying to find a balance in the design that avoids the need for this would be desirable.

You’d mostly need that when you’d want to impact the download priority of a single resource without adding “fetch-class” attributes on all other resources.

Yoav: could you expand on this idea of default fetch-classes with some examples of what this would mean for resources you’re not annotating with a “fetch-class” directly?

1 Like

Glad to see declarative ‘lazy-loading’ getting picked up again.

One thing I haven’t seen mentioned above is on-demand loading - i.e. don’t load the image until the user has scrolled a reasonable distance down the page. This is the behaviour that some js lazy-loading solutions choose. Depending on network conditions, it can be an antipattern, but can also be a huge bandwidth saver, so some kind of API to achieve this would be useful. It’d more be a case of having a declarative way in to something like IntersectionObserver, so perhaps not all that closely related to resource prioritising, but a unified API would be nice.

I’d prefer lower level (and likely verbose) APIs at first. Not because I relish defining dependency trees etc, but more because this debate reminds me a little of responsive images. From the complex yet flexible Picture element we learned a lot, and it led the way to the simple and good-enough-in-most-cases srcset + sizes API. There are so many pitfalls around resource loading it feels like we need to use something low level for a while in order to learn what we want.

I’ll echo a lot of the same points.

I like Jake’s proposal for exploring notion of groups. Conceptually, that’s how we think about different resources types under the hood in browsers today, and it translates well to “user-space” where you have different types of content that shares similar properties. The trick here is to define a mapping between browser + user-space groups, but perhaps that doesn’t have to be that complicated either… I think we could even think of Fetch#destination as groups. Once we have the Fetch concepts defined, mapping them to declarative markup should be relative simple.

I think we should work under assumption that “browser will do the right thing by default ™” and priority group controls are exceptional overrides that should be used sparingly. Similarly, specifying a group/priority is not a guarantee that the UA must respect it… UA has the situational knowledge of the environment and is free to optimize based on provided defaults and priority hints.

I’ve set up a WICG GH repo for this and wrote down the use cases mentioned here. Use case PRs welcome! :slight_smile: Let’s discuss the suggested solutions on issues there.

Probably both declarative and a fetch() argument.

I believe exposing re-prioritization to Web developers is an important use case. From a developer perspective, H1 also had some form of re-prioritization (e.g. viewport images in Chrome get “reprioritized” and leave the queue earlier).

Request destination does not map well to priorities. e.g. “script” destination in Chrome can be mapped to “high” when it’s a blocking script at the head, “medium” when it’s a blocking script at the bottom of the page (“discovered after an image” is the heuristic used) and “low” when it’s an async script. I think we’ll need to define other form of default groups if we go that route.