I think there would be value in exposing a way to manually specify the priority of resource requests. Both via HTML (perhaps using a priority attribute) and via JS (fetch() prioritization). The attribute based approach could be helpful for when you need to load resources with a particular priority before any JS is run at all.
Today, we have <link rel=preload>, however as is often not powerful enough to specify the correct priority. Browsers, even with the async attribute on scripts, don’t allow us to control the exact priority we may like these resources to be fetched with. <link priority>, <script priority> etc. could enable this.
Outside of the declarative approach to this problem, it would be useful to define priorities when fetch()ing resources in JS. Jake Archibald has kicked around some ideas for how to do this using a Fetch Controller (e.g fetchController.setPriority()).
Would manual priority control be a valuable capability to explore for the web?
I think the “how” of priority control is definitely up for debate. There may be more value in steering away from labels but offering a way to declare some resources as more or less important than others. That would allow the UA to decide how priorities should be set when priorities for H/2, QUIC etc are also to be considered.
The goal with this is to offer something lower-level than <img lazyload>.
I think something like this would be valuable for media streaming. Media streams are often downloaded in blocks, with a separate request for each block, and there is a natural priority order to those requests (the earlier in the stream a block is, the higher the priority). There is also value in having requests for several blocks outstanding at once, so that the connection from the server does not fall idle between blocks.
We’d be interested in experimenting with this on our site if there was a browser interested in implementing it.
I’m a big fan of this. Gonna cite my own tweetstorm but it just highlights that web developers (who understand resource priority) have to employ a lot of different hacks to trick the browser preloader. No two assets are the same.
Seems like all asset types (img/video/audio/css/js/iframes/3d-obj-models) could benefit from a unified priority API.
We leverage a for our own monetization modules, as we don’t want to block rendering for something “optional”, but this often queues the scripts for several seconds (on 3G) because they’re low priority.
However, we would prefer to explicitly set these to HIGH so they effectively get the “preload” behavior.
For reference, in our testing, this difference can account for 16% revenue impact!
This is more of an implicit assumption that can change anytime. Having an explicit declarative approach to set priority really helps.
What exactly is “priority” in this context? An int?
If <el1 priority=1> and <el2 priority=2>, how does this affect loading?
Does el1 block el2 request? Is there a strict dependency between el1 and el2?
What if I want to express a strict dependency? E.g. this stream segment before other segment.
Can I express relative priority in same group?
E.g. if two fetches have same priority, are they equal weight? Can I manipulate weights?
How do these priorities map to “default” (and unspecified :)) browser priorities?
How do I not shoot myself in the foot and give my resources too high of a priority, causing CRP regressions?
How do we expect these priorities to be mapped to H2 semantics? That is, what will the server see?
FWIW, I’d encourage folks to think about relative mappings, instead of assigning labels or int values… Instead, if we can express a tree “A is more important then B, but less important then Z”, then most of the above questions are addressed; If we can expose a set of primitives that communicate some resources more or less important than others (e.g. by saying this fetch ID > other fetch ID or element A > element B), then the UA can (and should) figure out the rest and map it to right set of H2 priorities / QUIC priorities / H1 connections, etc.
This is definitely something that I’ve been mulling over the last few weeks myself.
As I understand it, <link rel=preload /> will set a resource as a high priority AND move it to the top of the network queue.
If there was a mechanism to add priority to <link>, <script>, <img> etc
How would network queue order be determined? By source order?
A few disconnected thoughts:
As we know, when everything is priority, nothing is… Right now, a bunch of <link rel=preload /> tags create a little bit of restraint for authors (each asset must be added). Adding the ability to set priority on assets all the way through the page could make it rather easy to ruin any regular network conditioning that the browser would do.
<link rel=preload> also allows for media queries, which is hugely powerful. It’d be hard to do this with inline images, or images that used srcset.
Would setting priority granularity be useful? eg- <img priority="low" />
I feel like there’s so much education (and learning) to be done before greater ‘control’ is issued.
I’m very much for a way to provide the browser priority (‘low/high/highest/etc’) while fetching/preloading. Whether it’s during fetch, Image load or link preload.
<link rel="preload" as="image"> on Chrome at least preload at low priority. While trying to optimize image loading on Google Photos. I could not find a way to prefetch/preload images and control priority. If the browser can decide the priority based on various factors, I don’t see why we would not give developers the ability to tell the browser what the priority should be and not have it guess.
When loading a lot of images, some small for preloading below the fold, some high to be rendered on the screen. As soon as you don’t rely to have a bunch of <img> tags in the DOM (because as a developer you want full control on the lifecycle of the loading of the image), it’s impossible to leverage the priority browsers offer.
While trying to optimize image loading on Google Photos. I could not find a way to prefetch/preload images and control priority.
You can do that. Here are two proofs of concept with randomly inserted images. Images showing a 1 have priority 1, images showing a 2 priority 2 etc. Priority 1 = most important, priority 6 = least important.
The idea is to prevent Chrome to reorder the requests queue by evaluating the image positions. Using display;none is enough. Then we change all the data-src to src in the priority order, and Chrome will process the queue quite nicely.
Priority 1 have their src set in the HTML so the preloader can discover them.
The problem is : Firefox and Edge start to download as soon as the src has been set. So you’ve got a lot of parallel requests, and it turns out to be quite bad.
As I said, that’s just proofs of concept: images with priority 1 have their src set in the HTML so the preloader can discover them… but there is no limit to the number of parallel requests for them, and the queue of lower priority images is processed regardless of the download state of the images having priority 1.
You could also adapt the number of parallel downloads for big images, add a timer to evaluate the connection speed and adapt the number of parallelRequests accordingly, etc.
I’m aware of the priority being set based on screen positions. You’re solution is what I was trying (and failed, sorry) to quickly mention in the quote below:
This is not a good enough solution in my opinion as it includes DOM manipulation which impacts FPS, especially during scrolling with hundreds of images. In any case, having to reverse-engineering browser rules for priority does not seem like the best way to handle this compared to an opt-in option.
If you avoid reflows and use requestAnimationFrame, I don’t think DOM manipulations should be a big problem. The biggest problem I see is image decoding, but even with two simple img tags, you can clearly see that the problem exists.
Load it twice : the animated gif is cacheable, not the landscape photo.
On the second loading, set CPU: 20x slowdown. You can see that the problem is decoding.
Now load this page again in Fast 3G: the browser is decoding while the file is downloading. Smaller chunks to decode, no obvious freeze.
But if you change the src of the img tag, Chrome is decoding the image while it’s downloading, and you end up having the same behavior that the one you had with two simple img tags (try with 20x slowdown, Fast 3G):.
I can’t spot any difference between the two on my LG G3 (very slow redraws on scrolling in both cases) and my iPhone SE (smooth slow scrolling, noticeable redraws on fast scrolling). Except for the loading order, of course.
I believe it would encourage web application authors to break up their resources more thoughtfully if they could tell the browser when certain pieces are critical, not only on load, but in considering the user’s tuple of next actions. I feel this will become essential for interactivity involving animations as monitor refresh rates begin to eclipse the frame rates we are capable of achieving within milliseconds.
I think that even more important than high priority may be low priority. If I want to load critical data and show my user what they want quickly and smoothly, using the cache during the downtime is key.
Very good questions that no doubt we’ll need to answer eventually, but I think the initial discussion we need to have here is mostly around the use-cases, rather than the solutions. And it seems like there’s overwhelming support for the need here.
There are certainly a few hurdles on the way to supporting those use cases:
Resource priorities are, as you say, unspecified and there’s very little chance we’ll be able to make them so (and if we will, it’ll cripple future innovation by browser engines, so should be a non-goal).
I think we may be able to overcome that issue by defining keywords that upgrade/downgrade priorities rather than defining absolute ones. e.g. <img src=foo priority="+1"> (or somesuch) to indicate that the image is of high importance to the content.
Defining full dependency trees in markup can become complex fairly quickly, but maybe we can define some form of dependency, which can help the browser define smarter H2 dependency trees.
The basic assumption must be that the developer would not markup everything to express a full tree. I think the chances of that ever happening are slim.
We need to address the “B depends on A and browser never discovers A” case.
There’s currently no standard mapping between browser priorities and H2/QUIC dependencies/weights. I think we don’t need to define one for the purpose of tackling these use cases.
Can you elaborate on why is would be necessary? Currently there’s no standard mapping between how browsers perceive resource priority and H2 dependency trees. (e.g. Firefox and Chrome mark critical JS and async JS in very different ways, both in dependencies and weights)
I believe we can tackle the stated use cases by providing something simpler, such as “upgrade/downgrade” semantics as well as optional dependencies or deference indications (“critical” vs. “important” vs. “other”).
I agree that defining a full mapping of resources to H2 priorities would be potentially more powerful, but:
Servers will rarely have knowledge of all the resources a certain page will trigger, and won’t necessarily know the ideal dependency tree.
If they do, and all said resources come from a single server (again, rare), the server could apply those ideal dependency trees while ignoring the browser’s sent priorities.