[Proposal] fetch-maps

Right now, we hash files so we can treat URLs as ‘immutable’, and cache them for a long time.

Eg styles.a74fs3.css can be set to cache for a year. If the content of the CSS is changed, the URL to the CSS would change.

The same would happen for resources in the CSS:

html {
  background: url('/bg.8e3ac4.png');
}

However, if the background image changes, the URL above will change. And that means the content of the CSS file has changed, so the URL for the CSS also needs to change.

The user ends up having to redownload the CSS (a render-blocking resource) to find out about the background image change.

This is also a problem with JavaScript, which imports other bits of JavaScript, and references images, CSS, fonts, etc.

If the problem isn’t clear from the above, here’s a more detailed description.

Import maps solve this but only for JavaScript importing JavaScript. The solution doesn’t work for CSS, or for other resources referenced in JS.

There was a proposal to allow arbitrary fetches to be resolved via the import map using the import: scheme, but the layering feels wrong to me. Why should CSS have to do through a JS module mapper just to map fetches from one URL to another?

I propose that we introduce “fetch maps”, which allow URLs to be mapped from one to another.

Import maps will be layered on top, but will focus on module-specific features.

<script type="fetchmap">
{
  "urls": {
    "/styles.css": "/styles.a74fs3.css",
    "/bg.png": "/bg.8e3ac4.png"
  }
}
</script>
<link rel="stylesheet" href="/styles.css">

And the stylesheet:

html {
  background: url('/bg.png');
}

Now, to change the background image, only the map needs to be changed.

Two high-level questions in terms of scope:

(1) Would such a fetch map also support symbolic references (“bare specifiers”) or only rewrite already plausible URLs?

As an example, could I have a stylesheet that includes @import '<ref>best-fonts/comic-sans' or background: url('<ref>memes/shrug')? Or would that always require mapping them to some virtual URL space like @import '/deps/best-fonts/comic-sans' which may then be rewritten to unpkg etc.?

(2) Guy Bedford was recently digging into a preload manifest. I assume that the fetch map would be orthogonal to this?

The capability to prefetch the background image in your example or any potential @import references before knowing the style contents seems valuable. But maybe this can be ignored for non-import cases since the depth of the dependency tree is likely more shallow?

I think both sides of the map would be passed through new URL(url, document.baseURI).href or equivalent, so it’d be virtual URL space. However, for backwards compatibility, it makes sense to have the non-hashed-URLs be real, just max-age=0.

I’ll let Guy comment on that. It does feel useful though!

Gotcha. I assume that any kind of fetch/policy restrictions would apply to the rewritten URL, not the virtual/source code URL. So it would be possible to use symbolic URLs like bare:best-fonts/comis-sans even though bare: isn’t a supported protocol?

I hadn’t considered that, but yeah, that would be useful.

Interesting idea. I’m wondering, would this support scoped mappings? These are a pretty key feature of import-maps, but I’m not really sure how the referrer should be defined outside of the JS module concept (not that there’s no answer, there’s just multiple possible ones). OTOH I don’t have a clear idea of a use case for scoped mappings outside of JS.

Also wondering: Is this just a single-level mapping like import maps, or is it applied recursively? (Import maps are mapping module specifiers to URLs, so they are one level, but a URL-to-URL mapping would get to make this choice.)

I’m not sure about the scoped thing. I’ll have a think.

As for recursive, my gut feeling is to make it a single pass to avoid infinite loops.

Maybe a silly question, but how does the import map approach compare to an etag approach?

Whilst useful, import maps seem relatively complex to use and set up, requiring a sophisticated tool chain and constant dev maintenance as they add/remove resources.

With etags, a developer can just push a file and leave it to the browser and server to figure out of it was changed. What would be against etags and in favor of import maps? Some things I can think of:

• Needless 304 requests
• Server-side tracking potential
• Difficult to use in multi-server/load balancer/proxy setup?

This could be implemented by a Service Worker. I’ve done a related implementation for a slightly different case (serving local blobs for remote URLs for a game development IDE).

The only problem is that on the first run, you aren’t guaranteed to have the SW running yet, so you need a fallback. Perhaps there could be a way to say “this page needs the SW running first” (although I imagine that would cause too much of a negative performance impact). Or maybe there could be a special protocol like sw://bg.png which means “wait until the SW is ready, and then send a fetch event for bg.png to it”. Or there could be server-side logic to send a default/latest version when bg.png is requested (presumably with caching disabled to avoid the versioning problems on that…)

Another thing to work out is if fetchmaps as proposed are implemented, does the SW receive a fetch event for /bg.png or /bg.8e3ac4.png? What would be the rationale for picking one over the other?

Nah, that feels like a feature that should stay with import maps. Since fetch maps cover fetches triggered by HTML, I don’t think it’s possible to determine the source script accurately.

The fastest HTTP request is the one that isn’t made . To get a 304 you need to go through all the HTTP+TLS connection setup stuff, then make a request, then get a response. For smaller assets, that’s the majority of the work. More info on HTTP caching: https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching

The only other problem is updates . The service worker controls multiple pages at once, as such its update system is independent of any particular page. Whereas fetch maps are per context and immediately available/updated.

I kinda like your idea of a ‘sw’ scheme that waits for the service worker. Will keep that in mind for other use-cases.

Page-level things should happen before things that live outside the page, and fetch maps would be page-level. Other examples of page-level fetch control: CSP, MIX, client referrer policy.

I’ve only taken a glance, but my feeling is this would happen before step 2.1 https://fetch.spec.whatwg.org/#main-fetch. This means the mapping would happen before CSP, otherwise the map could effectively bypass CSP.

This means the service worker would receive a fetch for /bg.8e3ac4.png.

If the service worker had its own fetch map, it would apply when fetch() is called in the service worker. However, I’m not sure how a service worker would get a fetch map (import maps looked at the worker issue in https://github.com/WICG/import-maps/issues/2).

Open questions:

• Would a page’s fetch map apply to navigations, new Worker, new SharedWorker, serviceWorker.register?
• What happens with the query string and hash?
• Does this apply to all HTTP methods?

Just to understand, we’d have both import maps and fetch maps, coexisting? Namely, the output of the import map would be fed into fetch map for a secondary mapping?

Yeah. That’s the way I see it working if import maps shipped as is. Btw I originally raised this in https://github.com/WICG/import-maps/issues/211.

I don’t “feel” exactly the same way. Take the example of CSS. It does not typically result in deep dependencies and, nowadays, it is even less used, in favor of JS in CSS approaches. Still, for those who still use CSS in the form of “CSS in a CSS file”, it is frequently tied to the code of a certain “component” and loaded along with the code of that component. And, hence, the existing proposals to being able to directly import style from "foo.css and other common file types (JSON, CSS, HTML). Import maps are applied in these cases, so no real problem here. When we step into the CSS files, relative URLs can be used to refer to associated CSS files or images that belong to the same package, so no problem as well.

However, imagine if your CSS is an “extension” to a shared theme stylesheet, which is provided by a dependency package, which it @imports. It may reuse some shared definitions, override others and let the remaining untouched. Additionally, you need to refer to a particular version of the shared theme stylesheet. In this case, the @import rule would greatly benefit from using scopes and the existing import map resolution.

Ultimately, I think that the resolution, scoping and versioning needs are the same and I’m not seeing the conceptual benefits of a separate mechanism and specification. Quite the opposite, I identify a great possibility for confusion between the two maps.

If there is a problem with the name “import maps” limiting its possible uses, it could possibly be rebranded as “fetch maps”.

Concluding, I believe effort should be made in bringing back the import: protocol to import maps. Or, else, where am I thinking wrongly?

I agree that CSS dependencies are less deep, but it still seems weird to invalidate a CSS file (and any JS file that references it) because a background image changes.

It also seems weird to tell folks “if you want to solve this, use import: URLs, so your fetch goes through the system designed for JS imports”. Seems like incorrect layering.

I agree that JS benefits from the scoping thing, but I just don’t see how that works outside of JS imports. If I create a <link rel="stylesheet"> and give it an href in some JS, then append it into the body of an iframe, which scope applies?

Well, if you’d want it to be relative to the JS, you’d probably need to use the being designed import.meta.resolve(.) API to force local resolution. Otherwise, specifying an “import:” URL in the link’s href attribute would make the name be resolved against the containing html document’s URL. This seems reasonable to me. I know that there are a lot of corner cases, but I believe it should be possible to decide on sensible rules plus the help of the said API for when more control is needed.

More and more, I don’t see the JS’s module system as specific to the JS world. The tendency has been to start supporting other types of resources (to be consumed by JS, sure). Maybe who comes from the “HTML imports” discussions thinks otherwise. That’s why I said that maybe the name “import maps” could be changed to release it from the necessary connotation of “belongs to JS”… Generally, I don’t share the “oddness impression” of using “import:” in a CSS context. Let’s change the name so that it is less JS centric? The rest, conceptually, looks the same to me. I wouldn’t invent much differently than what has been.

The import: protocol (or any other suitable name) seems like an unavoidable necessity to avoid the ambiguity between plain/bare names and relative URLs. You’d need one such protocol in your proposal too, no? Or, would you not be supporting bare names?

I strongly believe in building the web from “packages”, in an NPM style, and so, for me, CSSs, JSs and images, all exist in some package. Import maps make this a reality, to a great extent. It would be great if the missing pieces wouldn’t use a completely different addressing scheme (unless for good reasons, of course )

Thanks, great discussion (for me at least)!