[Proposal] fetch-maps

Tags: #<Tag:0x00007f4019230e28>

The concept of “bare names” is a JS module thing. It doesn’t apply to the rest of resource fetching.

Both sides of the map would be passed through new URL(url, document.baseURI).href or equivalent, and URLs would map from one to the other.

There’s an alternate world where the relationship between fetch map and import map would be the reverse:

  • Fetch maps can do remappings of raw URLs. They may map to import: URLs.
  • Import maps only apply to bare specifiers in JS and explicit import: URLs in general. They happen after fetch maps and can be context-sensitive.

This means that relative JS imports would never pass through the import map. Fetch maps would rewrite URLs, import maps would further rewrite import: URLs only.

But it could, no? There’s nothing impeding us to walk that way, I believe.

Did you see my bit on using packages to organize all web artifacts? When deployed, artifacts would end up exposed in URLs which are organized by package name and version (somehow…). For example, http://test.com/packages/@my-org/themes@12.3.4/header.css. A fetch/import map which assigns a prefix name to a package as a whole handles any kind of resource “exported” by it.

The use of logical names for referring to dependency resources is as much a need for JS as it is for any other types of web resources which are able to refer to other resources (includes, imports, href, etc.). This simple concept enables independence of URL schemes, deployment and versioning (and also of things such as caching hashes :-), as you point out) for dependency resources.

I’m trying to understand why you don’t seem to agree with or support this more general and unifying approach. Do you consider it a bad pattern? Are JS resources “incompatible” with other resources w.r.t. to the used URL patterns, packaging and versioning, etc.?

Cheers!

<img src="foo.jpg"> works today. We can’t change that behaviour now because it looks like what JS modules call a ‘bare module name’.

JS module resolution isn’t more general than fetching. Fetching existed before JS modules. Once JS modules are resolved, it calls into fetch. JS module loading is a layer on top of fetch. Making, eg, a CSS image fetch go through JS module resolution only to go back through regular fetch seems like really weird layering.

About changing behavior, I didn’t mean that. That’s exactly why import: (or other protocol) would need to exist, to disambiguate between bare names and relative URLs in non-import contexts. You can find that explicitly in one of my previous messages :slight_smile:

About the fact that it is JS module imports that use fetch, and not the other way around, that’s understood. My point is more about whether import maps should be considered a part of JS Modules or of fetch. My point is that, conceptually/functionally, import maps provide all of the features I would like fetch would support. In that sense, import maps (or whatever other name it would use) could be moved to the fetch layer, so that it could be used to address all other types of resources.

You didn’t comment on my opinion about the benefits of bare names for other web resources, other than JS resources. What do you think of that idea? I should also mention that one of the products developed by the company I work for uses a similar packaging/URL scheme to that described above, plus a form of import maps (the “good” old RequireJS), to expose web packages and all of its resources of varied types to application(s).

Thanks for your patience!

So, iiuc

  • this can’t be used for html files referenced directly in the browser’s url bar, but it can be used for references to html files in other html files (iframe, etc)
  • if I am linked directly to /bg.png I only get /bg.8e3ac4.png if there’s also a corresponding redirect, but that’s needed for UA that do not support fetchmaps anyway
  • /bg.png in the same stylesheet can reference different files depending on the fetchmap of the page they’re loaded from (I’m sure this could be used for neat things but I don’t want to do neat things, I want to do simple things that work and that I can understand without a lot of effort)
  • on a static site, if I change an asset I need to regenerate all the html files instead of all the css/js files; and, to keep the size down, ideally this would do dead code elimination of a sort but only so much can be done as it wouldn’t necessarily know whether assets referenced in an included css file are used on a given page and the like. A dynamic site would have similar problems with which fraction of the total fetchmap to include.

The last two points could be handled with external, canonical fetchmaps included by src but then those could get big again so there should probably be a mechanism to delegate to other fetchmaps like “for requests in the /static/ directory see /static/fetch.map”.

I like the idea, but, given all that, it seems like it’s just pushing the problem around unless there’s a single canonical fetchmap at the root that’s grabbed before anything else (and that can delegate to other non-canoncially named fetchmaps). You’d still need all the redirects but they could be generated from the fetchmap(s), possibly even just by apache/nginx reading them directly.

They cover a lot of things that are specific to JS modules, so I don’t think you can consider them to be ‘just fetch’.

I think the system should operate on URLs as they exist today, rather than try and introduce new URL concepts along with it. Maybe new URL concepts could be added later, as a separate proposal.

I’m not sure about iframe. In service workers, the owner of the iframe destination URL gets to control the fetch. This is because service workers can return arbitrary bytes as a response, so you don’t want evil.com to be able to control what loads into a notevil.com iframe.

If this proposal remains a simple ‘map’ that acts like a redirect, it can work for iframes and even navigations / form submissions.

Yeah, you’d either have a redirect or just duplicate assets.

A page’s CSP or service worker can already influence this.

A build tool already has to build up a dependency graph in order to calculate the hash of the file, it just needs to export this. Rollup already does this for JavaScript.

I don’t think so. For performance you’d want each page to have an inlined fetch map, ideally containing the minimum required data. A dependency graph would make this easy.

Possibly, I’m unaware of the specifics you mention. Maybe, they’re some specification details? Could you please point me to one?

Why are fetch maps separate from import maps? Couldn’t this proposal be an import maps v2 instead of a completely separate proposal? My understanding is that the radical simplification of import maps made in August 2019 (which included removing import: urls) was an effort to make the proposal easier for browsers to comment on. It was not because import: was deemed out of scope for import maps. My understanding comes from this issue where Domenic commented on the slimming down of the import maps spec.

Am I missing something? Why can’t fetch maps just be an “import maps v2”?

This post started as an issue on import maps https://github.com/WICG/import-maps/issues/211.

I don’t think you can build fetch maps on top of import maps, given the JS-specific behaviours within import maps. Fetch maps would sit below import maps, since when a JS module is resolved, it will use fetch if it needs content from the network.

Thanks for the reply - what is the js-specific behavior in import maps? That they are slimmed down just to module specifiers? This post from Domenic talks about how import maps would be used for html/css/json modules (if they ever become real things).

My thought is that we could introduce the import: URL scheme as part of the fetch infrastructure, while still using import maps as the basis for the scheme. And call that an extension/addition to import maps instead of requiring a new mechanism (fetch maps) to accomplish it. Is that not possible?

See the original post for a comparison to import:

From the original post:

There was a proposal to allow arbitrary fetches to be resolved via the import map using the import: scheme, but the layering feels wrong to me. Why should CSS have to do through a JS module mapper just to map fetches from one URL to another?

Import maps are not js-specific - they are specific to modules (which may include css, json, html modules, etc in the future). For me, it’s not so much that “CSS has to go through a js module wrapper” as much as “CSS can map its modules, too.” Could you clarify what you mean by “the layering feels wrong to me”?

1 Like

Yeah, this would be awesome. I’m currently implementing this manually myself (in another language that compiles to JS), and it’s a pain (kinda like getting a cat out of a tree).

I basically have a global url conversion table file stored in json, load it up first, store as a map, and every time an url is loaded, I have a function call that looks up the hashed url.