A partial archive of discourse.wicg.io as of Saturday February 24, 2024.

[Proposal] Module Import Headers

Randy-Buchholz
2020-05-17

Introduction

In a module, import produces a GET to a server and processes the results in a specific way. While the browser is aware that an import is taking place, the server isn’t. Letting the server know the response will be processed as an import opens up many possibilities.

Commonly an import request is made for a static file with something like - import { Foo } from "/bar.mjs";. If the response contains something like export class Foo { }, import is happy.

Because the request is a standard GET, we can include query parameters:
import { Class1, Class2 } from "/ModuleService.mjs?one=Class1&two=Class2";

In this case, ModuleService represents an entry point into a process that dynamically builds the response. We can intercept the request in the request pipeline to do this. Again, as long as the response contains something like export Class1 { } export Class2 { }, import is happy.

This approach allows us to do things like manage small discrete modules in our source, but avoid having to import each required module individually. We can use this for things like implementing primitive forms of (pseudo) dependency injection.

import { Dependency1, Dependency2, Dependency3 } 
from "/diService.mjs?\
asm1.ns1=Dependency1&\
asm2.ns2=Dependency2,Dependency3";

export class OperationalClass {
  #dependency1;
  #dependency2;
  #dependency3;
  constructor() {
    this.#dependency1 = new Dependency1();
    this.#dependency2 = new Dependency2();
    this.#dependency3 = new Dependency3();
  }
  doStuff(){
    const x = this.#dependency1.invokeFunction();
  }
}

When an import of OperationalClass is requested, the server can parse the imports, and package all four modules/classes into a single file. This is only one possible use of this approach.

Problem

A problem with this is that while the browser knows it is expecting a module when it requests an import, the server doesn’t explicitly know it needs to return a response in the form of an import response. We can use tricks like flags in the query string to indicate an import and inspect the query string for these flags in the request processing pipeline, but this is cumbersome and error prone. The imbalance between what the browser knows and what the server knows is limiting and problematic.

Solution

A better approach is to indicate an import is being requested in the request header. This provides simple and explicit information the request pipeline can use to route the request.

Proposal

Proposed: When an import produces its GET request, it adds request header information to indicate an “import response” is being requested.

Details

This proposal does not specify the specific headers, but identifies the need for a “module import” header group, and two header items:

  1. Import Indicator - this item is used as the primary filter by the request pipeline.
    If the indicator is not present or “false” the request is not an import request. If the indicator is present or “true” the request is an import request. Optionally, the indicator can be valued for use as a switching parameter.
  2. Import Data - this item is a string or set of Key Value Pair that contains information used in processing the request. (e.g., the query string values in the example).

Implementation

The proposal can be implemented in two stages.

The first stage simply adds a single header item to the request. This is (essentially) a flag - without value or with a default value. This should be a low impact effort, as it creates no changes visible to the developer, requires little code to implement, and has the only effect of creating a new header entry. Since no data is being managed, the import statement is unchanged - data can still be sent in the query string. This alone provides significant value.

Stage two would alter the import statement to accept parameters. The details need to be discussed, but a simple version might be:

import { Item }
from "/target"
with [parameters];

Any information provided in the with parameters would be written to the Import Data header item. The example might look something like:

import { Dependency1, Dependency2, Dependency3 } 
from "/diService.mjs"
with [
   ["asm1.ns1", "Dependency1"],
   ["asm2.ns2", "Dependency2, Dependency3"]
];
kaizhu256
2020-05-19

i think using url-search queries are perfectly fine. url-construction requires less tooling and are easier to inspect than http-headers when things inevitably go wrong and you have to troubleshoot why a module you requested is incorrectly configured.

Randy-Buchholz
2020-05-23

There are several problems with using a query string, both conceptually and practically. And, I don’t think working with headers is any more difficult here. The “users” of this are primarily those writing request handlers on the server. Application developers wouldn’t really know this existed since import is doing this behind the scenes on the browser. In these cases we are usually already in the headers. Inspecting the query is really more work and requires building more tooling.

The conceptual problem is that the information here is essentially meta-data about the request, similar to content-type. This information is more “processing directives” than user data. This type of information belongs in the headers. Which leads to the practical problems…

When you start adding “processing directives” to a query string, and making the user responsible for them you complicate and restrict their work. They need to place boiler-plate parameters in the string which is more work, more error prone, and something that should be automated.

Worse though, is that you are creating “reserved words”. Assume that your handler is looking for imports to see if it should process some of these differently than “static file” requests or “static import” requests. You might choose to add a parameter dynamic to signify this. Now, you need to parse the query string of every request to look for that parameter. This is happening early in the pipeline (routing phase) when you shouldn’t really need to be looking at query parameters. This level of inspection slows down the entire system.

This is a “weak filter” too. Without knowing the request context (i.e., import request v “normal” request) any request with a “dynamic” parameter is an import candidate and requires additional processing. In essence, you are “reserving” the parameter “dynamic” in the sense that the developer needs to know to that it will trigger additional overhead in the pipeline.

Having the request builder add import headers eliminates this problem.

Change - Header Types

I originally suggested two header items - “Import Indicator” and “Import Data”. I’m having second thoughts on “Import Data”. This really is user data and likely should be in the query string. So in the example using with, that information would end up in the query string. But I see the need for a different header - “Public Interface” (description not title, lol).

Reasoning

Imported modules have two information sets in the code - public and private. Everything exported is public and everything else is private. An import declaration has two parts - the list of required exports (export { Foo, Bar }) and the sources that define these (from "FooBar.mjs";). One way to look at these is as Interface/Implementation pairs. The export segment defines the required interface, and from identifies the implementers.

A primary use case this proposal supports (for me) is Lean and “Just-In-Time” (JIT) modules. The basic idea is that imports are built dynamically from sets of components to only include the pieces needed by the context (Lean) and assembled on demand (JIT). Both pieces of information (interface & implementer) are useful when building “just-in-time” imports. Implementer less so though… sort of.

In a static environment, we request (import) a “named static” file that we expect to export the items specified. In a more dynamic environment, we can be more versatile - we can “inject” any file that realizes our interface. This decouples things to a large extent and makes it easier to build libraries of reusable components (for example). We can change our thinking from importing “files” to importing “types”. In some ways (and contexts), the Web is a giant “type system”. URL’s paths are namespaces and files are (can be seen as in some cases) “types”. Backing up a little, from one perspective, the “traditional” import and instantiation in ES:

import { Foo, Bar } from "/Sources/FooBarSource.mjs";
const foo = new Foo();
const bar = new Bar(foo);

appears much like, in some other languages

using Sources.FooBarSource;
var foo = new Foo();
var bar = new Bar(foo);

A subtle difference is that in one case (second) we are using “types” and in the first we are using “type definitions”. Consider:

import { Foo, Bar }
from "/Namespaces.mjs?qualifiedtypes=Sources.FooBarSource[Foo,Bar]";

This specifies “types” and not their definitions. The request pipeline resolves the “types” and returns the definitions (as required by the interpreted nature of ES). This decoupling provides flexibility that can support JIT or DI scenarios.

Returning to my case. The current implementation of import would send the “types” in the query string (from which we can get/create the implementer), but we don’t know the “Public Interface” to expose if we are jitting. Adding a header item that contains the interface (i.e., import { … }) gives the server a complete picture of the intent.

Proposal

Proposed (original): When an import produces its GET request, it adds request header information to indicate an “import response” is being requested.

Proposed (Added): One of the header items shall contain the list of items specified in the imports { … } list.

With “Import Headers” and a “Public Interface” item, developers of request handling pipelines are in a much better position to develop enhanced capabilities and flexibility for ES Modules. Additionally, this would improve performance (over query string approaches) in scenarios where requests need to be routed to “import handlers”.

bahrus
2020-05-24

At the risk of stating something you may be very familiar with, the proposal above seems highly related to the proposal-module-attributes, even down to use of “with.” I could not find any open or closed issues there that suggest sending the “with” object as headers, so maybe worth raising?

Randy-Buchholz
2020-05-24

Thanks bahrus, I wasn’t aware. Some similar ideas there and seems like a case that could benefit from having a header group for modules.

Created and item: