[Proposal] Set origin-wide policies via a manifest


#1

Strawman Draft: https://mikewest.github.io/origin-policy/

TL;DR: Developers can jam all 23k of their Content-Security-Policy into a manifest file at /.well-known/origin-policy/[name-goes-here], along with a number of other headers and configuration options, and instruct user agents to apply the metadata contained within to every response served from the origin.

Overview

Developers set a number of properties associated with resources on an origin by delivering resource-specific HTTP response headers and meta elements. This is becoming more common over time, and it’s quite normal these days to see multiple kilobytes at the beginning of every response dedicated to such metadata. Setting this metadata is valuable indeed, as it can have a large impact on performance, security, and privacy.

However, the existing delivery mechanism is ill-suited to the task, suffering from a clear mismatch between the resource-specific nature of the metadata declarations on the one hand, and the origin-wide intent of the metadata on the other. Take Strict-Transport-Security [RFC6797] and Public-Key-Pins [RFC7469], for example. These headers explicitly alters the state of an entire Moreover, many headers are deployed in such a way as to be practically static across all resources that an origin serves. Content-Security-Policy [CSP3], for instance, can be very granular indeed, but is commonly implemented by setting a single policy which is delivered for an entire application.

A number of implications follow:

  1. Servers are required to repeat themselves. At length. Content-Security-Policy alone can easily eat multiple kilobytes of each navigational response, bandwidth which could instead be dedicated to content a user might care about. This has obvious and direct impact on the delay a user experiences when navigating, but has less obvious knock-on effects that reduce performance further. HTTP/2’s HPACK [RFC7541] header compression is limited to ~4k of state for processing, for instance, meaning that these verbose headers can greatly reduce its effectiveness

  2. Servers are required to repeat themselves. Unerringly. When policy applies strictly to a resource, and not to the origin, then a server must send that policy down with every response. If the developer forgets a page (which is actually a very common occurrence: when’s the last time you thought about your 404 error page? How about your 417? Or 505?), then the policy’s protections don’t apply to that resource, leaving an exploitable hole.

Origin Policy introduces a new delivery mechanism for policies which are meant to apply to an entire origin. In short, a server will provide an Origin Policy Manifest file at a well-known location. This file contains all of the metadata the server would like to set for each response. User agents can be instructed to synchronously download and process this manifest before completing a navigation to an origin’s resources, ensuring that the policy contained therin will be safely applied to each resource, and allowing the server to skip the overhead of including the relevant headers with each response. Typically, the server can speed things up even more by using HTTP/2 Server Push ([RFC7540], section 8.2) to proactively send the manifest file along with the response to the user agent’s first request.

Example

MegaCorp, Inc. wishes to ensure that a baseline content security policy is applied to each of the pages on https://example.com, while avoiding the overhead associated with large response headers, and the uncertainty that they’ve really covered everything that lives on the origin.

When they see a request come in that contains a Origin-Policy header, they can respond in kind, pointing the client to a manifest file in a well-known location on their server. That is, given the following request:

GET / HTTP/1.1
Host: example.com
Connection: keep-alive
...
Origin-Policy: 0
...

MegaCorp, Inc. can respond with:

HTTP/1.1 200 OK
Content-Encoding: gzip
Accept-Ranges: bytes
Cache-Control: max-age=604800
Content-Type: text/html
...
Origin-Policy: "policy-1"
...

The client will parse the response headers, and synchronously request https://example.com/.well-known/origin-policy/policy-1 before completing the navigation. The policies contained in that file will be cached according to the normal HTTP caching rules, and applied to pages on https://example.com/ (including the current navigation).

{
  "headers": {
    "fallback": [
      {
        "name": "Content-Security-Policy",
        "value": "script-src 'self' https://cdn.example.com"
      },
      {
        "name": "Referrer-Policy",
        "value": "origin-when-cross-origin"
      }
    ],
    "baseline": [
      {
        "name": "Content-Security-Policy",
        "value": "object-src 'none'; frame-ancestors 'none'"
      },
      {
        "name": "Strict-Transport-Security",
        "value": "max-age=10886400; includeSubDomains; preload"
      },
      {
        "name": "X-Content-Type-Options",
        "value": "nosniff"
      }
    ]
  },
  "cors-preflight": { /* TODO(mkwst): Syntax? */ },
}

Subsequent requests from the same client will contain the version of the policy currently cached for the origin. In this case:

GET / HTTP/1.1
Host: example.com
Connection: keep-alive
...
Origin-Policy: "policy-1"
...

#2

Love the idea overall. This will result in huge savings on headers size.

But curious to know about the debugging experience and how browser/cli tools might show the metadata about the headers.


#3

I like the idea, though wondering if rather than origin-policy it uses the domain and path rules as applied to cookies rather than a blanket origin. This allows a bit more fine grained control, particularly where an application lives at the path rather than subdomain level.


#4

But curious to know about the debugging experience and how browser/cli tools might show the metadata about the headers.

What would you like to see in devtools? I could imagine this being rendered as a separate resource (similarly to the way Chrome treats Service Workers), or perhaps the injected headers being tagged in some way. What would you care about when debugging?


#5

I like the idea, though wondering if rather than origin-policy it uses the domain and path rules as applied to cookies rather than a blanket origin. This allows a bit more fine grained control, particularly where an application lives at the path rather than subdomain level.

I’m a little wary of doing this for two reasons:

  1. The origin is the security boundary that matters. The fact that cookies and service workers support path-level distinctions is, in my opinion, a historical mistake. In short, /path1 and /path2 have exactly the same capabilities. Supporting developers’ tendency to treat them distinctly seems like a footgun.

  2. Adding paths to the manifest syntax (as is being discussed in https://lists.w3.org/Archives/Public/public-webappsec/2016Jul/0061.html) is potentially valuable, but it would be a huge increase in complexity, and it’s not really clear to me that we can do any of this yet. :slight_smile: I’d like to get a minimal manifest out the door to prove that the concept works before we get too deep into the configuration weeds.


#6

The fact that cookies and service workers support path-level distinctions is, in my opinion, a historical mistake

The distinction in service worker isn’t a security boundary aside from the location of the service worker script somewhat determining the scope of the SW. We weren’t really happy with this, here’s the discussion https://github.com/slightlyoff/ServiceWorker/issues/468

We can lift this restriction if you don’t think it’s a security concern.


#7

The distinction in service worker isn’t a security boundary aside from the location of the service worker script somewhat determining the scope of the SW.

I agree, and I’m glad you do too, though I think the folks who wanted the change wished to treat it as such. As Alex noted in the first comment on that bug “There has been quite a lot of further discussion on this point. Looks like we’re going to sacrifice this goat to accommodate sites which, frankly, are already broken. Le sigh.” That’s pretty accurate.

If we were doing everything over again, I’d prefer the strict MIME-type checking and origin-wide scoping. Since we’re not, I’m not sure it makes sense to tilt against this particular windmill.


#8

Hey Mike,

It’s really good to see your proposal here. As you know, I have a similar spec that I’d been working on (not realising you were as well!).

I gave you some feedback in e-mail and never heard back, I imagine you’re busy. So I’m putting a slightly modified version here for wider discussion.

For others’ benefit, my proposal is this spec. With that as context:

  • Using JSON means that encoding / decoding needs to happen, which means that errors (especially authoring errors) are more likely. My inclination is to just reuse HTTP headers’ textual format; it’s the lowest-friction way to do this, rather than having to worry about converting UTF-8 to ASCII, etc.

  • Your spec adds origin-wide flags (cors-preflight and unsafe-cors-preflight-with-credentials) that are specific to this format. I’m not crazy about having yet another configuration mechanism that people need to understand; one of the big benefits of relating the well-known location’s contents to headers was that It’s Just Headers.

  • It’s important to consider what happens when responses that omit headers are cached; that probably involves requirements to emit Vary appropriately.

  • How does this interacts with intermediaries? Either it’s end-to-end (in which case, the omitted headers are effectively hidden from proxies, reverse proxies, CDNs, server-side frameworks, etc., unless they also implement this spec), or it’s hop-by-hop (in which case it’s harder to deploy, and tricky to get right; different in H1 vs H2). I went for the former. In your spec, this is probably just some advisory text.

  • I think you’re using different resources (e.g., /.well-known/origin-policy/policy-1) for versioning, not to allow multiple policies to be active for a site at the same time - correct? If so, isn’t this just re-inventing ETags? Using ETags will Just Work, and allows you to avoid an RT in some scenarios.

  • You use the same header field name (Sec-Origin-Policy) both as a request and a response header, with different semantics and syntax. Doing that has been confusing for developers and admins in the past; please consider using two header field names.

From a WICG perspective, I’d love to hear feedback from folks about my spec too, of course. I’m not too fussed about venue, but because this touches HTTP so closely, it’d be good to at least coordinate with the HTTP WG.

Cheers,


#9

A perhaps ignorant question, but how does this play with the existing manifest proposal?

It feels like there is no consistency in how boundaries (security or application) are being defined for applications on the Web and for developers it’s hard to know what tool to use in what context.

ServiceWorkers, cookies, apps defined by a manifest, good ol’ fashioned websites.

Is there a high level architecture somewhere that defines a desired end state we are aiming for that is driving all of this?

My concern is that the tribal knowledge around web security is getting deeper and harder to grasp and that in itself is bad for security.


#10

I gave you some feedback in e-mail and never heard back, I imagine you’re busy

Sorry for the delay, @mnot. As it happens, I have been the opposite of busy! Beaches are nice. :slight_smile:

Using JSON means that encoding / decoding needs to happen, which means that errors (especially authoring errors) are more likely. My inclination is to just reuse HTTP headers’ textual format; it’s the lowest-friction way to do this, rather than having to worry about converting UTF-8 to ASCII, etc.

I agree that plaintext ASCII is simpler to parse than JSON. I think I disagree that that complexity would introduce authoring errors, simply because JSON is very well-understood by the developer audience I’m targeting. There’s good tooling available for the format in itself, validators could be written, etc. From an authoring perspective, the difference between:

# foo
Header-Name: Header-Value

and

{
  "headers": [
    {
      "name": "Header-Name",
      "value": "Header-Value"
    }
  ]
}

is pretty trivial. It’s not nothing, certainly, but it doesn’t seem fatal.

This more structured format also enables the fallback/baseline distinction on a per-header basis, which seems pretty valuable in terms of allowing per-resource overrides of origin-level defaults. This could, of course, be added to your proposal via some structure. I don’t think the format is really the major difference in approach.

Your spec adds origin-wide flags (cors-preflight and unsafe-cors-preflight-with-credentials) that are specific to this format. I’m not crazy about having yet another configuration mechanism that people need to understand; one of the big benefits of relating the well-known location’s contents to headers was that It’s Just Headers.

This is the big difference, and as it turns out, this is actually the thing I like most about my proposal. :slight_smile:

I agree that “It’s Just Headers” is super-appealing. However, I don’t think we can/should build features like the CORS bits with headers. Headers are resource-specific, and allowing them to assert origin-wide properties is problematic.

If we want to create a mechanism for setting site-wide policy, I think we ought to do it explicitly by divorcing those policies from resource-specific headers. That is, I’d prefer to deprecate existing origin-wide headers like Strict-Transport-Security by moving them to this kind of inherently origin-wide policy declaration.

It’s important to consider what happens when responses that omit headers are cached; that probably involves requirements to emit Vary appropriately.

Very true. I didn’t think about that at all. I added a tiny note in https://github.com/mikewest/origin-policy/commit/6ad70deeb2bd3ea428d9b3f6711a60e3d23e274d#diff-117d6498d2aa8019cc0abf5eeb87a9fa, and I’ll expand upon it going forward.

How does this interacts with intermediaries? Either it’s end-to-end (in which case, the omitted headers are effectively hidden from proxies, reverse proxies, CDNs, server-side frameworks, etc., unless they also implement this spec), or it’s hop-by-hop (in which case it’s harder to deploy, and tricky to get right; different in H1 vs H2). I went for the former. In your spec, this is probably just some advisory text.

I’d envisioned this as end-to-end, and really only looked at it from the perspective of the server and the browser. I suspect you have some opinions about what this would mean for intermediaries that need to understand things like the omitted headers? :slight_smile:

I think you’re using different resources (e.g., /.well-known/origin-policy/policy-1) for versioning, not to allow multiple policies to be active for a site at the same time - correct?

Correct. One policy is set for the origin, and it’s just a resource like any other, with the same request semantics and caching rules as any other resource. That means that ETag would certainly be one way of versioning the policy, and it might work really well for lots of users.

I’m not sure it would be the only mechanism that works, however, given the tight binding between the manifest’s effects and the application-layer logic that determines the headers sent by the origin. My understanding here might be outdated, but my understanding is that infinite-lifetime, unique URLs are best practice for a site’s resources (images, etc). This feels like a pretty good model to support with the manifest as well.

You use the same header field name (Sec-Origin-Policy) both as a request and a response header, with different semantics and syntax. Doing that has been confusing for developers and admins in the past; please consider using two header field names.

Hrm. Ok. Inventing more headers seems more complicated, but I’m open to distinctions between the request and response if that’s helpful.

I’m not too fussed about venue, but because this touches HTTP so closely, it’d be good to at least coordinate with the HTTP WG.

Very happy to!

Thanks again for your feedback! I look forward to more conversation, especially about the origin-wide vs resource-specific distinction raised above.


#11

I tried to address this in https://mikewest.github.io/origin-policy/#app-manifest; it’s pretty much an open question. I could certainly see the rules defined here merging in one way or another with the properties defined in an application manifest. I’ll paste it here just so you don’t have to hop and we can have the whole conversation here:


#12

Right! Sorry for being lazy.

This was perhaps an ambiguous question but what I am really trying to understand is how the boundaries of a “Web application” are defined and what type of boundaries these are. Also, what tools exist to manage the meta-data of this application vs the origin in which it exists and how these interact.

This is my understanding…

A web application exists at an origin and a has path. This application boundary is respected by cookies, ServiceWorkers and the appmanifest which can all be “scoped” by a path.

Therefor one can theoretically have multiple applications at an origin differentiated by path.

Further, the application would now inherit other properties from the origin-wide manifest.

What is not clear to me is how these all interact because, as you state, the app manifest has a declarative registration/installation mechanism so developers can’t depend on it being applied which I think severely impacts it’s usefulness.

On the other hand, as an app developer I can manage cookies and ServiceWorkers with much more granular control.

Shouldn’t the installation/registration of the origin wide and app manifests be consistent? Would that make resolving issues like https://github.com/w3c/manifest/issues/161 easier?


#13

A web application exists at an origin and a has path. This application boundary is respected by cookies, ServiceWorkers and the appmanifest which can all be “scoped” by a path.

Therefor one can theoretically have multiple applications at an origin differentiated by path.

Further, the application would now inherit other properties from the origin-wide manifest.

My claim is that the path-based scoping was a bad idea, and I don’t think it’s at all compatible with security decisions. If you want separation between two applications, you simply must put them into separate origins (subdomains, for instance: mail.google.com vs plus.google.com)^.

That is, App1 at /path1 cannot protect itself from a cross-site scripting attack that targets App2 at /path2. Those applications are same-origin (modulo the Suborigin proposal at https://w3c.github.io/webappsec-suborigins/), have access to the same client- and server-side resources. There is no relevant security boundary between them that can’t be overcome by a clever attacker. Defining policy for a particular path only is ~the same as not defining a policy.

^ Smart folks are working on https://w3c.github.io/webappsec-suborigins/, which mitigates some of these risks, but it’s opt-in on a per-resource basis, and it’s not clear to me that it can solve this problem entirely.


#14

IIUC that would mean that any change in origin policy would require (at least) revalidation of all resources in that origin. Is that correct? Would that be something that happens often?


#15

IIUC that would mean that any change in origin policy would require (at least) revalidation of all resources in that origin. Is that correct?

For every resource which asserted the Vary, yes. There’s probably no need to do so for static resources, for example.

It’s probably a bit more complicated than that (in both models) because it’s also possible for Vary to target one of the headers controlled by the origin policy manifest. shrug These are things we need to think about together before moving much further.

Would that be something that happens often?

For most folks, probably not? For some folks, every build could be a new policy.


#16

Hi all

I have just had my first read through Mike’s draft and wanted to chime in briefly with some hopefully useful input.

I work in the UK at the BBC and WRT @ahopebailie’s points/concerns, we have a single (well 2, www.bbc.co.uk and www.bbc.com) origin, with many (i think we’re in the high 10’s or possibly 100’s) of separate websites (e.g. home page, news, sport, weather, travel, childrens, …), developed, managed and hosted independently by our various teams. This structure has lead to a showstopper on HSTS as we currently have some but not all websites available on HTTPS - this is a real shame and I am very keen to add our input/help wherever possible to try to ensure future specs don’t cause us the same issues. FWIW, I believe that quite a number of other media-type organisations also have path-based (rather than domain/hostname-based) inter-website boundaries so I would imagine they’d have the same issues/concerns.

The added complication we (and i am sure others) have is the distributed nature of our organisation, teams do talk to each other of course but getting absolute, failsafe synchonisation between them and also security (in terms of segregation of access) means that shared configuration files which demand such constant attention as something like CSP would be really problematic so it’d be ideal from our org’s perspective to be able to deploy policies independently, per team/website.

It seems to me, after an admittedly relatively brief initial read, that this draft could accommodate path-based boundaries by defining the scope of application/caching a little more granularly. Either that or perhaps by explicitly stating which path(s) a policy document applies to, though the latter could get unwieldy quite quickly.

If i can be of any use in giving any further info or in any other way, please let me know.

Cheers Neil


#17

Hey Neil! I appreciate the feedback. Google has similar problems with regard to multiple applications living on a single origin, so I’m super-sympathetic to the general problem.

That said, I think there are a set of configuration options which only make sense at the origin level, as the origin is the only security boundary we have.

For better or worse, HSTS is a good example of such a policy: if you wish to ensure that a network attacker can’t downgrade your users to insecure transport, then making that assertion on anything other than an origin level doesn’t sound like it would be an effective strategy. I understand that this increases the difficulty of deployment.

Likewise, if your goal is to prevent XSS on your origin, pinning a CSP to a specific path is helpful for resources under that path, but also means that you don’t get the benefit of a site-wide policy for resources you’ve forgotten about (error pages are a good example). Given that every page on an origin has the same privilege to act on the origin with the user’s credentials, you’re still in trouble.

It might very well be the case that there’s enough need for path-based categorizations that we’ll want to add that in the future. For the moment, though, I’m interested in ensuring that the concept itself works. Limiting ourselves to the simple case of an app on an origin seems like a reasonable way to get over that hill.


#18

After reviewing this proposal, I really like it, especially the aspect of being able to set some policies across an entire origin rather then worrying about covering every potential future request.

I really like the significant perf advantage upon connection establishment, where you don’t have to blow your init CWND TCP budget on security headers. (at least for non-first navigations)

HTTP/2’s HPACK [RFC7541] header compression is limited to ~4k of state for processing, for instance, meaning that these verbose headers can greatly reduce its effectiveness

Slight tangent, but why is HPACK’s window limited to 4K? Could we fix that for things that actually need to be on a per resource basis?

User agents can be instructed to synchronously download and process this manifest before completing a navigation to an origin’s resources

This requirement is making me slightly nervous… Is there a way to avoid it? H2 push can significantly mitigate it, and the advertisement of the current policy with the request headers makes this significantly easier to pull off, but I suspect that many sites wouldn’t push the manifest, adding an extra blocking resource. It’s also not clear to me what “completing a navigation” is? Can the UA send a request for the HTML resource in parallel?

The Vary situation also worries me as I suspect policy changes will result in ignoring unaffected cached resources. Due to the fact that Key is applicable to header values, it won’t be able to save us here and add granularity.

Can you give examples of policy changes that prevent cache reuse?


#19

Slight tangent, but why is HPACK’s window limited to 4K? Could we fix that for things that actually need to be on a per resource basis?

Talk to the HTTP WG. I don’t understand the constraints they’ve placed on various bits of the protocol. I’m sure there’s a reason, but I don’t know what it is.

User agents can be instructed to synchronously download and process this manifest before completing a navigation to an origin’s resources

This requirement is making me slightly nervous… Is there a way to avoid it? H2 push can significantly mitigate it, and the advertisement of the current policy with the request headers makes this significantly easier to pull off, but I suspect that many sites wouldn’t push the manifest, adding an extra blocking resource.

If you don’t care about the initial load (or you see that the request has an outdated-but-not-too-outdated manifest, Brad’s proposal in https://github.com/WICG/origin-policy/issues/10 will address your concern. I haven’t merged that basically because I’m waiting for us all to collectively decide if we want JSON headers or not. :slight_smile:

It’s also not clear to me what “completing a navigation” is? Can the UA send a request for the HTML resource in parallel?

Yes. Basically we’d end up with a nested fetch, as spelled out in https://wicg.github.io/origin-policy/#monkey-patching-fetch. We’d fetch the resource, see the header, fetch the manifest, and then return the initial response. The requests are more or less serialized. “Synchronous” is probably the wrong word… it scared Boris at TPAC as well.

Can you give examples of policy changes that prevent cache reuse?

Mark has some examples of things that might effect cached resources in his https://tools.ietf.org/html/draft-nottingham-site-wide-headers-00, and recommends avoiding them in a site-wide policy. Similar language in this spec might be enough?


#20

Yeah, adding some form of “don’t send resource-specific headers to the entire origin” language will probably do the trick.