Measuring site performance benefits from a close connection with publisher business outcomes. Establishing this connection is difficult and often confounded by concurrent changes to site UX, content and product offerings.
Causal connections between performance and business outcomes require randomized A/B testing on live traffic. Sites, frameworks and analytics providers may support such experiments within the scope of the components they control, e.g. by varying the image or javascript payloads, or delaying server response. However, these methods do not always have a direct relationship to user-observable performance and/or require large interventions affecting many users to produce measurable effects.
Internally, Chrome has used browser-based experiments that modify memory use and establish causal connections between memory use and user experience. These studies are known as ablation studies, and are common in other black- or grey-box testing environments like machine learning.
This is a proposal for a low-level mechanism to support ablation experiments when measuring web site performance.