Adopting contributor guidelines around testing and bugs in the WHATWG was a successful experiment, and something similar has now also been adopted by some W3C groups/specs:
My aspiration for Chromium/Blink is that we will write web-platform-tests wherever possible from the very beginning of feature development, as tests are being written anyway, and might as well be shared. A lot of new work happens in the WICG, and it would probably help to have a shared understanding and expectations.
Some open questions:
- When specs are ahead of implementations, does it still make sense to require tests, when they cannot be checked against any implementation?
- If not, how does one track what parts of a spec are still untested?
- When implementation is prototyped ahead of a spec, can we still write tests?
- If so, how do we track which parts of test are not yet supported by any spec language?
My thinking is that it’s best to start thinking about these things as soon as implementation begins, treating the test suite as a companion to the spec. But, it’s not clear if starting at day 1 is actually productive, thoughts?