[Proposal] HTMLVideoElement Decode and Display Frame Resolution

Problem

Chromium and Chrome implementation of video decoder does not display the correct value for the encoded frame width and height when the video is produced by W3C specification MediaRecorder ad the video codec is set to VP8 or VP9.

The result is various incorrect values being output for videoWidth , videoHeight , ImageBitmap.width , ImageBitmap.height , MediaStreamTrack.getSettings().width and MediaStreamTrack.getSettings().height and the tab crashing when HTMLMediaElement.captureStream() is called on a <video> element where the encoded frames have variable resolution through the track (tab crashes at first resize of the <video> element).

In brief the bug is described at https://bugs.chromium.org/p/chromium/issues/detail?id=1004018.

Solution

Add an attribute and property to HTML <video> decodeAndDisplayFrameResolution element which mandates the VP8 and VP9 video decoders decode each individual encoded frame and display the potentially variable resolution (width and height) frames at HTML <video> element during playback instead of only reading the initial pixel dimensions once from the metadata of the WebM file.

Adding the attribute and property will avoid ambiguity as to what width and height is expected to be displayed at the HTML <video> element meaning that when variable resolution frames are encoded there will be a means officially specified to decode and display the each encoded frame which could potentially be encoded with variable resolution within the media file instead of only decoding the initial frame or media file metadata resolution.

Related WPT issue https://github.com/web-platform-tests/wpt/issues/17821 where resize event is not dispatched at Chromium, Chrome for WebM file recorded at W3C specified MediaRecorder with VP8 or VP9 codec set.

a new attribute isn’t a great solution here; a better one is to tighten the spec (if needed) and fix Chrome’s implementation

Agree. Am banned AFAICT indefinitely from contributing to WHATWG. Therefore in lieu of that path filed this proposal following reading at an issue, where if recollect accurately, that WHATWG and W3C have settled whatever issues they had re HTML. Do not have a current count of the number of Chromium bugs have filed relevant to this issue, more than one exists and is observable across various subdirectories of “media” parent source code. This proposal is intended to illuminate the issue of lack of clarity at a given specification regarding the subject matter of encoding, decoding and displaying variable resolution video; and/or plausible assertion of ambiguity of a definition of the pertinent terms used re width and height in controlling primary source specification(s) relied upon for guidance for the meaning of the terms used in potentially dependent and/or concurrent specifications.