[Proposal] EXIF / image-resolution: (auto?) and from-image

Yes, exactly - since Inch stands for DPI, it would make sense that Pixel would stand for dppx. I believe there is no current use for Pixel as an EXIF Resolution Unit.

Yes. I tend to like the second one because it doesn’t have float rounding issues; However the first one might introduce fewer backward compatibility issues (maybe non at all). A study of current existence of EXIF in web-exposed image is probably a good next step :slight_smile:

Continuing my research, I’m thinking that adding an XMP attribute might work better than EXIF… EXIF is less extensible and is more photography-oriented, also the mentioned EXIF attributes are used by image viewers.

Some more info and more detailed alternatives after some research:

Potential metadata formats

  • EXIF: used mainly for photography data. Very concise and in wide use. Browsers today recognize it for image-orientation.
  • XMP: very extensible, used for things like photo-tagging. Verbose.
  • IPTC: used mainly for rights management, not very relevant.

Current resolution metadata identifiers:

See definitions

  • ImageWidth/ImageHeight: usually refer to the original size of the image when it was taken, before it was scaled by any program. Many photos on the web have different ImageWidth/ImageHeight values from their actual pixel dimensions.
  • XResolution/YResolution/ResolutionUnit: used almost exclusively for DPI, as a printing directive,. The default is 72dpi. The different resolution units are defined in the EXIF standard.
  • PixelXDimension/PixelYDimension: intended for a different use, padding of compressed image data.

Proposed alternatives

  1. Use ImageWidth / ImageHeight EXIF to override intrinsic dimensions

    These attributes are quite common in the wild, and are often different from the image pixel dimensions. That’s because they don’t change when a program changes the image. See explanation. Using them to compute intrinsic dimension as an override of the pixel data dimensions would make a lot of sense, but is also likely to modify the presentation of a small amount of existing images on the web.
  2. Use ImageResolution EXIF with Pixel as ResolutionUnit

    This maybe makes the most sense spec-wise, and would not break any existing web site, but would require a revision to the Exif standard, as the different values for ResolutionUnit are defined in that spec.
  3. Use ImageResolution EXIF with Inch as ResolutionUnit, and treat 72dpi as 1dppx

    This would not require any spec change, as Inch is already a defined ImageResolution value. However, it would change intrinsic size for some images currently on the web, where the dpi is defined with a value other than 72. That is not uncommon in the web today.
  4. Use ImageResolution or ImageWidth/Height EXIF, with CSS image-resolution

    Either (1) or (3) can work without “breaking the web”, by allowing it as a non-default value for image-resolution CSS attribute.
  5. Add a new EXIF attribute

    Maybe PreferredPixelDensity or DisplayWidth/DisplayHeight. Will require changing the EXIF spec, but would not introduce any backwards compatibility issues.
  6. Define an XMP property for this

    This can be done without too much standardization overhead, as it can be done in an XMP namespace. However, it would require user agents to incur the overhead of parsing inline XMP in image data, which they currently ignore.
1 Like

I see no reason to restrict this to still image formats. It appears to apply to video as well. Hence other container and tagging formats would need to be considered.

1 Like

XMP is supported in video files, and it can be used to embed EXIF properties. So I believe that what we come up with here can be translated to video without a big effort.

Some details from the research:

  • Created a crawler that goes to many websites (starting the crawl from here, downloads the images, and parses their EXIF. The source code for it is here.
  • Ran the crawler on my computer, capped at 3000. The results are here.
  • The analysis:
    • Out of 3000 images sampled, 189 have any EXIF info (6.3%).
    • Out of those 189 images, 55 (29%) are displayed with their natural width & height.
    • Out of those 55 images, 10 (18%) have a resolution other than 72dpi, and 16 (29%) have EXIF width/height different from their pixel width/height.

So the effect on current images in the wild of using existing EXIF properties, (considering that the sample is representing of the wild) would be on less than 0.3% of images in the case of resolution, and on less than 0.5% in the case of ImageWidth/ImageHeight.

1 Like

Well, 0.3% >> 0.01% :thinking: https://github.com/whatwg/html/issues/4495#issuecomment-484084805

A more rigorous check, of 5000 images from 1190 site URLs, also testing whether an image explicitly sets width/height and manual examination of automatically-found affected sites, found around 5 images out of 5000 to be effected. That’s ~0.1% of images, but ~0.4% of sites. Note that “affected” in all of these case would mean that the image would shrink by a ratio 1.2-4.

Examples found by the crawler and verified manually: https://www.e-unwto.org/doi/book/10.18111/9789284421084 (the picture with the houses) https://www.accountingfoundation.org/jsp/Foundation/Page/FAFSectionPage&cid=1176164598953 (the picture of the meeting) http://bellschool.anu.edu.au/regions-countries/australia-new-zealand (the picture of the magnifying glass).

Raw data: https://gist.github.com/noamr/ec771e4f5d0fbb5abe8a68297adaa41d

Thanks @noamr! 0.4% of sites seems pretty high to change image density by default. 1190 sites is a small data set, though, so the percentage is a bit uncertain.

So if I understand your analysis correctly, the 5 images found are using EXIF ImageWidth / ImageHeight / XResolution / YResolution, and those would change the rendered dimensions of the image if they were honored by default. Correct?

Yea, it’s a bit high. It’s a pretty random dataset built from the list of most used sites. Creating a bigger sample set (or a better crawler :slight_smile:) would take some effort.

The 0.4% is for sites with images that have XResolution / YResolution and rely on intrinsic size for layout. The percentage for ImageWidth & ImageHeight is higher - a lot of photos mark those properties when taken, and then don’t modify them when e.g. the image is cropped or scaled. It’s like “original image dimensions”

Have you considered adding a custom metric to HTTP Archive? That should give you data for ~5 million pages, next time it runs a crawl (twice a month). See e.g. https://github.com/HTTPArchive/legacy.httparchive.org/commit/9939498e07a1190720cb082d44e1a7090c3d931c and https://almanac.httparchive.org/en/2019/media

nice! but the aforementioned github repo is said to be deprecated :confused:

Yeah… I’m not sure what the deal is, but there’s a new repo here https://github.com/HTTPArchive/httparchive.org

Maybe file an issue there and ask how to add a new custom metric?

Great! Started a discussion topic: https://discuss.httparchive.org/t/usage-of-exif-in-images/1898 And in the meantime using the UX-report URL list with my crawler :slight_smile:

From the discussion there, it sounds like it might be easier to create a Chromium use-counter which reports what you want. Then we’d be able to collect data from both chromestatus.com as well as the HTTPArchive.

Feel free to ping me offline if you need help with that

I’ve come up with an alternate solution, that (hopefully) fulfills all the following requirements:

  1. No new EXIF tags, as CIPA standartization is quite stale nowadays.
  2. No XMP dependency, as XMP is very verbose.
  3. Less than ~100Bytes overhead
  4. Minimal to no breakage of existing web pages.

The solution is to use the dimension (0xa002 and 0xa003) and resolution (0x011a, 0x011b and 0x0128) EXIF tag as follows:

  1. Read XResolution, YResolution, ResolutionUnit, PixelXDimension and PixelYDimension for the image
  2. If XResolution given the image’s actual width would result in the value in PixelXDimension, let the intrinsic width of the image be PixelXDimension. Do the equivalent for the y axis.
  3. If only one axis is defined in this way, scale the other axis maintaining the aspect ratio
  4. If one axis checks out and the other is not, ignore dimension metadata for this image.

For example, an image with 120px actual pixel width, and 240 value in its PixelXDimension exif, would be treated as a 240px low-resolution image if the image’s XResolution equals 36dpi.

Checking this with 16000 images on the web based on Chrome UX report, has found zero affected web pages.

This looks very sensible, and avoids the common case of images “scaled for the Web” without themetadata being changed.

It might be necessary also to consider the exif rotation tag in this, and to determine whether height/width attributes or CSS height/width are expecting this to have been applied, and to swap height/width fields if the rotation is through 90 or 270 degrees (only multiples of 90 degrees are supported as i recall). But maybe that’s more complexity than is needed.

1 Like

Thanks! re orientation - it’s a good point, though I’m suggesting to have the dimensions correspond to the pre-rotated image dimension. Usually scaling comes before rotation (Scale, Rotate, Translate), so the equivalent here would be to scale according to resolution and only then rotate according to orientation.

I like this proposal quite a bit.

  • It seems to eliminate any and all web compatibility concerns.
  • The belt-and-suspenders approach ensures that the encoder that set this metadata knew what it was doing, and if image content “drifts” away from the metadata, the metadata is ignored.
  • It allows for pixel-perfect precision (unlike the original Content-DPR proposal).
  • The overhead is relatively tiny (very-small images like image placeholders are an important use case).
  • It doesn’t invent anything new, or do anything that goes against either the letter or spirit of the EXIF spec.

The only downside I can think of is:

  • It’s a little secret-decoder-ring-ish. You have to set two dials just so, in a way that is not intuitive from looking at the EXIF alone.

One tweak – I think it might make sense to require all four of PixelXDimension, PixelYDimension, ResolutionX, and ResolutionY (and ResolutionUnit).

Let’s say we have an 800×800 image with XResolution=144dpi and PixelXDimension=400. So, density-corrected intrinsic size = 400x400. So far, so good.

Then, let’s say someone crops it to 800×600, and does not update the metadata. Density-corrected intrinsic size = 400×300, I’m okay with this.

But, let’s say they instead cropped it to 600×800, without updating the metadata. Now the PixelXDimension and XResolution are out-of-sync with the actual image dimensions, so the metadata is ignored and the density-corrected intrinsic size becomes 600×800. The differing results when cropping horizontally or vertically seem weird.

I think this all would be somewhat less befuddling if any image editor that changed the dimensions of the image, but did not change the metadata in concert, triggered the metadata to be ignored. The cost is a few dozen extra bytes in every resource … but perhaps worth it.

1 Like