Black bars on Sony HIF thumbnails

[teletypo]

Here's a HIF-image of a Sony ILX-LR1: 20241120-214155-cam0-000-000004-0.zip. It's captured at full 61MP with a 3:2 aspect ratio (the only way to get the full 61MP). The HIF pixel size is 9504x6336.

heif_info shows that there's 3 thumbnails embedded:

MIME type: image/heic
main brand: heix
compatible brands: mif1, heix, miaf, MiHA, jpeg, SHIF

image: 9504x6336 (id=10), primary
  colorspace: YCbCr, 4:2:2
  bit depth: 10
  thumbnail: 1616x1080
  thumbnail: 320x212
  thumbnail: 160x120
  color profile: nclx
  alpha channel: no 
  depth channel: no
metadata:
  Exif: 94208 bytes
  XMP: 57344 bytes
transformations:
  angle (ccw): 0
region annotations:
  none
properties:

• 6336/9504 aspectratio 0.6666, primary HIF
• 1080/1616 aspectratio 0.6683, thumbnail 1, HIF encoded
• 212/320 aspectratio 0.6625, thumbnail 2, HIF encoded
• 120/160 aspectratio 0.75, thumbnail 3, JPEG encoded (pretty sure because I had to rebuild libheif with JPEG decoder function to extract this)

Now I don't know why Sony decided to create thumbnails in different aspect ratios, but it seems they also decided to just add black bar(s!), probably to make up for the mismatch:

heif-thumbnailer 20241120-214155-cam0-000-000004-0.HIF 20241120-214155-cam0-000-000004-0-heif-thumbnailer.jpg

I see the same in dolphin, KDE's file manager:

But when I invest 99 cents to install the HEVC Video Extensions in the windows app store on a windows 10 machine, I get proper thumbnails in windows:

Now, it could be that windows is bypassing the problem by just using thumbnail 1 instead. But the thumbnail shown above in windows explorer are about 245x162 pixels, so windows might have gotten those from thumbnail 2.

So...

• is Sony writing black pixels to thumbnails 2 and 3, or is this a HEIF decoding artifact?
• what's the best way to exclude black pixels (hopefully without having to detect-and-crop)?

Thanks!

[bradh]

Note for further investigation: maybe we need clap for the thumbnail.

[farindk]

I analyzed the file. It contains three thumbnails:

• ispe: 1616x1080 (HEVC), id 11, stored size: 1664x1088
• ispe: 320x212 (HEVC), id 12, stored size: 320x256
• ispe: 160x120 (JPEG), id 13, stored size: 160x120

None of the images has a clap.

Thus, the HEIF file is wrong. The ispe does not match the encoded image size. Probably, the Windows decoder crops the image to the ispe size, but it should not do so without a clap.

Image 11:

Image 12:

Image 13:

[L4cache]

I have a bad news for you, HEVC Extension is free (preinstalled) in Windows 11...

[teletypo]

Thanks @L4cache, I will ask Microsoft for a refund ;)

@bradh and @farindk, I have asked Sony to take a look at this and hope that we can soon report back with something. Thank you for your impressively fast responses!

[farindk]

If it is an error that affects many images, and I guess that the camera firmware is not easy to upgrade, we might consider to add an option to the thumbnailer to crop the images to the ispe size to offer a workaround to faulty input images.