Native HDR Output

[LordKobra]

General Issue

Currently the game output is tailored towards 8-bit sRGB format. It would be nice to support higher bit-depth output devices and wider color spaces.

To-Do

(under construction / subject to change)

• support 10-bit buffer
• adapt tonemapping to different color-spaces

[Try]

Since change most-likely will affect the engine, can you please provide api proposal on how swapchain should be created now?

Afaik in Metal it's as simple as setting metalLayer.wantsExtendedDynamicRangeContent = YES and provide some HDR format, when creating swapchain images.

In Vulkan, there is VK_EXT_hdr_metadata, but it seem to require more data, such as primary colors and white-point.

[Try]

Also renamed this issue to "Native HDR", for clarity sake

[LordKobra]

Just a collection of knowledge so far:

• HDR output seems to be 99% Rec. 2020/Bt.2020 with PQ curve https://www.color.org/chardata/rgb/BT2020.xalter
• UI elements (and for such an old game perhaps the assets too) are usually done in sRGB/BT.709 https://www.color.org/chardata/rgb/BT709.xalter
• to render sRGB UI elements we can use scRGB, because it uses the same color primaries as sRGB https://www.color.org/chardata/rgb/scrgb.xalter
• BT.2020 is 10/12bpc, scRGB is 16bpc, both uniform distributed, not like typical float (Note: Just because they have this bit depth, it doesnt mean we can't use more bit-depth before outputting to the swapchain, this way dithering can be delayed etc.)

So the plan is game internal -> 16bit scRGB & apply UI (& dither)-> 10bit BT.2020 for output

This makes it much easier, since i feared there would be many competing HDR outputs & devices to support. But that's not the case. However a 16bpc texture would be needed to support scRGB.

This also leaves a black box for the "game internal" color space, which i still need to figure out properly.

Edit 1: color.org specs maybe outdated, need to find current ones when it comes to format implementation

[Kaldaien]

So the plan is game internal -> 16bit scRGB & apply UI (& dither)-> 10bit BT.2020 for output

It's questionable whether you really want to tack that last bit onto things. Transforming from scRGB to the display's wire-format (ST.2084 + BT.2020, at 10-bpc) is something the driver can do for you with zero assistance.

If you leave the output in FP16 scRGB, then the driver can leverage its own dithering and third-party overlays and post-processing done to your game can be performed in a more friendly color space. I strongly believe that encoding for HDR10 manually when you're already in scRGB is an anti-pattern. scRGB's a good universal HDR container and you can leave quantization to HDR10 to the OS to handle and that's a perfectly valid solution.

HDR10 as SwapChain output only really sees practical application with DLSS3 Frame Generation. NVIDIA refuses to accept an scRGB input for motion flow because 16-bpc FP is too taxing on the GPU's hardware to get frame generation performance benefits.

[LordKobra]

Some additional infos regarding platforms:
Mac: probably has the smoothest HDR experience, because everything from monitor to graphics API is in-house. You said, you can take care of this, @Try.

Windows: Vulkan HDR only works when the swapchain is exchanged for a dxgi swapchain. See here: dxgi-swapchain-on-opengl-and-vulkan.
To quote @Kaldaien :

The only means of fixing the problem is either driver-specific APIs (which are deprecated) or layering Vulkan on DXGI.
Layering on DXGI's the proper solution for a handful of reasons. It's the fast-path to display output that gets you access to modern GPU features like Multiplane Overlays, VRR, HDR, GPU timeline semaphores, etc.

and regarding how to do it the modern way:

[...] Windows has gone even further though, they're a few steps ahead of Linux in the availability of Multiplane Overlays.
On RTX 2xxx+ and newer GPUs, you can mix-and-match traditional Desktop Window Managed composition for part of your screen and simultaneously have a game that's covering part of the screen render directly to a dedicated hardware scan-out plane.

In other words, the DWM can be active at the same time as a game is rendering directly to the screen w/ hardware overlays.

For DX12/DXGI we can find some docs here: HDR in DX12

Linux is a little tricky, with no official HDR support but working support in gamescope. Direct Render Manager needs to be used. I would say for now, Linux HDR implementation is an optional feature, we better focus on Windows and Mac.

[Try]

See here: dxgi-swapchain-on-opengl-and-vulkan.

This doesn't seem to be relevant to application: DXGI-swapchain is something what driver implementation detail, AFAIK vulkan application has no say about it.

Linux HDR implementation

I'm fine with keeping it optional - we likely have to expose 'caps' anyway in device-properties and have 2 paths in game-rendering. However Vulkan extension seem to generic - maybe it will "just work" on linux?