[Windows] HEVC Encodes Very Slow Unity Playback Under Direct3D12

[4dbob]

Unity version

2022.3.52f1 LTS

Unity editor platform

Windows

AVPro Video edition

Ultra

AVPro Video version

3.1.3

Device hardware

NVIDIA GeForce RTX 3070 Ti
8 GB VRam

Which Windows version are you using?

11

Graphics API

D3D 12

Video API

Media Foundation

Audio output

None_

Any other Media Player component configuration required to reproduce the issue.

Hardware Decoding ON
Renderer Sync ON

Which output component(s) are you using?

Apply to Mesh

Any other component configuration required to reproduce the issue.

No response

The issue

This issue: 30fps HEVC Main10-encoded movies using AVPro v3.1.3 in Unity 2022.3.52 LTS (HDRP) play back at only 4-5fps in both the Editor and in a Build when selecting Direct3D12. Switching to Direct3D11 restores predominantly smooth playback, but my project (an existing installation I wish to update) requires Direct3D12.

I'm emailing a link to an example Unity project reproducing the issue. Note - this playback problem is inherent regardless of encode bitrate (the project includes four different encodes of the same movie to step through using the Spacebar).

How to reproduce: Open the 'AVPro HEVC Test' project, which is set to use Direct3D11. Hit play and observe the predominantly smooth 30fps playback. Switch to Direct3D12 under Project Settings | Player | Other Settings | Graphics APIs for Windows. The Editor will shut down and restart. Hit Play. The movies on my test computer now only play back at what looks to be ~4-5 frames per second (even though MediaPlayer is reporting 30-ish).

Expected outcome: With the addition of the Media Foundation 'Renderer Sync' option, I was hoping to greatly reduce if not eliminate video stutter for silky-smooth playback.

Media information

The videos are a specific non-standard resolution (yet divisible by 16 in both dimensions) to match an LED wall of that exact resolution.

They use the HEVC codec using the MAIN10 Profile and Rec. 2020 Color Space, and for testing purposes are encoded at 100, 75, 50 and 25 Mbps Constant Bitrate. The existing installation uses 100Mbps encodes.

Log output

No response

[RichRH]

Hi @4dbob,

Thanks for the detailed report.

I have managed to reproduce this (using latest v3 code) and it's quite odd, maybe some interop issue between D3D11 and D3D12 as it's actually (in theory) still using D3D11 to decode the video for D3D12 (Unity) rendering.

I'll try and dig deeper but a few notes:

• 'Renderer Sync' shouldn't have any impact with D3D12 output currently (but does make D3D11 output far smoother - we use it for our own installations)
• 'Support Buffered Display' is actually being removed as it was basically trying to solve the issue 'Renderer Sync' now handles more succinctly
• Performance for me was much improved in D3D12 by just disabling 'Hardware Decoding' if that works for you and could act as a workaround?

Cheers,

[4dbob]

Hi Rich, thanks for continuing to dig for an answer. I'm happy to know you're seeing it on your side as well. Thanks for the workaround suggestion - we tried turning off hardware decoding on site, but it stuttered far worse. We've got a lot of CPU horsepower, but we're simultaneously pushing four videos totalling 13.5 x 1.5 K of video to this display, so the graphics card is superior.

As I mentioned in my email, we'd also be willing to try NotchLC for some particularly troublesome videos but that isn't working under D3D12 either. The Console window shows:
      'd3d12 error: Creating a default shader resource view with non compatible format (dxgi-fmt=0 for a resource that uses dxgi-fmt=24)'
followed by an unending stream of this error:
      'd3d12: Attempting to bind a NULL or empty texture. Perhaps the texture contains no image data or has an invalid size.'

The installation combines video with realtime graphics which benefitted by increased performance under D3D12, so we cannot switch back to D3D11.