va: add av1 profile2

[davidwuAMD]

Adding VAProfileAV1Profile2 to support 12bit AV1 decoding

[XinfengZhang]

is it enough to add a profile only? will there be a platform support this profile?

[davidwuAMD]

Hi Xinfeng,
I am working on enabling 12bit AV1 decoding in ffmpeg and Mesa - so far it is working with this patch on amdgpu vaapi use case - I hope once this patch is in I will enable/push those supports to upstream.
David

[davidwuAMD]

Hi Xinfeng,
any concern about letting this pull request merged? Let me know.
Thanks,
David

[XinfengZhang]

LGTM, once the patch in mesa and ffmpeg merged, please paste the link here, thanks.

[davidwuAMD]

https://gitlab.freedesktop.org/davidwu2/mesa/-/commit/8fa17b068a1fef987c53dff881b643bcf632e258
this is the link for 12bit AV1 support in mesa - I will need final touch once I know the new libva version - right now I am using 1.23 for testing but this needs to be 1.23+.
Wondering when there will a new version for libva?

[XinfengZhang]

release is 2.xx.x , such as https://github.com/intel/libva/releases/tag/2.22.0, it is latest release.
master is 2.xx+1.0.pre with newest patches, current , it is 2.23.0.pre, suppose this patch will be merged into master, and will be in release 2.23.0

[davidwuAMD]

thank you Xinfeng!

in meson.build (or configure.ac) we have
va_api_major_version = 1
va_api_minor_version = 23
va_api_micro_version = 0

We have a check in Mesa (and ffmpeg) for these info above. For 12bit AV1 support we will need a new version.
I expect it should be something like
va_api_major_version = 1
va_api_minor_version = 24
va_api_micro_version = 0
or
va_api_major_version = 1
va_api_minor_version = 23
va_api_micro_version = 1
in this case I can enable 12bit AV1 in Mesa and ffmpeg. Should I make a pull request to bump up these info?

Let me know if you have any question.
David

[XinfengZhang]

you just need 1.23.0 withouth "pre1"
you could use macro #if VA_CHECK_VERSION(1, 23, 0), the release version including your patch is 2.23.0, current master is 2.23.0.pre1, it is not a release, so it may cache some feature un-released.

[davidwuAMD]

I see - I did use the "#if VA_CHECK_VERSION(1, 23, 0)" in my mesa and ffmpeg - I thought 1.23.0 is has released - my bad
OK - sounds good -
thanks,

[fhvwy]

Based on the Mesa commits above this isn't actually what the decoder support needs? The specific case being implemented is for the subset of VAProfileAV1Profile2 which is 12-bit 4:2:0 only. A decoder which supports that only can't advertise profile 2 support because it doesn't cover all cases as required by the standard in section A.2.

I think this needs something else to advertise a subset of a profile. Maybe VAProfileAV1Profile2_420Only as a profile, or some other query could be added to check for additional support beyond the profiles which are fully supported (so advertise VAProfileAV1Profile0 since that profile is fully supported, but then have some extra attribute to say it works with 12-bit 4:2:0 as well).

[XinfengZhang]

how about the VASurfaceAttrib, if it is Profile2, and the RT format is P010, it may represent 12bit 420 only. if it also include Y210 or P208, it represent 422?

[davidwuAMD]

I realized the limitation for AV1 Profile2 - I will submit another patch to call something like VAProfileAV1Profile2_420Only or VAProfileAV1Profile2_420. thanks for pointing that out.

[davidwuAMD]

another thought - can we keep VAProfileAV1Profile2 - as is? then in Mesa we check for the bitstream and allow for 12bit 420 only. I think ffmpeg might need more changes to support it

[davidwuAMD]

I made another pull request - #832
I will investigate in ffmpeg and Mesa to make sure only 12bit 420 is allowed there - once done I will post the MR.

[davidwuAMD]

Hi @XinfengZhang,
Would you be able to help get this merged?
thanks,
David

[XinfengZhang]

hi DavidwuAMD, looks driver could report the capability by vaSurfaceAttrib, like my previous comments #828 (comment)

[davidwuAMD]

oh - the driver only reports P010 and did not report Y210 or P208 etc. As the Hardware only supports 12bit 420. So I think #832 is needed to make it more accurate - so please help with #832

[davidwuAMD]

oh - the driver only reports P010 and did not report Y210 or P208 etc. As the Hardware only supports 12bit 420. So I think #832 is needed to make it more accurate - so please help with #832

finally, I come back and continue on this topic,
If I use VAProfileAV1Profile2 as it is in libva and test vainfo, mesa and ffmpeg.
1 vainfo --all
show the support under VAProfileAV1Profile2 - it lists what it supports. It did not advertise YUV444, etc. so this is not about mis-advertising. The fact is it shows only 420 if you really want to figure out.

VAProfileAV1Profile2/VAEntrypointVLD
   VAConfigAttribRTFormat                 : VA_RT_FORMAT_YUV420
                                             VA_RT_FORMAT_YUV420_10
                                             VA_RT_FORMAT_YUV420_12
                                            VA_RT_FORMAT_YUV420_10BPP
2 ffmpeg exits with error using 12bit YUV444 format without using vaapi decoding.
3 mpv shows hardware accelerated AV1 is not supported for YUV444 video clip.

./mpv/build/mpv -hwdec=vaapi /home/david/Samples/AV1_12bit/AV1_444/444av1.mp4
\u25cf Video  --vid=1  --vlang=eng  (av1 1920x1080 25 fps) [default]
[ffmpeg/video] av1: Your platform doesn't support hardware accelerated AV1 decoding.
[ffmpeg/video] av1: Failed to get pixel format.
[ffmpeg/video] av1: Get current frame error
Error while decoding frame (hardware decoding)!
VO: [gpu] 1920x1080 yuv444p12
V: 00:00:03 / 00:00:04 (99%)
Exiting... (End of file)

My conclusion is we can use VAProfileAV1Profile2 as-is in libva. If agrees, I can drop #832

[davidwuAMD]

there is a solution in ffmpeg that will check RT format - when vaapi reports only 12bit YUV420 is supported and a YUV444 input will not use vaapi to decode a video, which is expected. This means a combination of profile + TR format works. To me any definition as Profile_abc_only is not needed - which does tell the true support but does not solve the underneath issue such as when Av1Profile2_420 and Av1Profile2_444 both enabled, what ffmpeg can do to cover a case where in mesa only Av1Profile2_420 is supported for a GPU. I.e application knows what RT formats are supported ( vainfo as an example) by querying Mesa, if it does not use it correctly then it is its own issue.