Clarification on mfxAISuperResolutionMode Support and Implementation in libvpl

[quamt]

Hello,I am seeking to better understand the mfxAISuperResolutionMode feature within Intel’s libvpl.

I would appreciate some clarification on the following points:

Driver and Architecture Support:

Which Intel GPU architectures and driver versions support the mfxAISuperResolutionMode feature?
Is there a resource or changelog available that details when this feature was introduced or enhanced across different platforms in libvpl?

Understanding Application Integration:

I understand that the libvpl dispatcher links the application with the appropriate runtime libraries. Could you explain the recommended process for initializing and checking the availability of mfxAISuperResolutionMode within an application?

Additionally, what considerations should be made if this feature is not supported on certain hardware or driver versions?

Are there suggested fallback mechanisms or best practices in such cases?

Thank you for your guidance.
Any documentation or resources that could further my understanding of this feature and its integration would be greatly appreciated.

[jonrecker]

Thank you for your interest in this feature. Some documentation and code snippets are available in the VPP programming guide and the API reference.

@FurongZhang could you please provide some additional info re. the above questions?

[FurongZhang]

1. VPL API: We added mfxAISuperResolutionMode in VPL v2.11 as experimental API, v2.13 move to production.
2. Driver version:
   a. The latest driver supports AI power super resolution (still experimental API) https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html. The officially released driver in 2024 Q2/Q3 drivers can support this feature.
   b. This feature is only supported on Windows, NO on Linux.
   c. This feature has been supported on TigerLake, Metorlake, LunarLake. A lot of platforms are compatible.
   d. As the programming guide shows, use query or init to check the capability of super resolution. If failed, recommend to fall back to the VPL hardware scaling path(no-AI).

[quamt]

@FurongZhang
Thank you for your explanation.

Does this also work on Intel Arc GPUs?

[FurongZhang]

Yes, I works on Intel Arc GPUs. I assume you are talking about DG2 series.

[quamt]

Yes, I works on Intel Arc GPUs. I assume you are talking about DG2 series.

Thank you for your reply.
Yes, correct the DG2 series.
Arc A750 for example.

I tested to check it with the 64-bit version of QSVENc from the git hub repository here:
https://github.com/rigaya/QSVEnc/releases 7.69

Tested with latest driver that I found:
GPU: Intel Arc A750 Graphics (448EU) 300-2400MHz (32.0.101.6078)

Here it shows as X for not supported
AI Super Res x

Supported VPP features:

Resize                o
AI Super Res          x
Deinterlace           o
Scaling Quality       o
Denoise               o
Denoise2              o
Mctf                  x
Rotate                o
Mirror                o
Detail Enhancement    o
Proc Amp.             o
Image Stabilization   x
Perceptual Pre Enc    o
Video Signal Info     o
FPS Conversion        o
FPS Conversion (Adv.) o
AI Frame Interp       o

Thats why I was curious on how the detection feature works.

[FurongZhang]

What is the input / output format you are using?

[quamt]

What is the input / output format you are using?

I didn't try to encode something yet, with that setting enabled in the parameters.
The setting listed to enable is:
--vpp-resize mfx-ai-superres

I usually encode with hevc 10-bit or avc as input and also as output.

[FurongZhang]

I did not mean encoding. I mean VPP input / output format.

[quamt]

I did not mean encoding. I mean VPP input / output format.

Can you please elaborate?

[FurongZhang]

The input and output of VPP for AI super resolution is NV12. Other formats might not be supported.

[quamt]

The input and output of VPP for AI super resolution is NV12. Other formats might not be supported.

Thank you for the clarification.
I need to investigate and get back on this.

[quamt]

@FurongZhang

It uses NV12.

QSVEnc 7.70 updates to latest libvpl to support API 2.13.

6078 driver support is up to API 2.12?
Could there be a driver problem?

Updated driver to 32.0.101.6078

QSVEncC (x64) 7.70 (r3442) by rigaya, Sep 24 2024 12:52:40 (VC 1941/Win)
Intel Media SDK API v2.13

QSV available.
Media SDK Version: Hardware API v2.12

[wizziwig]

@FurongZhang ,

What is the maximum output resolution for AI Super Resolution? I get "GPU_HANG" error when destination resolution is 3840x2160. I get "UNSUPPORTED" error when output resolution is 2560x1440. No errors when scaling from 1280x720 src to 1920x1080 destination. Anything larger than 1920x1080 destination fails.

Using Intel ARC A770 with 8GB VRAM on driver 32.0.101.6297. Windows 11. 64 GB System RAM.

Using the master version of sample_vpp.exe:

sample_vpp.exe -sw 1920 -sh 1080 -scc nv12 -dw 3840 -dh 2160 -dcc nv12 -sr -i 1080p.yuv -o 2160p.yuv

CONFIGURE LOADER: required implementation: hw
CONFIGURE LOADER: required implementation mfxAccelerationMode: MFX_ACCEL_MODE_VIA_D3D11
CONFIGURE LOADER: Use dispatcher's low-latency mode
Loaded Library configuration:
    Version: 2.13
    ImplName: mfx-gen
    Adapter number : 0
    Adapter type: discrete
Loaded module path: C:\Windows\System32\DriverStore\FileRepository\iigd_dch_d.inf_amd64_77faeffef4036c57\libmfx64-gen.dll

Input format    NV12
Resolution      1920x1088
Crop X,Y,W,H    0,0,1920,1080
Frame rate      30.00
PicStruct       progressive
Output format   NV12
Resolution      3840x2160
Crop X,Y,W,H    0,0,3840,2160
Frame rate      30.00
PicStruct       progressive

Video Enhancement Algorithms
Deinterlace     OFF
Signal info     OFF
Scaling         OFF
CromaSiting             OFF
Denoise         OFF
MCTF            OFF
ProcAmp         OFF
DetailEnh       OFF
GamutMapping    OFF
ColorSaturation OFF
ContrastEnh     OFF
SkinToneEnh     OFF
MVC mode        OFF
ImgStab         OFF

IOpattern type                  sys_to_sys
Number of asynchronious tasks   1
Time stamps checking            OFF
ROI checking                    OFF

MediaSDK impl   hw | VIA_D3D11
HW accelaration is enabled
MediaSDK ver    2.13
VPP started

[ERROR], sts=MFX_ERR_GPU_HANG(-21), OutputProcessFrame, MFX_ERR_NONE!=sts error at sample_vpp.cpp:199

[ERROR], sts=MFX_ERR_GPU_HANG(-21), sample_vpp_main, OutputProcessFrame failed at sample_vpp.cpp:899```

sample_vpp.exe -sw 1920 -sh 1080 -scc nv12 -dw 2560 -dh 1440 -dcc nv12 -sr -i 1080p.yuv -o 1440p.yuv

CONFIGURE LOADER: required implementation: hw
CONFIGURE LOADER: required implementation mfxAccelerationMode: MFX_ACCEL_MODE_VIA_D3D11
CONFIGURE LOADER: Use dispatcher's low-latency mode

[ERROR], sts=MFX_ERR_UNSUPPORTED(-3), InitMemoryAllocator, pProcessor->pmfxVPP->Query failed at sample_vpp_utils.cpp:796

[ERROR], sts=MFX_ERR_UNSUPPORTED(-3), InitResources, InitMemoryAllocator failed at sample_vpp_utils.cpp:849

[ERROR], sts=MFX_ERR_UNSUPPORTED(-3), sample_vpp_main, InitResources failed at sample_vpp.cpp:552