add HW assistant copy interface

[XinfengZhang]

Signed-off-by: XinfengZhang [email protected]

[fhvwy]

The mention of GPUs in the name doesn't seem relevant. Most systems (not Intel) implement video independently of any programmable GPU components, and indeed a GPU may well not be present at all. In those cases I would expect it this sort of thing to be implemented with some kind of separate DMA engine able to read/write to all relevant pieces of memory (this seems possible in the GPU case as well since such a DMA engine may be available there too).

The mention of acceleration isn't obviously right either. The intent is presumably that the copy should be offloaded completely to minimise CPU use, not that it should be faster than just doing it directly on the CPU?

[XinfengZhang]

@fhvwy hi Mark, I changed it to HW assistant copy. actually, most cases, it is faster than CPU side, for example, copy tile surface to linear memory. offload CPU usage is also an option.

[dmitryermilov]

Hi @XinfengZhang ,

I still have the following proposals :

1. within dGFX we should consider cross adapter (or at least cross vaDisplay) copy. So probably VACopyObject should include VADisplay for VASurfaceID/VABufferID.
2. We need an input rectangle and coordinates (a point) at dst surface where to put data.
3. We need to provide an example how to use this interface for video->system and system->video memory copy. Probably it even won't require interface change. E.g. system memory can be represented by vaSurfaceId with userptr.
4. We need to consider supporting cases when planes of surfaces in system memory are allocated by different malloc calls (ffmpeg, HB do it).

Without it, I'm afraid the interface won't be as powerful and useful as it can be. Do the proposals make sense to you?

[XinfengZhang]

I still have the following proposals :

1. within dGFX we should consider cross adapter (or at least cross vaDisplay) copy. So probably VACopyObject should include VADisplay for VASurfaceID/VABufferID.
2. We need an input rectangle and coordinates (a point) at dst surface where to put data.

it sound like composition? should we call VPP to handle it?

3. We need to provide an example how to use this interface for video->system and system->video memory copy. Probably it
   even won't require interface change. E.g. system memory can be represented by vaSurfaceId with userptr.

agree, we should add a sample app in https://github.com/intel/libva-utils

4. We need to consider supporting cases when planes of surfaces in system memory are allocated by different malloc calls (ffmpeg, HB do it).

there are a lot of restriction of systeam memory allocated by application through malloc. such as the start address should be aligned with page (4k)

Without it, I'm afraid the interface won't be as powerful and useful as it can be. Do the proposals make sense to you?

[dmitryermilov]

I still have the following proposals :

1. within dGFX we should consider cross adapter (or at least cross vaDisplay) copy. So probably VACopyObject should include VADisplay for VASurfaceID/VABufferID.
2. We need an input rectangle and coordinates (a point) at dst surface where to put data.

it sound like composition? should we call VPP to handle it?

Why VPP? Why composition? Please see how other similar interfaces are defined:
https://docs.microsoft.com/en-us/windows/win32/api/d3d11/nf-d3d11-id3d11devicecontext-copysubresourceregion

1. We need to consider supporting cases when planes of surfaces in system memory are allocated by different malloc calls (ffmpeg, HB do it).

there are a lot of restriction of systeam memory allocated by application through malloc. such as the start address should be aligned with page (4k)

And ? Do you want to say we won't be able all scenarios. It's okay. But we should handle the scenarios which we can handle. Plus anyway, this're discussing interfaces, not implementation. Interacts doesn't care about alignment,

[dvrogozh]

These ID types goes as:

typedef unsigned int VAGenericID;

which might have different size... Can we add some specific size to this union? Like:

    union
    {
        VASurfaceID surface_id;
        VABufferID  buffer_id;
        uint64_t reserved;
    } object;

[XinfengZhang]

to enlarge the size to 64bit? which compiler/platform will lead to different size of unsigned int?

[dvrogozh]

not sure, but I am paranoid:)

[XinfengZhang]

maybe we could consider about the "uint32_t"/"uint64_t" "unsigned long" for future, such as 3.0? , unsigned long will be changed adaptively with the compiler. uint64_t /uint32_t will be dedicate size.

[dvrogozh]

We've also planned to expose vaMapBuffer2 API w/ flags and maybe w/ copy options... See #377 for the early draft.

Can we, please, consider whether we can apply VACopyOption to the vaMapBuffer2 API as well? I worry about va_copy_sync usage... Thought it might still be useful in vase of vaUnmap when we might or might not to wait for the sync...

[dmitryermilov]

I worry about va_copy_sync usage... Thought it might still be useful in vase of vaUnmap when we might or might not to wait for the sync...

@dvrogozh , do you mean asynchronous writing to vaSurface?

[dvrogozh]

Possible usages:

vaMapBuffer2(VA_COPY_SYNC_NONBLOCK); // allocates (maybe) shadow copy, no
                              // copy from original surface due to VA_COPY_SYNC_NONBLOCK (we don't care about the contetn)
memcpy();
unmap(); // schedules copy operation form shadow copy to final surface, we don't wait due to VA_COPY_SYNC_NONBLOCK

[dmitryermilov]

Yes, it's async writing to vaSurface/vaBuffer. Above, you say "usages". To tell you the truth I don't see other examples.
But anyway, IMO, it indeed can be useful.

[XinfengZhang]

it means map is block call , but unmap is un-block call , right? the interface will be a little weird. need to re-consider it. maybe also a unmap2?