Buffer Pool API

[jeremyong-az]

Buffer Pools and You

TL;DR

• Suggest changing buffer pools to suballocate resources from memory pages, as opposed to suballocating ranges from pages allocated as resources
• Suggest relaxing the API to optionally move buffer usage requirements to the buffer init request, as opposed to the buffer pool descriptor.

The current buffer pool abstraction works as follows. To allocate buffers in Atom, we generally first make a buffer pool:

// A buffer pool is advertised with a descriptor
RHI::BufferPoolDescriptor desc;
// Which advertises several properties including:

// Memory flags (host local or device local)
desc.m_heapMemoryLevel = RHI::HeapMemoryLevel::Host;

// Some bind flags
desc.m_bindFlags = RHI::BufferBindFlags::ShaderRead | RHI::BufferBindFlags::CopyRead;

// Now we can create and initialize it
RHI::Ptr<RHI::BufferPool> pool = RHI::Factor::Get().CreateBufferPool();
pool->Init(*device, desc);

Afterwards, we can suballocate buffers from this pool using BufferPool::InitBuffer and map memory for readback or upload using BufferPool::MapBuffer. The buffer returned from an InitBuffer creation request with a user-managed lifetime. Buffers and buffer pools are based on the base Resource and ResourcePool types, to handle the various lifecycle events.

Internally, when a BufferPool is initialized, it stores the description and uses the memory and buffer bind flags to allocate pages of internally managed memory. Requests to create buffers from a pool suballocate from these pages, each of which corresponds to a dedicated resource allocation.

To manage transitions, entire pages are transitioned at once by tracking the resource state and marking a transition as needed when various operations occur (e.g. mapping memory, issuing GPU commands). This can be seen in the various platform-specific RHI implementations of the FrameGraphCompiler when the buffer is bound as a BufferFrameAttachment and in the BufferPool itself for the mapped memory case.

Abstraction Limitations

There are a few things to consider with the current abstraction:

• Barrier and resource state transition granularity
• Slack memory waste and alignment restrictions

The abstraction as provided is convenient for the case where we know read and write access patterns across an entire pool are uniform. That is, when we are OK transitioning resource states and issuing UAV barriers at page granularity. However, often we may want to transition individual buffers, for example, using a buffer as a UAV early in the frame to write indices after a compute culling operation, and later use the same memory as an index buffer. This is particularly relevant for transient buffer management, for which we need some memory on a given frame but don't really care about retaining it for use later (except to save the allocation). An example might be on one frame, we use some scratch memory to build various BLAS structures, and on a different frame the same memory is repurposed for some terrain work.

The second issue is that we need a new pool for every usage type conceivable, which is a union of the various buffer usage state bits. While we can collapse this some by creating pools with more permissive usage bits, this exacerbates the resource state granularity issue mentioned above. In practice, we would need a separate pool for vertex streams, UAVs, scratch memory for AS builds, acceleration structures, etc., even though in reality much of this memory could live on the same pages.

Suggested Changes

The buffer pool can get a new usage option in its descriptor: BufferBindFlags::Unknown, which indicates that the bind flags must be specified when InitBuffer is called. For buffer pools that are created with unknown usage, memory is allocated per page, but as VkMemory or ID3D12Heap allocations, not as VkBuffer or ID3D12Resource objects. Subsequent suballocations are performed from those memory allocations as actual resources via API calls such as ID3D12Device::CreatePlacedResource. As before, when suballocations are made, care must be taken to ensure the alignment restrictions of the requested buffer are honored (256B constant buffers, shader tables, scratch buffers, etc.).

Within the BufferPool and the FrameGraphCompiler itself, the resource to be transitioned in this case is the buffer resource itself, which is now no longer shared with other buffers on a page for the BufferBindFlags::Unknown pool type.

Future

With the BufferBindFlags::Unknown pool, we can more flexibly manage transient per-frame buffer data from a single pool, providing potentially considerable memory savings.

[moudgils]

Do you think we would still want to keep the current way of using committed resources for pages around? Or should we just remove that all together. Maybe we can keep it around for some time and phase it out over time if left unused.

[jeremyong-az]

Yea I think the idea with my proposed API above is that if you specify BufferBindFlags that aren't Unknown with the current usage, the pool would function as before.

[jeremyong-az]

As for whether we want to keep that functionality in the far future, I'm not too convinced it's necessarily going to be needed, except for backwards compatibility. Someone could argue that we don't want to pay for barriers for each resource in a memory page when we could transition the entire page but this just translates to a global cache flush anyways, so there's no impact on the GPU timeline.

[moudgils]

This will also make gpu captures less confusing as the buffer would be named appropriately and not some arbitrary BufferPageXX :)

[jeremyong-az]

Yea resource naming and also the buffer format views in tools that let you specify how a resource is laid out (Pix, RenderDoc, etc.)