Allow indirect rendering, allow multimesh without transform for each instance

[LiorArbel]

Hey, I'm writing a game that needs to spawn a lot of vegetation, and I would really want it to be as efficient as possible to work on mobile.
Currently, multimesh holds a Transform3D\2D for each instance it has, and this is mandatory by the engine (if you set it's buffer to a size smaller than instance_count*sizeofTransform3D\2D it throws)
That takes a lot of memory and reduces performance when one wants to get to very high number of instances, of very simple and predictable geometry

I recently came across this video where he demonstrates optimizations for rendering a whole lot of triangles
One of them is sending a lot less data in the buffer by not sending unneeded things like UVs, vertex positions and more, because they are all calculate-able on the GPU from a list of much less data.
Or if I understood correctly - he is drawing the mesh indirectly on the GPU (I'm not proficient in graphics programming so I hope I'm using the correct term)

Currently in Godot, it is quite simple to achieve the shader side of this approach - each vertex gets a VERTEX_ID which would let us position it accordingly without using any existing transform data.
But there is no option to opt-out of sending the whole transform in the buffer, or even having it created automatically by the engine when increasing the instance count.

And of course ideally the best capability would be to be able to send any user-defined information in the buffer, and then position vertices accordingly in a shader, thus conserving memory and runtime.

With correct optimizations by the user, that would allow for creating very large or detailed worlds - be it terrain with a lot of vertices as seen in the video; or a lot of repeating, mostly-uniform instances as in my vegetation case

If there is any existing way or workaround to achieve a similar result in the existing state of the engine I would also really appreciate hearing about it!

[dnsiltei]

I have a similar discussion and was looking a little bit more into it, apparently the way to do this type of stuff for now in Godot is with particles.

[LiorArbel]

Can you expand more?
From what I researched, GPU particles take a lot more unneeded parameters and are limited to a maximum of 1 million, and from just a short try it looked like performance of 1 million particles was much much worse (running at almost 1fps) compared to 1 million multimesh instances running at 144fps

[Calinou]

There's no hard cap of 1 million particles. In 2D in particular, it's possible to render 1 million particles at 300+ FPS on modern GPUs.

What GPU and resolution did you test this with? Did particles use an unshaded material with shadow casting disabled on the GPUParticles3D node? This is important, as per-pixel shading and shadow casting for particles is expensive.

[dnsiltei]

what I meant was just that when people want GPU instancing in Godot, and don't want to deal with transforms on the CPU side, apparently particles are the go-to

[Calinou]

@Khasehemwy Please don't bump issues without contributing significant new information. Use the 👍 reaction button on the first post instead.

[ODtian]

currently gpu instancing in godot is wrapped in multimesh and most of its functions are written at core level, which means user can't use it for advanced usage, although it does support 4 float custom data.
I saw it is able to set the whole buffer for multimesh, so why don't we create a base class for multimesh where let user manage all the data for instancing and inherit by multimesh for more basic usage?

[nekotogd]

Like the first post said, I think indirect rendering could be a very useful addition for grass vegetation, and the current multimesh solution is somewhat inadequate**. Let me elaborate:

The Problem

Multimesh works great for rendering the same object many times in a single draw call. However, generating a multimesh primarily requires two pieces of information: the number of instances to draw, and the transform of each instance.

You have to calculate these transforms on the CPU*** which is not as suited to large parallel computing tasks as the GPU. Also calculating the positions / transforms for each grass instance is independent of any information of the other grass instances, making this perfectly suited for the GPU.

***You can calculate the transforms in parallel on the GPU using a compute shader for example, but you still have to pass back this huge buffer of transforms to the CPU, only for it to be passed back again to the GPU for drawing, which seems kind of unnecessary?

Furthermore, the CPU really isn't doing anything with the data once it receives it, it simply generates the multimesh to be sent back to the GPU, but doesn't need to modify the data. Also this data readback from the compute shader and generating the multimesh is rather expensive.

**Is this a Big Problem?

If you're alright with just culling grass based on multimesh chunks, then you won't have to generate a new multimesh that often. Only when the player moves into a new chunk, you'll have to update and maybe generate new multimeshes.

However, if you're worried about visibility culling the instances, technically, if even a small amount of the bounding box from a large multimesh grass chunk enters the view frustum, the whole multimesh is rendered. To mitigate this you'll have to calculate what grass positions are on screen every frame, which means re-generating the multimesh every frame...

Potential Solution

An indirect rendering solution would solve this because if the developer wants to move these transformation calculations to the GPU, and carry out frustum culling on each frame using a compute shader, they won't be bottlenecked with the current GPU>CPU>GPU design, and would instead be able to point the GPU to the drawing data it needs indirectly.

[Calinou]

Did you look into using a custom particle shader for grass rendering? There are a few examples of this out there such as https://www.reddit.com/r/godot/comments/1etkapf/grass_rendering_in_godot/.