An integrated LOD system

[unfa]

The problem

Godot 4 has two LOD features right now:

1. The Automatic LOD system (I'll call it AutoLOD for short)
2. The manual Visibility Range settings (I'll call it VisRange for short)

Both have limitations and strengths, and I think they should be merged together with some new features added to create a complete and integrated LOD system.

AutoLOD

Upsides

1. Does not require any manual work and works on mesh-level
2. Because it is based on pixel density, it takes FOV and screen resolution into account.
3. It allows per-asset tweaks to offset the desired amount of detail for less or more important models (characters vs environment for example)

Downsides

1. The quality and efficiency of the generated LOD models is limited, as of course the system will not be able to discard and optimize geometry as well as an artist could, plus it's limited to re-using the same textures. It could maybe be possible to have it re-bake textures from the original model to preserve more detail in normal maps and reduce the triangle count even more etc but that'd be a very complex feature.

2. There's no way to preview or tweak the generated LOD models or to even know how many are generated.

Bugs

Currently in my project AutoLOD does not work at all. LOD models are either not generated or not utilized. This gave me a false sense of security and made me use mid-poly placeholder assets where I should've been using low-poly ones.

VisRange

Upsides

1. Allows superior quality and efficiency due to allowing the use of custom, highly optimized LOD models with unique textures for preserving maximum detail with less rendering cost in exchange for more artist work and possibly more asset storage required (due to optional use of unique LOD textures, not sure how much extra storage AutoLOD costs).

2. Allows for smooth fading between LOD models to reduce the LOD popping.

Downsides

1. Requires manually importing and configuring multiple MeshInstance3D nodes this means that whenever you want to re-use a model that has LODs you need to copy and entire scene to utilize that, instead of just selecting a different mesh resource.
2. Cannot be set up in the advanced import dialogue to be re-configured automatically on reimporting a scene
3. No global LOD distance controls
4. The system is entirely based on object distance, not taking into account the camera FOV or viewport resolution. This means that to use this system at all, the user should actually script their own solution to adjust the distances for different screen resolutions and camera FOV. In an FPS game where iron sights or sniper scopes are common, this is an absolute must, or players would see ugly LOD models every time they used a sniper scope or binoculars in game.
5. The system requires multiple MeshInstance nodes carefully configured to work together

Bugs

Fading doesn't seem to work correctly, or I don't know how to set it up.

What I think we need

AutoLOD and VisRange could be merged to utilize the strenghts of both and eliminate their weaknesses. It would require a few new features though.

1. A new resource (let's call it MipMesh for now) should be created that would aggregate multiple manual LOD meshes with their materials as well as an LOD configuration. AutoLOD would be utilized to provide LOD models if there are none or fill the gaps between manual LOD models. It could also replace the current Mesh resource, though the LOD features should be made optional for cases where LOD is not used.
2. A visual editor for LOD ranges needs to be made for easily configuring and testing the MipMesh resource in the game context.
3. Manual LOD models should be automatically analyzed for detail density and a proposed LOD factor should be estimated*
4. A visual editor for LOD distance thresholds and fade

• esitmating custom LOD model detail density is more difficult than with AutoLOD due to unique LOD textures playing a role. I think it should be possible to do by rendering the LOD models in a fixed environement and camera from multiple angles and then comparing the resulting rendered images for amount of difference between pixels. Calculate the difference of the two images and then average all pixels to come with a single total difference value. This should allow us to derive a detail factor, proposing how the manual LOD modesl will fit on the LOD0 <---> CULL scale.

The engine would allow objects in the imported scene to use the same name with _LOD0, _LOD1 etc suffixes to group the objects together into a single MipMesh resource. If there are no LOD models present, a regular Mesh resource is used instead.

Material resources could also be analyzed to deduplicate ones that differ only by used textures.
MipMap generation and imported texture resolution could be optimized for LOD models to not waste resources on detail that will not be visible.

Say my LOD0 model uses a 2K texure, but by accident my LOD1 model also uses 2K texture, that will never be rendered itsel, - only it's mipmaps. Then the texel density of LOD0 can be estimated and an optimal textrue resolution for LOD1 can be derived, then the LOD1 textures can be imported in lower resolution, saving storage and video memory without any visual impact.
Similarly for LOD0 there is no point generating mipmaps that will never be rendered if LOD1 kicks in earlier.

If LOD models use the same material, the textures could probably be atlassed like MipMaps to optimize the rendering, especially when two LOD models are rendered at one during fading. This makes even more sense to use a custom resource and handle this during the import process - manual LOD setup could never do things like this, but a deep engine integration can provide such benefits.

If we go further with this idea - the MipMeshes and MipMaps can work together, optimizing rendering and memory to use lower-fidelity meshes with their custom textures on a lower level.

As long-term improvement, the AutoLOD models could be automatically UV unwrapped and have custom textures baked based on the LOD0 model - this could provide even better visual fidelity with lower rasterization impact, ultimately providing fully automatic advanced LOD generation that can be optimizeed with manual assets if need be, but in many instances would provide great looks out of the box. Althought obviously baking PBR maps on import would probably take some time and could be very error prone.

The AutoLOD could also take a few passes to optimize meshes further by testing visual difference vs polycount vs rendering time in passes to tweak the settings per-LOD mesh to optimize the results in exchange for extended import post-processing time.

What are your thoughs?

[Calinou]

I feel this adds a lot of complexity that may not be effectively taken advantage of by most users. I value having a simpler but less powerful system, rather than a big and flexible solution that is difficult to wrangle. What I'd just do is:

• Make sure visibility ranges can be set up from the Advanced Import Settings dialog. Since numeric values must be specified, this might be difficult to do via object names though (if you want to set up visibility ranges in Blender in advance).
• Adjust visibility range selection based on camera FOV (and possibly screen resolution?).
  • There is a well-known formula for this, similar to the one you'd use to adjust mouse sensitivity based on FOV in a FPS game.
  • For this to work, a reference FOV must be configured in the project settings. It would default to the Camera3D default FOV, which is 75 degrees (vertical FOV).
  • If resolution-dependent LOD selection is desired, the default window size can be used in the Project Settings.
  • In the editor, the actual LOD distance thresholds (based on editor camera FOV and viewport resolution) should be displayed in the top bar when selecting a GeometryInstance3D node.
• Add a global multiplier for all distance fade values, configurable as a project setting and with a Viewport property (for run-time changes). This can also be used to achieve FOV/resolution adjustments if we end up not implementing either of those in core.
  • There should probably be several properties, actually: one for GeometryInstance3D distance fade, one for Decal distance fade, one for OmniLight3D/SpotLight3D distance fade.
  • Should the multiplier also affect margins (hysteresis/fade)? I'd say yes, unless there is a significant reason not to do that.
• Implement Add triangle / vertex count display for MeshInstances #248.

All those items can be implemented independently from each other. However, we should probably fix godotengine/godot#58513 before implementing this in the Light3D and Decal distance fade systems.

Bugs

Fading doesn't seem to work correctly, or I don't know how to set it up.

Fading requires you to set the begin/end margin to values greater than 0. Also, materials will go through the transparent pipeline during fading, which means that transparency sorting issues can occur. To avoid this, there are plans to allow forcing dithering on entire materials with a single toggle, but it hasn't been implemented yet.

[unfa]

Yeah, if we could just have VisRanges of all LOD meshes for an object be editable in a sane visual way and kept between scene reimports, that'd already do a lot. I guess the AutoLOD would still be doing some work between the manual LODs either way.

[fire]

I proposed making the autolod measurement of screen pixel area the default, but it was not done. So there's two different systems and the measurements for sizing are different.

[unfa]

@reduz Has told me that AutoLOD will be extended to allow geometry streaming in the future, so it makes no sense merging it with VisRange.

I guess then what does make sense is:

1. Create a new resource type that governs LOD group behaviour (distances for each LOD level, fading, culling etc)

Why? To allow for multiple configurations to be reusable and globally controllable. Say you want to decrase the LOD distances for world models, but not character models - with LOD managing resources as long as you have them separate you can esily alter the resource in one place and everything updates. With every node having it;'s own settings this would require a lot of extra scripting, signals and manual updating for each node.

2. Create a visual editor/widget (abased on gradient editor?) for the LOD managing resource and make provide something to debug the LODs (show current displayed model polycount from the LOD group?)

Why? Right now setting up VisRange distances and making them match between meshes is a completely manual process, that is wasteful and error-prone. Also very tedious to tweak and test.

3. Provide a way to specify what objects should be considered an LOD group during scene import, and have that be possible to do with the scene layout / naming etc in the user's DCC of choice.

This could be done with hierarchy and naming patterns. For example:

Gun # implicit LOD0
|- Gun_LOD1 # explicit LOD1
`- Gun_LOD2 # explicit LOD2

If a MeshInstance node has a child that has the same name only with LODn suffix - then the models are treated as an LOD group.
This pattern could also work fro more complex scenes with multiple models, each having separate LODs.

Gun
|- Scope
|  |- Scope_LOD1
|  `- Scope_LOD2
|- Mag
|  |- Mag_LOD1
|  `- Mag_LOD2
|- Hand
|  |- Hand_LOD1
|  `- Hand_LOD2
|- Gun_LOD1
`- Gun_LOD2

In the above example every part of the model has it's own LOD models.

Child LOD groups (Scope, Mag, Hand) are governed by the parent LOD group (Gun) and will follow it's configured LOD distances, unless another LOD configuration is provided as an override. The LOD managing resource could specify if the submodels in the LOD group should be considered as a single mesh (all visiblity changes follow LOD0 - useful fro small scenes like character models) or work independently - each part keeps track of it's distance from camera and active LOD level separately (useful for very large scenes like entire environments or massive spaceships etc.).

Note that dynamically adding models into this hierarchy would also make them follow the Gun's LOD policy - that'd be useful for cosmetics on player models, modular gun models etc.

Why? Having to manually mark what objects belong to a common LOD group and which object defines what LOD level will be extremely tedious and error prone. It'd also allow much more UI work in the scene import dialog. Handling this via scene hierarchy and object naming is both easier for Godot devs and easier for users (as long as it's documented).

4. Define a reference FOV and viewport resolution in the project settings and correct the LOD level evaluation based on that, as @Calinou suggested. Also provide global controls so games can easily affect the LOD distances to balance visual quality and performance.

Why? Right now LOD distances do not care for camera FOV and screen resolution, this can lead to either rendering too much detail (on low resolutions or wider FOVs) or too little details (on higher resolutions or lower FOVs).

[NineAlexT]

I really like the idea of LOD group resources. It also greatly simplifies the workflow of my suggestion at #4286 on using glTF extras for setting the visibility range settings, as instead of setting Custom Properties in Blender for "vis_range_begin", "vis_range_begin_margin" etc you could set a Custom Property string named "lod_group" (or similar) containing a string "weapon", "character" or "hlod" and have Godot automatically check for a LOD group resource with that name on import. That moves the work of setting visibility distances or similar into Godot while still making the settings automatic.

[azur-wolve]

What about have for LODs:

• global config / various profiles (that you can use for each LOD resource)
• a "local" to resource/scene config, for allow specific use-case

currently GUI themes do similar stuff, by allowing both general configs and specific configs

[Calinou]

I started working on visibility range automatic FOV adjustment in a branch: https://github.com/Calinou/godot/tree/visibility-range-add-fov-adjust

[fire]

Currently in my project AutoLOD does not work at all.

Can you send some bug reports on this?