# Terrain generation

To create realistic environments, a skybox is not enough, and a terrain must be added.
The main difficulty of this part is to match the terrain with the skybox, in order to get a good rendering.
As we consider that the skybox is already good, it can be used to generate a terrain.

## Image-to-terrain model

As far as I know in (2024-09), there is no good open-source model to generate a terrain from the image.
This solution would be the more direct, but we will have to create our own pipeline.

## Matching 2D image and 3D terrain: depth generation

A terrain is essentially a 3D surface.
A simple transition from an image to a 3D environment is to use a depth generation model (
[Models on Hugging Face](https://huggingface.co/models?pipeline_tag=depth-estimation&sort=trending)
).
The details on this technique are documented at [RGBD image generation](./RGBD%20image.md).
From now on, we consider that we have RGBD (D for depth) data for the scene.

## Terrain segmentation

How to separate the voxels that represent the terrain from the other? This is a segmentation task.
The segmentation is explained at [RGBD image#Segmentation](RGBD%20image.md#Segmentation).

## From RGBD image to RGBH terrain

A terrain is often represented as a combination of a texture and an heightmap.
Both elements are viewed from the top with an orthographic projection, while the RGBD image uses a perspective projection.
To change between projection systems, we use the image gradient to complete "the shape of the visible terrain".
In the future, a real heightmap would be preferable.

When the terrain is segmented (and completed), it is still a RGBD segment.
The projection change is the best way of generating a real terrain, but this task is not finished yet.