Future Pull-Request for Adding Group of Objects to Existing Scene

[Inv4lidn4m3]

Hello Sionna's team,

With @guillaume-larue, we are currently working on a potential pull-request for Sionna RT.
The fork (work in progress) is located here: https://github.com/guillaume-larue/sionna/tree/ISAC

The objective is to be able to add to an existing scene what we call assets (objects or group of objects) and then be able to manipulate and remove them.

Our purpose for this feature is to be able to automatically generate scenes dataset for machine learning.

We are currently working on the notebook, tests and fixing some bug.

The pull request would be ready in a few days to a few week, depending on the time we can allocate to it.

Would this be of interest to you?

Thanks in advance!

[jhoydis]

Hi @Inv4lidn4m3,

Thanks for sharing this idea and reaching out to us before preparing a pull request.

We have been thinking about adding a similar feature for some time but could not find a satisfactory solution yet.

Could you explain how your feature would work in a nutshell?

Thanks

[guillaume-larue]

Hi @jhoydis,

In essence, we have introduced a new class called AssetObject. This class points to an asset XML descriptor and a set of PLY mesh files, similar to how scenes are exported from Blender. When an AssetObject instance is added to a scene using the scene.add() method, the elements from the asset XML are appended to the scene (temporary) XML file, and the scene is reloaded. The AssetObject class keeps track of references to the newly created SceneObject s (i.e., shapes in the scene generated by adding the asset). It offers an abstract interface to these SceneObject instances. For example, you can move the entire asset by setting my_asset.position, which will update the positions of all SceneObject instances related to that asset accordingly.

Reloading the scene involves clearing all scene objects, reloading the scene using Mitsuba from the new XML, reinstantiating the RT solvers and reloading scene objects. Importantly, this process keeps all other Sionna scene-related properties (e.g., TX/RX, RIS, Material) unchanged. One drawback of this approach is that it probably breaks any differentiability with respect to the previous scene, as it is almost equivalent to loading a new scene. However, it remains useful for programmatically defining datasets and scenarios while maintaining all other scene properties. This approach is straightforward, though there is room for further refinement such as avoiding the need to reload all scene objects with each asset addition or removal.

We are currently finalising the work on asset’s radio material management and the associated tests. When this will be finished we will also provide a tutorial notebook.

Thanks

[jhoydis]

Hi @guillaume-larue,

Thanks for sharing this explanation. We have experimented with similar ideas in the past, but felt that it was a bit "hacky". Unfortunately, we do not have a better solution yet. We'll review your PR once it is ready, but it might take a bit of time.

In any case, thanks for this useful fork!

[Inv4lidn4m3]

Hello @jhoydis,

Thanks for your support!

As far as we know, there is currently no way to add or remove a mesh from an existing scene in Mitsuba. We couldn't find any relevant information in the API reference or online questions.
Do you confirm this? We have a limited understanding of Mitsuba itself, so we may have missed something.
This is why we rely on reloading the scene from scratch each time, based on a new xml file. Also, this easily ensure that everything related to Mitsuba and other Sionna's features work smoothly.

[jhoydis]

For now this is indeed the case. We are in contact with the Mitsuba developers to see if there is something which could be done.

[Inv4lidn4m3]

@jhoydis See the related pull-request

[Fedomer]

Hello Sionnateams and Mr @guillaume-larue thank for your effort

Sorry but if you create a shape like this in mitsuba it works but:

import mitsuba as mi

# Inizializza Mitsuba con il plugin corretto
mi.set_variant('scalar_rgb')

# Definizione di un cubo come shape Mitsuba
mi_shape = mi.load_dict({
    "type": "cube",  # Tipo di forma Mitsuba
    "to_world": mi.ScalarTransform4f.translate([0, 0, 0]) @ mi.ScalarTransform4f.scale([5, 2, 1.5])  # Set Dimension
})
car = sionna.rt.SceneObject(name="Car01",
                             object_id='car01',
                            mi_shape = mi_shape
                             )
scene.add(car)

it seems that the creation of the object was blocked by a control.

is it possible to just have at first just a basic type of element like a box (usefull for car, truck or why not human):
myBox1 = Scene.Box(name,position,orientation,scale,material,color)
scene.add(myBox1)

Step 2 - Load Object

Why not have function like LoadObject(file=XMLFile) for a type sionna.rt.SceneObject and the you can add at a scene any object exported in mitsuba file.