ViewNode - Rendering problem for the second time

[konrad-thoc]

val viewNode = ViewNode(
            engine = sceneView.engine,
            modelLoader = sceneView.modelLoader,
            viewAttachmentManager = viewAttachmentManager,
        ).apply {
            disable()
            scale = Scale(-5f, 5f, 1f)
        }

        viewNode
            .loadView(
                context = this@ARActivity,
                layoutResId = R.layout.tracker_ar_view,
                onLoaded = { _, view ->
            
                    anchorNode.addChildNode(viewNode)

                    // Remove duplicates
                    val index = sceneView.childNodes.indexOfFirst { it.name == DESTINATION_NODE_NAME }
                    if (index != -1) sceneView.removeChildNode(sceneView.childNodes[index])

                    sceneView.addChildNode(anchorNode)
                    destinationNode = anchorNode
                },
                onError = {
                 
                }
            )

I have this code for rendering XML view, when I enter this screen for the first time everything works ok, however after the 2nd time I enter the screen I get an error.

[rawello]

hi, I tried to reproduce your error, created an application using ARSceneView, restarted the scene many times, but apart from the Filament double free error, I got nothing else. Can I ask you to provide me with more data to reproduce the error?

[konrad-thoc]

Yeah sure, the problem is when navigating back to a Fragment/Activity where you previously loaded an XML view using ViewNode.

Here is Initial App Activity, when we pressed the button we are redirected to MainActivity. After the ViewNode is loaded in MainActivity, you need to navigate back (in the example you should use the system navigation) and then navigate to MainActivity again, this causes an error

class SecondActivity : AppCompatActivity() {

    private lateinit var binding: ActivitySecondBinding

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        enableEdgeToEdge()
        binding = ActivitySecondBinding.inflate(layoutInflater)
        setContentView(binding.root)

        binding.apply {
            button.setOnClickListener {
                val intent = Intent(this@SecondActivity, MainActivity::class.java)
                startActivity(intent)
            }
        }
    }
}

Here is MainActivity

class MainActivity : AppCompatActivity() {

    private lateinit var binding: ActivityMainBinding
    private lateinit var viewAttachmentManager: ViewAttachmentManager

    private var trackerViewBinding: TrackerViewBinding? = null

    private var destinationNode: AnchorNode? = null
    private var directionArrow: AnchorNode? = null

    private var trackerLat: Double = 0.0 // REPLACE WITH YOUR COORDINATES
    private var trackerLng: Double = 0.0 // // REPLACE WITH YOUR COORDINATES
    private val distanceState = MutableStateFlow<Double>(0.0)
    private var lastDestinationRotation: Rotation? = null

    override fun onResume() {
        super.onResume()
        viewAttachmentManager.onResume()
    }

    override fun onPause() {
        super.onPause()
        viewAttachmentManager.onPause()
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        enableEdgeToEdge()
        binding = ActivityMainBinding.inflate(layoutInflater)
        setContentView(binding.root)

        setupView()
    }

    private fun setupView() {
        binding.apply {
            sceneView.apply {
                lifecycle = [email protected]
                planeRenderer.isEnabled = true
                this.mainLightNode
                // Set rendering distance to 200m
                cameraNode.far = 200f
                configureSession { session, config ->
                    config.focusMode = Config.FocusMode.AUTO
                    config.lightEstimationMode = Config.LightEstimationMode.DISABLED
                    config.depthMode = Config.DepthMode.DISABLED
                    config.planeFindingMode = Config.PlaneFindingMode.DISABLED
                    config.updateMode = Config.UpdateMode.LATEST_CAMERA_IMAGE
                    if (session.isGeospatialModeSupported(Config.GeospatialMode.ENABLED)) {
                        config.geospatialMode = Config.GeospatialMode.ENABLED
                    } else {
                        showErrorMessage("Geospital mode not supported on this device")
                    }
                }
                onSessionUpdated = { session, frame ->
                    updateStatusText(session)

                    drawDestinationTracker(session)
                    drawDirectionArrow(session, frame)
                }
                onTrackingFailureChanged = { reason ->
                    updateInstructions(reason)
                }
                viewAttachmentManager = ViewAttachmentManager(this@MainActivity, this)
            }
        }
    }


    private fun drawDestinationTracker(session: Session) {
        val sceneView = binding.sceneView

        if (destinationNode != null) {
            // Rotate node to camera
            val cameraNode = sceneView.cameraNode
            if (distanceState.value < 1 && lastDestinationRotation != null) {
                destinationNode!!.rotation = lastDestinationRotation!!
            } else {
                destinationNode!!.lookAt(cameraNode)
            }
            lastDestinationRotation = destinationNode!!.rotation
            return
        }

        val anchor = createEarthAnchor(session) ?: return
        val anchorNode = AnchorNode(sceneView.engine, anchor)
        // Set the node identifier
        anchorNode.name = Constants.DESTINATION_NODE_NAME
        ViewRenderable.builder()
            .setView(sceneView.context, R.layout.tracker_view)
            .build(sceneView.engine)
            .thenAccept { renderable ->
                trackerViewBinding = TrackerViewBinding.bind(renderable.view)

                val viewNode = ViewNode(
                    engine = sceneView.engine,
                    modelLoader = sceneView.modelLoader,
                    viewAttachmentManager = viewAttachmentManager
                ).apply {
                    setRenderable(renderable)
                    disable()
                    scale = Scale(-10f, 10f, 1f)
                }

                anchorNode.addChildNode(viewNode)

                // Remove duplicates
                val index = sceneView.childNodes.indexOfFirst { it.name == Constants.DESTINATION_NODE_NAME }
                if (index != -1) sceneView.removeChildNode(sceneView.childNodes[index])

                sceneView.addChildNode(anchorNode)
                destinationNode = anchorNode
            }
    }

    private fun createEarthAnchor(session: Session): Anchor? {
        val earth = session.earth ?: return null
        if (earth.trackingState != TrackingState.TRACKING) return null

        val altitude = earth.cameraGeospatialPose.altitude - 1

        return earth.createAnchor(
            trackerLat,
            trackerLng,
            altitude,
            0f, 0f, 0f, 1f,
        )
    }

    private fun drawDirectionArrow(session: Session, frame: Frame) {
        if (destinationNode == null) return
        if (frame.camera.trackingState == TrackingState.TRACKING) {
            val pose = frame.camera.pose
                .compose(Pose.makeTranslation(0.3f, 0f, -1f))
                .extractTranslation()
            val anchor = session.createAnchor(pose)

            if (directionArrow != null) {
                directionArrow!!.anchor = anchor
                // Rotate node to destination
                directionArrow!!.lookAt(destinationNode!!)
                return
            }

            val sceneView = binding.sceneView
            val anchorNode = AnchorNode(sceneView.engine, anchor)
            // Set the node identifier
            anchorNode.name = Constants.DIRECTION_ARROW_NODE_NAME
            lifecycleScope.launch {
                val modelInstance =
                    sceneView.modelLoader.loadModelInstance(Constants.DIRECTION_ARROW_FILE)
                if (modelInstance != null) {
                    val modelNode = ModelNode(
                        modelInstance = modelInstance,
                        scaleToUnits = 0.3f,
                        centerOrigin = Position(y = -0.5f),
                    )
                    modelNode.disable()
                    anchorNode.addChildNode(modelNode)

                    // Remove duplicates
                    val index = sceneView.childNodes.indexOfFirst { it.name == Constants.DIRECTION_ARROW_NODE_NAME }
                    if (index != -1) sceneView.removeChildNode(sceneView.childNodes[index])

                    sceneView.addChildNode(anchorNode)
                    anchorNode.lookAt(destinationNode!!)
                    directionArrow = anchorNode
                } else {
                    showErrorMessage("Unable to load model")
                }
            }
        }
    }

    private fun updateStatusText(session: Session) {
        val earth = session.earth
        if (earth?.trackingState == TrackingState.TRACKING) {
            val cameraGeospatialPose = earth.cameraGeospatialPose
            val geospitalPose = getString(
                R.string.geospatial_pose,
                cameraGeospatialPose.latitude,
                cameraGeospatialPose.longitude,
                cameraGeospatialPose.horizontalAccuracy,
                cameraGeospatialPose.altitude,
                cameraGeospatialPose.verticalAccuracy,
                cameraGeospatialPose.heading,
                cameraGeospatialPose.headingAccuracy,
            )
            binding.statusText.text = resources.getString(
                R.string.earth_state,
                earth.earthState.toString(),
                earth.trackingState.toString(),
                geospitalPose,
            )

//            val distanceToTracker = DistanceCalculator.calculateDistance(
//                cameraGeospatialPose.latitude,
//                cameraGeospatialPose.longitude,
//                trackerLat,
//                trackerLng,
//            )
//            distanceState.value = distanceToTracker
//
//            binding.accuracyText.text = "Accuracy: ${cameraGeospatialPose.getAccuracy()}"
//            trackerViewBinding?.trackerDistance?.text = "${String.format("%.1f", distanceToTracker)} m"
        }
    }

    private fun updateInstructions(reason: TrackingFailureReason?) {
        binding.instructionText.text = reason?.getDescription(this) ?: ""
    }

    private fun showErrorMessage(error: String) {
        Toast.makeText(this, error, Toast.LENGTH_SHORT).show()
    }
}

[rawello]

Sorry for late answer, did you try full debug step by step drawDestinationTracker in ViewRenderable builder part?

[konrad-thoc]

No, but I'm sure there is a problem with ViewRenderable, because after replacing it and loading the 3D model, everything works fine. It is also easy to reproduce, you have to create 2 activities, and on the second one you have to load the view node using ViewRenderable, then you have to press the system back button, and then you have to go to the second activity again. Then there is a crash which cannot even be caught

[Lorenzoarc]

I have the same issue.Any news?

[konrad-thoc]

no new information so far

[konrad-thoc]

@rawello are you able to tell if it can be fixed?

[rawello]

@rawello are you able to tell if it can be fixed?

Hi man, I have deadlines at work, last time when I tested this code, I was unable to reproduce the error, a little later I will try to solve your problem if this will still relevant, I really want to help and do PR but so much work

[rawello]

@rawello are you able to tell if it can be fixed?

@KonradTHOC
Hello man, I downloaded the sources, installed them in my project, there was an error before that when swiping back in “Compose” ARScene you could get a crash, I looked through the code and came to strange solutions, maybe I missed the point somewhere, but, to begin with updated all libraries except "Filament" and in some cases in ARScene change LocalLifecycleOwner to androidx.lifecycle.compose.LocalLifecycleOwner . Actually, I uncommented one line, wrapped part of the code in “Try-catch” and added a couple of lines to “ARScene”. So
in ARScene.kt am added

onRelease = { sceneView ->
                try {
                    Log.d("ARScene", "Releasing ARSceneView")
                    sceneView.arCore.destroy()
                    sceneView.session?.close()
                    sceneView.scene.removeEntities(sceneView.scene.entities)
                    sceneView.destroy()
                }
                catch (e:Exception){
                    e.message?.let { Log.e("error arscene", it) }
                }
            }

And in SceneView.kt im uncommented runCatching { ResourceManager.getInstance().destroyAllResources() }

ps, sorry for my leg in video xd)

video_2024-11-17_19-40-24.mp4

upd: in video 6 modelnodes connected each others with a lot of ViewNode

[konrad-thoc]

@rawello I am using XML views so I don't have such a method. What method should I use instead of that?

[rawello]

@konrad-thoc
oh I'm using compose, try adding similar code to onDestroy() with the order of destroy as I presented

[konrad-thoc]

@rawello yeah it seems to be working with onDestroy method. Thanks!!

    private fun destroySceneView() {
        try {
            val sceneView = binding.arSceneView
            sceneView.arCore.destroy()
            sceneView.session?.close()
            sceneView.scene.removeEntities(sceneView.scene.entities)
            sceneView.destroy()
        } catch (e: Exception) {
            Timber.d("$this - cannot destroy SceneView")
        }
    }

[rawello]

@konrad-thoc if its will work, notify me, i will to PR

[konrad-thoc]

problem solved, thanks