Enhancements and Refactoring for XR Runtimes Integration #149
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The function `GetMappedAction()` is added to the vtkVRInteractorStyle base class through MR-10784. See https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10784
Removes "GestureEnabledButtons" ivar that became obsolete following these commits: * c5c6d6f ("BUG: Restore complex gesture support", 2023-01-29), * 22eba9f ("BUG: Fix integration with updated Slicer event delegation and VTK OpenVR API (KitwareMedical#131)", 2023-12-21) * 0fa6102 ("BUG: Ensure handling of gesture triggers the "End" event", 2023-12-20) * 512efe6 ("BUG: Do not report "Unrecognized device" if handling complex gesture", 2023-12-20) For reference, it was originally introduced in 43bef4b ("ENH: Make trigger button configurable", 2019-01-11).
The function `GetCurrentGesture()` is added to the vtkVRRenderWindowInteractor base class through MR-10786. See https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10786
Since OpenXR and OpenVR runtime each require their own instance of interactor style, this commit relocates is a step toward reducing the amount of code that will be duplicated.
Since OpenXR and OpenVR runtime each require their own instance of interactor style, this commit is a step toward reducing the amount of code specific to the interactor. The is made possible after the following vtkOpenVRRenderWindowInteractor updates: * Declare AddAction() functions as virtual. See https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10785 * Mark ComplexGesture recognition functions as public. See https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10786 For reference, the "GestureButton" configuration functions were introduced through these commites: * 43bef4b ("ENH: Make trigger button configurable", 2019-01-11) * e268168 ("ENH: Add python-accessible functions to capture vr controller events, associate buttons with gestures and enable/disable interaction.", 2019-05-30)
Update the RenderWindowInteractor to use the recognizer. This will allow to have dedicated interactor for OpenVR and OpenXR runtime wihout any code duplication.
Following 22eba9f (BUG: Fix integration with updated Slicer event delegation and VTK OpenVR API), the VirtualReality interactor style derives from `vtkOpenVRInteractorStyle` and all the upstream capabilities are now available.
Also removes redundant call `InteractorStyle->SetInteractor(this->Interactor)` Calling `Interactor->SetInteractorStyle(this->InteractorStyle)` is sufficient.
Update the InteractorStyle to use a delegate implementing the common logic related to processing of the "Pinch3D" complex gesture, handling of the "PositionProp" interaction and update of the view magnification. This will allow to have dedicated interactor styles for OpenVR and OpenXR runtime without code duplication.
Group headers and remove unused ones. Consistently use "VR [Logic|MRML|MRMLDM|Widgets] includes" comment to introduce headers from this extension. Re-order include groups and sort alphabetically within groups. Remove `vr::TrackedDevicePose_t` forward declaration that became obsolete following commit 0c8c2e8 ("ENH: Decouple update of transform node matrix and attributes", 2023-12-26)
This is done anticipating subsequent refactoring in which the computation of the ActionManifestPath will be moved to the logic.
Update VirtualReality logic adding `ComputeActionManifestPath()` utility function and update VirtualRealityView to compute the path based on the XR runtime. To support computing the path based on the module install state, the `ModuleInstalled` property was also added.
…base on XRRuntime
This commit streamlines the process of creating a RenderWindow in the qMRMLVirtualRealityView class by making several improvements: - Reorganize the update of QCursor in the `createRenderWindow()` function around the time-consuming `RenderWindow->Initialize()` call. - Set the view node's Error status directly within `createRenderWindow()`, and check for view node errors before returning from `updateWidgetFromMRML()`. - Enhance `destroyRenderWindow()` to gracefully handle calls without instantiated objects. - Update `createRenderWindow()` to set a view node error if the XR runtime is undefined and RenderWindow creation did not occur.
Since we systematically attempt to initialize the runtime if it is "UndefinedXRRuntime" or if it has changed, this commit also adds support for checking for the maximum initialization attempts and avoid recursion.
Users can now conveniently select and update the current XR Runtime.
This commit removes the explicit dependency on the "openvr.h" header in the TransformWidget. However, it's essential to note that tracking results are not standardized, and the status will consistently be reported as "off" for XR runtimes other than OpenVR.
For reference, the function was originally introduced in commit 46b252d ("Added progressive rendering", 2018-04-20) Also renamed ivar from `viewUpDirectionChangeSpeed` to `viewUpChangeSpeed`
…d()` Since having the head-mounted display (HMD) connected may not always be a good proxy to check if the initialization succeeded, we instead swith to the dedicated `GetVRInitialized` function available since commit Kitware/VTK@bddd94efd3 ("Improve OpenVR/OpenXR module and test and enable OpenXR in CI", 2023-01-11) Moreover, the determination of whether the event loop should run has been refined by relying on the GetVRInitialized function as the primary criterion. In the case of the OpenVR runtime, the use of GetHMD is retained through the introduction of the `hmdConnected` internal variable for enhanced flexibility.
- Introduce the CMake option `SlicerVirtualReality_HAS_OPENVR_SUPPORT`, initialized to OFF on macOS. - Introduce `vtkMRMLVirtualRealityConfigure.h`, a configurable header facilitating conditional compilation based on the presence of the macro `SlicerVirtualReality_HAS_OPENVR_SUPPORT`. - Update `vtkVirtualRealityViewInteractorObserver::GetInteractorStyleDelegate` to dynamically return the current delegate based on the availability of OpenVR support. - Ensure that when OpenVR support is disabled, proper linkage against the `VTK::RenderingVR` target is maintained, providing a backend-agnostic interface.
- Add SlicerVirtualReality_HAS_OPENXR_SUPPORT CMake option - Add OpenXR-SDK and vtkRenderingOpenXR external projects - Add OpenXR specialized interactor style class for. - Add OpenXR specialized interactor class. Thanks to the shared `vtkVirtualRealityComplexGestureRecognizer`, there is no duplicated code. - Add OpenXR specialized interactor style class. Thanks to the shared `vtkVirtualRealityViewInteractorStyleDelegate`, there is no duplicated code. - Update vtkSlicerVirtualRealityLogic::ComputeActionManifestPath support for `OpenXR`. - Update `vtkMRMLVirtualRealityViewNode` adding `OpenXR` to `XRRuntimeType` enum - Update `qMRMLVirtualRealityView::createRenderWindow` to instantiate `vtkOpenXR*` classes - Update `qMRMLVirtualRealityView::currentXRRuntime` to check for OpenXR
This was
linked to
issues
Dec 30, 2023
This was referenced Jan 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These changes enable the integration of XR runtimes (OpenXR and OpenVR) into SlicerVirtualReality extension. Below is a summary of the key improvements, simplifications, and cleanup:
Improvements:
SlicerVirtualReality_HAS_OPENVR_SUPPORT) for toggling OpenVR support.ComputeActionManifestPath()) for computing the path based on the selected XR runtime.ModuleInstalledproperty for checking the install state of the module.vtkVirtualRealityViewInteractorStyleDelegatefor common logic related to gesture processing.vtkVirtualRealityViewInteractor.Simplifications:
shouldConsiderQuickViewMotion()function for improved readability.viewUpDirectionChangeSpeedtoviewUpChangeSpeedfor better clarity.CalculateCombinedControllerPosefunction to the logic class for better organization.Cleanup & Removal:
CreateRenderWindow()function for better readability.GetCurrentGesture()from the base class, streamlining the inheritance hierarchy.GestureEnabledButtonsivar for codebase simplification.vtkVirtualRealityViewInteractorStyle.Support for both Slicer
StableandPreviewThese changes are supported in both the latest
StableSlicer release (5.6.1) and the currentPreviewbuild. The relevant changes have been backported to a dedicated branches for both thePreviewbuild1 and theStable2 release.For the release, the corresponding VTK sources have been downloaded in the corresponding Slicer stable build trees located in the official build machine used to build the extensions daily.
For reference, here are the pull requests for the
Slicer/VTKchanges used for theRelease:Relatedimprovements and fixes
See detail below.
Acknowledgment
This significant endeavor was made possible through the support of @adamrankin and his team. Their dedication has been instrumental in shaping and advancing the project. Special thanks also go to @LucasGandel and @sankhesh for their invaluable technical support and guidance. 🙏 🚀
Footnotes
slicer-v9.2.20230607-1ff325c54-2: https://github.com/Slicer/VTK/tree/slicer-v9.2.20230607-1ff325c54-2 ↩slicer-5.6-v9.2.20230607-1ff325c54-2: https://github.com/Slicer/VTK/tree/slicer-5.6-v9.2.20230607-1ff325c54-2 ↩