Camera and manipulators

[dlyr]

Currently (and in upcoming PR) camera are detached from their manipulator
A CameraManipulator has two goals:

• bind input event to camera transformation
• compute the transformation according to camera data and some manipulator data (such as target point or scene radius)

A viewer has one active manipulator at a time, and only one manipulator is stored in memory.
Viewer's manipulator can be changed on the fly (the old one is deleted) with setCameraManipulator

In PR #740, camera data management is updated such that manipulator's camera always update "activeCamera", active camera switch is not well define (how should the manipulator's own data should be updated ?)

We can discuss how to design camera and camera management here.

[dlyr]

I think Camera and CameraManager are fine in #740
The first thing I have in mind is to move out of manipulator the "manipulation functions" and to move them in a function collection each manipulator can call.
The second thing is that the viewer could store manipulator data along with each camera's so that camera switch will also switch back manipulators ...

[nmellado]

I am not sure to understand what's wrong/ill-designed. Could you detail a bit more what are actual problems ?

[MathiasPaulin]

Perhaps we can have only one manipulator with several mode that shares parameters but do not use/update them in the same way.
By mode, I imagine the following

1. Observe : that rotate around a fixed point, i.e. the center of gravity of the focused object or the center of the scene, always looking at this point. Translations in the Camera frame are constrained to Z-translation only leaving the center of rotation untouched. Useful to examine a single object
2. RotateAround : that rotates around a movable point, always keeping this point at the same position in the 2D image plane, with the ability for the user to change this point by selecting it, moving it, ... Translations will be in the in the XY, XZ or YZ planes of the camera frame, translating the center of rotation the same way, useful for multi-object inspection
3. WalkAround : that rotates the camera around its own center, always keeping the same orientation around the Z-axis (keep the "up" direction unchanged", translations are free, if a point was previously focused for RotateAround, it stay fixed in the camera Frame
4. FlyMode : With the ability to rotate also around the Z axis

A mode is just a set of functions acting on some "manipulated frame" that, in turn, serve to modify/recompute the camera Frame.

The manipulator could store an association between a Camera and its manipulated frame (mode, center of rotation, XYZ frame...) and each time we switch to a camera, we get back its old manipulation parameter or, with a special switch command, recompute the parameter to respect the constraints of the manipulation mode on the left camera.

This is similar to what one can find in the old QGlViewer toolkit.

[nmellado]

In any case we need to keep the possibility to create new manipulators (or modes as you call them), as done here:

    viewer->setCameraManipulator(
        new CameraManipulator2D( *( viewer->getCameraManipulator() ) ) );

https://github.com/STORM-IRIT/Radium-Apps/blob/master/ShaderEditor/main.cpp#L106

[dlyr]

Well in my current view of the problem, mode == manipulator, more or less
In libQGLViewer, there is the following inheritance ManipulatedCameraFrame-> ManipulatedFrame ->Frame.
PivotPoint (which is slightly different from target) is a member of ManipualtedFrame, and also a frame is a tranlation + orientation (as quat) whcih ease computation and lessen numerical drift.
Then a camera has a manipulatedCameraFrame + camera parameters (fov, scene center, focus dist, screen size...)

Do we need to have scale in camera frame ? if so could we have quat + translation + scale represented explicitly in the camera data ?

In your suggestion @MathiasPaulin I do not get who store what.

[MathiasPaulin]

The camera store its frame, The manipulator associates the data to manipulate frame (manipulation data, e.g. pivot point, target ...), a reference to the camera to access its frame and the mode that use the manipulation data to modify the camera frame. Then a mode is just a functor that takes as parameter a frame and the manipulation data and that applies modification on them according to its behavior. The user can define its own mode by calling, e.g,

viewer->getCameraManipulator()->setMode( myManipulatorMode );

with myManipulatorMode being a std::function<void(Frame &, ManipulationData &)>. A user defined mode can then be a lambda, an callable object (with void operator()(Frame &, ManipulationData &) ), a compatible std::bind(...) call wrapper ...

But, and I forgot it, it is unclear how to take actions and context from KeyMappingManager. This might restrict a mode to be a callable objects on which the manipulator can set the action (in a handleMousePressEvent) so that the mode can use it to transform a (dx, dy) 2D movement to the right operation when the manipulator call the mode in a handleMouseMoveEvent.

To conclude, I'm not sure if my proposal is something we can do ...

[nmellado]

Something I don't get with this proposal: what are the ManipulationData ? What if MyManipulatorMode needs something specific ?

[nmellado]

After discussing with @MathiasPaulin, I may have a suggestion.

From what I see, a Camera should store the minimal set of parameters required to init any Manipulator (or mode, or whatever).
The constructor of any Manipulator should take the Camera as input, and it is the job of the constructor to initialize the internal state of the Manipulator so there is no change of property for the current Camera (e.g. position or target jump). Each Manipulator class then define its own interface and user control that affect the Camera state.
With this design we don't have to write explicit conversion between each Manipulator types (and thus avoid the combinatorial explosion coming with this kind of things).

From what I see, a Camera only needs to store a frame (describing its position and orientation) and a focus distance, expressed in the z axis.
Both the Trackball and the Flying manipulators can be initialized from these information, and both of them can recompute these attributes when the user interact. For Trackball specifically, the target point can be initialized from the Camera frame and focus distance.

[dlyr]

The difficult point is "both of them can recompute these attributes when the user interact".

For instance if you change a "rotate around point" of the trackball camera (with user interaction for instance).
If you switch active camera, should you use the old rotate around ? recompute a new one ?
if you switch back, there is no way of getting the old "rotate around point" from the camera data itself, at best you can compute a new one from camera data.

[nmellado]

In the general setting where the user/application create a new manipulator at each change, all the manipulator attributes need to be recomputed from the camera, so yes, this might cause a change of the trackball behavior if another manipulator has changed the focus distance in the meantime.

If you want to allow to switch back to a previous state, then you need to store multiple manipulators in your app and switch between these objects, instead of having having only one Manipulator. Switching back to a previous state means that you do not trust the current camera state and want to revert to something older. Note that would make sense in an undo/redo mechanism, but we don't have that in Radium yet.

[dlyr]

Another question related to manipulator, in it's current implementation, is there any way to access renderer/viewer to read depth under mouse ?
By current implem, I mean I don't want to create too huge changes to achieve that, I have already create the ForwardRenderer::readDepth(x, y) method, but I'm not sure how to access it from a camera manipulator. Any hints ?

[dlyr]

ping @nmellado @MathiasPaulin

[MathiasPaulin]

@dlyr is this question still opened ?

[dlyr]

Yes, it's link to PR #765 for these lines