wayland: implement wl_touch #177

Sporif · 2024-04-21T14:59:46Z

I'm testing this on a touchscreen laptop and trying to implement basic support:

Send events to the active client at the correct position (tested by tapping, scrolling and zooming in Konsole and Firefox)
Hide the cursor on touch down
Switch the active client on touch down
Switch client on titlebar touch down
Switch workspace on bar touch down
Resize clients with touch motion on the edges/corners

Future work:

calibration-matrix option for switching the orientation of a touchscreen
touch gestures/bindings

Closes #115

mahkoh · 2024-04-21T16:24:10Z

Thanks for working on this.

I'm not familiar with how other compositors implement touch input, but here are some thoughts regarding how I would imagine it:

There are two touch owners:

struct DefaultTouchOwner;

struct GrabTouchOwner {
    node: Rc<dyn Node>,
    down_ids: AHashSet<i32>,
}

The DefaultTouchOwner ignores all events except down. Once it gets a down event, it finds the node under the touch position and replaces itself by GrabTouchOwner.

The down_ids field contains all ids that are currently down. When this set becomes empty, the GrabTouchOwner replaces itself by the DefaultTouchOwner.

When the GrabTouchOwner receives a down event, that event is always sent to the node, regardless of where on the screen it occurs. This means that you can press with one hand on an application and then with another hand anywhere else on the screen and the second event is sent to the node that was hit by the first press.

The first down event should also switch the keyboard focus to that node if it is an application window. (Using the same logic used by mouse button presses.)

mahkoh · 2024-04-21T16:42:31Z

Alternatively, each touch id could have its own touch owner. This seems to be how it works on android.

Sporif · 2024-04-22T00:01:53Z

I've implemented your first suggestion, what do you think?

Also, I was trying to hide the cursor with seat.set_app_cursor(None), but then it doesn't show up again until it enters another surface. What's the best way of implementing that feature?

src/ifs/wl_seat/event_handling.rs

src/backends/metal/input.rs

src/ifs/wl_seat.rs

src/ifs/wl_seat/touch_owner.rs

mahkoh · 2024-04-22T13:49:31Z

I've implemented your first suggestion, what do you think?

LGTM. Have you checked how other compositors handle multi-touch?

Also, I was trying to hide the cursor with seat.set_app_cursor(None), but then it doesn't show up again until it enters another surface. What's the best way of implementing that feature?

Why do you want to hide the cursor?

Sporif · 2024-04-22T21:25:38Z

LGTM. Have you checked how other compositors handle multi-touch?

I haven't looked at the code but KWin seems to do it the way you suggested. Sway only sends touch down events that fall within the boundaries of the initial surface after the first touch down, but it always sends motion events.

Why do you want to hide the cursor?

KWin, Sway and Windows all hide the cursor on touch. I find it can be distracting when you're not using it.

mahkoh · 2024-04-23T10:42:38Z

KWin, Sway and Windows all hide the cursor on touch. I find it can be distracting when you're not using it.

Maybe it's best to simply not render the cursor in that case. See GfxFramebuffer::create_render_pass.

Edit: But this is not going to work for hardware cursors. Maybe functions disable_cursor/enable_cursor on WlSeatGlobal that are called when the first touch down/the last touch up occurs and that does the necessary work while preserving the cursor that should be shown otherwise.

mahkoh · 2024-04-29T18:01:00Z

I'm currently working on adding tablet support. I'll continue the review once that is done.

mahkoh · 2024-05-02T16:17:42Z

The tablet PR #190 has significant overlap with this MR:

It adds a mechanism to map input devices to outputs.
It adds a CursorUserGroup type with a deactivate function that hides the cursor.
It adds the necessary code for tablets to interact with compositor GUI elements.

Sporif · 2024-05-03T12:36:45Z

The tablet PR #190 has significant overlap with this MR:

It adds a mechanism to map input devices to outputs.

It adds a CursorUserGroup type with a deactivate function that hides the cursor.

It adds the necessary code for tablets to interact with compositor GUI elements.

So should I wait for you to merge that PR and then rework this one?

mahkoh · 2024-05-03T15:56:53Z

Sounds good.

mahkoh · 2024-05-04T08:15:49Z

#190 has been merged.

Sporif · 2024-05-05T14:31:55Z

I've now implemented all the basic features. There's still some more future work but for now I think this is enough.

mahkoh · 2024-05-05T15:02:35Z

src/backend.rs

+pub enum TouchEvent {
+    Down { pos: TouchPosition },
+    Up,
+    Motion { pos: TouchPosition },
+    Cancel,
+    Frame,
+}


This should be flattened into the InputEvent enum and the fields of TouchPosition should be flattened into the Down and Motion variants.

mahkoh · 2024-05-05T15:03:06Z

src/backend.rs

@@ -272,6 +289,12 @@ pub enum InputEvent {
        state: KeyState,
    },

+    Touch {
+        seat_slot: i32,


This should be named something else due to the existing meaning of seat.

mahkoh · 2024-05-05T15:04:33Z

src/cli/seat_test.rs

@@ -583,6 +584,63 @@ async fn run(seat_test: Rc<SeatTest>) {
        }
        println!();
    });
+    let st = seat_test.clone();
+    TouchDown::handle(tc, se, (), move |_, ev| {
+        if all || ev.seat == seat {


The events should be collected and only be printed on the next frame event. Similar to the tablet frame event.

mahkoh · 2024-05-05T15:06:30Z

src/ifs/wl_seat.rs

+        let mut capabilities = POINTER | KEYBOARD;
+        let handlers = &self.state.input_device_handlers;
+        for (_, d) in handlers.borrow().iter() {
+            let dev = &d.data.device;
+            if dev.has_capability(InputDeviceCapability::Touch) {
+                capabilities |= TOUCH;
+                break;
+            }
+        }


The capabilities of a seat should only consider the input devices connected to a seat. These capabilities can also change at runtime when a device is attached or detached and then they will have to be sent to the client again.

mahkoh · 2024-05-05T15:07:21Z

src/ifs/wl_seat.rs

+            self.client.protocol_error(
+                self,
+                MISSING_CAPABILITY,
+                &format!(
+                    "wl_seat {} .get_touch called when no touch capability has existed",
+                    self.id
+                ),
+            );


This can be removed.

mahkoh · 2024-05-05T15:17:03Z

src/ifs/wl_seat/touch_owner.rs

+    }
+
+    fn motion(&self, seat: &Rc<WlSeatGlobal>, time_usec: u64, id: i32, x: Fixed, y: Fixed) {
+        self.down_ids.borrow_mut().insert(id);


This should be removed.

mahkoh · 2024-05-05T15:20:45Z

src/libinput/event.rs

+    pub fn x(&self) -> f64 {
+        unsafe { libinput_event_touch_get_x(self.event) }
+    }
+
+    pub fn y(&self) -> f64 {
+        unsafe { libinput_event_touch_get_y(self.event) }
+    }


I don't think we need these.

mahkoh · 2024-05-05T15:23:01Z

src/tree.rs

+    fn node_on_touch_down(
+        self: Rc<Self>,
+        seat: &Rc<WlSeatGlobal>,
+        time_usec: u64,
+        id: i32,
+        x: Fixed,
+        y: Fixed,
+    ) {
+        let _ = seat;
+        let _ = time_usec;
+        let _ = id;
+        let _ = x;
+        let _ = y;
+    }
+
+    fn node_on_touch_up(self: Rc<Self>, seat: &Rc<WlSeatGlobal>, time_usec: u64, id: i32) {
+        let _ = seat;
+        let _ = time_usec;
+        let _ = id;
+    }
+
+    fn node_on_touch_motion(
+        self: Rc<Self>,
+        seat: &WlSeatGlobal,
+        time_usec: u64,
+        id: i32,
+        x: Fixed,
+        y: Fixed,
+    ) {
+        let _ = seat;
+        let _ = time_usec;
+        let _ = id;
+        let _ = x;
+        let _ = y;
+    }
+
+    fn node_on_touch_frame(&self, seat: &WlSeatGlobal) {
+        let _ = seat;
+    }


I think this should work more like node_on_axis_event where the contents of a frame are collected first and then passed via a single call.

mahkoh · 2024-05-05T15:27:04Z

src/ifs/wl_seat/touch_owner.rs

+    }
+}
+
+impl TouchOwnerHolder {


This also needs to handle the case where the node that we're interacting with changes its position. See WlSeatGlobal::apply_changes.

What changes should be applied in such a case? There's no touch cursor, and for the duration of the touch sequence the node should stay the same no matter where it is.

Edit: And the latest node position is used for every touch down or motion.

I think we'll have to store the current absolute positions of the touch ids in compositor space and then send synthetic motion events to the node when the node moves.

mahkoh · 2024-05-05T15:27:50Z

src/tree/container.rs

+        x: Fixed,
+        y: Fixed,
+    ) {
+        self.pointer_move(CursorType::Touch(id), seat.pointer_cursor(), x, y, false);


pointer_move should be refactor so that the cursor is optional.

mahkoh · 2024-07-21T00:05:53Z

I've opened #230 to fix the remaining issues with this PR.

One thing I've removed is integration with the compositor UI. There were some serious issues with multi-touch and I'm only testing this with a graphics tablet hacked to work as a touch screen, so I don't have the fine control to properly design this. This can get added back in the future.

Sporif force-pushed the wl_touch branch from d393e11 to e5b8f05 Compare April 21, 2024 23:57