Gesture customization UX (#99) · Issues · Teams / KDE Visual Design Group / Issues · GitLab
Admin message
Join us at
Akademy
to celebrate KDE's 30th anniversary!
Travel support requests
are open till May 31st.
Register now
Gesture customization UX
CC @nclarius, @duha, @redstrate, @zamundaaa
Here is a UX draft for configuring touchpad gestures. It is also meant to apply to touchscreen gestures, stroke gestures (a.k.a. mouse gestures), and scroll gestures (internally known as axis gestures) with few changes.
This is the result of several days of discussion with @nclarius and mock-up creation work. It is not meant as a final state for the UI, but it did come from a process of iterative improvements. I figure there is more work to be done.
I'm leaving for about 3 weeks of travel as of now; it's not the best time to kick off a larger UX discussion. But I want to put it out there before I forget all the arguments that led to this. Hopefully @nclarius can get a clear picture of what she needs to work on in the next little bit.
## Requirements
During our [gesture customization mini-sprint](https://blogs.kde.org/2025/06/12/gesture-customization-mini-sprint/) last week, Natalie and I surveyed the set of gestures (more generically: triggers) and assignable actions that Plasma currently supports or may want to support in the future. This has now moved to the community wiki: https://community.kde.org/Goals/Input/Triggers_and_Actions
In a nutshell, actions are largely what already exists in the "Shortcuts" KCM, and gestures are largely what libinput provides through its API. Compared to the current internal gesture registration support in KWin, we also added:
* "Rotate" and "Hold" gestures,
* Diagonal swipe directions (like in libinput-gestures), as well as
* Optionally context requirements across the board, namely pressed modifier keys and targeting particular windows.
We are not looking to support all of these initially, but the design should allow for it.
From the UX side, we want to give people an overview of which gestures are available, what action they will perform when triggered, and reassign with ease. A user may want to perform the same action on a different trigger, or assign a different action to a given trigger.
Here's what gesture settings seem to look like on other platforms, based mostly on screenshots around the web:
| macOS | Windows | GNOME (Gesture Improvements extension) | Mouse Actions (3rd-party app) |
|-------|---------|----------------------------------------|-------------------------------|
|  |  |  |  |
## Proposal in its current state
### Gesture overview for each input device KCM
If the user wants to view/edit touchpad gestures, she will likely start looking in the "Touchpad" KCM. Stroke gestures are a natural fit for the "Mouse" KCM. At the bare minimum, these KCMs must provide an entrypoint to link to gesture overview & configuration pages. We decided to embed gesture overview & configuration as a sub-page. Here is an example for the [anticipated unified](https://invent.kde.org/plasma/plasma-desktop/-/issues/117) "Mouse & Touchpad" KCM.
| KCM front page | Gesture overview sub-page |
|----------------|---------------------------|
|  |  |
The overview for each device type may look different; for instance, stroke gestures should be shown with a preview of the drawn path, whereas touchpad and touchscreen gestures can use pre-defined gesture icons.
The "Touchscreen Gestures" KCM may not use tabs, or perhaps uses tabs to switch between touchscreen edge gestures and touchscreen multi-touch gestures. By adding gesture pages for each KCM individually, we can make sure they work well for that input device type and we can complete one gesture type after another.
With a `ListView` for the gesture overview, we can strike a nice balance between showing what's available on one hand, and leaving enough flexibility for context-specific variations of gestures (i.e. apply only if modifiers pressed and/or if specific window is active). It's also widely used elsewhere in System Settings.
The view will be automatically sorted ~~by gesture type~~ [by component and action](https://invent.kde.org/teams/vdg/issues/-/issues/99#note_1242146). If the same action is assigned to two separate gestures, it will be listed twice with different gesture preview / subtitle. We discussed making one unified `ListView` for all gesture types, with optional filter, but we thought that tabs for Stroke/Touchpad/Scroll/etc. gestures are more intuitive, plus a single list will end up too long anyway.
### Assigning an action for a gesture
macOS and Windows offer a pre-defined set of gestures for use with a pre-defined set of actions that fit neatly into a ComboBox. We would like to offer more flexibility, which is paid for with higher levels of nesting.
In our design, there is one form for selecting a gesture, and a separate sub-page for selecting the action to assign to the gesture. We are not 100% decided which one should go first when the user clicks "Add Gesture..."; however, it seems [we're leaning toward selecting the action first](https://invent.kde.org/teams/vdg/issues/-/issues/99#note_1242106). (The initial design can be found [in a thread below](https://invent.kde.org/teams/vdg/issues/-/issues/99#note_1385051).)
A button called "Add..." is located to each action list item on this action selection sub-page, analogous to the shortcut key assignment in "Shortcuts". Pressing the Back button returns to the gesture overview without having assigned any action to a gesture.
| Assigning emulated key presses | Assigning regular actions |
|--------------------------------|---------------------------|
|  |  |
Clicking "Add..." will open the gesture selection dialog, mentioning the selected action that will be assigned.
| "Add Gesture" dialog | Gesture is not yet assigned | Gesture is already assigned |
|----------------------|-----------------------------|-----------------------------|
| Mock-up: |  |  |
| Confirm button: | "Add Gesture" | "Replace Assigned Action" |
I used a dialog, rather than a sub-page, because it provides clear guidance for either "Add Gesture" or "Cancel" at the bottom. In contrast, a KCM sub-page always provides a Back button and optional contextual page actions on the top right. It's not clear that the Back button would mean cancel, rather than "Done, this is good". For the gesture selection flow, I want the user to first select all aspects of the gesture, then confirm "atomically" in one go. Hence the dialog.
After confirming the dialog, the action selection sub-page will close and the user is back at the gesture overview sub-page. (If pressing "Cancel" in the dialog, nothing gets assigned but the user remains on the action selection sub-page. Perhaps they want to assign a different action.)
### Editing an existing gesture
Instead of pressing "Add Gesture..." on the gesture overview page, the user can also press the pencil "Edit" button of a given list item. For example, the touchpad gesture "Swipe up with 3 fingers" is assigned to "Cycle through Window Overview and Desktop Grid" by default.
Pressing the "Edit" button will also take the user into the action selection sub-page, but now it's in edit mode. This changes two elements:
1. An informational `InlineMessage` at the top tells the user which existing gesture assignment they're looking at. The `InlineMessage` also contains a "Change gesture..." button to modify the gesture assignment rather than the triggered action.
2. Each action list item now has an "Assign" button associated, instead of "Add...". This will reassign the action for the edited gesture immediately and take you right back to the gesture overview.
| "Edit" view with actions | Gesture is not yet assigned | Gesture is already assigned |
|--------------------------|-----------------------------|-----------------------------|
|  "Change Gesture..." opens gesture dialog |  |  |
|  "Assign" - back to overview | | |
The "Change Gesture" dialog allows changing the gesture of this action, possibly unassigning another action from the selected gesture. The dialog confirmation button will present the user with what they intended to do, without allowing any half-modified state when exiting the dialog.
The Back button on the action selection sub-page takes you back to the overview without having modified the gesture or its assigned action.
## Changes worth considering
#### Should we allow the "Edit" button to reassign an action, or is "Remove" + "Add..." enough?
The initially proposed flow lets the user edit both aspects of the gesture/action mapping, in the same order that both were added: first action selection, then gesture selection. We wanted a straightforward way both to change the gesture that's assigned to an action, as well as to change the action that's assigned to a gesture.
However, revisiting the flow makes me wonder if it would be simpler to only "Edit" the gesture itself, and fix the action after it gets added to the list. Assigning a new action to the same gesture would then have to happen in a different way, e.g.:
* (a) If the newly assigned action is already listed in the gesture overview, "Edit" the gesture of that item and take over the previously assigned action by re-specifying the gesture in the gesture edit dialog.
* (b) If the newly assigned action isn't yet listed in the gesture overview, go through the full "Add Gesture..." flow, specifying both the action and re-specifying the gesture in the gesture edit dialog.
* (c) Perhaps we could add a triple-dot contextual menu to each list item in addition to the "Edit" and "Remove" list action icons, with a single menu entry "Reassign Gesture to Different Action..."? Then that would open the action selection dialog, but it wouldn't have to fulfill a double role like the combined action/gesture edit flow is designed for.
* A triple-dot menu could also contain an entry "Swap Gestures..." which lets you select a different action to swap with.
#### "Add Gesture..." button in "Shortcuts" KCM?
Just because we decided not to make the "Shortcuts" KCM the main hub, doesn't mean we can't add a link from Shortcuts' action view to the Mouse/Touchpad/Touchscreen/Edge/Scroll gesture overview and pre-select the action there for assignment. We should also ideally list in each of these action listings the various shortcuts and gestures that are currently assigned to an action, regardless of which KCM and flow we're in at the moment.
## What we decided not to do
#### Shortcuts KCM as main gesture hub
We initially explored if we should take the "Shortcuts" KCM (a.k.a. `kcm_keys`) out of the "Keyboard" section and generalize it from just keyboard shortcuts to any kind of shortcuts. In addition to the "Add..." button for entering keyboard shortcuts, there would be an "Assign Gesture..." button or different ones for different gesture types.
We decided against adding to the "Shortcuts" KCM for the time being. The main reason is that we really want a gesture overview within the "Mouse", "Touchpad", and "Touchscreen Gestures" KCMs, probably also within "Screen Edges". If we start from such an overview page, navigating to the separate "Shortcuts" KCM will be problematic (e.g. requiring "Apply" before switching), and the Back button will be broken. There would also be a remaining challenge for how to tailor the "Shortcuts" UI to add specifically a touchpad gesture, for example. Furthermore, "Shortcuts" seems quite intuitive inside the "Keyboard" section where it is right now, compared to looking for keyboard shortcuts or gesture shortcuts outside of the respective categories.
So, re-purposing the "Shortcuts" KCM seems like a larger UX challenge. Instead, we decided that extracting QML/C++ components from "Shortcuts" is more promising. We can then tailor a version of it to assign an action to a gesture within a different KCM.
#### "Edit Gesture" button next to "Edit Action" button
Each list item features an "Edit" button to modify the gesture. A user might want to either change the gesture for the assigned action, or change the assigned action for a given gesture, so theoretically it might make sense to provide an "Edit" button for each of these.
However, we don't have two different "Edit" buttons for gesture edits vs. action edits. We could try drawing specific ones, but I don't think they would make the difference very clear.
We considered splitting each item in the gesture overview list into two columns, one for the action name, the other one for the gesture. That way we could put a regular "Edit" button next to each column and make the difference clear through the button position, rather than through the icon. I don't think that's going to work out either though, both because of text width concerns, and because it's rather different than established conventions for what list items should look like.
#### Combined page for gesture and action selection
Sticking with a single "Edit" button, I tried fitting both gesture selection and action selection on a single page. It turned out weird though, form label alignment is very challenging and looked bad, vertical space would quickly feel crammed as well. So if we don't want to stick to a comprehensive list of pre-defined gestures, which could be very long given modifier/window variants and diagonal swipe directions, then I'm convinced we need a separate form for selecting a gesture.
## Try your own hand
The code for these mock-ups is currently available in plasma/plasma-desktop, `work/jpetso/gesture-settings-mockup` branch. It's just good enough to show the content for these screenshots, but not in any way functional. I abused existing code from the "Mouse" and "Shortcuts" KCM, so those will be broken if you apply this branch. Perhaps a workable base for further UX experiments though.
Cheers and sorry for missing out on some of the discussion going forward, until I'm back in full swing!
issue