GNOME Bugzilla – Bug 755183
Mouse/touchpad protocol enhancements
Last modified: 2021-06-10 15:05:44 UTC
I'm using a touchpad all the time, and I think mouse/touchpad handling really deserves the following updates. We should cooperate with Thomas and others to come up with a reasonable extension. (Unfortunately there's no public bugtracker for xterm, so I'm filing it here, lest we forget it, and perhaps for a first round of comments before we reach out to him.) 1. Smooth scroll reporting. On normal screen (when dragging the scrollbar), or on alternate screen with no mouse reporting (e.g. "less" when we convert the wheel to Up/Down keypresses) we already recognize smooth movements and send events one by one, rather than multiple of them at once after reaching 1 old unit of scrolling. This is, however, not true for apps that handle the mouse themselves, e.g. mcview. Here scrolling is rough, by 3-4 lines at a time. The mouse protocol should be extended, so that if an app requests it, we should be able to report finegrained motion. 2. Horizontal scrolling. We have vertical, why should also have horizontal just as equally. (Actually, when running "less" where we synthesize keypresses, we could already do this on our own without extending the protocol. But of course I'd also like mcview and friends to have this feature.) 3. Selective events. For some simpler apps (e.g. nano-like text editors) it might make sense to request scroll events, yet ignore clicks and let the terminal handle them the way it wants to. 4. Offscreen values. Pointed out here: http://sourceforge.net/p/joe-editor/patches/111/ an app should be able to recognize when the mouse is being dragged out of the window while its button is held down. This is so that they can handle somehow when the mouse reaches the top or bottom row (e.g. highlight the text), and handle it differently when it moves beyond (e.g. start scrolling). Okay, the 4th one is not about touchpad, but the first three are. A decade or two ago the mouse wheel was something new and special (let alone touchpads with an easy way of scrolling), something extra compared to the two or three buttons. Nowadays, with laptops with large and good quality touchpad, and with tons of content to consume, I feel that precise scrolling with two fingers became perhaps an even more basic operation that clicking. But at the very least a first class citizen along the lines of clicking or pressing a key. We should treat it accordingly.
Agreed on all 4 points.
5. Finer grained coordinates. At least double precision is important for decent highlighting and (drag-n-)dropping experience implemented by an app. When clicking on an button in an ncurses app, or using a UI like mc's main screen, or double-clicking to select a word, what you're interested in is which cell the click occurs in, this is what we have now. When highlighting text, or rearranging text by drag-n-dropping in an editor, however, what we're interested in (horizontally) is which boundary between two letters is the closest. E.g. the text is "abcdef" and you want to select "cdef". You should do it by trying to click between b and c (rather than somewhere over c), which might happen to fall on either the right side of b or the left side of c. Either one should begin selecting "cdef". Clicking on the right side of c should begin selecting "def" just as the left side of d, etc. That's how it works on every UI, and even in vte's selection (when the app doesn't know about it). Apps should receive enough information to be able to implement this behavior too. (Note: This probably makes the most sense if you're using I-Beam cursor rather than rectangle/underscore.) (As a bonus, double precision vertically would allow applications to have a one-liner number entry box followed by up+down arrows in a single charcell; too bad the ideal Unicode character for that (two triangles above each other) doesn't seem to exist.)
6. If you scroll from activities overview to get to previous or next workspace, it's way too fast and inaccurate. 7. For scrolling to be really smooth & natural, the scrollable panel should not be sticked to your fingers. Try in any Android app and drag quickly in one direction and then release your finger; the panel keeps scrolling based on the strength of your movement and then it stops later. I was expecting to have the same experience when two-finger-scrolling with my laptop, but window sticks to fingers when you release them.
9. Pinch in/out zoom, everywhere, for touchpads that support it.
8. As per bug 769696, mouse-aware terminal applications should receive enough information to be able to scroll exactly following the fingers on a touchscreen device.
10. The mouse button and the keyboard modifier should be encoded in separate parameters, see https://gitlab.gnome.org/GNOME/vte/issues/93 for rationale.
11. And there shouldn't be an off-by-one jump (the value of 3 encoded as 2, the value of 4 encoded as 4 – see at the same link again).
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/vte/-/issues/2226.