GNOME Bugzilla – Bug 778328
Pressure sensitivity lost after removing pen from screen (Surface Book/MyPaint)
Last modified: 2017-02-19 07:21:19 UTC
Created attachment 345175 [details] MyPaint Screenshot showing displacement and loss of pressure sensitivity and weird stroke behaviour -First time submittal, I'll try my best.- I am using MyPaint on the Surface Book. For general Pressure Sensitivity I have the necessary WinTab driver and C++ Redistribution 2010 installed, both for 64-bit Windows version. The pen is recognised and has continuous Pressure Sensitivity in Gimp and Krita. In MyPaint, however, the pressure sensitivity gets lost after I remove the pen from the screen after the first stroke/input on canvas. The first stroke is fine. It is to note, that MyPaint still recognises the pen as stylus afterwards. Additionally, the position of the stroke drawn and the point of input have quite a distance. Shifting to the right and slightly lower. See attached image. But this may be a separate bug which I'd be happy to report separately. This issue has been discussed beforehand here: https://www.bountysource.com/issues/28004023-surface-pro-3-4-n-trig-hardware-pressure-sensitivity-stops-working-after-moving-pen-away-from-screen I attach two screenshots of MyPaint showing how it looks like, as well as some, forgive me, debugging stuff I tried to do with MSYS2 and the GTK3 demo (as was recommended in the link above). Please let me know if I need to add more information along with a little help as to where I would find said info. I appreciate any help and thanks so much for your time! :) Kind Regards, Jannie (Windmaedchen)
Created attachment 345176 [details] MyPaint recognising the pen properly
Created attachment 345177 [details] Report from MSYS2 I've copied everything until a few lines of apparent movement tracking.
MyPaint dev here. We think this is a bug in GTK3, given the apparent regression against what GIMP does (GTK2 still, I assume). The little grey rectangles around the drawn line are an unrelated HiDPI MyPaint bug: this bug is purely a GDK-level input mishandling. This is MyPaint v1.2.1 (our most recent release, and the first to ship with a bundled GTK3-demo). Its bundle uses GTK v3.22.7. I'm willing and able to do MSYS2 package builds of stable (3.22.x) GTK patched with whatever the GTK devs want, and distribute them via https://filebin.net/. I co-maintain https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-gtk3-git which is what I would be basing the builds on. However I do not own the affected hardware. In the past, throwing --gdk-debug flags at gtk3-demo has proven useful for sorting though silly input values. If a GTK dev wants to tell us what to use there, that would be good. ---------------- Windmaedchen: it would be useful to confirm whether this is triggered by the end of a simple pendown-draw-penup sequence, or by a proximity-out message. Proximity events happen for devices that let you "hover" the cursor when the pen is near the screen or tablet. Please let us know if you see a difference between 2 strokes when the pen is kept real close to but not touching the screen between them, and 2 strokes when the pen is taken far far away between them.
I've been reminded on IRC about https://gist.github.com/achadwick/54e5b111e5ac908becf1 which is some test code I wrote for debugging a different bug. It draws cute graphs of the tablet axis data that GTK sees. It may be helpful for discovering stuff about discontinuities and device changes (or not...) To use it: click the [Raw] button and save the into axischart.py inside your MSYS2 install's tmp. Then run python2 /tmp/axischart.py from a MINGW64 or MINGW32 shell, as appropriate.
(In reply to Andrew Chadwick from comment #3) > Windmaedchen: it would be useful to confirm whether this is triggered by the > end of a simple pendown-draw-penup sequence, or by a proximity-out message. Provided I stay close enough to the screen (about 2-3cm) the pressure sensitivity isn't lost inbetween strokes. I have to remove the pen more than that in order to loose pressure sensitivity (and the ability to draw in an actual stroke and not this weird mess of lines). Would you need any "proof" or is my word enough?
(In reply to Windmaedchen from comment #5) No, that's fine. That confirms nicely that you're seeing https://github.com/mypaint/mypaint/issues/484, and it makes it even clearer that it's the proximity (in/out) events that are wrecking subsequent motion events (not necessarily just ones with pressure). Thank you! Looks like the bit of code that handles it is https://github.com/GNOME/gtk/blob/gtk-3-22/gdk/win32/gdkdevicemanager-win32.c#L1199 It may be instructive for us to compare with the equivalent bit in the working GTK2-era code: https://github.com/GNOME/gtk/blob/gtk-2-24/gdk/win32/gdkevents-win32.c#L3589 It looks different, sure. But I don't understand the difference in any detail D:
Created attachment 345261 [details] AxisChart Python Thing Result As suggested by Andrew Chadwick. "I've been reminded on IRC about https://gist.github.com/achadwick/54e5b111e5ac908becf1 which is some test code I wrote for debugging a different bug. It draws cute graphs of the tablet axis data that GTK sees. It may be helpful for discovering stuff about discontinuities and device changes (or not...)"
According to the documentation[1], LOWORD (msg->lParam) == 0 when cursor leaves your context, and is some other value when it enters your context (HIWORD is the same, but for the whole hardware; not sure what it means in physical terms). My first impression is that the gtk3 code handles the "leaving" part (by switching back to the core pointer?), but does not handle the "entering" part. There's the WT_CSRCHANGE handling, which is sent when a new cursor enters your context - it switches off the core pointer and makes the non-core device the active one, it seems. My first uneducated guess is that WT_CSRCHANGE is sent only once, processed, wintab device is activated, then cursor leaves the proximity of the context, WT_PROXIMITY is sent, and the wintab device is switched off, and no more WT_CSRCHANGE messages are sent. I would suggest running gtk with a different set of debug flags to make it print out the messages it receives. Though i'm not sure whether GTK is aware of the WT_* names (it might very well just print message numbers instead). Since i don't have this kind of hardware, i can only speculate. If my guess is correct, then extending WT_PROXIMITY handling to make it activate the device again would fix the problem. [1] http://www.wacomeng.com/windows/docs/WacomWindevFAQ.html
LRN: Unfortunately I have no programming background. I am happy to do code-meddling but I'd need more specific instructions, please. Luckily, I sit in the midst of a nest of programmers every day and I am sure one of them is happy to guide me additionally.
Instructions like that would benefit everybody ☺ I've just written up a cheatsheet for making debug builds of GTK on MSYS2, to be used when somebody proposes patches. I'll try to keep it up to date. https://github.com/mypaint/mypaint/wiki/Building-GTK3-from-git-on-Windows
Windmaedchen: can you attach the output of [in a MINGW64 or MINGW32 shell] $ gtk3-demo --gdk-debug=INPUT,EVENTS --run=event_axes as well? No need to recompile anything just yet, you can do it with stock MSYS2 gtk3-demo. Just the bit around the first proximity-out until a few motions after the subsequent proximity-in, please! The EVENTS level of debugging is a little noisy. I'll try and do the same for my main tablet, which I know from the Linux side sends proximity stuff.
Created attachment 345362 [details] MSYS2 run=event_axes log I have left it as was. Even though I recognised the tons of just "here is my position" events I feared I'd remove something important... Hope this helps anyway. I also apologise for the gaps, I tried to copy/paste several times, it just came out like this.
This seems to support my hypothesis. However, i'm not sure how to handle WT_PROXIMITY correctly. The problem is that there are multiple cursors (theoretically; not sure how many you actually have) within the same context, and GDK treats each cursor as a separate "device". Googling suggests that cursor 0 is "puck", cursor 1 is "pen" and cursor 2 is "eraser". WT_PROXIMITY only has the context. It has neither serial number of a packet (which can be used to look up the cursor number), nor the cursor number itself. Also, googling suggests that multiple cursors could be in use simultaneously. I'm not even sure that GDK supports that. Thus it does not seem to be possible to know which cursor left the context proximity. We could probably attach a counter to _gdk_input_ignore_core and only switch it when the counter goes from 0 to 1 or from 1 to 0. My guess is that in practice it would mean that core pointer will be ignored as long as at least one cursor is still in proximity. However, the question remains: which cursor should be made active (in GDK terms) instead of core pointer when one cursor re-enters the proximity? I might be able to work around this by delaying the core pointer disabling until the next WT_PACKET, but this is getting kind of complicated...
Created attachment 345390 [details] Intuos5_WORKING.zip More WT_PROXIMITY captures from real hardware, this time from an Intuos 5 tablet on Windows 7. Details in the zipfile. I have been careful to exclude extraneous motions from other devices.
^ I should add: the Intuos 5 is *not* affected by this. Those captures are just for comparison with a working tablet.
Created attachment 345446 [details] [review] GDK W32: Handle drivers that do not send WT_CSRCHANGE after WT_PROXIMITY Some drivers don't do that (not sure whether that is the correct behaviour or not). Remember each WT_PROXIMITY with LOWORD(lParam) != 0 that we get, then look for a WT_CSRCHANGE. If WT_CSRCHANGE doesn't come, but a WT_PACKET does, assume that this device is the one that sent WT_PROXIMITY. This logic is slightly off when we have two devices and one leaves the proximity then re-enters the proximity, but the next packet we get is from the *other* device. This way we'll give the wrong device to _gdk_device_virtual_set_active(), although it's ambiguous as to which device *should* be "active" if two are used simultaneously. Another problem is that we, presumably, get WT_PROXIMITY for all devices, even disabled ones, but we only bump _gdk_input_ignore_core for enabled devices. As a result, it might be possible to put an enabled device into the proximity, then put a disabled device into the proximity, then remove the disabled device, which will result in _gdk_input_ignore_core being bumped back to 0 and system pointer being set active for the core pointer.
Created attachment 345460 [details] [review] GDK W32: Handle drivers that do not send WT_CSRCHANGE after WT_PROXIMITY (alternative) Also include fallback code to ensure that WT_PACKETs for an enabled device disable the system pointer, because WT_PROXIMITY handler might have enabled it by mistake, since it's not possible to know which device left the proximity (it might have been a disabled device). I *think* that this patch obsoletes attachment 345446 [details] [review], but i'm not sure. The changes here are somewhat more extensive, and i can't test either of the patches due to not owning a stylus-input device.
Temporary debug binaries of gtk3 for MSYS2 built from the stable branch (tag 3.22.7 + 46 git revisions at 8f01d6c5f4) plus the patch from attachment 345460 [details] [review]. I'll be testing these myself tonight, but I do not have a Surface Pro. https://filebin.net/lfkci1x3cgri60fm Install them into an MSYS2 installation with: $ pacman -U mingw-w64-*-gtk3-git-3.22.7.46.g8f01d6c5f4.345460-1-any.pkg.tar.xz They conflict with and will replace the non-git version from the MSYS2 repositories. You will be asked about this, say yes when prompted. To uninstall (revert to the repository stable version) with: $ pacman -Sy $ pacman -S mingw-w64-{i686,x86_64}-gtk3 SHA256SUMS below, signed. -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 e61c955cf4b85f1a4e8148a2e28abc1903e4b3a2a5f10cabc8306b338badb974 *mingw-w64-i686-gtk3-git-3.22.7.46.g8f01d6c5f4.345460-1-any.pkg.tar.xz 1e0bcc584873663c719bc897f3ba5dab53b233293c8d204885f7a2b3358c93d6 *mingw-w64-x86_64-gtk3-git-3.22.7.46.g8f01d6c5f4.345460-1-any.pkg.tar.xz -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEwCOR9Luk8OKyfGv/bjA34Sh4spkFAlid73wACgkQbjA34Sh4 spmMbw/+Ky2IsULUmQqHtoOwglu1+JeHSkAX+WgDcUleZPvUBMnr6qjVI0h4rOHA GbRGIrG/3v8dAFAL6gWPjhfiCGgvV74IFJ7+VD9I/7XL5eGP4IxYatgj176K9Z+5 HeamXnmcCOe7PV4fP2sAhP5UoGWfP0Vgd4OE/zk1ISQwNGpZIbSV85HHoLbPTB/V +6xNNlb44lU82oWwGvZrUSxJerdwybh2sRjKSFJfPcRQoUf3VNNVKJL4O2MkRXOl Sz7bAKgltUh/Wn7IDIwubIwNIRWY85g8bJv2ODf5mgqZkmiNmvEmXGaxpDzJLIcO obDxY/NjG3PIUDZDTZraF7Ta+opube5tqIt1hmA3CrULDWYFs+hxAZ/sqHM6BEnq 7GGuevdEnUAZrReNfTw4Vbbxu9dJH2pgsoWv3qzXItOxFxaLKGy0RK6yxVwPwivu G9nMflddhxSMdJpxOgqQsuL3dsRGQJ9QhgbJ4RO4s/Zqk4EumjRlHgwZooYxNy8H L1buTMNnwmlGk+9SBpKKycSos1l+LRyvoavxIBxIQ87tWN/hS6Kgdx+gep2meoGc IE40Io441kcojpFRbglJKzuL61O7wKu3i7/LoKfXD2CumdCoxRHOFSze8veGQ3ll 4wrBnqpuEzwCfMFt+HdgUEcugx4oZH2PaOurESVuovMWeIPB14k= =r7Pv -----END PGP SIGNATURE-----
Created attachment 345474 [details] MSYS2 Event log with patch345460 Same event log as before, this time with the patch from attachment 345460 [details] [review]. (Applied as Andrew in comment #18 described. This time I left the list of keyboard buttons out.
Created attachment 345475 [details] Axis Chart Image with patch345460 While I couldn't do anything with the event log, I like this sight! I moved the pen out of proximity several times (>10cm) and still had pressure afterwards. At least according to this graph.
(In reply to Windmaedchen from comment #20) > I like this sight! I moved > the pen out of proximity several times (>10cm) and still had pressure > afterwards. At least according to this graph. I'm not sure whether you mean "this fixes the bug", or "this does not fix the bug", or "this fixes the bug, but there are other issues that prevent the pressure from working"?
(In reply to LRN from comment #21) > I'm not sure whether you mean "this fixes the bug", or "this does not fix > the bug", or "this fixes the bug, but there are other issues that prevent > the pressure from working"? Apologies. I like seeing pressure where there wasn't before, but Andrew mentioned in an email to me that the result seen in this graph may not be conclusive = I am nor sure if it is fixed. Also, it doesn't work with my regular installed MyPaint, although that is probably because that isn't actually patched. Andrew: So, can I open MyPaint via MSYS2 somehow? To see this patch in action? In your instructions of comment #11, when that EVENT Windows opens, I can't actually see anything when making any pen input (no red blob or anything), so I am not yet really "seeing" any fix from my artist point-of-view.
The image in attachment 345475 [details] looks like a "this fixes the bug" to me. Smooth and continuous X and Y during pressure>0 motions, even after WT_PROXIMITY out/in pairs. And the existence of pressure>0 *at all* after the first proximity-out/in pair is a really good sign. Windmaedchen are the blobs in "gtk3-demo --run=event_axes" still responding to pressure after taking the stylus out of range and back in a few times? I can confirm that my Intuos setup still works after this patch, under 64-bit Windows 7. I've asked via the @MyPaintApp twitter to try and get more affected users testing. Also in the downstream report.
(In reply to Windmaedchen from comment #22) > Andrew: So, can I open MyPaint via MSYS2 somehow? To see this patch in > action? In your instructions of comment #11, when that EVENT Windows opens, > I can't actually see anything when making any pen input (no red blob or > anything), so I am not yet really "seeing" any fix from my artist > point-of-view. Absolutely! Alexey has started doing builds of it now, starting with 1.2.1, so with the "gtk3-git" packages installed, you should be able to do [in either the MINGW32 or the MINGW64 shell/environment] $ pacman -S mingw-w64-{i686,x86_64}-mypaint # install both builds from the repo $ mypaint -c /tmp/cleanconfig # for a clean config you can delete later $ mypaint # to use the config in your Windows user profile then compare it with the situation without the patched gtk3-git packages and normal unpatched gtk3 installed. This is an independent copy of MyPaint that's stored entirely under your c:\msys32 or c:\msys64 area.
Created attachment 345479 [details] Screenshot PatchedMyPaint via MSYS2 This looks good! The patched version via MSYS2 gives me pressure sensitivity even after leaving proximity several times. In comparison, my normal install of MyPaint still has the problem as before. Andrew: Oh, my mistake! In the Event Window of "gtk3-demo --run=event_axes" I CAN see the red blob and it reacts nicely to pressure input, even after removing the pen. It was just out of view due to the displacement of input point on screen and appearing stroke on the canvas. I, too, think this fixes the issue of losing pressure sensitivity. Yet, the issue of the displacement remains.
(In reply to Windmaedchen from comment #25) > > Yet, the issue of the displacement remains. This is indeed a separate issue, I think down to https://git.gnome.org//browse/gtk+/tree/gdk/win32/gdkdevicemanager-win32.c#n1014 not scaling down the x/y coordinates on hidpi setups. It is probably better to handle it in a separate bug.
I would very much like to open another bug then, for even if THIS bug is as good as fixed, I still can't use MyPaint. Would here be the right place for this? Can I just copy the information (product, component, version, etc.) I have given for this bug or does it go somewhere else? Anything I can make better with my second bug report?
(In reply to Windmaedchen from comment #27) > I would very much like to open another bug then, for even if THIS bug is as > good as fixed, I still can't use MyPaint. > > Would here be the right place for this? Can I just copy the information > (product, component, version, etc.) I have given for this bug or does it go > somewhere else? Anything I can make better with my second bug report? This one was just perfect :). The bug information would indeed be the same, and in the bug description you can just mention "Originally reported together with bug #778328" to avoid providing further screenshots or the same detailed info again.
(Regarding the offset/displacement) I've got my eye on the existing bug 776883, but that is a multi-monitor scale+offset thing. Also tablet-only. Fixing this bug may also have fixed 745697, or have proven it to be a duplicate. I'm asking the reporter of that bug to do some more digging. I've been playing around and looking through old MyPaint reports, and think I've managed to reproduce something like this offset for a single monitor by tweaking GDK_SCALE. It looks like a productive way forward. I'll report back with a separate bug number so that people can Cc: themselves onto it. --------- Windmaedchen: echoing Carlos here. This is the perfect level of detail and testing feedback. Thank you! Thanks also to LRN and the rest of the GTK team for a lovely, fast fix. LRN/everyone: do we need to test the maybe-obsoleted earlier patch as well, or are you happy to go ahead and merge?
IMO this should be merged for stable and for gtk4, if it's good and that's possible. To date, all testing has taken place on stable only, due to build issues on Win32.
(In reply to Andrew Chadwick from comment #29) > LRN/everyone: do we need to test the maybe-obsoleted earlier patch as well, > or are you happy to go ahead and merge? Probably not. If the alternative patch works (and does not break anything), then that's what we're going to use. The difference is mostly about the handling of multiple devices (AFAIU it means a tablet with a stylus and a puck; AFAIU, "inverted stylus" and "stylus" are mutually-exclusive) and the handling of multiple devices when one of them is programmatically disabled by the application (it is done at GDK level, so the driver will keep sending us messages about that device). This is highly theoretical at this point, as none of us has the hardware to test this code. Another untested configuration is a system which has multiple tablets connected. Speaking of which, i've noticed that all wintab devices are given GDK_SOURCE_PEN type. There's a GDK_SOURCE_ERASER type, which we might be able to detect and assign, if anyone's interested. Not sure what the impact of that will be.
(In reply to Andrew Chadwick from comment #30) > IMO this should be merged for stable and for gtk4, if it's good and that's > possible. To date, all testing has taken place on stable only, due to build > issues on Win32. That's the reason why i'm only merging anything into gtk-3-22. Most likely, there will be a moment when master is stable enough and we'll have to forward-port every useful patch from gtk-3-22 to master. At least that's what i'm going to do.
(The offset issue is being handled in bug 778508) (In reply to LRN from comment #32) > we'll have to > forward-port every useful patch from gtk-3-22 to master. At least that's > what i'm going to do. LRN: Thanks. I'll keep an eye on https://git.gnome.org/browse/gtk+/log/?h=gtk-3-22 What mechanisms exist to make sure it doesn't get forgotten for master, out of interest? (In reply to LRN from comment #31) > Probably not. If the alternative patch works (and does not break anything), > then that's what we're going to use. The difference is mostly about the > handling of multiple devices (AFAIU it means a tablet with a stylus and a > puck; AFAIU, "inverted stylus" and "stylus" are mutually-exclusive) and the > handling of multiple devices when one of them is programmatically disabled > by the application (it is done at GDK level, so the driver will keep sending > us messages about that device). This is highly theoretical at this point, as > none of us has the hardware to test this code. > > Another untested configuration is a system which has multiple tablets > connected. As far as I understand it, there can only be one manufacturer's wintab.dll on the system. However, it could be multi-tablet aware, so that might not be a guarantee. Sadly I don't have more than one Wacom in my testing stash. > Speaking of which, i've noticed that all wintab devices are given > GDK_SOURCE_PEN type. There's a GDK_SOURCE_ERASER type, which we might be > able to detect and assign, if anyone's interested. Not sure what the impact > of that will be. It'll be good for art apps like MyPaint, but it isn't essential. Although MyPaint spots GDK_SOURCE_ERASER devices, it also looks for /eraser/ in the device's name when it needs to make an initial choice of brush definition for an unfamiliar device. After that, we handle it like any other unique device by associating it with its last-used bush definition.
(In reply to Andrew Chadwick from comment #33) > (The offset issue is being handled in bug 778508) > > (In reply to LRN from comment #32) > > we'll have to > > forward-port every useful patch from gtk-3-22 to master. At least that's > > what i'm going to do. > > LRN: Thanks. I'll keep an eye on > https://git.gnome.org/browse/gtk+/log/?h=gtk-3-22 > What mechanisms exist to make sure it doesn't get forgotten for master, out > of interest? > None. I'll just find the point where gtk-3-22 was branched off from master, and will go over all commits in gtk-3-22 since that point.
Created attachment 345562 [details] Eraser-Behaviour I just wanted to add something, now that I see you mention the ERASER... With both, patched and unpatched/stable GTK3 version, the eraser side of my pen has constant set, unchangeable pressure and properly working x/y axes, no matter how often I leave proximity and switch between pen and eraser sides. On the image you can see the (cute) graph with the stable, unpatched GTK3 version. First input is the pen, second as well but x/y is broken and then comes the eraser side, several times, finishing with the pen again. It looks the same on the patched version, for the eraser. I don't use the eraser side at all, so I didn't check in my previous tests.
If you have any access to alternative (not based on GTK/GDK) pieces of software that handle pressure-sensitive pen input, could you please try the eraser there and see if it is actually pressure-sensitive (maybe it just works that way...)? Either way, for the eraser sensitivity problem file a separate bug.
Googling suggests that "SP4" pen (whatever that is; this is what i get by googling "Surface Book") has no pressure sensitivity on eraser (it is said that previous pen generation had no eraser on the opposite end; it had a button instead, which could be used to transform the normal pen tip into an eraser, with the usual pressure sensitivity). So - please confirm that your eraser is actually pressure-sensitive before filing a but about that.
Review of attachment 345460 [details] [review]: Note positive test results with the affected hardware at comment 25. Is this good to go (I think so!)
Comment on attachment 345460 [details] [review] GDK W32: Handle drivers that do not send WT_CSRCHANGE after WT_PROXIMITY (alternative) attachment 345460 [details] [review] pushed into gtk-3-22 branch as commit f9df0fc94c7b83f598b1dabedf5b45e09c0c6c47