[KDE 6] Alt drag/zoom stops working after a short amount of time

[aidanhadley]

Describe the bug
When using RNote on KDE 6, when using alt to scroll or to zoom, it stop working after a short amount of time, instead making a stroke instead

To Reproduce
Steps to reproduce the behavior:

1. Install RNote and KDE 6
2. Stay on draw mode
3. use ALT to drag round
4. Observe the behaviour

Expected behavior
It shouldn't stop scrolling/zooming.

Console Output
Run the app in the terminal with flatpak --env=RUST_LOG=rnote=debug --env=RUST_BACKTRACE=1 run com.github.flxzt.rnote
and post the output.

$ flatpak --env=RUST_LOG=rnote=debug --env=RUST_BACKTRACE=1 run com.github.flxzt.rnote

Note that the directories 

'/var/lib/flatpak/exports/share'
'/home/ldas/.local/share/flatpak/exports/share'

are not in the search path set by the XDG_DATA_DIRS environment variable, so
applications installed by Flatpak may not appear on your desktop until the
session is restarted.

2024-04-08T18:58:10.782024Z DEBUG rnote: .. tracing subscriber initialized.

(rnote:2): Gtk-CRITICAL **: 14:58:23.888: The layout manager of type GtkBoxLayout 0x59b50ebc2080 does not create GtkLayoutChild instances

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: EndeavourOS/Arch Linux
• App Version: 0.10.2 (?)
• Installation Source: AUR
• Desktop Environment: KDE 6 (latest)
• Display Server: Wayland
• Input Source: External Tablet + keyboard

Let me know if I've missed anything

[Doublonmousse]

You're using alt + pen to drag around right ? Can reproduce on windows, seems like it has to do with event handling.
The weird thing is that ALT is still there and seemingly detected and I'm not yet sure what's the reason the behavior suddenly changes.

Testing the other alt + drag possibility (alt + mouse and alt + touch with touch drawing enabled doesn't seem to have the same issue).

[aidanhadley]

yes, i'm using alt+pen, as well as alt+shift+pen. tested it on SwayWM and it has the same issue. probably, as you said, event handling issues.

[camradex]

same issue, disabling windows ink fixes the problem but i lose pressure sensitivity, as mentioned on the duplicate issue

OS: Win 11
App Version: 0.10.2
Installation Source: .exe
Input Source: External Graphics Tablet + keyboard

[mrWinston]

Same issue for me, but i'm on linux, not windows:
OS: Fedora 40, Wayland, Hyprland
App Version: 0.11.0
Installation: Flatpak
Input Source: External Tablet + Keyboard

Is there anything I can do to help debug the issue further? I'm able to run rnote from source as well and could collect some additional debug information if needed.

[Doublonmousse]

@mrWinston The relevant code is here :

rnote/crates/rnote-ui/src/canvaswrapper.rs

Lines 496 to 525 in 2e0d148

	self.canvas_alt_drag_gesture.connect_drag_begin(clone!(
	@strong offset_start,
	@weak obj as canvaswrapper => move |gesture, _, _| {
	let modifiers = gesture.current_event_state();
	// At the start BUTTON1_MASK is not included
	if modifiers == gdk::ModifierType::ALT_MASK {
	gesture.set_state(EventSequenceState::Claimed);
	offset_start.set(canvaswrapper.canvas().engine_ref().camera.offset());
	} else {
	gesture.set_state(EventSequenceState::Denied);
	}
	}));
	self.canvas_alt_drag_gesture.connect_drag_update(
	clone!(@strong offset_start, @weak obj as canvaswrapper => move |_, offset_x, offset_y| {
	let canvas = canvaswrapper.canvas();
	let new_offset = offset_start.get() - na::vector![offset_x, offset_y];
	let widget_flags = canvas.engine_mut().camera_set_offset_expand(new_offset);
	canvas.emit_handle_widget_flags(widget_flags);
	})
	);
	self.canvas_alt_drag_gesture.connect_drag_end(
	clone!(@weak obj as canvaswrapper => move |_, _, _| {
	let widget_flags = canvaswrapper.canvas().engine_mut().update_rendering_current_viewport();
	canvaswrapper.canvas().emit_handle_widget_flags(widget_flags);
	}),
	);
	}

Line 501 may be why though. For subsequent drags BUTTON1_MASK may be part of it or there might be other modifiers getting in here.

Maybe changing to something like gdk_modifiers.contains(gdk::ModifierType::ALT_MASK) but not containing shift or variations of this (alt + shift + drag is also another operation).

And looking at the same time at traces from crates/rnote-ui/src/canvas/input.rs for catching inputs that wouldn't be captured by the GestureDrag part.

I haven't looked in detail on why this happen.
It's kinda working weird and in different ways depending on the target OS as well. On windows alt + pen drag can start drawing when changing the pressure applied.

[esensar]

I have tried to fix this, but failed. I gathered some information though. The drag gesture itself seems to end (at least in my case and changing it to gdk_modifiers.contains doesn't help, because the modifier is there the whole time). What I noticed is that I get an additional event right after I start dragging, which seems to be detected as a mouse event (it has a different device ID and it has pressure set to exactly 0.5) - not really sure if this helps, it also seems like this might be an issue on the OS side, or maybe GTK side.

Not sure if this helps at all, but when I tap on the titlebar, next drag works as expected and after that it gets broken again until I tap on the titlebar again.

[Doublonmousse]

Yeah, this is helpful (if it resets on clicking on the titlebar, it's possible there's some internal gtk state that becomes inconsistent causing the issue and tapping on the titlebar resets the gesture which is tied to the canvas).

Logging all signals related to the gesture, I get

  15.049325900s DEBUG rnote::canvaswrapper::imp: cancel signal, None
  15.049439100s DEBUG rnote::canvaswrapper::imp: end signal, None
  15.049513800s DEBUG rnote::canvaswrapper::imp: connect drag end
  15.049684200s TRACE rnote::canvas::input: canvas event MotionNotify - gdk_modifiers: ModifierType(ALT_MASK), is_stylus: false
  15.049760100s TRACE rnote::canvas::input: element=Element { pos: [[515.0, 560.0]], pressure: 0.5 } pen_state=Proximity modifier_keys={KeyboardAlt} pen_mode=None event_time_delta=0ns msg="handle pen event element"
  15.058772100s TRACE rnote::canvas::input: canvas event MotionNotify - gdk_modifiers: ModifierType(ALT_MASK | BUTTON1_MASK), is_stylus: true

The MotionNotify seems weird with its is_stylus: false status. There are some additional ButtonPress/ButtonRelease events that can happen when using the stylus (for proximity in/out and transition to and from hovering to pressing on the screen) but they all have the stylus attribute to true, and same for the MotionNotify.

@esensar Are you on linux ? On windows the behavior is somehow different (the alt gesture always starts correctly but stops at some point and the stylus begins writing instead).
I can push a flatpak nightly with the additional log for the gesture if you want, chances are this event happens in the same place (just after the drag end : the gesture probably stops there and the event that caused it is propagated further)

[esensar]

Yeah, this is helpful (if it resets on clicking on the titlebar, it's possible there's some internal gtk state that becomes inconsistent causing the issue and tapping on the titlebar resets the gesture which is tied to the canvas).

Logging all signals related to the gesture, I get

  15.049325900s DEBUG rnote::canvaswrapper::imp: cancel signal, None
  15.049439100s DEBUG rnote::canvaswrapper::imp: end signal, None
  15.049513800s DEBUG rnote::canvaswrapper::imp: connect drag end
  15.049684200s TRACE rnote::canvas::input: canvas event MotionNotify - gdk_modifiers: ModifierType(ALT_MASK), is_stylus: false
  15.049760100s TRACE rnote::canvas::input: element=Element { pos: [[515.0, 560.0]], pressure: 0.5 } pen_state=Proximity modifier_keys={KeyboardAlt} pen_mode=None event_time_delta=0ns msg="handle pen event element"
  15.058772100s TRACE rnote::canvas::input: canvas event MotionNotify - gdk_modifiers: ModifierType(ALT_MASK | BUTTON1_MASK), is_stylus: true

The MotionNotify seems weird with its is_stylus: false status. There are some additional ButtonPress/ButtonRelease events that can happen when using the stylus (for proximity in/out and transition to and from hovering to pressing on the screen) but they all have the stylus attribute to true, and same for the MotionNotify.

I have noticed similar logs in my case too. It looks as if a mouse event was triggered right at the start and that it stops the gesture. I have no idea how GTK handles these as I haven't really worked with it, but I can see that no events are logged from canvas when the gesture works (because gesture captures these events before they reach canvas I guess?).

@esensar Are you on linux ? On windows the behavior is somehow different (the alt gesture always starts correctly but stops at some point and the stylus begins writing instead). I can push a flatpak nightly with the additional log for the gesture if you want, chances are this event happens in the same place (just after the drag end : the gesture probably stops there and the event that caused it is propagated further)

I am on Linux. I see this issue when I am on Sway WM. I have also tried GNOME and it works as expected there.

The only reliable way to get it to work has been to tap on the titlebar before always, so I guess there might be some issue with the canvas taking focus, or something like that (again, I haven't worked with GTK, so I am just guessing 😄 ).

[Doublonmousse]

I'm running with

GDK_DEBUG=EVENTS RUST_LOG=TRACE rnote

on windows and I see on windows a

WM_SYSKEYDOWN ... Alt ch:12 KF_REPEAT KF_ALTDOWN sc:56 rep:1
 ==> GDK_KEY_PRESS ...
TRACE : canvas event key pressed - gdk_key: Key(65513),
DEBUG : connect drag end
TRACE canvas event MotionNotify - gdk_Modifiers: ModifierType(0x0), is_stylus: false
TRACE element : Element { pos : [], pressure: 0.5} ...

Can you check with GDK_DEBUG=EVENTS if you see something similar on linux. On window it seems to be that an event can be sent when the alt key stays pressed, which somehow creates a MotionNotify event with an is_stylus:false value and cancels the grab event this way (which doesn't happen with a mouse because it's always is_stylus:false)

[esensar]

It is similar, it does not send the event with is_stylus:false right away, but it is sent shortly after starting the drag:

Allocating a new GdkButtonEvent for event type GDK_BUTTON_PRESS
tablet frame, time 11557215
2025-02-27T20:46:19.609885Z TRACE rnote::canvas::input: canvas event MotionNotify - gdk_modifiers: ModifierType(ALT_MASK), is_stylus: true
2025-02-27T20:46:19.609893Z TRACE rnote::canvas::input: handle pen event element - element: Element { pos: [[234.0390625, -73.32814025878906]], pressure: 0.0 }, pen_state: Proximity, event_time_delta: 14ms, modifier_keys: {KeyboardAlt}, pen_mode: Some(Pen)
2025-02-27T20:46:19.609899Z TRACE rnote::canvas::input: handle pen event element - element: Element { pos: [[233.87109375, -69.96095275878906]], pressure: 0.0 }, pen_state: Proximity, event_time_delta: 8ms, modifier_keys: {KeyboardAlt}, pen_mode: Some(Pen)
2025-02-27T20:46:19.609901Z TRACE rnote::canvas::input: handle pen event element - element: Element { pos: [[234.71484375, -60.53126525878906]], pressure: 0.0 }, pen_state: Proximity, event_time_delta: 0ns, modifier_keys: {KeyboardAlt}, pen_mode: Some(Pen)
frame 0x55b4240e2100
tablet motion 332.070312 356.472656
Allocating a new GdkMotionEvent for event type GDK_MOTION_NOTIFY
tablet tool 1 pressure 25244
tablet frame, time 11557223
frame 0x55b4240e2100
tablet motion 334.089844 361.355469
Allocating a new GdkMotionEvent for event type GDK_MOTION_NOTIFY
tablet tool 1 pressure 29931
tablet frame, time 11557229
frame 0x55b4240e2100
Allocating a new GdkMotionEvent for event type GDK_MOTION_NOTIFY
2025-02-27T20:46:19.628644Z TRACE rnote::canvas::input: canvas event MotionNotify - gdk_modifiers: ModifierType(ALT_MASK), is_stylus: false
2025-02-27T20:46:19.628650Z TRACE rnote::canvas::input: handle pen event element - element: Element { pos: [[293.32421875, -36.61720275878906]], pressure: 0.5 }, pen_state: Proximity, event_time_delta: 0ns, modifier_keys: {KeyboardAlt}, pen_mode: None
frame 0x55b4240e2100

[Doublonmousse]

I've opened an upstream issue on gtk4's gitlab https://gitlab.gnome.org/GNOME/gtk/-/issues/7383

Edit: making good progress on this, at least where the issue originates in gtk4's code.
Can someone test on linux as well? The modification would be to debug print the button information when the if in the linked code is triggered
https://gitlab.gnome.org/GNOME/gtk/-/blob/main/gtk/gtkgesturesingle.c?ref_type=heads#L228-238 and see if it happens at the same time as the gesture getting cancelled.