Enable Gboard CJK keyboards

[knyipab]

Relates to #115 and #147.

Add a preference option "Workaround to enable CJK Gboard".

• The issue of Gboard IME showing up as QWERTY affects mostly only CJK keyboards.
• Tested on my Samsung Z Fold5, Android 14. Gboard 14.1.04.621126403-release-arm64-v8a.
• It is made as a preferences.xml option because the impact on other IMEs is not well known and there may be side effects. API to detect current IME in use InputMethodManager#getCurrentInputMethodInfo() is not available until recent API level (34).

[knyipab]

Incorrect text UTF8 text sometimes sent to X clients. Not a bug of this commit, instead there are likely some problems in mInjector.sendTextEvent(). Lemme investigate and will reopen after fixing.

[knyipab]

Cannot reproduce the aforementioned bug after re-runtermux-x11 command in Termux. In addition, it's not a bug of this PR, so I re-open this PR.

Only for record: as far as I can tell, there may be bug in XCB receiver side. Log by Java_com_termux_x11_LorieView_sendTextEvent shows that correct character is sent. However, incorrect characters shown in XFCE DE is some characters that was typed before. I tested that termux-x11 session for more than a day and perhaps changes some internal state of XCB receiver side, causing the bug. Anyways, I can't reproduce that at the moment.

[knyipab]

Sorry for confusion. I will stop editing here and look for @twaik your comments on this "Workaround to enable CJK Gboard" option.

More testing are done as below.

Device	Android Ver.	GBoard Ver.	LangTag detection to activate CJK Gboard
Samsung Z Fold5	14	14.1.04.621126403-release-arm64-v8a	Working
Samsung S21	14	14.0.11.612796517-release-arm64-v8a	Working
Pixel 4a	13	14.0.11.612796517-release-arm64-v8a	Not working, blank getLanguageTag()
Pixel 4a	13	14.1.04.621126403-release-arm64-v8a	Not working, blank getLanguageTag()
Emulated (AVD Pixel 4)	13	14.1.04.621126403-release-x86_64	Not working, blank getLanguageTag()
Emulated (AVD Pixel 8)	14	12.4.05.482060964-preload-x86_64	Not working, blank getLanguageTag() and requestCursorUpdates() not even called
Emulated (AVD Pixel 8)	14	14.1.04.621126403-release-x86_64	Working

This workaround is by no mean elegant but probably the only existing workaround siince being last discussed in 2021 termux/termux-app#1539. Bit more context of the problem and welcome smarter workaround: the TYPE_TEXT_VARIATION_NORMAL can enable CJK, emoji and clipboard input from Gboard, but at the same time activates Gboard word suggestion causing backspace not working and the word suggestion can be suppressed through restartInput()/invalidateInput(), but does not work well for non-CJK Gboard such as English (when type fast and two keypresses commitText() so close in time, one of them will be ignored). Therefore, the getLanguageTag() detection is a workaround to set TYPE_TEXT_VARIATION_NORMAL only when it is needed (i.e. CJK IME).

Grateful for merging given importance to Gboard CJK users. I don't mind putting it onto an "Experimental" section of perefernces.xml. Thanks.

[twaik]

Relates to #115 and #147.

Not related.
That happened because Xwayland (used a long time ago) did not have any instruments to input custom Unicode symbols which are not in current keymap.

Only for record: as far as I can tell, there may be bug in XCB receiver side.

There is no "XCB receiver" in termux-x11. There is some custom light weight protocol to avoid XCB dependency (which adds 1MB+ for every architecture variant and 4.5MB to universal build).

However, incorrect characters shown in XFCE DE is some characters that was typed before.

Termux:X11 sends characters the same way tigervnc-server and x11vnc do. It maps characters to keycodes with no associated keysyms (

termux-x11/app/src/main/cpp/lorie/InputXKB.c

Lines 52 to 54 in 6fbd332

	static const KeyCode fakeKeys[] = {
	92, 203, 204, 205, 206, 207
	};

) and sends key event to application. This method is not really reliable but it works for most text editors and does not interfere with existing hotkey handling implementations.

I did not find better way to implement this, x11vnc way is the best option I found.

[knyipab]

Thanks. Since the bug is really not a problem of this PR and neither Gboard, I opened #625 for test result and further discussion.

Back to this PR, I tested together with #617 (which makes available the 3-finger swipe up gesture to activate soft keyboard in DeX mode) and commit another workaround for DeX mode in which InputConnection#requestCursorUpdates() is not called by Gboard at all.

Any adjustment to this PR you would like me to do before merging?

[813ethan]

any chance this would be merged? working perfectly for me with xfce4 for this branch with traditional chinese gboard

[knyipab]

twaik may have his/her own consideration.

After months of use, apart from the requirement of "Android 14 + Gboard 14" and the input character issue (which is unrelated to Gboard), there are found to be three issues but I haven't had time to look into and solve:

• Alt is sometimes locked when using non-English IME with hardware keyboard (DeX mode), switching back to English IME can temporarily resolve.
• Sometimes IME is not showing up nor working when phone screen OFF in DeX mode + hardware keyboard + onscreen soft keyboard. Can be resolved by turning on phone screen.
• Very rarely it can't detect IME subtype therefore not activate IME, the temporary solution is reopen Termux:X11

Still, I always think that it would be better to have option than none. I rather to write in the option description to tell user that there is buggy behaviour and may lock your Alt key.

[twaik]

Currently I am working on some other changes, I'll review it again a bit later. Also both this and #617 must be rebased (and the latter one has unresolved reviews).
Termux:X11 code has been pretty much changed since prs were opened.

[knyipab]

Rebased.

The Alt issue can be resolved by user going to Gboard settings to disable "Emoji with physical keyboard / Physical Alt to show emoji keyboard".

[knyipab]

Indeed, for those Gboard users who turn off suggestion and autocompletion for their English keyboard (and vaguely other letter-based IMEs), the lang subtype detection (mIMM.getCurrentInputMethodSubtype().getLanguageTag()) and mImeCJK would be unnecessary and LorieView can simply set InputType.TYPE_TEXT_VARIATION_NORMAL, and then users can enjoy voice input and emoji input. Edited: And that won't require Android 14 + Gboard 14.

However, I am not that kind of no-suggestion & no-autocompletion user. I am just saying it and see if you may want to extend an option for those users to force InputType.TYPE_TEXT_VARIATION_NORMAL.

Thanks for your time and consideration.

[twaik]

Pay attention to the review please.

[twaik]

Probably it can be fixed bytusing IBus methods. AFAIK it can let us send unicode symbols without mapping them to keys and avoid this weird behaviour.

[knyipab]

It seems to require ibus to be on the deps of termux-x11-nightly package and start an ibus daemon. Is that right?

Another info: previous test results in #625 show that the implementation tigervnc + realvnc viewer can be 100% reproducible and output is reliable, unlike Termux:X11's unpredictable weird output. (Grateful for letting me know if their approach has sacrificed any features, but) Their approach is almost perfect except that the support to some specific language characters is lacking, which doesn't seem to me to be an implementation limitation. Anything we can possibly learn from their approach?

[twaik]

It is pretty much weird, because I compared termux-x11's implementation with the original one and did not find any differences except function names, dropped upper-lower feature in lorieAddKeysym (because most likely there will be no difference), pressKey replaced by QueueKeyboardEvents, some logging stuff changed, dropped Shift+Alt mapping to META stuff. But generally they are equal.

[knyipab]

No expertise, perhaps about the frontend side (RealVNC) (??).

But I think clues can be found in the xev log files in #625: x11-zh-sc.log and vnc-zh-sc.log. One observable difference is that termux-x11 sends some events between each characters:

MappingNotify event, serial 51, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 203, count 1

MappingNotify event, serial 51, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 203, count 1

There may be more difference but I haven't probed into.

[twaik]

MappingEvent is event telling programs to associate some keycode (one in this case) to specific unicode symbol.

[knyipab]

Nonetheless, seems that no MappingNotify (but MapNotify) is seen in the log of tigervnc. Where is MappingNotify produced in the termux-x11 code base? I can remove it and test.

[twaik]

Main entry point for unicode input is lorieKeysymKeyboardEvent. But in the case tou remove it keyboard input will be possible only with Prefer scancodes option.