See also HildonGtkInputMode:

 http://maemo.gitorious.org/hildon/gtk/blobs/master/gtk/gtkenums.h#line559

Having the "don't spellcheck" flag (UITextAutocorrectionTypeNo) is also important, eg, for entering usernames. (No, I didn't mean "dawn".)

(There's no plan to implement inline spelling checking for 3.2, but if we're going to get people tagging their entries, we should define all the tags/flags now.)

As you can see in the linked API docs, there are more types (auto-correction type, auto-capitalisation, the name of the return key, etc.).

Android's equivalent:
http://developer.android.com/reference/android/text/InputType.html

InputScope for Windows Phone:
http://msdn.microsoft.com/en-us/library/ms538181(VS.85).aspx

Needs a concrete api proposal. It should answer a few questions, such as
- How will the entry 'mode' be made known to the OSK ?
- Will the mode affect any behaviours of GtkEntry itself ?

(In reply to comment #6)
> Needs a concrete api proposal.

Yes. I was figuring that we'd play around with the corresponding API in StEntry first, and then propose gtk API once we had more idea what we wanted.

> - How will the entry 'mode' be made known to the OSK ?

Current vague theory: we should expose the entry mode to the input method (since it would be useful to some IMs too), and we should expose the input method information to the OSK (since in addition to wanting to know the entry mode, the OSK may want to know where on the screen the cursor is too).

Not sure what the best way to do that is. Should the OSK hook itself in as a Gtk IM somehow? Or should gtk broadcast the information in a way that the OSK can find? (Eg, additional properties on the _NET_WM_USER_TIME_WINDOW.)

There's a bunch of things we could do with this information inside GtkEntry:
- provide correctness information next to the entry (think a warning icon if the email/url is wrong)
- provide validation (to make forms code easier to write)
- provide hilighting (think URLs in Chrome)
- provide additional clues to accessibility ("please enter a valid email address")
- and from a11y there to the OSK?

That of course leads to the question if part of that information is also useful in cell renderers or labels (in particular hilighting) and if it's useful to hook other machinery into it (email entries could automatically be hooked up into an entry completion feeding from my address book) and if it should be possible to extend it (in particular to provide hilighting or an abstraction for autocomplete in complex libraries/applications.

That said, we can probably start with an enum and go from there. :)

PS: Should "password" be a part of this enum or should the boolean toggle for "display as password" keep being a separate thing?

Oh, and there's of course HTML 5:

http://dev.w3.org/html5/spec/Overview.html#attr-input-type

which is way more exhaustive than all the options here. It also contains a bunch of things that GTK only ships buttons for.

(In reply to comment #8)
> - and from a11y there to the OSK?

The OSK currently works via a gtkmodule that snoops focus and works with gtkwidgets directly. So it can just read the property directly off the GtkEntry.

Dan, are you actually working on this?

no, afaik no one is actively working on this

(In reply to comment #10)
> (In reply to comment #8)
> > - and from a11y there to the OSK?
> 
> The OSK currently works via a gtkmodule that snoops focus and works with
> gtkwidgets directly. So it can just read the property directly off the
> GtkEntry.

What do you mean, does it not use GtkIMContext? Do you have a link to the code?

> What do you mean, does it not use GtkIMContext? Do you have a link to the code?

Here is the code, though I'm not sure how it works:
http://git.gnome.org/browse/caribou/tree/modules/gtk3/caribou-gtk-module.vala
(Thanks to Jan Arne Petersen for finding that.)

Surely a sane on-screen keyboard (or other input method) would use the GTK+ input method system, so I guess we'll also need some way for GtkIMContext modules to get the widget, or at least get this information about the widget.

(In reply to comment #14)
> I guess we'll also need some way for GtkIMContext
> modules to get the widget, or at least get this information about the widget.

Maemo Fremantle had a GTK+ patch to add this:
  http://maemo.org/api_refs/5.0/beta/gtk/GtkIMContext.html#GtkIMContext--hildon-input-default
and that should be somewhere here:
  https://maemo.gitorious.org/hildon/gtk/

(In reply to comment #14)
> Here is the code, though I'm not sure how it works:
> http://git.gnome.org/browse/caribou/tree/modules/gtk3/caribou-gtk-module.vala

Caribou installs a file that tells gnome-settings-daemon to load this gtk module when the on-screen-keyboard gsettings key is enabled. The module installs a gdk event filter to snoop on events and figure out what the currently-focused widget is.

(The original plan was for the module to use Atk interfaces to snoop on the focus, rather than using a gdk event filter, but this doesn't currently work in gtk3 because of bug 655828.)

> Surely a sane on-screen keyboard (or other input method) would use the GTK+
> input method system

Yes, it was originally an input method. We switched because:

  a) g-s-d already had the infrastructure to load arbitrary gtkmodules based
     on gsettings; it was more complicated to get it to override the choice
     of IM module

  b) it would in theory be possible to use both the shell OSK and an input
     method, so that, eg, you could type latin characters with the OSK and
     have an IM turn them into kanji. (Obviously this would be unnecessary
     if it was possible to type kanji directly from the OSK.)

(In reply to comment #16)
> Yes, it was originally an input method. We switched because:
> 
>   a) g-s-d already had the infrastructure to load arbitrary gtkmodules based
>      on gsettings; it was more complicated to get it to override the choice
>      of IM module

Is there any discussion of that somewhere? It sounds like a problem that should be solved instead of worked around so bluntly. Though I don't understand why gnome-system-daemon needs to change the IM Module any more in gnome-shell than it did before.

>   b) it would in theory be possible to use both the shell OSK and an input
>      method, so that, eg, you could type latin characters with the OSK and
>      have an IM turn them into kanji. (Obviously this would be unnecessary
>      if it was possible to type kanji directly from the OSK.)

Typing CJK characters phonetically via latin characters is very common - Pinyin and Canjii, for instance. Maliit does that  and also uses non-latin (Zhuyin/Bopomofo) phonetic input (though these need closed-source engines currently), though I don't know if it reuses the regular Meego Keyboard. But breaking out of the standard GtkIMContext system does not seem like the best way to reuse keyboard code.

(In reply to comment #17)
> (In reply to comment #16)
> > Yes, it was originally an input method. We switched because:
> > 
> >   a) g-s-d already had the infrastructure to load arbitrary gtkmodules based
> >      on gsettings; it was more complicated to get it to override the choice
> >      of IM module
> 
> Is there any discussion of that somewhere? It sounds like a problem that should
> be solved instead of worked around so bluntly. Though I don't understand why
> gnome-system-daemon needs to change the IM Module any more in gnome-shell than
> it did before.
> 
GTK modules are - like any support for arbitrary modules in any software system (see Firefox add-ons or nautilus, gedit, epiphany and soon shell extensions) - a desaster waiting to happen. If it were my decision, we'd stop supporting this craziness.
That said, I think that the Caribou approach is currently the best solution that we have.

I have no idea how XKB, a11y, IM, OSK, magic keys and regular keyboards should interact on a desktop and I doubt anyone has a clear design in mind on how to handle keyboard input properly. All solutions range somewhere from "do it in the kernel" to "do it in the toolkit". And usually you can find any combination of problem and solution out there in the wild.

> I doubt anyone has a clear design in mind 

Well, the Maliit team have a great deal of experience in this field and have spent a great deal of time coming up with a framework that works. Nobody else in the open-source world seems to come close. So it would be good if some way could be found to let Maliit work in gnome-shell so that work can continue towards the perfection you maybe think still needs to be achieved.

(Yes, this bug report is getting way off track. I'll blog about this general issue tomorrow and that will be a slightly better place to discuss it.)

Patches to add use hints to GtkEntry will be welcome here, as will be thoughts on how best to expose that information to the outside world.

Please keep your push for maalit out of gtk bugzilla.

(In reply to comment #20)
> Please keep your push for maalit out of gtk bugzilla.

Sorry. We just took the opportunity to learn more about how gnome-shell's keyboard works since it was brought up here. We should have a patch for the hints soon.

(In reply to comment #21)

> We should have a patch for the hints soon.

Cool, looking forward to that.

Created attachment 201671 [details] [review]
Tentative patch for input mode.

Attached tentative patch for input mode to check if direction is right. It is based on Hildon.

Review of attachment 201671 [details] [review]:

Documentation for what the various flags are supposed to mean would be essential, here. I have no idea what 'tele' or 'autocap' might mean, for instance.

I would have expected things like 'url', 'email address', 'phone #' to be available as hints.

I think the property should maybe be called 'input-mode-hint', since it is not actually enforcing anything, just hinting im and a11y technologies about the type of input that his expected.

(In reply to comment #24)
> I think the property should maybe be called 'input-mode-hint', since it is not
> actually enforcing anything,
>
Should it enforce things? Because naming it "hint" and then later doing validation based on it is bad. And adding a different property for validation later is bad, too...

I don't want to see code for validation of phone number formats in gtk, thats clearly application territory...

Created attachment 201795 [details] [review]
Tentative patch for input mode, v2.

I documented the enumeration and got rid of its two members. Also, `[-_]hints' is appended in some places.

As to have `url', `phone #' or `email address' in hints it is worth to think whether it is sensible to enumerate all possible uses of entry. I think that hints about allowed characters would be enough and application themselves should implement the purpose of entry with its validation.

+ * @GTK_INPUT_MODE_ALPHA: Allow alphabetic characters
+ * @GTK_INPUT_MODE_NUMERIC: Allow numeric characters
+ * @GTK_INPUT_MODE_SPECIAL: Allow non-numeric, non-alphabetic characters.
+ * @GTK_INPUT_MODE_FULL: Allow any character
+ * @GTK_INPUT_MODE_NO_SCREEN_PLUGINS: No input mode plugins should be displayed
+ * @GTK_INPUT_MODE_MULTILINE: Edited field can hold several lines
+ * @GTK_INPUT_MODE_INVISIBLE: Edited field is hidden
+ * @GTK_INPUT_MODE_AUTOCAP: Capitalize words at the beginning of sentence
+ * @GTK_INPUT_MODE_DICTIONARY: Check for word in dictionary (spellchecking)

This is the main thing that needs to be discussed. I guess that we should start with an even smaller set that we know will be useful. So I guess we can remove:
  GTK_INPUT_MODE_NO_SCREEN_PLUGINS
because its description is vague and it is probably special-purpose.

I doubt that these need to be here, because they duplicate GTK+ API elsewhere:
  GTK_INPUT_MODE_MULTILINE: Edited field can hold several lines
  GTK_INPUT_MODE_INVISIBLE: Edited field is hidden

I also wonder why this needs to be here:
  GTK_INPUT_MODE_DICTIONARY: Check for word in dictionary (spellchecking)

So, for an email address input, we want to have dots and an @ sign in the OSK, but we don't need slashes. Would we use GTK_INPUT_MODE_SPECIAL there? What about GTK_INPUT_MODE_NUMERIC? My email has no numbers, but I know of people who do.

What about URL entries? / and . is definitely necessary to enter them, so do we want GTK_INPUT_MODE_SPECIAL? What about GTK_INPUT_MODE_NUMERIC?

Phone numbers need GTK_INPUT_MODE_NUMERIC and + and -, maybe # and *.

So I guess what I'm wondering is: Do these flags make sense for the cases we care about? Afaik the iphone and android keyboards use a regular or numeric keyboard and add a few extra keys depending on usecase. For those extra keys GTK_INPUT_MODE_SPECIAL is not fine-grained enough.

(In reply to comment #28)
> I also wonder why this needs to be here:
>   GTK_INPUT_MODE_DICTIONARY: Check for word in dictionary (spellchecking)
How else will input method know if it should try to provide spell-checking or word-prediction?

An important password usecase should perhaps be explicitly supported: IM must not collect the characters typed in password fields. And why not let IM-provided spell-checking be opt-out instead of opt-in?

In general, I think content types like phone number or email should have their own flags. IM cannot know the intent of a field that has _MODE_NUMERIC + _MODE_SPECIAL, it could be phone number or it could be product numbers.

Usecases: Showing a dialing-pad for phone-number fields (a dialing-pad is typically different from something only for numbers).

(In reply to comment #27)
> Created an attachment (id=201795) [details] [review]
> Tentative patch for input mode, v2.

>+   * GtkEntry:input-mode:

Does GtkTextView need to implement this as well? Or is it safe to assume that all textviews are full-keyboard-and-spellcheck-please?

(In reply to comment #25)
> (In reply to comment #24)
> > I think the property should maybe be called 'input-mode-hint', since it is not
> > actually enforcing anything,
> >
> Should it enforce things?

I'd say no, because if they were enforcing, then adding a new flag later would be an ABI break against old keyboard implementations.

(In reply to comment #29)
> GTK_INPUT_MODE_SPECIAL is not fine-grained enough.

agreed

(In reply to comment #30)
> An important password usecase should perhaps be explicitly supported: IM must
> not collect the characters typed in password fields.

passwordness is already indicated by a different GtkEntry property, although yes, we should make sure that that property gets exposed to the IM as well.

> And why not let
> IM-provided spell-checking be opt-out instead of opt-in?

Assuming that the default value of the hints is going to be "0", then it depends on whether we want spellcheck/autocomplete to be on by default or not... It's basically on-by-default in iOS, but I'm not sure that that necessarily means it's the better option in existing gtk apps too.

I think more semantic hints like 'phone nr', 'email address', 'url' are going to be much more useful, and we should have them (maybe in addition to those more basic 'allow-such-and-such-chars' modes). E.g for email addresses, one could offer smart completion based on contacts.

Created attachment 202700 [details] [review]
Tentative patch for input mode, v3.

Added flags for phone, url and email. Removed the multiline flag for now, as this bug is about GtkEntries, not GtkTextViews - it can be added back later. Also, I was thinking about adding flags for time and date, but probably different widgets should be used for them (GtkComboBox or GtkSpinButton).

Review of attachment 202700 [details] [review]:

::: gtk/gtkenums.h
@@ +951,3 @@
+  GTK_INPUT_MODE_AUTOCAP           = 1 << 30,
+  GTK_INPUT_MODE_DICTIONARY        = 1 << 31
+} GtkInputMode;

Comparing this to some of the prior art cited in the bug, I think we could do a bit better here.

Proposals:

- Make it explicit in the docs that additions to this enumeration are possible, and state that consumers should ignore values that they don't know.

- Make it explicit which of these values are 'variations' or 'modifiers' and which are complete modes. Something like 'email|numeric' probably doesn't make sense, and the documentation should make that clear. My guess is that the following grouping makes sense:

character set restrictions: alpha/numeric/special. these can be freely combined
complete 'semantic' modes: full/phone/url/email. doesn't make sense to combine
  with character set restrictions or each other
additional behaviour hints: invisible/autocap/dictionary: these can be added to
  any of the others

- Other apis discriminate between sentence/word/character autocapitalization, so we should probably rename AUTOCAP to CAP_SENTENCE to leave room for adding more

- What is the intended semantics of DICTIONARY ? Other apis have AUTO_CORRECT - is that the intention ? Dan pointed out the importance of having a NO_AUTO_CORRECT counterpart to make entering names non-annoying. An alternative would be to add a NAME mode

- Back to the charset restrictions: does alpha include space ? does numeric include hex or decimal ? should punctuation be a separate class ?

Review of attachment 202700 [details] [review]:

::: gtk/gtkenums.h
@@ +951,3 @@
+  GTK_INPUT_MODE_AUTOCAP           = 1 << 30,
+  GTK_INPUT_MODE_DICTIONARY        = 1 << 31
+} GtkInputMode;

Comparing this to some of the prior art cited in the bug, I think we could do a bit better here.

Proposals:

- Make it explicit in the docs that additions to this enumeration are possible, and state that consumers should ignore values that they don't know.

- Make it explicit which of these values are 'variations' or 'modifiers' and which are complete modes. Something like 'email|numeric' probably doesn't make sense, and the documentation should make that clear. My guess is that the following grouping makes sense:

character set restrictions: alpha/numeric/special. these can be freely combined
complete 'semantic' modes: full/phone/url/email. doesn't make sense to combine
  with character set restrictions or each other
additional behaviour hints: invisible/autocap/dictionary: these can be added to
  any of the others

- Other apis discriminate between sentence/word/character autocapitalization, so we should probably rename AUTOCAP to CAP_SENTENCE to leave room for adding more

- What is the intended semantics of DICTIONARY ? Other apis have AUTO_CORRECT - is that the intention ? Dan pointed out the importance of having a NO_AUTO_CORRECT counterpart to make entering names non-annoying. An alternative would be to add a NAME mode

- Back to the charset restrictions: does alpha include space ? does numeric include hex or decimal ? should punctuation be a separate class ?

Should this maybe be different kinds of properties?

I mean, email addresses or phone numbers do have a limitation on allowed character set. They also have a generally agreed upon capitalization behavior.

Same for auto-correction - I would imagine auto-correction to find typos in names, emails and phone numbers based on my address book and in URLs based on my browser history if it was turned on.

Hi,
Usually, such kinds of hint are separate into two different group.
One is "Preferred" (no real effect, but a hint) and the other is "Restricted" (subset of character is limited).

If we don't want to separate to two types then we may explicitly write better document.

# 36
What we discussion here is the hint for input method, not for the application itself.

So auto-correction actually usually means input method(or on-screen keyboard if you don't quite understand) provides some hint, for example, if you type, "app" and there is hint about "apple".

This kinds of completion usually could not apply to email field (for example: how can "prw325@gmail.com" be a real word?). But email look up is still up to application itself.

Review of attachment 202700 [details] [review]:

So from my point of view
@GTK_INPUT_MODE_ALPHA: Allow alphabetic characters
@GTK_INPUT_MODE_NUMERIC: Allow numeric characters
@GTK_INPUT_MODE_SPECIAL: Allow non-numeric, non-alphabetic characters
@GTK_INPUT_MODE_FULL: Allow any character
@GTK_INPUT_MODE_PHONE: Edited field expects phone number
@GTK_INPUT_MODE_URL: Edited field expects URL
@GTK_INPUT_MODE_EMAIL: Edited field expects email address
These should be "Restricted", no matter input method exist or not or what they do, it will always affects typing.

@GTK_INPUT_MODE_INVISIBLE: Input characters are hidden (password)
@GTK_INPUT_MODE_AUTOCAP: Capitalize words at the beginning of sentence
@GTK_INPUT_MODE_DICTIONARY: Check for word in dictionary (spellchecking)
These should be categorized to "hints".
GTK_INPUT_MODE_DICTIONARY is named "Qt::ImhNoPredictiveText" under Qt, maybe it's more accurate and covers more case.

And Qt have something more:

Qt::ImhPreferNumbers  Numbers are preferred (but not required).
Qt::ImhPreferUppercase Upper case letters are preferred (but not required).
Qt::ImhPreferLowercase Lower case letters are preferred (but not required).

Which also seems to be viable, add those should be no harm.

Created attachment 215183 [details] [review]
Tentative patch for input mode, v4.

New patch. For now it only introduces new enum and flags, but they are not used anywhere.

Notes and questions:
GtkInputPurpose is an enum describing purpose and may be supplemented with GtkInputHints. Still some common sense is needed when using them.

GtkInputPurpose should be considered as "should" by applications and input methods while GtkInputHints as "may".

GtkInputPurpose is not a closed list - it will be extended for sure with some new purposes I haven't thought of.

I don't know whether setting GtkInputPurpose on GtkEntry should somehow change behaviour of GtkEntry - we probably do not want url validation in toolkit, but on the other hand setting GTK_INPUT_PURPOSE_{PASSWORD,PIN} could modify character visibility setting. I am rather striving toward information role of both purpose and hints.

 * @GTK_INPUT_PURPOSE_ALPHA: Allow only alphabetic characters.
 * @GTK_INPUT_PURPOSE_DIGITS: Allow only digits.
 * @GTK_INPUT_PURPOSE_NUMBER: Edited field expects numbers.

Should those be purposes, a hints, or removed? I am quite uncomfortable with them as there are too many questions involving them (like what about spaces? does GTK_INPUT_PURPOSE_NUMBER allow for several number or only for one? what may be their usecase?). They are probably too general, too "lowlevel".

I like the idea of splitting this into 2 properties.

A few comments:

- It should be explicitly said that these enumerations may be extended in the future and that unknown purpose values should be interpreted as 'free form', while unknown hint values should be ignored.

- The documentation for the purpose enum should say that it is meant to hint on-screen keyboards about the appropriate keys to show

- I wonder what the value of separating password/pin from freeform/digits is ? Input methods can already check the 'visibility' property of entries to know whether this field is password-like

- As mentioned above, there should be a 'no spellcheck' hint (and perhaps the name hint should be defined as 'no spellcheck'+'capitalize words')

Created attachment 221443 [details] [review]
enums: Add purpose enum and hints flags.

This commit adds two enumerations that will be used
to pass additional information to input methods, in
the subsequent commits.

Created attachment 221444 [details] [review]
GtkIMContext: Add purpose and hints

Add input-purpose and input-hints properties to GtkIMContext.

Created attachment 221445 [details] [review]
GtkEntry: Add input purpose and hints

Add input-purpose and input-hints properties to GtkEntry,
and pass these on to GtkIMContext.

Created attachment 221446 [details] [review]
GtkTextView: Add input purpose and hints

Add input-purpose and input-hints properties and pass these through
to the GtkIMContext.

Created attachment 221447 [details] [review]
Add documentation

Document the new API and properties, and their interaction with
existing properties.

Here is what I intend to commit soon. Comments appreciated.

The initial patch is based on Krzesimirs. I've reshuffled the enums a bit, and expanded the documentation with what I think makes sense.

The subsequent patches add GtkIMContext, GtkEntry, GtkTextView api to use these new enums.

After looking at the hildon code, I have not copied their approach of syncing 'invisible' with input purpose - it seems wrong to me that the purpose should change just because I check a 'show password text' checkbox below the entry...

I have also not copied the filtering according to purpose inside gtkentry that hildon has. If anything, that belongs into GtkIMContextSimple - we can do that later.

Is possible add a case where the developer want avoid the display of the on screen keyboard?

Our use case is simple, we have a Calculator (in the Sugar environment), and the activity already provide his own keyboard.

Calculate screenshots: http://activities.sugarlabs.org/en-US/sugar/addon/4076

Created attachment 222360 [details] [review]
patch, add GTK_INPUT_HINT_INHIBIT

This extra flag would allow implementing what's been suggested in comment 47

BTW, the changes in master look pretty good, and the flags fairly complete, just a couple of observations:

* GtkIMMultiContext might also need forwarding those settings to child IM contexts

* There's a minor indenting glitch that was unfortunately pasted several times:
 http://git.gnome.org/browse/gtk+/tree/gtk/gtkentry.c#n10388
 http://git.gnome.org/browse/gtk+/tree/gtk/gtkentry.c#n10437
 http://git.gnome.org/browse/gtk+/tree/gtk/gtktextview.c#n9715
 http://git.gnome.org/browse/gtk+/tree/gtk/gtktextview.c#n9766

Review of attachment 222360 [details] [review]:

::: gtk/gtkenums.h
@@ +1011,3 @@
  * @GTK_INPUT_HINT_UPPERCASE_SENTENCES: Suggest to capitalize the
  *     first word of each sentence
+ * @GTK_INPUT_HINT_INHIBIT: Suggest to inhibit the input method

I think this should be more precise, and talk explicitly about onscreen keyboards, like

@GTK_INPUT_HINT_INHIBIT_OSK: Suggest to not show an onscreen keyboard 
    (e.g for a calculator that already has all the keys).

(In reply to comment #49)

> 
> * GtkIMMultiContext might also need forwarding those settings to child IM
> contexts

Good point, you are probably right. Does hildon have code that we can steal ?

> * There's a minor indenting glitch that was unfortunately pasted several times:

oops. If you want to fix that, please go ahead.

All fixed now