Created attachment 198883 [details] [review]
Proposed fix

I thought we started doing something like this in the glibmm or gtkmm initialization, though I can't find the email now.

The following program crashes:

#include <sstream>
#include <glibmm.h>

int main(void)
{
  Glib::init();
  Glib::ustring str("ó");
  std::ostringstream oss;
  oss << str;
  return 0;
}

Setting the global C++ locale solves the problem.

Note that this causes an exception on some (strange?) systems. See bug #619445 , for instance.

I also seem to remember that we discussed doing something like this in glibmm or gtkmm's initialization, but I can't find the change or the discussion now. Can someone remember?

Created attachment 199758 [details]
More complicated test case

I tested with the attached test case on my Ubuntu 11.04 system and different
values of the environment variable LANG.

LANG=sv_SE.utf8
---------------
The test case in comment 3 throws a Glib::ConvertError exception (Invalid byte
sequence in conversion input), and crashes since the exception is not caught.

oss.imbue(std::locale("")), as recommended in the present ustring
documentation, is of no avail. The same exception.

std::locale::global(std::locale("")), as recommended in this bug, or
setlocale(LC_ALL, ""), are better. No exception.

new Gtk::Main(argc, argv) is fine. No exception.
Gtk::Main's constructor calls gtk_init(), which calls setlocale(LC_ALL, "").

LANG=C
------
The Glib::ConvertError exception is thrown in all cases.

LANG=abc or LANG=hu_HU.utf8 (non-existent language)
---------------------------
The Glib::ConvertError exception is thrown, and there are also other errors.

gtk_init() prints
  (process:2056): Gtk-WARNING **: Locale not supported by C library.
       Using the fallback 'C' locale.

setlocale(LC_ALL, "") returns NULL, indicating failure.

std::locale("") throws a std::runtime_error exception, saying
  locale::facet::_S_create_c_locale name not valid


Conclusions:
- If the environment variable LANG specifies a language that is not installed
  in the system, there's probably no better alternative than to keep the
  "C" locale, with which every C or C++ program starts.
- Provided LANG is correctly set, there's no problem in a gtkmm program,
  initialized in the usual way by creating an instance of Gtk::Main.
- In a pure glibmm program, the locale used by C functions like
  g_locale_from_utf8() and g_locale_to_utf8() must be set, if a Glib::ustring
  with non-Ascii characters shall be sent to a stream.
  The locale can be set with std::locale::global(std::locale("")) or
  setlocale(LC_ALL, ""). Be aware that std::locale("") can throw an exception.

A solution that breaks ABI and API: Change the Glib::init() prototype from
Glib::init() to Glib::init(bool set_locale = true).

Created attachment 228484 [details] [review]
patch: Glib::init(): Set global locale.

This patch makes Glib::init() set the global C++ and C locale to the user's
preferred locale, usually found in the environment variable LANG.
It also corrects the description of Glib::ustring.

It's very reasonable to do this, but I wonder if it counts as an API break.
The behaviour of glibmm after calling Glib::init() will not be exactly the same
as without the patch, and the present behaviour can't be called a bug. Nor can
it be called a feature.

If it's considered an API break, it can't be pushed now. Then Debarshi's patch
in comment 1 should be pushed instead. It only corrects the description of
Glib::ustring.

I've pushed Debarshi's patch (actually a very similar patch) that fixes the
ustring description.

I suppose that the proposed modification of Glib::init() shall wait until the
next API break. Therefore I keep this bug open, but change its title and
component.

Created attachment 332095 [details]
testinit.cc

This program shows the C and C++ global locales before Glib::init(), after
Glib::init(), and after Gtk::Application::create(). The result on my system is

  Before Glib::init()
  std::setlocale(LC_ALL, nullptr)=C
  std::locale().name()=C

  After Glib::init()
  std::setlocale(LC_ALL, nullptr)=C
  std::locale().name()=C

  After Gtk::Application::create()
  std::setlocale(LC_ALL, nullptr)=sv_SE.UTF-8
  std::locale().name()=C

As Bjarne Stroustrup says: "In a mixed C and C++ program, having the C global
locale differ from global() is error prone." (The C++ Programming Language,
4th ed, section 39.2)
That's the situation after a call to Gtk::Application::create().

The most important modification is perhaps to have Gtk::Application::create()
set the C++ global locale to what gtk_init() has set the C global locale, rather
than to have Glib::init() set the global locale to the user's preferred locale.

See also gtkmm bug 765044 comment 8 and some following comments.

I guess we need to look at this again now that we are breaking glibmm API.

Glib and gtk+ have different approaches to setlocale().

The glib documentation says: "A number of interfaces in GLib depend on the
current locale in which an application is running. Therefore, most GLib-using
applications should call setlocale (LC_ALL, "") to set up the current locale."
https://developer.gnome.org/glib/stable/glib-running.html

gtk_init(), on the other hand, calls setlocale(LC_ALL, "") unless
gtk_disable_setlocale() is called before gtk_init().

I suggest that we let glibmm behave similarly to gtk+: Let Glib::init() call
std::locale::global(std::locale("")) unless an appropriate function is called
before Glib::init(). We can add
  void Glib::set_init_to_users_preferred_locale(bool state);
  bool Glib::get_init_to_users_preferred_locale();
with state == true, if the set-function is not called.
(Better function names are welcome. It's difficult to find names that are not
misleading. These functions don't set or get a locale, they just tell what
other functions shall do.)

The constructors of Gtk::Application should do
  if (!Glib::get_init_to_users_preferred_locale())
    gtk_disable_setlocale();

In any case both glibmm and gtkmm should initialize the C++ global locale to be
the same as the C global locale.

I can make patches, one for glibmm and one for gtkmm, if you think this is the
way to go.

Thanks. That seems reasonable.

How might this affect existing applications that are ported to gtkmm 4? If developers need to do something, maybe you can suggest some text for this part of a porting guide. We should put that here:
https://wiki.gnome.org/Projects/gtkmm

I have pushed patches to glibmm and gtkmm.
It remains to add some info to the gtkmm wiki and the gtkmm tutorial.

I have added a "Changes in gtkmm 4" chapter to the gtkmm tutorial. It contains
a paragraph that explains the change in locale handling.
That's perhaps enough for now. I suppose that the wiki and the tutorial will be
synchronized later on.

I'm no longer allowed to modify the wiki pages. Only persons on the list of
Trusted Editors can do that. Perhaps someone can add me to that list?
The info in the wiki says that anyone already on the list can add other persons
to the list.