Created attachment 27721 [details] [review]
Patch to support G_USE_HOME_FOR_HOMEDIR env var

what is traditionally done ? does HOME take preference over pwd or not ?
if it traditionally does, then I think glib should do the same. If it doesn't,
then a simpler approach might be separate G_HOME variable which would take
preference over pwd.

It is traditional (though not universal) to let HOME take preference over the
passwd entry, because that is more flexible.

From reviewing the relevant mail threads, I believe the primary reason for
glib2's decision is that GNOME programs write as well as read user
configuration, and if you have just run "su", it is inconvenient to have root
writing files into your ~/.gconf or ~/.gnome2 directories.  (Other possible
solutions involve chowning newly-created configuration files to match the
directory owner, or making the homedir behavior conditional on whether geteuid()
matches getuid() or on whether geteuid() matches the owner of $HOME; I can see
why any of those alternatives might be considered distasteful because of their
complexity.)

Even if it's better for glib2 to honor $HOME over the passwd entry, there is
also a potential compatibility issue from changing the behavior in the middle of
the 2.x line.  Changing the behavior from 1.x to 2.x was more easily justifiable.

I'm okay with G_HOME instead of G_USE_HOME_FOR_HOMEDIR if people like that better.

After checking with Owen, I just clarified the relation of g_get_home_dir() and
$HOME in the docs.

Why did you close the bug report without giving a reason for not applying the
patch or the G_HOME variation on it?

Why did you add a comment to gutils.c stating that "If there is a HOME
environment variable, its value is returned, otherwise use some system-dependent
way of finding it out" when you were, in the very same commit, documenting that
the reverse was true in misc_utils.sgml?  (Note that the actual ordering varies
between Windows and Unix.)

I closed the bug because Owen and I came to the conclusion that we don't want
to give $HOME preference or add $G_HOME.

I removed the comment, not added it, since it didn't reflect what the code was
doing any longer. Instead I added a more correct comment in the docs.

Can you state a rationale for not wanting to add G_HOME, rather than just coming
to a private consensus and closing the bug report?  G_HOME would be very useful
for people who have network-mounted home directories and want to run
homedir-intensive GNOME programs such as Evolution against a local homedir.

(Sorry for misinterpreting the gutils.c change.  Removing the comment amkes
perfect sense.)

*** Bug 161887 has been marked as a duplicate of this bug. ***

*** Bug 309244 has been marked as a duplicate of this bug. ***

This breaks Gentoo portage sandboxed installs any time eg. gnomevfs is called during the build process, see https://bugs.gentoo.org/show_bug.cgi?id=128289 as example.

It seems that people expect that they can set $HOME and seem to regard it as a bug if setting $HOME doesn't work (especially in the case that the /etc/passwd
value doesn't exist): e.g. see bug report http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=415640.

The obvious way for applications to respond (as someone recommends in that bug report) is to test $HOME before calling g_get_home_dir -- thus negating whatever glib is trying to accomplish by not testing $HOME.  So at the least,  it seems that the documentation ought to either recommend checking $HOME or to explain why software shouldn't check $HOME.

My preferred resolution is for g_get_home_dir to give $HOME priority as people expect and apparently want.  As for su, su provides a -/-l/--login option for this very purpose; people who don't use that option presumably want the existing $HOME to remain in effect, and would regard it as a bug for glib-using applications not to honour that.

For those looking for rationale, bug #2311 and 
http://mail.gnome.org/archives/gtk-devel-list/2002-March/msg00066.html
pretty much already hashed it out extensively, including discussion in that archive link of when apps should test $HOME themselves. Perhaps someone should copy some of this into the docs, I haven't looked at what the docs say right now.

This is one of those things where revisiting it every few years is worse than just picking something and sticking to it, I would say. It's also not really backward compatible to change, if the above-mentioned mailing list post is correct in saying that apps need special handling (as I would say they do) to deal with $HOME != current user's home.

On a more editorial note, I think the setup mentioned in the debian bug where there's no sane homedir in the passwd entry is pretty much a pathological/broken situation. "don't do that then" comes to mind, though I admit I don't understand why it's being done.

Thanks for digging out that reference, Havoc. I'll try to convert it into a 
doc addition.

su/sudo/... and other software are setting $HOME to where they think applications should treat as their home directory; one would expect a function called g_get_home_dir to return the directory that applications should treat as their home directory.

If you feel that any software is setting $HOME to a bad value (or wrongly failing to set $HOME), then please consider filing a bug with that software.

I missed the discussion of where apps should test $HOME for themselves.  If you mean the suggestion that apps should test $HOME only if they're prepared to deal with the case of their config file not existing yet or not being writable, then see below.

> Complicated solutions ($HOME unless it is not writeable) are
> almost always wrong because they aren't predictable and people
> don't understand how they work.

I suggest that complicated solutions like some glib-using programs using getpwent and most other programs testing $HOME is wrong, for similar reasons.

> Basically, there are two things we could provide as the result
> of the function call:
> 
>  - A directory that might be the place where we want to 
>    look for settings, but that might not be suitable as a place
>    to get settings in some circumstances, either becuase
>    we don't have permissions, because it isn't suitable 
>    for what we are doing, or because it shouldn't be trusted.
> 
>  - The current user's home directory

A problem with this analysis is that the above options aren't mutually exclusive.  It's relatively common in corporate environments and places where many humans share one account (libraries, public access terminals etc.) to make configuration files read-only.  Ignoring $HOME doesn't mean applications don't need to handle the case of config files not existing or not being writable.

The $HOME environment variable allows users to tell applications what the right location is, even when the desired location is somewhere other than what getpwent says.  Why does the $HOME environment variable exist, if not to be consulted?  There will always be programs consulting $HOME to find what directory they should treat as the home directory.  If any software isn't setting $HOME correctly, then that software should be fixed, so that programs consulting $HOME do the right thing.

It's true that changing g_get_home_dir to respect $HOME at this point would be a bit traumatic.  It was traumatic when it changed to ignore $HOME (even in the context of a major version bump), and changing back would be equally so.  Apps which blindly store stuff into g_get_home_dir (not just configuration) would cause permission issues within their own data stores if they get run after an su without the - option.  In glib 1, you could blame it on the apps or on libgnome for not providing enough machinery for the apps.  But glib 2 has taken responsibility for this issue by defining the homedir differently from the way traditional UNIX apps do.

So, while I think glib 2 made the wrong decision, I don't advocate fixing it.  I do advocate adding a $G_HOME override as a compatible workaround for glib2's mistake.

A collected set of use cases where failing to respect $HOME creates problems:

  * Your homedir is remote and performs poorly, so you want to use a local store for an app which is very I/O-intensive on the homedir.

  * Your passwd entry does not exist due to an exotic setup.  (Arguably, exotic = broken, so I don't think this is a strong use case.)

  * You're inventorying what changes an app makes to the homedir and want a clean starting point.

  * You want to try making some large configuration change to an app to see what effect it has, and you don't want the change to be permanent.

  * You su without the - option and, contrary to the prior expected usage of su, the settings of the apps you run are not taken from the old user's homedir.

It's legitimate to say "those are all broken or marginal, and we don't care," but I don't agree.

Something to keep in mind is that most apps rely on something of a "process swarm" (session manager, window manager, configuration daemon, various other per-session daemons potentially) - these all have to pretty much agree on DISPLAY and the homedir or stuff breaks.

Configuration files in GNOME are not even generally read by the app itself, but by a separate daemon that would not have the HOME variable you set.

So, it's only going to work reliably to set the homedir for an entire session. And at that point creating a test account or just changing your passwd entry seems pretty plausible.

Changing g_get_homedir() just isn't enough. Code has to be written with this case in mind, and it just isn't, and it's the sort of thing that's hard and obscure enough that it probably never will be.

Several people are referring to situations where changing the behaviour back to $HOME taking precedence is going to cause problems. I'd really like to hear about a scenario where a real application will break.

Because of this bug, most glib applications fail to work when run in a chroot whose passwd file is not synchronised with the master system, or whose home directories are not present. That includes every single mono application (at least using mono 1.9), since mono tries to write a .wapi directory. This is terribly annoying and I am tired of locally fixing applications so that they work.

The big picture here is that chroot is a poor form of virtualization.  Use KVM if you're using Linux, containers if on OpenSolaris.  It will be far more reliable.

Alternatively if this is for the 32 bit Firefox on 64 bit system issue, well, Debian could adopt multilib.  

Or you could just keep the password file in sync.

-- Colin, original author of Debian+chroot guide who now thinks KVM, multilib, and SELinux are better solutions for virtualization, application compatibility, and security respectively

I don't want the argument to slip towards the benefits of chroots, but KVM is a big no-no here because it is pathetically slow to boot, while "dchroot etch <command>" is instantaneous and lets me test software on numerous distributions without having to wait for a VM to boot or wasting expensive resources.

Another situation where $HOME not having precedence causes breakage is the use of fakeroot. It is not a standard utility, but it is standard enough across Debian and Ubuntu platforms for package builds.

Created attachment 197281 [details] [review]
04_homedir_env.patch

This is debian patch, that would be nice to see upstreamed to fix, for example, pygtk tests failing due trying to write at wrong location:
https://245103.bugs.gentoo.org/attachment.cgi?id=287377

Something like Debian's $G_HOME patch would immensely help me for writing e. g. gvfs tests, too, i. e. tests which can be run with "make check" at e. g. jhbuild time and thus don't have any privileges to do fancy things like unshare, bind mounting, or virtualization.

Right now I have to put temporary files and directories into the real user $HOME, as some things (such as trash://) do not work (or work differently) outside g_get_home_dir(). The Debian patch has been there for ages and not causing any problems, but of course I can't rely on it for upstream tests.

Created attachment 230447 [details] [review]
g_get_home_dir(): honour $HOME

If the $HOME environment variable is set, prefer that to the entry in
/etc/passwd.

This brings us in line with almost every other utility and library on
UNIX-like systems while avoiding some of the more complicated
possibilities that have been suggested.

This incompatible change has been petitioned for quite some time by
many, and in particular from the Debian world, which carries a patch
that adds a new G_HOME environment variable with the same meaning as
this patch now assigns to HOME.

The primary motivation for the change was to increase the testability of
GLib-based programs from 'make check' types of frameworks: it is now
possible to set HOME to a temp directory to avoid the testsuite
modifying the user's real home directory.

The change also brings us increased compliance with the XDG Base
Directory Specification.  The specification specifically states that the
default values should be computed based on the HOME environment
variable, whereas we were basing them on the value from /etc/passwd.

The change was agreed to by all in attendence at the November 29 Gtk+
developer meeting.

Review of attachment 230447 [details] [review]:

Nothing major here, just some general thoughts, and one wording suggestion:

Will this affect the autofs case, where /home/user is automatically mounted on first access?  It seems possible to me that autofs might say ENOENT for an access() system call, but do the automount for everything else.  I'm not sure if this is the case, but it's likely worth checking.

I don't quite get what we're buying by doing the check for existence, and returning something undefined if it doesn't.  I mean...if HOME is wrong, then the system calls to access it will fail, and hopefully the app will show that as a sensible error?  So why not just say "Returns the value of $HOME if set" (period?)

::: glib/gutils.c
@@ +985,3 @@
+ *   increase the compatibility of GLib with other programs (and the XDG
+ *   basedir specification) and to increase testability of programs
+ *   based on GLib (by making it easier to run them from test rigs).

"rigs" is a bit informal - how about "frameworks"?

Honestly, I only kept the code that did the checks because it was already there (although I moved it out of a Windows-only block...)

I'm happy to drop that (or move it back inside the #ifdef).

By the way, none of my comments should be construed as blocking this patch, since it was approved at the meeting.  I was just adding some thoughts which you may or may not address as you see fit.

I agree with both comments -- in particular, I don't think trying to deal with an invalid $HOME actually buys us anything and it is possible that it could cause the issues you mention -- plus, it's slower.

I've moved the check back inside the #ifdef.

Attachment 230447 [details] pushed as 7523c04 - g_get_home_dir(): honour $HOME