GNOME Bugzilla – Bug 314482
g_io_channel_unix_new fails if C runtimes don't match
Last modified: 2018-05-24 10:36:21 UTC
Trying to use g_io_channel_unix_new from code compiled with VC++ 7.x doesn't work because the file descriptor is unknown to the runtime the glib uses. This occurs if pygtk is compiled for Python 2.4. I think a g_io_channel_win32_file_handle function that takes a file handle is needed to work around this.
In theory, yes, such a function would be relatively trivial to add. But do we really want to "approve" using different C runtimes by adding such a function? Aren't there subtle ways in which GLib's use of C runtime specific data can cause unexpected behaviour if an application that links to GLib uses another C runtime? For instance, where does g_print(), g_printerr() and g_log() output go? At least any dup2() or freopen() you might do to fds 1 and 2, or stdout and stderr in the non-msvcrt using code won't affect GLib's use of them.
I think g_io_channel_win32_file_handle is useful in its own right because I eventually would like to add support for taking advantage of overlapping I/O on NT class os's. I also think that multiple C runtimes is a fact of life -- I'm actually running into this bug because I have a mix of the debug version of the VC++ 6 runtime (msvcrtd.dll) and the non-debug version (msvcrt.dll). This is especially true since the gtk stack is difficult to rebuild. BTW, I've found stdio redirection to be unreliable, even with only 1 C runtime library.
(In reply to comment #1) > In theory, yes, such a function would be relatively trivial to add. But do we > really want to "approve" using different C runtimes by adding such a function? We already have that, in pygtk, with or without any "approval" from anyone, and as result python folks can't use io_watch on win32. They are certainly not going to recompile python nor pygtk (and even if they would, users get python and glib from different sources, and it's unlikely Tor starts doing what python folks do or vice versa). So, bad or not, having a method for a handle would help. It would suck, but it would suck less than missing functionality. After all it's not the first thing which can't be done on win32 in the same way as on unix.
Could you come up with a patch then?
(In reply to comment #3) > So, bad or not, having a method for a handle would help. It would suck, but it > would suck less than missing functionality. After all it's not the first thing > which can't be done on win32 in the same way as on unix. Actually I'm not sure it would help with pygobject, since python's os.pipe() returns a file descriptor, not a win32 handle, and I don't know any python function that can convert a descriptor to a handle. For instance, the Python wrapper for os.pipe in win32 is: HANDLE read, write; int read_fd, write_fd; BOOL ok; Py_BEGIN_ALLOW_THREADS ok = CreatePipe(&read, &write, NULL, 0); Py_END_ALLOW_THREADS if (!ok) return win32_error("CreatePipe", NULL); read_fd = _open_osfhandle((Py_intptr_t)read, 0); write_fd = _open_osfhandle((Py_intptr_t)write, 1); return Py_BuildValue("(ii)", read_fd, write_fd); Since GLib does not have (AFAIK) a function to create a pipe, the only way to get this working is to have GLib accept HANDLE, and make Python wrap _get_osfhandle somehow in order to convert a python file descriptor into a handle which is passed to GLib. Honestly I don't see any solution, but I'm pretty sure this bug report will only solve half of the problem. Python and Glib should just be using the same C runtime. Any way to work around this requirement is probably going to involve more work than the requirement itself.
Oops, never mind my last comment. There _is_ a python msvcrt module wrapping get_osfhandle. Therefore a GLib API that accepts a handle would solve the problem and be most welcome.
http://munt.mine.nu:8080/files/iowin32.patch I am trying to test it, win32 pygobject lately doesn't want to play with me.
(In reply to comment #7) > http://munt.mine.nu:8080/files/iowin32.patch > I am trying to test it, win32 pygobject lately doesn't want to play with me. > I hope that is only a quick test, not a real patch for pygobject, because you can't assume gobject.IOChannel() will receive a python file descriptor; this will fail if the file descriptor was created by some library linking to a different runtime. In my opinion the best way to go is for glib[1] to provide g_win32_fd_to_handle and g_win32_handle_to_fd convertion functions, because some day g_io_channel_unix_new may not be the only glib API accepting file descriptors. Then the python code looks like: pipe_in, pipe_out = os.pipe() iochannel = gobject.IOChannel(gobject.win32_handle_to_fd(msvcrt.get_osfhandle(pipe_out))) A bit long but "Explicit is better than implicit.". [1] or if we can assume pygobject runtime is always == glib runtime, maybe put it just in pygobject is ok too.
(In reply to comment #8) > (In reply to comment #7) > > http://munt.mine.nu:8080/files/iowin32.patch > > I am trying to test it, win32 pygobject lately doesn't want to play with me. > > > > I hope that is only a quick test, not a real patch for pygobject, because you > can't assume gobject.IOChannel() will receive a python file descriptor; this > will fail if the file descriptor was created by some library linking to a > different runtime. It indeed is. And python folks should use handles explicitely if they mix stuff like that. Treating integer fd as coming from python dll is simply a convenience for usual code which uses python file api (and socket, most importantly). As to the patch, you might have noticed where it is and how it's posted here, it is not a real patch for pygobject. > In my opinion the best way to go is for glib[1] to provide g_win32_fd_to_handle > and g_win32_handle_to_fd convertion functions, because some day > g_io_channel_unix_new may not be the only glib API accepting file descriptors. > Then the python code looks like: > > pipe_in, pipe_out = os.pipe() > iochannel = > gobject.IOChannel(gobject.win32_handle_to_fd(msvcrt.get_osfhandle(pipe_out))) Doesn't make sense, IMO, if we can do that under the hood. If one wants handle (e.g. file coming from fifth library linked to tenth C runtime) let him use handle. In any case my intent was to try it out and see what happens, without starting discussions about python in a glib bug report ;)
(In reply to comment #9) > > In my opinion the best way to go is for glib[1] to provide g_win32_fd_to_handle > > and g_win32_handle_to_fd convertion functions, because some day > > g_io_channel_unix_new may not be the only glib API accepting file descriptors. > > Then the python code looks like: > > > > pipe_in, pipe_out = os.pipe() > > iochannel = > > gobject.IOChannel(gobject.win32_handle_to_fd(msvcrt.get_osfhandle(pipe_out))) > > Doesn't make sense, IMO, if we can do that under the hood. If one wants handle > (e.g. file coming from fifth library linked to tenth C runtime) let him use > handle. In any case my intent was to try it out and see what happens, without > starting discussions about python in a glib bug report ;) OK, I guess that's acceptable too. So: pipe_in, pipe_out = os.pipe() handle_out = msvcrt.get_osfhandle(pipe_out) iochannel = gobject.IOChannel(handle_out)
Created attachment 105522 [details] [review] patch A very simple patch, works with pipes. Is anybody still interested in this?
The new function Looks simple and if you say you know this is useful, I don't lobject it going in. The documentation needs some love though. I am not sure one can say that the function does "Open a file handle", as it after all is passed an already open file HANDLE. Looking at the C runtime sources, I think I see that after calling _open_osfhandle() on a HANDLE, then indeed when you call close() on the returned fd, CloseHandle() gets called on the HANDLE. This might actually be a bit unexpected, so maybe a warning is needed. I would also like some text spelling out what kind of HANDLEs can and should be passed to the function.
(In reply to comment #12) > The new function Looks simple and if you say you know this is useful, That's the problem, I don't know if it's useful. I thought for some time that gio is going to replace GIOChannel, but I see now that that's not going to happen at least until the next after next glib release. So perhaps it could be useful for pygtk, I don't know. > I don't > lobject it going in. The documentation needs some love though. I am not sure > one can say that the function does "Open a file handle", as it after all is > passed an already open file HANDLE. > > Looking at the C runtime sources, I think I see that after calling > _open_osfhandle() on a HANDLE, then indeed when you call close() on the > returned fd, CloseHandle() gets called on the HANDLE. I think it's backwards, if you close the handle then the file descriptor becomes invalid (at least I got segfault when closed the handle after creating a watch on it). So glib could duplicate the handle, or it could keep the handle around so that set_close_on_unref could close it. I guess the former is better, what do you think?
Created attachment 105598 [details] [review] patch A better patch.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/glib/issues/34.