After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 638899 - Remove gio static bindings
Remove gio static bindings
Status: RESOLVED FIXED
Product: pygobject
Classification: Bindings
Component: introspection
unspecified
Other All
: Urgent blocker
: ---
Assigned To: Nobody's working on this now (help wanted and appreciated)
Python bindings maintainers
: 640103 (view as bug list)
Depends on:
Blocks: 623278
 
 
Reported: 2011-01-07 11:11 UTC by Tomeu Vizoso
Modified: 2011-02-11 21:16 UTC
See Also:
GNOME target: 3.0
GNOME version: ---


Attachments
Remove gio static bindings (516.89 KB, patch)
2011-01-07 11:11 UTC, Tomeu Vizoso
committed Details | Review
Put back static gio bindings (516.51 KB, patch)
2011-02-02 14:33 UTC, Martin Pitt
committed Details | Review
Run gio tests separately when enabled (1.68 KB, application/x-shellscript)
2011-02-02 14:35 UTC, Martin Pitt
  Details
Remove GIO documentation (940.84 KB, patch)
2011-02-11 21:12 UTC, Simon van der Linden
committed Details | Review
Remove last GIO-related bits (6.48 KB, patch)
2011-02-11 21:12 UTC, Simon van der Linden
committed Details | Review

Description Tomeu Vizoso 2011-01-07 11:11:43 UTC
Because we cannot use in the same process the static and the introspected bindings.
Comment 1 Tomeu Vizoso 2011-01-07 11:11:47 UTC
Created attachment 177740 [details] [review]
Remove gio static bindings
Comment 2 Paolo Borelli 2011-01-16 11:38:59 UTC
Why do we need to remove them? Can we split things so that the module is not loaded by pygi apps?
Comment 3 Tomeu Vizoso 2011-01-17 10:26:27 UTC
(In reply to comment #2)
> Why do we need to remove them? Can we split things so that the module is not
> loaded by pygi apps?

There's also the problem with Python 3. The only static API that should be inside pygobject is the low level API in glib that supports all the rest.
Comment 4 Paolo Borelli 2011-01-17 10:58:06 UTC
I am still confused, why cannot gio behave like the old pygtk and just be a separate binary that does not get loaded by pygi apps?
Comment 5 Tomeu Vizoso 2011-01-17 12:43:15 UTC
(In reply to comment #4)
> I am still confused, why cannot gio behave like the old pygtk and just be a
> separate binary that does not get loaded by pygi apps?

Ah, sure, I thought you were suggesting keeping them inside the pygobject tree but just do some hacking to avoid them being loaded during the tests.

They of course can be moved to a separate module (or to pygtk itself) if someone is willing to move them and maintain them. This ticket is only about removing them from pygobject, if someone is willing to do the work, they can move them to wherever they want. Gian Mario has told me that he won't have cycles any time soon to take care of it.
Comment 6 Ignacio Casal Quinteiro (nacho) 2011-01-19 13:48:27 UTC
Pushed. If somebody complains we can move this to a separate module.
Comment 7 Dieter Verfaillie 2011-01-19 14:10:54 UTC
Sorry to react this late but isn't this going to break the next release for
mswindows (PyGObject-2.28/PyGTK-2.24)?

Note, I don't think we can depend on having any gi support on mswindows
before GTK+ 3, so we'll most probably still need static bindings until
a gtk+-bundle supporting gobject introspection becomes available...
Comment 8 Tomeu Vizoso 2011-01-20 10:02:14 UTC
(In reply to comment #7)
> Sorry to react this late but isn't this going to break the next release for
> mswindows (PyGObject-2.28/PyGTK-2.24)?
> 
> Note, I don't think we can depend on having any gi support on mswindows
> before GTK+ 3, so we'll most probably still need static bindings until
> a gtk+-bundle supporting gobject introspection becomes available...

What about creating a new module and put there the bindings? Another possibility is moving them to PyGtk.
Comment 9 John Stowers 2011-01-20 11:34:32 UTC
Nak. This is far too much of an API/ABI break this late in the cycle.

Can't we detect at runtime whether PyGI is loaded, or vice-versa; detect if someone does "import gio" after using PyGI and error gracefully.

This will break all existing users of the old bindings.

Even putting this in a new module will not solve the problem (although I agree it is the best result if you won't revert this). Old code will still do "import gio", the program will still crash even if gio static bindings live in a different module.
Comment 10 John Stowers 2011-01-20 11:36:39 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > I am still confused, why cannot gio behave like the old pygtk and just be a
> > separate binary that does not get loaded by pygi apps?
> 
> Ah, sure, I thought you were suggesting keeping them inside the pygobject tree
> but just do some hacking to avoid them being loaded during the tests.

^^^

I am suggesting this, at least this late in the cycle.
Comment 11 Tomeu Vizoso 2011-01-20 14:27:34 UTC
(In reply to comment #9)
> Nak. This is far too much of an API/ABI break this late in the cycle.

Well, it's not that clear to me that this can be called an ABI break as any applications will keep happily run on a system that has the gio bindings available.

Availability of the static bindings is related to this only to the point that packagers need to do some extra work in order to get the gio bindings packaged.

> Can't we detect at runtime whether PyGI is loaded, or vice-versa; detect if
> someone does "import gio" after using PyGI and error gracefully.

We can do a lot of things, but these hacks are proving themselves expensive. There's quite some weird hacks in pygobject that nobody remembers any more why they were added. This makes things very fragile and increase the cost of moving forward.

> This will break all existing users of the old bindings.

Not if the static gio bindings continue to get packaged.

> Even putting this in a new module will not solve the problem (although I agree
> it is the best result if you won't revert this). Old code will still do "import
> gio", the program will still crash even if gio static bindings live in a
> different module.

Sure, but the main motivation for this removal has been the addition of Gio tests, which failed because of clashes with the static wrappers. As a plus we would bbe able to remove some of the hacks that J5 had to add when added Pytohn 3 support to the rest of PyGObject.

My current view of the issue is that we have three possibilities:

1. put back the gio bindings and either don't put back the tests or take them out from the test suite. (this means work for upstream now and in the future)

2. put the gio bindings inside pygtk (this means packagers need to update the pygobject and pygtk packaging to reflect this)

3. put the gio bindings inside a new module (this means that someone needs to do the move upstreams and then packagers need to update the change in the source package)

Note that 1. also means that in a not-so-distant future the gio bindings will get removed again from pygobject, so packagers will have to do the same work anyway.

I appreciate the work that you win32 guys are doing and I'm willing to find the time to do 1 myself, but I'm really unsure this will benefit the project as a whole.
Comment 12 John Stowers 2011-01-20 21:19:32 UTC
I'm sorry Tomeu etc, but I find that reply unacceptable.

First, let me lay out my assumptions;
a) Multiple versions of PyGObject 2.x are not parallel installable
b) GIO static bindings have been removed from PyGObject
c) Users of PyGTK also use [static] gio

If my assumptions are correct, and PyGObject was to be released today then *all* *existing* *users* of gio would be broken. This is a complete API/ABI break and I fail to see how such a change could be allowed to happen with such little discussion. I understand you guys are on the hackfest making progress, but this is unacceptable for existing users of the static bindings.

This is not solely a win32 issue; as PyGtk maintainer you just made all my users code stop working. Breaking 10s/100s of applications with one commit cannot benefit the project as a whole.

You have held me (and Dieter) hostage to your decision. You have given me only two choices, create a pygio module or import the code into pygtk. I have very little time to do this, and more worryingly I am disappointed in how willingly you break existing code without a API/ABI compatible solution in place.
Comment 13 Dieter Verfaillie 2011-01-20 22:15:09 UTC
I understand full well the need to go forward and am grateful for all the work
that's being put in pygobject's gi support. I'm looking forward to enjoy the
improvements that have been and are going to be done in the future, even when
I'm mostly (but not always) working from an OS that mostly seems to limp
behind ;)

I also understand the need to sometimes clean away old cruft to be able to go
forward, as long as doing so does not break any existing code that depends on
such old cruft. Everyone that has worked on any complex software project has
been in such situations more than once and yes, it sucks...

On the other hand, I simply don't understand the sometimes "perceived" need to
remove old cruft just to make it easier to support some shiny new feature
without actually providing a complete solution for existing projects depending
on such old cruft. Hand waving of the type of "if someone is willing to move
them and maintain them" just doesn't work and in the case of nobody rising to
actually do the work would have been an ABI/API break from the point of view
of a static gio bindings user. And the PyGTK user (this change breaks building
PyGTK as already demonstrated in bug 640103. I hope it is clear that such
breaks are simply unacceptable.

Up until Comment 11, where you volunteered to do that work, we where heading
towards such a situation. Please understand I'm not trying to be difficult
without the intent to actually do something about it. If it ever comes to a
split pygio package I will happily help out where I can (write a setup.py
file for mswindows support and do the tests, build .exe and .msi installers
and include it in the aio installer). But be prepared to take the
responsibility for your actions and maintain such a separate package with
regards to the autotools voodoo required to actually make it work wherever
pygobject/pygtk/whatever works today. Doesn't sound like fun to me, doesn't
sound possible to complete before the next release (april 2001?), and I do
have another proposal described below.

My concerns are more general than the static gio bindings alone. As a
GNOME/Python bindings user I am concerned about the future availability of
PyGTK and friends (what we've been calling "static bindings") in general.

Consider the following:
1) GTK+-3 will be parallel installable with GTK+-2;
2) pygobject is the foundation of Python bindings for both ABI/API versions
   of GTK+;
2) Most people are going to be migrating to a GNOME 3 desktop at one time
   or another;
3) There's huge amounts of code out there using PyGTK and friends that by
   definition depends on PyGTK/GTK+-2. Either in packaged projects for
   whatever distro you're using or as code snippets plastered all over the
   web;
4) Introspected GTK+ bindings are not compatible with GTK+-2. In other
   words, pygobject's gi support is not a drop-in replacement and I don't
   think that's the goal of the project unless I'm misunderstanding something;
5) One way or the other, some people will still need to run such software
   written in Python that depends on PyGTK/GTK+-2 on their shiny new
   GNOME 3 desktop for some years to come. And don't be fooled, it took
   some projects years to migrate from GTK+-1 to GTK+-2 (GnuCash can serve
   as an example here). With the current popularity of PyGTK you can be sure
   we'll see similar situations. And that's only talking about packages
   we know about through today's package managers. Let's nog forget there's a
   whole lot of "internal" code around the globe that's going to be affected
   by whatever decision taken.

Now, if it would have been an application written in c, linked against
libgtk+-2.0 there would be no problem. The distro's package manager simply
pulls in GTK+-2(.24) and it's dependencies or the user requests its
installation and you're done.

As it stands now, Python applications are going to be a different matter.
In some years from now, for example when pygobject-2.38 is the version in
use, pygobject will still need to support the static bindings we use today
and yes, that means what will become PyGTK 2.24, etc.

The easiest solution I can come up with to solve these problems is to release
a "static bindings" specific pygobject-2.28 (together with matching pygtk-2.24
in april?) and a parallel installable pygobject-3.0 where all the gi specific
shiny new cleanup work can be done safely, without ever having to worry about
PyGTK and friends again. Or we could maybe stick with pygobject-2.26 for static
bindings and make what's currently master pygobject-3.0 and be done with it.

Such a 3.0 release could even provide *some* static bindings support for
*some* time if it proves necessary for very specific projects (as the
gst devs seem to have already requested?). You'd mostly be freed of a whole
lot of extra complexity created by *maintaining* two separate bindings systems
where people that know about the inner details of one of those systems are
already extremely thinly spread and over Python2 and Python3. Like you
described in Comment 11.

I apologize if this reply gets interpreted as overly strong or something.
That has not been my intent. I've spent hours composing this message and
going over translations from Dutch to English and can still only hope it
is understood as an attempt at being constructive to the benefit of *all*
GNOME/Python bindings users, whether "static vs dynamic" and "now vs in a
couple of years".

Regards,
Dieter
Comment 14 Dieter Verfaillie 2011-01-20 22:16:55 UTC
*** Bug 640103 has been marked as a duplicate of this bug. ***
Comment 15 John Stowers 2011-01-21 05:27:27 UTC
As an experiment I spent a few hours creating a pygio module and getting it to build (did not succeed yet). If you insist on forcing this out of the PyGObject tree then PyGObject needs to

* Install pyglib-compat and 
* Install pygsource and _glib_la system wide

So basically libpyglib and the currently private _glib_la will likely need to be public, perhaps with a new pc file, certainly/hopefully with some sort of API/ABI guarantee.

I'm not sure this will be less work then just reverting the removal.
Comment 16 Ignacio Casal Quinteiro (nacho) 2011-01-21 09:29:30 UTC
Hey John, we are fine with exporting the bits needed to have a different module for it. Do you have the needed patches for it, or should we do it? ;)
Comment 17 Tomeu Vizoso 2011-01-21 10:22:41 UTC
(In reply to comment #12)
> I'm sorry Tomeu etc, but I find that reply unacceptable.
> 
> First, let me lay out my assumptions;
> a) Multiple versions of PyGObject 2.x are not parallel installable
> b) GIO static bindings have been removed from PyGObject
> c) Users of PyGTK also use [static] gio
> 
> If my assumptions are correct, and PyGObject was to be released today then
> *all* *existing* *users* of gio would be broken. This is a complete API/ABI
> break and I fail to see how such a change could be allowed to happen with such
> little discussion. I understand you guys are on the hackfest making progress,
> but this is unacceptable for existing users of the static bindings.
> 
> This is not solely a win32 issue; as PyGtk maintainer you just made all my
> users code stop working. Breaking 10s/100s of applications with one commit
> cannot benefit the project as a whole.
> 
> You have held me (and Dieter) hostage to your decision. You have given me only
> two choices, create a pygio module or import the code into pygtk. I have very
> little time to do this, and more worryingly I am disappointed in how willingly
> you break existing code without a API/ABI compatible solution in place.

I don't think this is a realistic appreciation of the situation.

If you read again my last comment, I have offered myself to put back the static bindings and write any hacks that will allow them to coexist with the work going on with Gio and introspection. Also note that there hasn't been a release with this change yet.

About if we can call it an API/ABI break, the consensus in this room is that nor pygobject, pygtk nor pygio are breaking API as long as the upstream sources keep being packaged. To stop packaging the gio bindings is a downstream decision.

But again, this shouldn't be a discussion about words but about practical issues, and that's why I have made that offer even if I don't agree with your point of view.
Comment 18 John Stowers 2011-01-22 03:22:57 UTC
Debating what exactly constitutes an API break is missing the wood for the trees. 

I am most concerned that this happened late in the cycle, with little discussion. Even Comment 11 seems like an off-handed remark, not something considered beforehand. I do not get the impression that the PyGObject team is concerned with keeping existing PyGTK (+static) code working. I also don't think offloading this to downstream is constructive.

So I am suggesting reverting the commit until either a standalone pygio works on all platforms, pygio is moved into pygtk, or another agreement is reached.

I have started a standalone pygio here
https://github.com/nzjrs/pygio

Modifications for pygobject will be pushed here
https://github.com/nzjrs/pygobject/tree/for-pygio
Comment 19 John Stowers 2011-01-22 03:24:10 UTC
(In reply to comment #16)
> Hey John, we are fine with exporting the bits needed to have a different module
> for it. Do you have the needed patches for it, or should we do it? ;)

I'm working on this. Can you think of any other users of pyglib out of tree? I might make pyglib single include, so I will need to change these other users too.
Comment 20 Paolo Borelli 2011-01-22 08:40:22 UTC
given the stop-gap nature of pygio, can't you just duplicate the bits of pyglib etc you need? ideally by also changing the prefixes wit sed so that there are no symbol conflicts. IIRC the pyglib and glib_la were just a way to share code among different modules and on windows they are already treated differently since loading a dll from a module is not trivial
Comment 21 Tomeu Vizoso 2011-01-24 16:53:07 UTC
(In reply to comment #20)
> given the stop-gap nature of pygio, can't you just duplicate the bits of pyglib
> etc you need? ideally by also changing the prefixes wit sed so that there are
> no symbol conflicts. IIRC the pyglib and glib_la were just a way to share code
> among different modules and on windows they are already treated differently
> since loading a dll from a module is not trivial

This sounds best to me but I'm also ok with adding this API to pygobject even if we deprecate it in a few months.

Nacho has also offered to find a new place for pygio, guess it's best if John and he agree on a plan and an execution.
Comment 22 Ignacio Casal Quinteiro (nacho) 2011-01-26 14:38:32 UTC
So John, how is this going? I like also the idea of copying the things needed into pygio so the module is a bit more independent.
Comment 23 John Stowers 2011-01-26 19:43:51 UTC
(In reply to comment #22)
> So John, how is this going? I like also the idea of copying the things needed
> into pygio so the module is a bit more independent.

Work is ongoing in github, me and Dieter are co-ordinating. Things currently work but there is still things to be fixed so it builds on windows, and there is some additional packaging/autotools work I need to complete.

I don't expect us to finish this in the next few days, so if you guys want to do a pygobject release this week please revert the removal until we have pygio standalone back to the level of completeness it was before removal.

I'm not a fan of just copying the code across because I don't want the possibility of things getting out of sync. Looking at the changes, it is not too much work to export pyglib API.
Comment 24 Dieter Verfaillie 2011-01-26 21:58:39 UTC
Tomeu, Nacho, John and everybody else who's interesting in this:
is it OK if I create http://live.gnome.org/PyGIO and start documenting
the split over there, starting with a todo list?
Comment 25 John Stowers 2011-01-26 23:41:43 UTC
(In reply to comment #24)
> Tomeu, Nacho, John and everybody else who's interesting in this:
> is it OK if I create http://live.gnome.org/PyGIO and start documenting
> the split over there, starting with a todo list?

Please do. You may as well start with the most recent TODO you emailed me, changed to reflect the current status (now it seems to work)
Comment 26 Tomeu Vizoso 2011-01-27 09:23:38 UTC
(In reply to comment #24)
> Tomeu, Nacho, John and everybody else who's interesting in this:
> is it OK if I create http://live.gnome.org/PyGIO and start documenting
> the split over there, starting with a todo list?

Sure!. Thanks a lot for your effort, I'll see what I can do from that list.

J5: I'm leaning towards making sure that we don't release PyGObject without the gio static bindings until PyGIO is ready to be released. Easiest may be reverting the commit that removed them and disabling the gio tests, if we cannot hold releasing for long enough.
Comment 27 Dieter Verfaillie 2011-01-27 10:35:35 UTC
The http://live.gnome.org/PyGIO page has been created. Feel free to comment,
edit, improve and maybe even assign some items on the todo list to yourself ;)
Comment 28 Ignacio Casal Quinteiro (nacho) 2011-01-27 10:46:25 UTC
I'm going to push the pygio branch to the gnome server.
Comment 29 Ignacio Casal Quinteiro (nacho) 2011-01-27 10:49:17 UTC
Done: http://git.gnome.org/browse/pygio
Comment 30 Ignacio Casal Quinteiro (nacho) 2011-01-27 10:56:47 UTC
Remember to push the doap file.
Comment 31 Tomeu Vizoso 2011-01-27 10:58:39 UTC
 ||Create pygio bugzilla product and move relevant bugs from pygobject, pygtk, etc || ||<: #ff8080> todo ||

Is this really needed? Maybe we can keep using pygobject/gio unless there's some rule about the bugzilla product matching git modules.
Comment 32 Dieter Verfaillie 2011-01-27 11:03:42 UTC
Nacho: a pygio.doap file already exists :)

Tomeu: I don't know if there's any rules about that, just thought it would be
more clear if the split pygio module got it's own bugzilla product. I'm fine
with both solutions, as long at it gets clearly documented somewhere...
Comment 33 Ignacio Casal Quinteiro (nacho) 2011-01-27 11:05:39 UTC
Ups, right, I must be blind :D
Comment 34 Martin Pitt 2011-02-02 11:57:56 UTC
(In reply to comment #26)
> J5: I'm leaning towards making sure that we don't release PyGObject without the
> gio static bindings until PyGIO is ready to be released. Easiest may be
> reverting the commit that removed them and disabling the gio tests, if we
> cannot hold releasing for long enough.

Alternatively we could also change Makefile.am to run the gio tests separately?

As this is currently blocking the upgrade to a newer pygobject, want me to look into this and propose a patch?
Comment 35 Martin Pitt 2011-02-02 12:13:09 UTC
Summary from discussion in #introspection:

- General feeling is that the better medium-term solution is to split out the bindings (pygio, or slurp them into pygtk).

- However, we are a bit pressed to release a new pygobject soon, as GTK3 will land RSN and we need a new pygobject to provide a good PyGI support.

- It seems the path of least resistance and dependencies is to temporarily add back the static bindings into pygobject, and drop them again when either

  * the standalone pygio is released

  or

  * pygobject gets released and branched to a stable 2-28-branch; then they could be removed again from master.

Major objections to this?
Comment 36 Martin Pitt 2011-02-02 14:33:32 UTC
Created attachment 179881 [details] [review]
Put back static gio bindings

In case we want to revert the removal for now, this first patch is the reversion of bca5834fc8fa342149e0e (git revert mostly worked, just had a small conflict in configure.ac).
Comment 37 Martin Pitt 2011-02-02 14:35:06 UTC
Created attachment 179882 [details]
Run gio tests separately when enabled

... and run the gio tests in a separate process, so that the Gio tests can work as well.

Ignacio said he'd try to get pygio working this week, so let's hold off on those two for now. But putting them here in case we need them (I need them urgently to untangle our packages, so I use them locally for now).
Comment 38 John Stowers 2011-02-02 23:37:42 UTC
> Ignacio said he'd try to get pygio working this week, so let's hold off on
> those two for now. But putting them here in case we need them (I need them
> urgently to untangle our packages, so I use them locally for now).

Just to be clear, standalone pygio currently *works* in the sense that it builds, runs, and things seem to perform as before.

There is a bit of work to do inorder to make it a proper package, adding back the documentation support, tests, examples, etc.

Ignacio is welcome to help with this. As I said earlier, dieter and I seem to be a little busy with other things at the moment.
Comment 39 Ignacio Casal Quinteiro (nacho) 2011-02-02 23:40:06 UTC
So is there any reason to keep the pygobject patches in your branch and not to push them?
Comment 40 John Stowers 2011-02-02 23:52:40 UTC
(In reply to comment #39)
> So is there any reason to keep the pygobject patches in your branch and not to
> push them?

a) agreement from the pygobject maintainers that this is the right thing to do (seperate pygio)
b) confirmation it works on windows (waiting for dieter)
c) movement of the pygio tests to pygio
Comment 41 Ignacio Casal Quinteiro (nacho) 2011-02-02 23:54:15 UTC
a) afaik the maintainers are agree about this.
b) ok
c) I can try to make this unless there is anybody else working on it.
Comment 42 Ignacio Casal Quinteiro (nacho) 2011-02-02 23:55:16 UTC
btw, I'd like to see the pygobject patches in pygobject master before starting to work on anything of this. So first we focus on having everything ready in pygobject.
Comment 43 John Stowers 2011-02-02 23:58:07 UTC
Two more things, for general feedback

J5, do you see any massive shortcomings in pyglib wrt python 2/3 compatibility
that will see it radically change in future? I would like to think that
libpyglib is sensibly stable, or if any changes are needed in it, please let us
know first so we may keep pygio working.

Secondly, do we forsee a similar thing happening in future if/when static glib
is removed?
Comment 44 John Stowers 2011-02-03 00:00:39 UTC
(In reply to comment #42)
> btw, I'd like to see the pygobject patches in pygobject master before starting
> to work on anything of this. So first we focus on having everything ready in
> pygobject.

Sorry, which changes?

Would you like to see the revert land before helping us finish pygio, or you would like to see the for-pygio branch land on pygobject (and hence we do not do the revert?)
Comment 45 Ignacio Casal Quinteiro (nacho) 2011-02-03 09:52:40 UTC
I would like to see the for-pygio branch land on pygobject.
Comment 46 John Stowers 2011-02-04 01:02:26 UTC
(In reply to comment #45)
> I would like to see the for-pygio branch land on pygobject.

Ok. Just waiting for dieter to confirm things work on windows with the split.
Comment 47 Dieter Verfaillie 2011-02-04 15:02:02 UTC
My apologies for the silence, this week has been exceptionally busy...
I managed to advance a little on the windows related changes today (nothing
really works yet) and plan to finish it starting monday as the weekend is
reserved for fosdem ;)
Comment 48 Dieter Verfaillie 2011-02-06 20:18:20 UTC
While trying to explain the current situation, this comment has grown
quite large, so again, my apologies...

Status
======

I've pushed the windows related work already done to the following
repos/branches [1]:

pygobject: https://github.com/dieterv/pygobject/tree/for-pygio-windows
pygio    : https://github.com/dieterv/pygio/tree/for-pygio-windows

Nothing works yet and there's some bad news and (maybe?) some good news:

The bad news
============

Unless I missed something, we're stuck on pygobject where we need to build
and distribute libpyglib as a shared library (.dll) because pygio needs it
and can only be built after pygobject has been installed. The problem:

1) Python's distutils build_clib command only supports building static
libraries and as an extra bonus does not even support installing those.
Comment from build_clib.py: 
# This might be confusing: both build-clib and build-temp default
# to build-temp as defined by the "build" command.  This is because
# I think that C libraries are really just temporary build
# by-products, at least from the point of view of building Python
# extensions -- but I want to keep my options open.

2) Python's distutils build_ext command obviously supports building shared
libraries and also installing those, but it looks like it only supports
building Python extension modules and not plain old "normal" shared
libraries.

The good news (maybe)
=====================

I'm not known for giving up easily and out of nowhere tried pulling
build_shlib through google (that would have been the distutils command name
I'd use when writing such a monster). So, guess I was surprised to learn that
not only we're not the first ones to encounter such a limitation in distutils
but also that it has already been (partly) solved [2].

NIST's OOF2 project contains distutils build_shlib and install_shlib commands,
that seem to be able to do what we require to build libpyglib as a shared
library. Only, it does not work out of the box on windows with MinGW (and let's
not even begin talking about what would happen with those other windows
compilers [3]).

The OOF2 project lives here: http://www.ctcms.nist.gov/oof/oof2/.
The distutils commands mentioned above and related files come from:
http://www.ctcms.nist.gov/oof/oof2/source/oof2-2.0.5a11.tar.gz (in the
shlib directory)

The maybe part???
=================

The required code (build_shlib.py, install_shlib.py and setup_shlib.py)
is marked as follows:
# This software was produced by NIST, an agency of the U.S. government,
# and by statute is not subject to copyright in the United States.
# Recipients of this software assume all responsibilities associated
# with its operation, modification and maintenance. However, to
# facilitate maintenance we ask that before distributing modified
# versions of this software, you first contact the authors at
# oof_manager@ctcms.nist.gov.

I do not know if it is OK to include and (heavily?) modify such code, if it's
license (if it can be called a license?) is compatible with the LGPL and how
it possibly affects the pygobject project in general. IANAL ;)

I'm willing to continue to invest the time and effort to try to get this
working (for MinGW/MSYS only!) and properly integrate it into dsextras if:
- it is guaranteed that static bindings will continue to be supported
throughout pygobject's 2.X cycle, taking into account that there's little
maintenance on some and mostly no more maintenance whatsoever on other
static bindings projects;
- we are allowed to take the OOF2 code mentioned above and base whatever's
needed on that;
- permission is given to refactor dsextras into a package (keeping backwards
compatibility, of course).

Regards,
Dieter

[1] For those who are wondering why this stuff lives on github: I'm living
most of my online life behind an extremely restrictive corporate firewall
(basically port 80 & 443 and yeah, it sucks) and sadly git.gnome.org does not
yet seem to provide access through the http protocol. At home I can't currently
count on having enough time to properly maintain branches etc on git.gnome.org.
Besides, I'm not yet confident enough with git to not screw things up which,
if I do, is no big deal on github ;)

[2] More proof that if it's not on pypi, it doesn't exist ;)

[3] Visual Studio's/some MS compiler used to be a requirement for building the
official pyg* packages. Glad that restriction has finally been lifted :)
Comment 49 Martin Pitt 2011-02-09 17:03:06 UTC
Comment on attachment 179881 [details] [review]
Put back static gio bindings

I applied the two patches to the pygobject-2-28 stable branch, as discussed on IRC. They won't get back to master, as the longer-term solution is the separate pygio project and GI.
Comment 50 Martin Pitt 2011-02-09 17:07:54 UTC
I think from pygobject's perspective this is done now. 2.28.x will have the gio bindings, and future major releases will drop support for GTK2/static bindings/etc. anyway.

Do you want to keep this open against a to-be-created pygio project? if so, please do and reassign.

Thanks to everyone involved!
Comment 51 Dieter Verfaillie 2011-02-09 19:49:38 UTC
Thanks!

I have one question left, when such a future pygobject release drops support for
GTK2/static bindings, is that version going to provide for parallel
installation like was done with pygtk-0.6.x/pygtk-2.x? If yes, then there would
be no need for a split PyGIO package and everything could continue to
coexist peacefully for as long as it is needed (without requiring extra work
on legacy static bindings projects)...
Comment 52 Martin Pitt 2011-02-10 05:35:27 UTC
Parallel installability was mentioned in yesterday's discussion. master doesn't do that yet, but is meant to be for the next major release.
Comment 53 Dieter Verfaillie 2011-02-10 10:44:25 UTC
Are there any ideas or plans on how it is going to be done?

For the windows builds, I was playing with the idea of holding of the
changes to the installation layout for pygobject-2.28 [1] and keep
installing things as before:
-site-packages
  - gtk-2.0
    - glib
    - gobject
    - ...
  - pygtk.py
  - pygtk.pth

and only changing it for pygobject-2.30, wich will also need a change in
it's package name, for example pygobject2-2.30 [2]:
-site-packages
  - glib
  - gobject
  - gi
  ! no more pygtk.p*, yaay :)

An aio installer for 2.30 can then detect the existence of the
pygtk.pth file and complain loudly if it's there. The separate
package installer can't offer such protection (due to distutils
bdist_msi limitations), but those are for "advanced" users
anyway imho...

This has the effect that when both pygobject-2.28 and pygobject2-2.30
are installed (and pygtk.pth does not exist):
- If you want to use static bindings, you're required to "import pygtk;
  pygtk.require('2.0')" before importing glib, gobject, gtk, etc.
  This has always been a requirement, but due to the presence of the .pth
  file by default some people may have forgotten about that and we risk
  breaking a small amount of scripts here and there...
- when using introspection, you simply don't do the "require" dance and
  import glib, gobject and gi straight away...

Anyway, just a tought...

[1] done in commit 792e679c06df4357843fd310c7953a931172fc99
[2] we can only have 1 "package" with the same name installed at the same time
Comment 54 Simon van der Linden 2011-02-10 12:56:14 UTC
Dieter, I was thinking about that this morning, since we can require specific versions directly with g-i. Maybe a bug can be filled with this proposal?
Comment 55 Simon van der Linden 2011-02-11 16:32:51 UTC
The doc was not actually removed from the tree:

docs/reference/pygio-appinfo.xml
docs/reference/pygio-applaunchcontext.xml
docs/reference/pygio-asyncresult.xml
docs/reference/pygio-bufferedinputstream.xml
docs/reference/pygio-bufferedoutputstream.xml
docs/reference/pygio-cancellable.xml
docs/reference/pygio-classes.xml
docs/reference/pygio-constants.xml
docs/reference/pygio-datainputstream.xml
docs/reference/pygio-dataoutputstream.xml
docs/reference/pygio-drive.xml
docs/reference/pygio-emblemedicon.xml
docs/reference/pygio-emblem.xml
docs/reference/pygio-fileattributeinfo.xml
docs/reference/pygio-fileenumerator.xml
docs/reference/pygio-fileicon.xml
docs/reference/pygio-fileinfo.xml
docs/reference/pygio-fileinputstream.xml
docs/reference/pygio-filemonitor.xml
docs/reference/pygio-fileoutputstream.xml
docs/reference/pygio-file.xml
docs/reference/pygio-filterinputstream.xml
docs/reference/pygio-filteroutputstream.xml
docs/reference/pygio-functions.xml
docs/reference/pygio-icon.xml
docs/reference/pygio-inputstream.xml
docs/reference/pygio-loadableicon.xml
docs/reference/pygio-memoryinputstream.xml
docs/reference/pygio-memoryoutputstream.xml
docs/reference/pygio-mountoperation.xml
docs/reference/pygio-mount.xml
docs/reference/pygio-outputstream.xml
docs/reference/pygio-seekable.xml
docs/reference/pygio-simpleasyncresult.xml
docs/reference/pygio-themedicon.xml
docs/reference/pygiounix-classes.xml
docs/reference/pygio-unixinputstream.xml
docs/reference/pygio-unixoutputstream.xml
docs/reference/pygio-volumemonitor.xml
docs/reference/pygio-volume.xml

Unless I did not get some point, I'm reopening this bug.
Comment 56 Simon van der Linden 2011-02-11 21:12:09 UTC
Created attachment 180687 [details] [review]
Remove GIO documentation
Comment 57 Simon van der Linden 2011-02-11 21:12:21 UTC
Created attachment 180688 [details] [review]
Remove last GIO-related bits
Comment 58 Simon van der Linden 2011-02-11 21:16:32 UTC
Attachment 180687 [details] pushed as e4ebbd7 - Remove GIO documentation
Attachment 180688 [details] pushed as 0c20977 - Remove last GIO-related bits