GNOME Bugzilla – Bug 642048
Parallel installation for pygobject-2.28 and pygobject-2.28+1
Last modified: 2011-08-13 10:06:39 UTC
While discussing the removal of the static gio bindings from pygobject (see bug 638899), a remark was made about the possible parallel installation of what will become pygobject-2.28 (gi + static bindings support including gio) and what I assume will become pygobject-2.30 (gi only). I was asked to file a new bug report for an idea I've been working on that could ensure that ability. For the windows builds (but the technique should also apply elsewhere), I have been testing out the idea of holding back the changes to the installation layout for pygobject-2.28 [1] and keep installing things as before: -site-packages - gtk-2.0 - glib - gobject - gtk - ... - pygtk.py - pygtk.pth and only changing it for pygobject-2.30, which will also need a change in it's package name, for example pygobject2-2.30 [2]: -site-packages - glib - gobject - gi ! no more pygtk.py and pygtk.pth, yaay :) This has the effect that when both pygobject-2.28 and pygobject2-2.30 are installed (and pygtk.pth does not exist): - If you want to use static bindings, you're required to "import pygtk; pygtk.require('2.0')" before importing glib, gobject, gtk, etc. This has always been a requirement, but due to the presence of the .pth file by default, some people may have forgotten about that and we risk breaking a small amount of scripts here and there... - when using introspection, you simply don't do the "require" dance and import glib, gobject and gi straight away... An example session illustrating the idea: ============================================================ $ ls /d/bin/Python27/Lib/site-packages/pygtk.pth ls: /d/bin/Python27/Lib/site-packages/pygtk.pth: No such file or directory $ cat /d/bin/Python27/Lib/site-packages/gtk-2.0/gobject/__init__.py # -*- Mode: Python; py-indent-offset: 4 -*- # pygobject - Python bindings for the GObject library # Copyright (C) 2006 Johan Dahlin # # gobject/__init__.py: initialisation file for gobject module ... $ cat /d/bin/Python27/Lib/site-packages/gobject/__init__.py print('pygobject2-2.30 introspection goodness') $ python -c "import pygtk; pygtk.require('2.0'); import gobject; print(gobject.GObject())" <gobject.GObject object at 0xbd6b98 (GObject at 0xd34000)> $ python -c "import gobject" pygobject2-2.30 introspection goodness ============================================================ A possible way to get rid of the pygtk.pth file when 2.30 releases could maybe look like this: pygobject-2.28.0 would still install pygtk.pth so gi users can import glib/gobject/gi strait away (and no import statements will need to be changed when uprading to 2.30) and a 2.28.1 release could then be made together with pygobject2-2.30 that no longer installs pygtk.pth by default (but creates a pygtk.pth.example file with a big warning in it). 2.30 would only allow 2.28.1 to be installed in parallel and that's that :) Anyway, just a thought... [1] done in commit 792e679c06df4357843fd310c7953a931172fc99 [2] we can only have 1 "package" with the same name installed at the same time
I've published some branches implementing this idea for the distutils based setup.py and dsextras.py (used to create windows binaries). https://github.com/dieterv/pygobject/tree/pygobject-2-28/ -> branched from the pygobject-2-28 branch on git.gnome.org + bdist_wininst related patch https://github.com/dieterv/pygobject/tree/pygobject-2-28-1/ -> branched from my pygobject-2-28 branch described above + rename pygtk.pth to pygtk.pth.example https://github.com/dieterv/pygobject/tree/pygobject-2-30/ -> branched from the master branch on git.gnome.org + bdist_wininst related patch + setup.py/dsextras.py catching up to the current state of things + rename to pygobject2 ! No autotools related files have been touched during this process, so don't expect ./configre && ./make to work... The zip files published on https://github.com/dieterv/pygobject/downloads/ have been created from the branches described above by doing the followoing (from an MSYS bash console on Windows XP): $ git checkout pygobject-2-28 $ rm -rf build/* && python setup.py bdist --format=zip $ mv dist/pygobject-2.27.91.win32.zip dist/bug642048_pygobject-2.28.0.win32.zip $ git checkout pygobject-2-28-1 $ rm -rf build/* && python setup.py bdist --format=zip $ mv dist/pygobject-2.27.91.win32.zip dist/bug642048_pygobject-2.28.1.win32.zip $ git checkout pygobject-2-30 $ rm -rf build/* && python setup.py bdist --format=zip $ mv dist/pygobject2-2.90.1.win32.zip dist/bug642048_pygobject2-2.30.0.win32.zip
This won't work as it breaks ABI for the static bindings. Right now there is a pth file that does not require the addition of the import pygtk magic. I would think for 3.0 we would internalize all static modules such that you need to import them from the registry and we namespace it - from gi3.repository import GObject BTW, the next version will be pygobject-3 since we are breaking ABI here so it warrents a major version bump.
Oh, that's even better! If pygobject-3.0 only installs gi3 into site-packages (or at least nothing that would conflict with pygobject-2.28's glib, gobject and gi), then indeed nothing needs to be done and this can be closed as invalid I suppose. Thanks for the info btw!
How does this intersect with Python 3? I am also frightened by the prospect of having 3 different Python bindings here: * Old pygtk * pygobject2 + gi * pygobject3 But please - with my GNOME release team hat on, can we do unstable work in branches, and only land to master when it's "ready"? Where "ready" is at least "ready for testing, may have bugs".
This is how it fleshed out in order to effectively transition. In hind sight we should have just kept with the fork but there were issues there also. Namely we weren't getting fixes in to PyGObject fast enough. As for master/branch issue for GI the master branch is stable. For legacy it does break ABI. This is a brief pain point. I understand your issues with us doing it this way but for us master represents the branch we want developers hacking on and testing. After we get this bug put to bed we should be in better shape but PyGTK still needs to rely on pygobject-2-28 branch in jhbuild whether or not we make this parallel installable. We will try better in the future to keep things in sync.
Created attachment 187311 [details] [review] Bump version to 3.0 for partial parallel installability This isn't complete, but it's a start.
Also can you explain the python2/3 plans? Will this new version support both Python 2 and 3?
Actually the pc files will mostly go away. Nothing should be compiling against pygobject 3 except internal modules. The module name needs to be installed to gi3 and gobject needs to go under _gobject and installed inside gi3, same with glib. All examples and tests need to be moved to use gi3 and remnants of static bindings purged. All versions from 2.28 on support both Python 2 and 3 out of the box.
Actually we most likely should keep the gi namespace and just say you need to turn off introspection in pygobject2 to do a parallel install. This would make app developers lives a lot easier at the expense of packager confusion.
(In reply to comment #9) > Actually we most likely should keep the gi namespace and just say you need to > turn off introspection in pygobject2 to do a parallel install. This would make > app developers lives a lot easier at the expense of packager confusion. this sounds like a good solution actually - I hope the packager pushback won't be too hard.
Created attachment 192483 [details] [review] up platform version to 3.0
Created attachment 192484 [details] [review] introspection is no longer optional
Created attachment 192485 [details] [review] remove pygtk.py
Created attachment 192486 [details] [review] up platform version to 3.0
Created attachment 192487 [details] [review] introspection is no longer optional
Created attachment 192488 [details] [review] remove pygtk.py
Created attachment 192489 [details] [review] move the static bits internal to gi and refactor build files * the glib module now becomes the gi._glib module * the gobject module now becomes the gi._gobject module * we do this so we can install in parallel with PyGObject 2
Created attachment 192490 [details] [review] refactor gi module to import and use internal _gobject module
Created attachment 192491 [details] [review] refactor to use the new internal _glib and _gobject modules * use relative imports instead of aboslute * fix the C imports to import the internal _gobject libs * add a check to see if the PyGObject 2 gobject module was already imported
Created attachment 192492 [details] [review] refactor the internal _glib module to import correct modules
Created attachment 192493 [details] [review] refactor tests to only use PyGObject 3 syntax * for PyGObject 3 we want to discourage the use of legacy interfaces * Using interfaces like from gi.repository import GObject makes sure that the internal _gobject module is loaded and not PyGObject 2's gobject module which would cause the application to not work correctly
Created attachment 192494 [details] [review] rename the pyglib shared library so we don't load the old one
Created attachment 192495 [details] [review] refactor, add objects and types to the correct internal module
Created attachment 192496 [details] [review] add DynamicGLibModule which works like DynamicGObjectModule
Created attachment 192497 [details] [review] fix tests to use the new GLib module
Created attachment 192498 [details] [review] avoid dependency issue by importing the internal gobject
Created attachment 192499 [details] [review] make GObject and GLib able to take overrides * derive directly from DynamicModule instead of InterfaceModule
These patches need to be applied in sequence. Together they make PyGObject master parallelable installable with 2.28
Tested 2.28 without introspection and master with these patches applied and it sure looks good :) Out of curiosity, why does libpyglib-gi-2.0-@PYTHON_BASENAME@ not use @PLATFORM_VERSION@ (ie 3.0) instead of 2.0?
Because GLib and GObject are still at version 2.0. Thanks for the testing
all committed