GNOME Bugzilla – Bug 788570
Appveyor build file with pygobject-win32 binaries
Last modified: 2017-12-08 10:44:59 UTC
Created attachment 360984 [details] [review] appveyor.yml There is a patch for building meld's installer via appveyor, though the commands for actual build are not appveyour-specific I've selected pygobject-win32 PyGI AIO instead of mingw binaries because I was running meld 3.17+ from a checkout with those binaries on python 3.4 (64-bit) for many months and doesn't have any major problems, but has small incomatibilities while trying to use meld checkout with mingw binaries. The build depends on initial patches for cx_Freeze 5 from Kai - https://mail.gnome.org/archives/meld-list/2017-August/msg00012.html And on patch for fixing dll list for latest pygobject-win32 3.24.1. It looks that installer installs meld in a proper way, but unfortunately running the installed meld doesn't work as fine as pure meld checkout. 1. The major issue is that resulting binary starts only with current directory being the directory of the Meld.exe, starting from any another directory (including start menu shortcut) leads to exception window about LoadLibary-related problem. 2. Another major issue is no inner-word diff highlighting in file comparison (looks like an exception was thrown and cought, but no console to see it) 3. After installing on a machine with no gtk installed previously the first execution shows Meld window only after several minutes (because of constructing the font cache?). All next executions works fast. 4. Trying to use version control view pops up lots of black console windows while executig VCs. Those issues doesn't look to be related to appveyor integration, so I created this issue as patch set for appveyor building, assuming it is working ok. Patches are also at https://github.com/galkinvv/meld/commits/appveyor-pygi-aio
Created attachment 360985 [details] [review] latest pygobject-win32 DLL list Adding required patch for DLL list. The build takes 2-6 minutes depending on appveyour load. Example of succeed build log: https://ci.appveyor.com/api/buildjobs/4jx4a9e0c34x6mq5/log
Created attachment 360986 [details] required patches from Kai Willadsen Adding base patches to have all required data in gnome.org bugzilla.
Created attachment 361289 [details] [review] (0005)patch for working from any current directory It appears that initialization problems was due to change of cx_freeze version. cx_freeze5 by default doesn't zip packages. Which has better compatibility for most packages but for gtk + cairo referencing same gobject dlls breaking is in the other direction (non-zipping). Attaching patch making cx_freeze 5 behave like previous version. Using zipped packages is generally faster since less filesystem access is performed. It was disabled by default in cx_freeze 5 only because zipping breaks some packages.
Created attachment 361290 [details] [review] (0006)Fix multiprocessing and freezing compatibility After this patch a seemingly-working-fine unofficial installer was built! https://ci.appveyor.com/api/buildjobs/affg705qdf7ycji6/artifacts/dist%2FMeld-3.18.01-win32.msi This doesn't resolve very-long-first-start problem https://bugzilla.gnome.org/show_bug.cgi?id=785313 and console windows popups while running VCs but it is mostly working fine. It has version 3.18.01 (extra zero) to distinguish from future official builds. github with patches - https://github.com/galkinvv/meld/tree/cxfreeze5-fixing succeeded build - https://ci.appveyor.com/project/galkinvv/meld/build/cxfreeze5-fixing-8d46a0a.30
I just wanted to let you know that I can't review or test these at the moment. I definitely will! It will just take me a week or two to get to them. I do appreciate the work you've done here, and I'm cautiously okay with continuing to use the all-in-one installer for this build.
*** Bug 785314 has been marked as a duplicate of this bug. ***
I've pushed your patches to master. I also took the opportunity to reformat and break it up a bit, just because I was finding it a bit difficult to read. I'll look at grafting the commits from master to 3.18 soon and making a real 3.18 release for Windows. The currently configured project is at https://ci.appveyor.com/project/kaiw/meld, and the most recent build artifact... should work. I haven't actually figured out what using these releases is going to look like yet, but I'm extremely happy that this part of the process at least is all good. Thank you so much!
Okay these changes are now on 3.18 as well, so I'm going to close this bug. I don't think we'll know how this is going to work until I've cut a release on it, but in theory the appveyor build is set up to trigger on tags, at which point I can just download the artifact, test it, and push to GNOME hosting. At some point in the future, we could consider switching to S3 hosting so that it was easy to hook up auto-deployment. I'd also really like it if I wasn't the only person who could do these things, but I don't currently have the energy to do the admin necessary to set up anything more complicated. In theory, it would be good to have an account on S3 and AppVeyor for Meld as a project. I'd also maybe consider moving the web hosting away from GitHub on to a static S3 site or something.
I noted that current master now requires gtk 3.20+ this would break the windows build since now pygobject-win32 provides only gtk 3.18. Fortunately msys2-mingw64 provides gtk+3.22. I tried it today again and now it mostly works with meld checkout: after commenting out os.environ['XDG_DATA_DIRS'] = data_dir in conf.py meld prints a lot of Gtk-Criticals during startup but continues working fine. I switched my windows workflow to this version to estimate its stability. So I think while pygobject-win32 build compatibility is strong enough for building 3.18 installer with it, the future switch to mingw64 DLLs would be already unavoidable with next meld version.
(In reply to Vasily Galkin from comment #9) > I noted that current master now requires gtk 3.20+ > > this would break the windows build since now pygobject-win32 provides only > gtk 3.18. arg, I'm sorry! I assumed but didn't check that the pygi-aio numbers matched GTK+ releases, and that we'd got a GTK+ upgrade in there. That's disappointing. So for a start, I'm actually okay to downgrade our requirement to 3.18 if this becomes a real problem. Other than a minor GtkSourceView feature, these were mainly just removing hacks for earlier GTK+ versions. > Fortunately msys2-mingw64 provides gtk+3.22. I tried it today again and now > it mostly works with meld checkout: > after commenting out os.environ['XDG_DATA_DIRS'] = data_dir in conf.py > meld prints a lot of Gtk-Criticals during startup but continues working fine. > > I switched my windows workflow to this version to estimate its stability. > > So I think while pygobject-win32 build compatibility is strong enough for > building 3.18 installer with it, the future switch to mingw64 DLLs would be > already unavoidable with next meld version. Honestly, I think this was long-term unavoidable anyway. I had significant problems trying to get the msys2 build going, but I also have no experience in doing this. While looking I found a few projects also using appveyor and pygobject, all doing slightly different things, e.g., https://github.com/mypaint/mypaint/ https://github.com/quodlibet/quodlibet
Do you know if pygobject-win32 does anything special than what msys2-mingw64 does? Sorry for the dumb question, I'm rather ignorant on GTK/PyGTK.
I think the main difference is the compiler used for python (and maybe for python extensions). pygobject-win32 expects official python build with VisualStudio10, and msys2-mingw64 uses mingw. Also pygobject-win32 use a lot of non-upstream patches: https://github.com/tumagonx/pygi-mingw-patches/