GNOME Bugzilla – Bug 775281
Allow simply setup an uninstalled environment based on msys2
Last modified: 2016-12-13 18:00:18 UTC
Created attachment 340939 [details] [review] Patch 1 I think it would be convenient to let people work on GStreamer using gst-build using a simple setup based on msys2 (as it provides a very simple way to install our dependencies). I started implementing it in our setup script, and the user can get started installing python3 > 3.5 having it in PATH and latest msys2[0]. It is still a bit hacky and ugly but I wonder if you generaly agree with that approach? This all started thanks to Scott D Phillips who explained his method in a ML thread. [0] http://msys2.github.io
Created attachment 340940 [details] [review] uninstalled: Properly setup uninstalled env on windows
Created attachment 340941 [details] [review] Work around issue using msys2 provided zlib
Created attachment 340942 [details] [review] Use mesonintrospect to set library path this way the uninstalled target can be used before the libraries are built
Created attachment 340943 [details] [review] setup: Allow setting up a developement invironment based on msys2 This environment is meant to be setup in a Visual Studio 'Native X64 tools' terminal, and takes for granted that msys2 has been installed, Visual Studio has C(++) support has been installed and python3 is installed and is present in PATH.
Review of attachment 340943 [details] [review]: ::: setup @@ +92,3 @@ + + def get_libname(self, dll_name): + MESON_GIT = 'https://github.com/mesonbuild/meson.git' change \d to \d+ and then you can delete the LIBNAME_EXCEPTIONS than have to do with `dash number`. @@ +101,3 @@ + def get_configs(self): + super(Msys2Configurer, self).get_config() + \ + 'mingw-w64-x86_64-libxml2', get_config should be get_configs, but rtsp_server is building ok for me so you can probably just delete this. @@ +108,3 @@ + lib_dir = lib_dir[:-3] + 'lib' + + 'mingw-w64-x86_64-json-glib'] self.get_libname @@ +137,3 @@ + print('Generating .lib file for %s ...' % f, end='') + sys.stdout.flush() + self.make_lib @@ +169,3 @@ + return False + + lib_dir = lib_dir[:-3] + 'lib' put mingw64/bin before usr/bin to prefer the mingw64 executables
maybe add something like this: > def should_gen_lib(self, dll): > lib_dir, dll_name = os.path.split(dll) > if lib_dir.endswith('bin'): > lib_dir = lib_dir[:-3] + 'lib' > > lib_name = self.get_libname(dll_name) > lib = os.path.join(lib_dir, lib_name) > if not os.path.exists(lib): > return True > return os.stat(dll).st_mtime_ns >= os.stat(lib).st_mtime_ns and then you can change the check in make_libs to something like: if f.endswith('.dll') and self.should_gen_lib(os.path.join(base, f)): as an optimization to avoid unnecessary building of .lib files.
This is cool in principle, but from a support perspective it worries me. Windows is already difficult enough to support, with this we'd add the possibility of external glib/ffmpeg builds into the mix.
I think MSYS2 isn't good as a long-term option for many reasons (we should only use Python3 and Visual Studio), but since it's our best option right now I can understand why Scott would want this. I will go with whatever Tim decides. My concern right now is that the gst-uninstalled script is becoming this monolithic chunk of code. I think we should try to split functionality out of it otherwise it is a large hurdle for people to grok it and add to it.
(In reply to Nirbheek Chauhan from comment #8) > I think MSYS2 isn't good as a long-term option for many reasons (we should > only use Python3 and Visual Studio), One thing to consider is that the glib project recommends you use the msys2 binaries: https://www.gtk.org/download/windows.php > The preferred way to obtain GTK+ and its dependencies in binary form > for Windows is to use the MSYS2 project. (In reply to Nirbheek Chauhan from comment #8) > My concern right now is that the gst-uninstalled script is becoming this > monolithic chunk of code. I think we should try to split functionality out > of it otherwise it is a large hurdle for people to grok it and add to it. Good point, the msys2 stuff might have a happier home in a totally separate script.
> (In reply to Nirbheek Chauhan from comment #8) > > I think MSYS2 isn't good as a long-term option for many reasons (we should > > only use Python3 and Visual Studio), > One thing to consider is that the glib project recommends you use the msys2 binaries: > https://www.gtk.org/download/windows.php I think that is partly because no one on the GTK+ team has the time or the resources to provide GTK+ releases on Windows. However, that is an aside. What I mean is that in the long term we want to build everything from source with Visual Studio instead of relying on MSYS2 for prebuilt binaries. In fact, we shouldn't even need to use the MSYS2 environment[1]. I think the goal of building all of GTK+ and deps with Visual Studio (+ Meson) is achievable. 1. My experience with MSYS2 has left me with serious doubts as to the quality of the MSYS2 distribution.
I don't see any reason not to go ahead with this, I was just mentioning this now. We may decide to remove support for this again in future. The glib project mostly doesn't care about Windows, and upstream glib is more often than not problematic from a GStreamer point of view. There are very very few people hacking GLib on Windows and let's just say my impression is their workloads differ from what GStreamer does. In any case, what is written on the home page does not really mean much for us one way or another IMO.
(In reply to Tim-Philipp Müller from comment #11) > I don't see any reason not to go ahead with this, I was just mentioning this > now. We may decide to remove support for this again in future. > > The glib project mostly doesn't care about Windows, and upstream glib is > more often than not problematic from a GStreamer point of view. There are > very very few people hacking GLib on Windows and let's just say my > impression is their workloads differ from what GStreamer does. In any case, > what is written on the home page does not really mean much for us one way or > another IMO. Point taken. Sounds like having the msys2 stuff in a separate script off to the side is even more of a good idea given that.
> (In reply to Nirbheek Chauhan from comment #8) > > My concern right now is that the gst-uninstalled script is becoming this > > monolithic chunk of code. I think we should try to split functionality out > > of it otherwise it is a large hurdle for people to grok it and add to it. > > Good point, the msys2 stuff might have a happier home in a totally separate > script. I started with a totally separate script but in the end there was quite a lot of duplication so decided to go and factor it in. I could now actually move it to a separate script again just moving the subclass there.
(In reply to Nirbheek Chauhan from comment #8) > I think MSYS2 isn't good as a long-term option for many reasons (we should > only use Python3 and Visual Studio), but since it's our best option right > now I can understand why Scott would want this. I will go with whatever Tim > decides. > > My concern right now is that the gst-uninstalled script is becoming this > monolithic chunk of code. I think we should try to split functionality out > of it otherwise it is a large hurdle for people to grok it and add to it. Actually I almost did not touch gst-uninstalled at all for that. I simply refactored it sensibly so we do not depend on build to be done to set the environment but instead rely on the meson introspection information to do so, which is much nicer in any case. Also I do not expect much more thing to be needed in there in the future tbh. Problem was about the setup script, and I am coming up with a separate script for it (rebuilding from scratch to check it still works).
Review of attachment 340943 [details] [review]: ::: setup @@ +140,3 @@ + print('DONE') + + def get_libname(self, dll_name): I submitted patches in bug 775292 and bug 775293 for dtls and schroedinger build problems. I didn't see any problem with openjpeg. Hopefully with those you can just totally remove these hacks.
Created attachment 340954 [details] [review] Add a script to setup a developement environment based on msys2 This environment is meant to be setup in a Visual Studio 'Native X64 tools' terminal, and takes for granted that msys2 has been installed, Visual Studio has C(++) support has been installed and python3 is installed and is present in PATH.
(In reply to Scott D Phillips from comment #15) > Review of attachment 340943 [details] [review] [review]: > > ::: setup > @@ +140,3 @@ > + print('DONE') > + > + def get_libname(self, dll_name): > > I submitted patches in bug 775292 and bug 775293 for dtls and schroedinger > build problems. I didn't see any problem with openjpeg. Hopefully with those > you can just totally remove these hacks. Cool, will try with those patches, I had some new issue building qtdemux this time, will look at that tomorrow. (In reply to Thibault Saunier from comment #16) > Created attachment 340954 [details] [review] [review] > Add a script to setup a developement environment based on msys2 > > This environment is meant to be setup in a Visual Studio > 'Native X64 tools' terminal, and takes for granted that > msys2 has been installed, Visual Studio has C(++) support > has been installed and python3 is installed and is present > in PATH. In that new version I should have addressed all you comments, thanks for the review!
Review of attachment 340954 [details] [review]: maybe rename setup_msys2.py to msys2_setup.py. I like being able to type ./s<tab><enter> for setup :) Apart from that and the hacks which I think you'll be able to drop soon all patches lgtm.
(In reply to Scott D Phillips from comment #18) > Review of attachment 340954 [details] [review] [review]: > > maybe rename setup_msys2.py to msys2_setup.py. I like being able to type > ./s<tab><enter> for setup :) Apart from that and the hacks which I think > you'll be able to drop soon all patches lgtm. OK done locally, I am waiting for #775292 to be fixed so I can remove the last hack and push :)
Attachment 340940 [details] pushed as d3c029f - uninstalled: Properly setup uninstalled env on windows Attachment 340941 [details] pushed as 2b74d4d - Work around issue using msys2 provided zlib Attachment 340942 [details] pushed as 954c276 - Use mesonintrospect to set library path Attachment 340954 [details] pushed as 1c46eb2 - Add a script to setup a developement environment based on msys2