After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 789316 - Add Meson support in cerbero
Add Meson support in cerbero
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: cerbero
1.12.x
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
: 768028 (view as bug list)
Depends on: 733067 789663 789726
Blocks: 788273 791670 794604
 
 
Reported: 2017-10-22 10:00 UTC by Xavier Claessens
Modified: 2018-05-02 17:44 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Add call_env arg to shell.call() (1.44 KB, patch)
2017-10-22 10:04 UTC, Xavier Claessens
none Details | Review
Config: Add modify_env_for_python3 helper (1021 bytes, patch)
2017-10-22 10:04 UTC, Xavier Claessens
none Details | Review
Fix meson recipe (939 bytes, patch)
2017-10-22 10:05 UTC, Xavier Claessens
none Details | Review
Update meson version to 0.43.0 (924 bytes, patch)
2017-10-22 10:18 UTC, Xavier Claessens
none Details | Review
Drop meson recipe and add it as part of bootstrap (2.58 KB, patch)
2017-10-22 12:58 UTC, Xavier Claessens
none Details | Review
Drop meson recipe and add it as part of bootstrap (2.60 KB, patch)
2017-10-22 14:10 UTC, Xavier Claessens
none Details | Review
Drop meson recipe and add it as part of bootstrap (3.18 KB, patch)
2017-10-22 22:03 UTC, Xavier Claessens
none Details | Review
Meson: Drop meson recipe and add it as part of bootstrap (5.55 KB, patch)
2017-10-24 19:33 UTC, Xavier Claessens
none Details | Review
Meson: extract ModifyEnvBase class out of MakefilesBase (5.37 KB, patch)
2017-10-24 19:33 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (2.63 KB, patch)
2017-10-24 19:33 UTC, Xavier Claessens
none Details | Review
Meson: Port gst-transcoder recipe (1.36 KB, patch)
2017-10-24 19:33 UTC, Xavier Claessens
none Details | Review
Meson: Drop meson recipe and add it as part of bootstrap (6.12 KB, patch)
2017-10-26 16:11 UTC, Xavier Claessens
none Details | Review
Meson: Drop meson recipe and add it as part of bootstrap (6.12 KB, patch)
2017-10-26 16:12 UTC, Xavier Claessens
none Details | Review
Meson: extract ModifyEnvBase class out of MakefilesBase (5.37 KB, patch)
2017-10-26 16:12 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (6.19 KB, patch)
2017-10-26 16:12 UTC, Xavier Claessens
none Details | Review
Meson: Add exe_wrapper support while cross-compiling (3.04 KB, patch)
2017-10-26 16:12 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (6.39 KB, patch)
2017-10-26 19:07 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (6.30 KB, patch)
2017-10-27 16:52 UTC, Xavier Claessens
none Details | Review
Meson: Drop meson recipe and add it as part of bootstrap (6.12 KB, patch)
2017-10-30 16:11 UTC, Xavier Claessens
none Details | Review
Meson: extract ModifyEnvBase class out of MakefilesBase (5.37 KB, patch)
2017-10-30 16:11 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (6.30 KB, patch)
2017-10-30 16:11 UTC, Xavier Claessens
none Details | Review
Meson: Add exe_wrapper support while cross-compiling (3.04 KB, patch)
2017-10-30 16:11 UTC, Xavier Claessens
none Details | Review
Meson: Add 'meson' variant disabled by default (1.66 KB, patch)
2017-10-30 16:11 UTC, Xavier Claessens
none Details | Review
Meson: Port gst-transcoder recipe (1.36 KB, patch)
2017-10-30 16:12 UTC, Xavier Claessens
none Details | Review
Meson: Port GStreamer Core recipe (1.68 KB, patch)
2017-10-30 16:12 UTC, Xavier Claessens
none Details | Review
Meson: Port GStreamer Base recipe (2.46 KB, patch)
2017-10-30 16:12 UTC, Xavier Claessens
none Details | Review
Meson: Drop meson recipe and add it as part of bootstrap (6.12 KB, patch)
2017-10-31 20:18 UTC, Xavier Claessens
none Details | Review
Meson: extract ModifyEnvBase class out of MakefilesBase (5.37 KB, patch)
2017-10-31 20:18 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (7.03 KB, patch)
2017-10-31 20:18 UTC, Xavier Claessens
none Details | Review
Meson: Add exe_wrapper support while cross-compiling (2.96 KB, patch)
2017-10-31 20:19 UTC, Xavier Claessens
none Details | Review
Meson: Add 'meson' variant disabled by default (1.66 KB, patch)
2017-10-31 20:19 UTC, Xavier Claessens
none Details | Review
Meson: Port gst-transcoder recipe (1.36 KB, patch)
2017-10-31 20:19 UTC, Xavier Claessens
none Details | Review
Meson: Port gstreamer-1.0 recipe (1.67 KB, patch)
2017-10-31 20:19 UTC, Xavier Claessens
none Details | Review
Meson: Port gst-plugins-base-1.0 recipe (1.49 KB, patch)
2017-10-31 20:19 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (7.08 KB, patch)
2017-10-31 20:34 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (7.06 KB, patch)
2017-10-31 20:42 UTC, Xavier Claessens
none Details | Review
Meson: Drop meson recipe and add it as part of bootstrap (6.12 KB, patch)
2017-12-13 16:43 UTC, Xavier Claessens
none Details | Review
Meson: extract ModifyEnvBase class out of MakefilesBase (5.37 KB, patch)
2017-12-13 16:43 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (7.06 KB, patch)
2017-12-13 16:43 UTC, Xavier Claessens
none Details | Review
Meson: Add exe_wrapper support while cross-compiling (2.96 KB, patch)
2017-12-13 16:43 UTC, Xavier Claessens
none Details | Review
Meson: Port gst-transcoder recipe (1.36 KB, patch)
2017-12-13 16:43 UTC, Xavier Claessens
none Details | Review
Meson: Bump version to 0.44.0 (2.86 KB, patch)
2017-12-20 19:33 UTC, Xavier Claessens
none Details | Review
Meson: extract ModifyEnvBase class out of MakefilesBase (5.36 KB, patch)
2017-12-20 19:33 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (7.04 KB, patch)
2017-12-20 19:33 UTC, Xavier Claessens
none Details | Review
Meson: Port gst-transcoder recipe (1.48 KB, patch)
2017-12-20 19:34 UTC, Xavier Claessens
none Details | Review
Meson: Bump version to 0.44.0 (3.46 KB, patch)
2017-12-20 21:19 UTC, Xavier Claessens
none Details | Review
Fix the meson recipe and bump version to 0.45.1 (3.74 KB, patch)
2018-03-28 01:43 UTC, Xavier Claessens
none Details | Review
Meson: extract ModifyEnvBase class out of MakefilesBase (5.36 KB, patch)
2018-03-28 01:43 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (6.98 KB, patch)
2018-03-28 01:43 UTC, Xavier Claessens
none Details | Review
Meson: Port gst-transcoder recipe (1.48 KB, patch)
2018-03-28 01:43 UTC, Xavier Claessens
none Details | Review
Meson: Add BuildType.Meson to compile recipes (6.98 KB, patch)
2018-03-28 13:23 UTC, Xavier Claessens
none Details | Review
Fix the meson recipe and bump version to 0.45.1 (3.74 KB, patch)
2018-04-06 15:15 UTC, Xavier Claessens
committed Details | Review
Meson: extract ModifyEnvBase class out of MakefilesBase (5.36 KB, patch)
2018-04-06 15:16 UTC, Xavier Claessens
committed Details | Review
Meson: Add BuildType.Meson to compile recipes (6.90 KB, patch)
2018-04-06 15:16 UTC, Xavier Claessens
none Details | Review
Meson: Port gst-transcoder recipe (1.48 KB, patch)
2018-04-06 15:16 UTC, Xavier Claessens
committed Details | Review
Meson: Let recipes define cross compilation properties (2.14 KB, patch)
2018-04-06 15:16 UTC, Xavier Claessens
committed Details | Review
Meson: Add BuildType.Meson to compile recipes (6.40 KB, patch)
2018-04-17 15:11 UTC, Xavier Claessens
committed Details | Review
Fetch meson and ninja from tarball instead of git (1.64 KB, patch)
2018-04-17 15:12 UTC, Xavier Claessens
committed Details | Review

Description Xavier Claessens 2017-10-22 10:00:58 UTC
It invokes ./install_meson.py command that seems to not exist anymore. I think it should be using the standard setup.py instead, but that's tricky because cerbero is python2 and meson is python3, so the PYTHONPATH must be modified, unless I'm mistaken?
Comment 1 Xavier Claessens 2017-10-22 10:03:24 UTC
The following fixes have been pushed:
Comment 2 Xavier Claessens 2017-10-22 10:04:52 UTC
Created attachment 362037 [details] [review]
Add call_env arg to shell.call()
Comment 3 Xavier Claessens 2017-10-22 10:04:56 UTC
Created attachment 362038 [details] [review]
Config: Add modify_env_for_python3 helper

When cerbero needs to call python3 commands, PYTHONPATH must be modified
in the environ.
Comment 4 Xavier Claessens 2017-10-22 10:05:01 UTC
Created attachment 362039 [details] [review]
Fix meson recipe
Comment 5 Xavier Claessens 2017-10-22 10:18:47 UTC
Created attachment 362043 [details] [review]
Update meson version to 0.43.0
Comment 6 Nirbheek Chauhan 2017-10-22 11:20:38 UTC
It seems like the only thing that needs meson in cerbero is gst-transcoder? I think we should remove this recipe and install it as a system dependency in bootstrap instead. Every distro ships Python3 and that's the only dependency of Meson, so it's not going to be a significant burden.

If you want to add meson support to recipes within Cerbero, you will really want to rebase the patches in https://github.com/centricular/cerbero. A lot of them are about adding MSVC support, but the commits are discrete and you should be able to pull in only the Meson stuff. I haven't gotten to it yet because there was a lot of churn during 1.12 especially around the -static recipes.
Comment 7 Xavier Claessens 2017-10-22 12:58:22 UTC
Created attachment 362051 [details] [review]
Drop meson recipe and add it as part of bootstrap
Comment 8 Xavier Claessens 2017-10-22 13:01:15 UTC
This patch is missing windows/ios bootstrap, I have no idea how it they work, and have no way to test. Also missing the package name for pip3 on rh/fedora/etc. I tested on Ubuntu 16.04.
Comment 9 Nirbheek Chauhan 2017-10-22 13:20:30 UTC
Review of attachment 362051 [details] [review]:

::: cerbero/bootstrap/linux.py
@@ +52,3 @@
+
+        if meson_needed:
+            shell.call('pip3 install meson')

pip3 should only be used on older distros and on Windows/macOS.

I think this should also be `pip3 install --user` on Linux because people will not appreciate Cerbero installing anything outside of $HOME.

@@ +71,3 @@
                 'libxtst-dev', 'libxrandr-dev', 'libglu1-mesa-dev',
+                'libegl1-mesa-dev', 'git', 'subversion', 'xutils-dev',
+                'python3-pip']

I think we should use the distro-provided meson for recent distros (such as debian testing and unstable, fedora 25/26, ubuntu artful, etc).
Comment 10 Xavier Claessens 2017-10-22 14:10:29 UTC
Created attachment 362053 [details] [review]
Drop meson recipe and add it as part of bootstrap
Comment 11 Xavier Claessens 2017-10-22 14:12:23 UTC
I'm too lazy to check distro-by-distro which version of meson they have, so this patch just always install meson and python3-pip and if "meson --version" says it's too old it will pip install it.
Comment 12 Nirbheek Chauhan 2017-10-22 21:10:43 UTC
(In reply to Xavier Claessens from comment #11)
> I'm too lazy to check distro-by-distro which version of meson they have, so
> this patch just always install meson and python3-pip and if "meson
> --version" says it's too old it will pip install it.

This is probably the better solution since it's future-proof too. It also means that we will use whatever meson the user has in their PATH.

Just one more thing: using pip3 install --user means we need to ensure that ~/.local/bin is in our PATH. Does that require changes to our config?
Comment 13 Xavier Claessens 2017-10-22 22:03:37 UTC
Created attachment 362062 [details] [review]
Drop meson recipe and add it as part of bootstrap
Comment 14 Xavier Claessens 2017-10-22 22:05:29 UTC
I don't know if all distro has .local/bin in PATH by default. I don't know what needs to be added in PATH for windows and ios though.
Comment 15 Xavier Claessens 2017-10-24 19:33:45 UTC
Created attachment 362215 [details] [review]
Meson: Drop meson recipe and add it as part of bootstrap
Comment 16 Xavier Claessens 2017-10-24 19:33:49 UTC
Created attachment 362216 [details] [review]
Meson: extract ModifyEnvBase class out of MakefilesBase

This is the common part between Makefile based build and the future
Meson based build.
Comment 17 Xavier Claessens 2017-10-24 19:33:53 UTC
Created attachment 362217 [details] [review]
Meson: Add BuildType.Meson to compile recipes
Comment 18 Xavier Claessens 2017-10-24 19:33:58 UTC
Created attachment 362218 [details] [review]
Meson: Port gst-transcoder recipe
Comment 19 Xavier Claessens 2017-10-24 19:38:41 UTC
Here is a first step at making Meson recipes first class citizen. I chosed to port only gst-transcoder for now because it is the only one that is already being built using meson.

I updated Windows instructions in README to install Meson and python3. I tested those steps on a fresh install of Win10 in a VM.

On my Ubuntu 16.04, gst-transcoder fail to build with that error:

Error in gtkdoc helper script:
'gtkdoc-scangobj' failed with status 1
ld: unrecognized option '-Wl,-rpath,/home/xclaesse/programmation/cerbero/build/sources/linux_x86_64/gst-transcoder-1.9/cerbero-build-dir/'
ld: use the --help option for usage information
Linking of scanner failed: 


I get the exact same error when trying to build gstreamer-1.0 recipe ported to Meson. Any Idea?
Comment 20 Xavier Claessens 2017-10-24 19:39:14 UTC
Review of attachment 362218 [details] [review]:

Marking this commit as needs-work because it fails to build for me.
Comment 21 Xavier Claessens 2017-10-26 16:11:55 UTC
Created attachment 362353 [details] [review]
Meson: Drop meson recipe and add it as part of bootstrap
Comment 22 Xavier Claessens 2017-10-26 16:12:12 UTC
Created attachment 362354 [details] [review]
Meson: Drop meson recipe and add it as part of bootstrap
Comment 23 Xavier Claessens 2017-10-26 16:12:30 UTC
Created attachment 362355 [details] [review]
Meson: extract ModifyEnvBase class out of MakefilesBase

This is the common part between Makefile based build and the future
Meson based build.
Comment 24 Xavier Claessens 2017-10-26 16:12:39 UTC
Created attachment 362356 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 25 Xavier Claessens 2017-10-26 16:12:48 UTC
Created attachment 362357 [details] [review]
Meson: Add exe_wrapper support while cross-compiling

This allows us to specify the wrapper that Meson should use to run
executables while cross-compiling. Currently just has cross-win*
support, but more can be added later including qemu for arm targets,
etc.
Comment 26 Xavier Claessens 2017-10-26 19:07:26 UTC
Created attachment 362369 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 27 Justin Kim 2017-10-27 09:33:58 UTC
(In reply to Xavier Claessens from comment #19)
> Here is a first step at making Meson recipes first class citizen. I chosed
> to port only gst-transcoder for now because it is the only one that is
> already being built using meson.
> 
> I updated Windows instructions in README to install Meson and python3. I
> tested those steps on a fresh install of Win10 in a VM.
> 
> On my Ubuntu 16.04, gst-transcoder fail to build with that error:
> 
> Error in gtkdoc helper script:
> 'gtkdoc-scangobj' failed with status 1
> ld: unrecognized option
> '-Wl,-rpath,/home/xclaesse/programmation/cerbero/build/sources/linux_x86_64/
> gst-transcoder-1.9/cerbero-build-dir/'
> ld: use the --help option for usage information
> Linking of scanner failed: 
> 
> 
> I get the exact same error when trying to build gstreamer-1.0 recipe ported
> to Meson. Any Idea?

How about just disabling document generation by adding the below?

+    def configure(self):
+        self.meson_configure['disable_doc'] = 'true'
+        super(recipe.Recipe, self).configure()
Comment 28 Xavier Claessens 2017-10-27 10:35:19 UTC
That works of course, but I would like to get it working rather than hiding the bug. But I'm out of ideas to fix this one...
Comment 29 Xavier Claessens 2017-10-27 14:26:53 UTC
Comment on attachment 362218 [details] [review]
Meson: Port gst-transcoder recipe

This recipe builds fine once this meson pull request gets merged:
https://github.com/mesonbuild/meson/issues/2540
Comment 30 Xavier Claessens 2017-10-27 16:52:40 UTC
Created attachment 362422 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 31 Xavier Claessens 2017-10-30 16:11:31 UTC
Created attachment 362555 [details] [review]
Meson: Drop meson recipe and add it as part of bootstrap
Comment 32 Xavier Claessens 2017-10-30 16:11:38 UTC
Created attachment 362556 [details] [review]
Meson: extract ModifyEnvBase class out of MakefilesBase

This is the common part between Makefile based build and the future
Meson based build.
Comment 33 Xavier Claessens 2017-10-30 16:11:45 UTC
Created attachment 362557 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 34 Xavier Claessens 2017-10-30 16:11:51 UTC
Created attachment 362558 [details] [review]
Meson: Add exe_wrapper support while cross-compiling

This allows us to specify the wrapper that Meson should use to run
executables while cross-compiling. Currently just has cross-win*
support, but more can be added later including qemu for arm targets,
etc.
Comment 35 Xavier Claessens 2017-10-30 16:11:57 UTC
Created attachment 362559 [details] [review]
Meson: Add 'meson' variant disabled by default

When enabling this variant, recipes supporting it will be built using
Meson instead of Autotools.
Comment 36 Xavier Claessens 2017-10-30 16:12:04 UTC
Created attachment 362560 [details] [review]
Meson: Port gst-transcoder recipe
Comment 37 Xavier Claessens 2017-10-30 16:12:10 UTC
Created attachment 362561 [details] [review]
Meson: Port GStreamer Core recipe
Comment 38 Xavier Claessens 2017-10-30 16:12:15 UTC
Created attachment 362562 [details] [review]
Meson: Port GStreamer Base recipe
Comment 39 Xavier Claessens 2017-10-30 16:15:17 UTC
I think this is a really good start, you can add `variants = ['meson']` into your ~/.cerbero/cerbero.cbc to make gst core/base build with meson. I think we should start setuping CI that uses that variant to ensure GStreamer always builds fine using meson.

Of course more recipes should be ported.
Comment 40 Nicolas Dufresne (ndufresne) 2017-10-30 18:24:28 UTC
(In reply to Xavier Claessens from comment #39)
> I think this is a really good start, you can add `variants = ['meson']` into
> your ~/.cerbero/cerbero.cbc to make gst core/base build with meson. I think
> we should start setuping CI that uses that variant to ensure GStreamer
> always builds fine using meson.

Note, we already have meson builds on ci.gstreamer.net, just not using cerbero atm. I overall agree this is a nice intermediate step.
Comment 41 Xavier Claessens 2017-10-30 18:58:14 UTC
(In reply to Nicolas Dufresne (stormer) from comment #40)
> Note, we already have meson builds on ci.gstreamer.net, just not using
> cerbero atm. I overall agree this is a nice intermediate step.

But not for cross builds, right?
Comment 42 Thibault Saunier 2017-10-31 20:12:49 UTC
(In reply to Xavier Claessens from comment #41)
> (In reply to Nicolas Dufresne (stormer) from comment #40)
> > Note, we already have meson builds on ci.gstreamer.net, just not using
> > cerbero atm. I overall agree this is a nice intermediate step.
> 
> But not for cross builds, right?

No, only native builds right now, so it would indeed be a nice intermediary addition fmpov.
Comment 43 Xavier Claessens 2017-10-31 20:18:44 UTC
Created attachment 362693 [details] [review]
Meson: Drop meson recipe and add it as part of bootstrap
Comment 44 Xavier Claessens 2017-10-31 20:18:50 UTC
Created attachment 362694 [details] [review]
Meson: extract ModifyEnvBase class out of MakefilesBase

This is the common part between Makefile based build and the future
Meson based build.
Comment 45 Xavier Claessens 2017-10-31 20:18:56 UTC
Created attachment 362695 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 46 Xavier Claessens 2017-10-31 20:19:03 UTC
Created attachment 362696 [details] [review]
Meson: Add exe_wrapper support while cross-compiling

This allows us to specify the wrapper that Meson should use to run
executables while cross-compiling. Currently just has cross-win*
support, but more can be added later including qemu for arm targets,
etc.

Based on code from:
Nirbheek Chauhan <nirbheek@centricular.com>
Justin Kim <justin.kim@collabora.com>,
Comment 47 Xavier Claessens 2017-10-31 20:19:10 UTC
Created attachment 362697 [details] [review]
Meson: Add 'meson' variant disabled by default

When enabling this variant, recipes supporting it will be built using
Meson instead of Autotools.
Comment 48 Xavier Claessens 2017-10-31 20:19:17 UTC
Created attachment 362698 [details] [review]
Meson: Port gst-transcoder recipe
Comment 49 Xavier Claessens 2017-10-31 20:19:29 UTC
Created attachment 362699 [details] [review]
Meson: Port gstreamer-1.0 recipe
Comment 50 Xavier Claessens 2017-10-31 20:19:38 UTC
Created attachment 362700 [details] [review]
Meson: Port gst-plugins-base-1.0 recipe
Comment 51 Xavier Claessens 2017-10-31 20:20:09 UTC
I can now cross build for android at least gstreamer core and base.
Comment 52 Xavier Claessens 2017-10-31 20:34:42 UTC
Created attachment 362701 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 53 Xavier Claessens 2017-10-31 20:42:38 UTC
Created attachment 362704 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 54 Xavier Claessens 2017-11-08 14:48:01 UTC
The 2 patches needed in Meson to get this working got merged now.
Comment 55 Tim-Philipp Müller 2017-11-08 22:26:31 UTC
Ideally they'd also get into a release so we don't have to use a random git snapshot of Meson :)
Comment 56 Xavier Claessens 2017-11-08 23:49:53 UTC
There is a release every month, 0.44.0 should be rsn.
Comment 57 Justin Kim 2017-12-11 05:24:02 UTC
meson 0.44.0 is coming today. :)
Comment 58 Xavier Claessens 2017-12-13 16:43:24 UTC
Created attachment 365499 [details] [review]
Meson: Drop meson recipe and add it as part of bootstrap
Comment 59 Xavier Claessens 2017-12-13 16:43:30 UTC
Created attachment 365500 [details] [review]
Meson: extract ModifyEnvBase class out of MakefilesBase

This is the common part between Makefile based build and the future
Meson based build.
Comment 60 Xavier Claessens 2017-12-13 16:43:36 UTC
Created attachment 365501 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 61 Xavier Claessens 2017-12-13 16:43:41 UTC
Created attachment 365502 [details] [review]
Meson: Add exe_wrapper support while cross-compiling

This allows us to specify the wrapper that Meson should use to run
executables while cross-compiling. Currently just has cross-win*
support, but more can be added later including qemu for arm targets,
etc.

Based on code from:
Nirbheek Chauhan <nirbheek@centricular.com>
Justin Kim <justin.kim@collabora.com>,
Comment 62 Xavier Claessens 2017-12-13 16:43:46 UTC
Created attachment 365503 [details] [review]
Meson: Port gst-transcoder recipe
Comment 63 Xavier Claessens 2017-12-13 16:45:02 UTC
Rebased on master, dropped patches that port recipes to meson because that's probably premature. Only kept gst-transcoder because that one is already meson-only.
Comment 64 Sebastian Dröge (slomo) 2017-12-14 08:29:22 UTC
Review of attachment 365499 [details] [review]:

::: README
@@ +37,3 @@
 -------
 The initial setup on Windows is a little bit longer, but only a few programs are required.
+  * Python2.7 AND Python3.6: https://www.python.org/downloads/

This should be merged after porting to Python 3 IMHO. Requiring two versions of Python is just silly

::: cerbero/bootstrap/linux.py
@@ +52,3 @@
+
+        if meson_needed:
+            shell.call('pip3 install --user meson')

How would meson installed on non-Unix? Seems to be missing

Why do we install via PIP instead of having a recipe? Updating with PIP or ensuring that at least a specific version is used seems less than ideal
Comment 65 Sebastian Dröge (slomo) 2017-12-14 08:33:09 UTC
Review of attachment 365501 [details] [review]:

::: cerbero/build/build.py
@@ +408,3 @@
+        ModifyEnvBase.__init__(self)
+
+        self.meson_dir = os.path.join(self.build_dir, "cerbero-build-dir")

Make this a bit shorter please, think of poor Windows with its limited capability of handling long paths :)

@@ +413,3 @@
+        if self.config.cross_compiling():
+            self.new_env['CC'] = None
+            self.new_env['CXX'] = None

Why? Maybe the user wanted to build with clang instead of gcc?

@@ +459,3 @@
+        # Take CC and CXX from _old_env because we modified env to make them be
+        # the native toolchain. If the variable is not set, we assume we want
+        # MSVC

Why? on Windows this might make sense, but not in general
Comment 66 Sebastian Dröge (slomo) 2017-12-14 08:34:14 UTC
Comment on attachment 365502 [details] [review]
Meson: Add exe_wrapper support while cross-compiling

This requires the bootstrap for the cross-win* targets to install wine then
Comment 67 Xavier Claessens 2017-12-14 17:47:23 UTC
(In reply to Sebastian Dröge (slomo) from comment #64)
> +  * Python2.7 AND Python3.6: https://www.python.org/downloads/
> 
> This should be merged after porting to Python 3 IMHO. Requiring two versions
> of Python is just silly

I agree but it seems we are not there yet, sadly. I don't think we should block on py3 porting, do we?

> ::: cerbero/bootstrap/linux.py
> @@ +52,3 @@
> +
> +        if meson_needed:
> +            shell.call('pip3 install --user meson')
> 
> How would meson installed on non-Unix? Seems to be missing

Up to the user to pip install it, like said in README. I guess we could add similar code in other OS bootstrap for convenience.

> Why do we install via PIP instead of having a recipe? Updating with PIP or
> ensuring that at least a specific version is used seems less than ideal

It is actually complicated to have it as recipe because we mix python2 and python3 here and we can only set PYTHON_PATH to one of them. It can be work around (my initial patches did that) but nirbeek said we should just consider meson comes from your system.

(In reply to Sebastian Dröge (slomo) from comment #65)
> +        self.meson_dir = os.path.join(self.build_dir, "cerbero-build-dir")
> Make this a bit shorter please, think of poor Windows with its limited
> capability of handling long paths :)

Maybe just "builddir" ? We should make sure to not be too generic otherwise it would clash with a folder that projects has in git. Like glib has a "build" folder.

> @@ +413,3 @@
> +        if self.config.cross_compiling():
> +            self.new_env['CC'] = None
> +            self.new_env['CXX'] = None
> 
> Why? Maybe the user wanted to build with clang instead of gcc?

Yeah, that's a bit a quick hack... The problem is with meson you must have the native compiler in env and set the target compiler in meson-cross-file.txt. Unfortunately cerbero always set the target compiler in env.

See how in write_meson_cross_file() it takes the compiler from self._old_env to write it in the file.

Ideally in config.py we should load both native and target envs, then have a way to switch between them depending on the type of recipe you're building. That's much more work and I'm not exactly sure how to do that.

So yes, that's an ugly hack, but it makes stuff work. Do you think that's merge blocker?

> @@ +459,3 @@
> +        # Take CC and CXX from _old_env because we modified env to make
> them be
> +        # the native toolchain. If the variable is not set, we assume we
> want
> +        # MSVC
> 
> Why? on Windows this might make sense, but not in general

That's from nirbeek branch, it defaults to MSVC. CMake default to "gcc" which is equaly wrong I guess... We should not even need a default there, in practice you should always have a compiler set in your env, no?

(In reply to Sebastian Dröge (slomo) from comment #66)
> This requires the bootstrap for the cross-win* targets to install wine then

Do we even ever run target platform binaries? That wasn't supported before, not sure we need this. That's from nirbeek. IMO wine is a big and unreliable dependency to have. I guess we should test if cross build still works without that.
Comment 68 Sebastian Dröge (slomo) 2017-12-15 08:28:05 UTC
(In reply to Xavier Claessens from comment #67)
> (In reply to Sebastian Dröge (slomo) from comment #64)
> > +  * Python2.7 AND Python3.6: https://www.python.org/downloads/
> > 
> > This should be merged after porting to Python 3 IMHO. Requiring two versions
> > of Python is just silly
> 
> I agree but it seems we are not there yet, sadly. I don't think we should
> block on py3 porting, do we?

I'm fine with requiring both for a short while until the py3 port is finished (work is being done on that currently AFIAU), as long as you can confirm that there are no weird problems with using both versions in cerbero :)
Comment 69 Sebastian Dröge (slomo) 2017-12-15 08:32:47 UTC
(In reply to Xavier Claessens from comment #67)

> > Why do we install via PIP instead of having a recipe? Updating with PIP or
> > ensuring that at least a specific version is used seems less than ideal
> 
> It is actually complicated to have it as recipe because we mix python2 and
> python3 here and we can only set PYTHON_PATH to one of them. It can be work
> around (my initial patches did that) but nirbeek said we should just
> consider meson comes from your system.

I'm fine with that until the py3 port is in, but then please add a recipe for that.

> (In reply to Sebastian Dröge (slomo) from comment #65)
> > +        self.meson_dir = os.path.join(self.build_dir, "cerbero-build-dir")
> > Make this a bit shorter please, think of poor Windows with its limited
> > capability of handling long paths :)
> 
> Maybe just "builddir" ? We should make sure to not be too generic otherwise
> it would clash with a folder that projects has in git. Like glib has a
> "build" folder.

Maybe "build", and if existing already "build_", "build__", etc? "builddir" could equally conflict

> So yes, that's an ugly hack, but it makes stuff work. Do you think that's
> merge blocker?

No, but open another Bugzilla bug about that so it's not forgotten. That should be fixed sooner or later, it's ugly :)

It also seems weird that meson needs the host compiler in the environment. I thought meson hates environment variables for everything...

> > @@ +459,3 @@
> > +        # Take CC and CXX from _old_env because we modified env to make
> > them be
> > +        # the native toolchain. If the variable is not set, we assume we
> > want
> > +        # MSVC
> > 
> > Why? on Windows this might make sense, but not in general
> 
> That's from nirbeek branch, it defaults to MSVC. CMake default to "gcc"
> which is equaly wrong I guess... We should not even need a default there, in
> practice you should always have a compiler set in your env, no?

Yes, or cerbero should make sure that this is the case. Is this part needed, and if so, for what exactly?

> (In reply to Sebastian Dröge (slomo) from comment #66)
> > This requires the bootstrap for the cross-win* targets to install wine then
> 
> Do we even ever run target platform binaries? That wasn't supported before,
> not sure we need this. That's from nirbeek. IMO wine is a big and unreliable
> dependency to have. I guess we should test if cross build still works
> without that.

AFAIU meson might do that. autotools does not.
Comment 70 Xavier Claessens 2017-12-15 23:42:37 UTC
When cerbero is running with python3 it is really easy to build meson inside cerbero instead of requiring to pip install it. So let's wait for those patches to get merged first.
Comment 71 Xavier Claessens 2017-12-20 19:33:39 UTC
Created attachment 365801 [details] [review]
Meson: Bump version to 0.44.0

Now that cerbero is running with python3, PYTHONPATH is refering to
python3 path. This makes the meson built inside cerbero usable with our
build env.
Comment 72 Xavier Claessens 2017-12-20 19:33:48 UTC
Created attachment 365802 [details] [review]
Meson: extract ModifyEnvBase class out of MakefilesBase

This is the common part between Makefile based build and the future
Meson based build.
Comment 73 Xavier Claessens 2017-12-20 19:33:55 UTC
Created attachment 365803 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 74 Xavier Claessens 2017-12-20 19:34:01 UTC
Created attachment 365804 [details] [review]
Meson: Port gst-transcoder recipe
Comment 75 Xavier Claessens 2017-12-20 19:45:12 UTC
(In reply to Sebastian Dröge (slomo) from comment #69)
> (In reply to Xavier Claessens from comment #67)
> 
> > > Why do we install via PIP instead of having a recipe? Updating with PIP or
> > > ensuring that at least a specific version is used seems less than ideal
> > 
> > It is actually complicated to have it as recipe because we mix python2 and
> > python3 here and we can only set PYTHON_PATH to one of them. It can be work
> > around (my initial patches did that) but nirbeek said we should just
> > consider meson comes from your system.
> 
> I'm fine with that until the py3 port is in, but then please add a recipe
> for that.

Rebased my patchset on top of my python3 branch (bug #733067). Meson is now built inside cerbero. No need to pip install it anymore.


> > (In reply to Sebastian Dröge (slomo) from comment #65)
> > > +        self.meson_dir = os.path.join(self.build_dir, "cerbero-build-dir")
> > > Make this a bit shorter please, think of poor Windows with its limited
> > > capability of handling long paths :)
> > 
> > Maybe just "builddir" ? We should make sure to not be too generic otherwise
> > it would clash with a folder that projects has in git. Like glib has a
> > "build" folder.
> 
> Maybe "build", and if existing already "build_", "build__", etc? "builddir"
> could equally conflict

It cannot just add _ until it finds a dir that doesn't exists because if we run again the compile step it must reuse the same dir that previous run.

I opted for "_builddir" which should be unique, short and sort first when you browse the folder.

> > So yes, that's an ugly hack, but it makes stuff work. Do you think that's
> > merge blocker?
> 
> No, but open another Bugzilla bug about that so it's not forgotten. That
> should be fixed sooner or later, it's ugly :)

Opened bug #791670 and linked to it in a comment.

> > That's from nirbeek branch, it defaults to MSVC. CMake default to "gcc"
> > which is equaly wrong I guess... We should not even need a default there, in
> > practice you should always have a compiler set in your env, no?
> 
> Yes, or cerbero should make sure that this is the case. Is this part needed,
> and if so, for what exactly?

I removed the default value.

> > (In reply to Sebastian Dröge (slomo) from comment #66)
> > > This requires the bootstrap for the cross-win* targets to install wine then
> > 
> > Do we even ever run target platform binaries? That wasn't supported before,
> > not sure we need this. That's from nirbeek. IMO wine is a big and unreliable
> > dependency to have. I guess we should test if cross build still works
> > without that.
> 
> AFAIU meson might do that. autotools does not.

Dropped that patch, it doesn't seems to be needed, for now at least.
Comment 76 Xavier Claessens 2017-12-20 21:19:20 UTC
Created attachment 365813 [details] [review]
Meson: Bump version to 0.44.0

Now that cerbero is running with python3, PYTHONPATH is refering to
python3 path. This makes the meson built inside cerbero usable with our
build env.
Comment 77 Tim-Philipp Müller 2018-01-16 11:54:36 UTC
*** Bug 768028 has been marked as a duplicate of this bug. ***
Comment 78 Xavier Claessens 2018-03-28 01:43:22 UTC
Created attachment 370221 [details] [review]
Fix the meson recipe and bump version to 0.45.1

Now that cerbero is running with python3, PYTHONPATH is refering to
python3 path. This makes the meson built inside cerbero usable with our
build env.
Comment 79 Xavier Claessens 2018-03-28 01:43:37 UTC
Created attachment 370222 [details] [review]
Meson: extract ModifyEnvBase class out of MakefilesBase

This is the common part between Makefile based build and the future
Meson based build.
Comment 80 Xavier Claessens 2018-03-28 01:43:48 UTC
Created attachment 370223 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 81 Xavier Claessens 2018-03-28 01:43:56 UTC
Created attachment 370224 [details] [review]
Meson: Port gst-transcoder recipe
Comment 82 Xavier Claessens 2018-03-28 13:23:05 UTC
Created attachment 370243 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 83 Olivier Crête 2018-03-28 18:27:00 UTC
Review of attachment 370243 [details] [review]:

::: cerbero/build/build.py
@@ +516,3 @@
+
+        for (key, value) in self.meson_options.items():
+            meson_cmd += ' -D%s=%s' % (key, str(value))

I wonder if we shouldn't always write a meson file if any of those values are set? In case one wants to override the CC, etc even when not cross-compiling really. For example, to use clang instead of gcc, or to use icc.
Comment 84 Xavier Claessens 2018-03-28 18:43:50 UTC
You can set CC and CXX in your env for that. Meson takes the native toolchain from your env, and the cross toolchain from that file.
Comment 85 Xavier Claessens 2018-04-06 15:15:58 UTC
Created attachment 370591 [details] [review]
Fix the meson recipe and bump version to 0.45.1

Now that cerbero is running with python3, PYTHONPATH is refering to
python3 path. This makes the meson built inside cerbero usable with our
build env.
Comment 86 Xavier Claessens 2018-04-06 15:16:07 UTC
Created attachment 370592 [details] [review]
Meson: extract ModifyEnvBase class out of MakefilesBase

This is the common part between Makefile based build and the future
Meson based build.
Comment 87 Xavier Claessens 2018-04-06 15:16:18 UTC
Created attachment 370594 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 88 Xavier Claessens 2018-04-06 15:16:27 UTC
Created attachment 370596 [details] [review]
Meson: Port gst-transcoder recipe
Comment 89 Xavier Claessens 2018-04-06 15:16:37 UTC
Created attachment 370597 [details] [review]
Meson: Let recipes define cross compilation properties

With autoconf we used to define variables like glib_cv_stack_grows in
the environ, but with meson those variables needs to be defined inside
the cross-file we generate. For example GLib will be using those
properties: https://bugzilla.gnome.org/show_bug.cgi?id=794898.
Comment 90 Justin Kim 2018-04-11 07:49:03 UTC
Review of attachment 370594 [details] [review]:

::: cerbero/build/build.py
@@ +425,3 @@
+        # Find Meson
+        if not self.meson_sh:
+    make_install = None

There's a corner case when building on Mac. If meson is installed by Homebrew on Mac OS X, it should be 'meson.py' because 'meson' is a batch shell.
Comment 91 Xavier Claessens 2018-04-11 11:05:55 UTC
We don't use system meson, cerbero install it's own, so I think it should work on macos too. Did you try?
Comment 92 Justin Kim 2018-04-12 01:02:55 UTC
(In reply to Xavier Claessens from comment #91)
> We don't use system meson, cerbero install it's own, so I think it should
> work on macos too. Did you try?

Ah, yes. I forgot to run `bootstrap` while checking.
Comment 93 Nirbheek Chauhan 2018-04-17 11:33:04 UTC
Review of attachment 370591 [details] [review]:

lgtm
Comment 94 Nirbheek Chauhan 2018-04-17 11:39:38 UTC
Review of attachment 370592 [details] [review]:

lgtm
Comment 95 Nirbheek Chauhan 2018-04-17 12:07:33 UTC
Review of attachment 370594 [details] [review]:

::: cerbero/build/build.py
@@ +399,3 @@
+            --default-library=%(default-library)s --buildtype=%(buildtype)s \
+            --backend=%(backend)s ..'
+    default_library = 'shared'

Do we want to change this? This made sense when we had separate recipes for static and shared, and this value would be overriden in the static recipes.

@@ +425,3 @@
+        # Find Meson
+        if not self.meson_sh:
+            self.meson_sh = self.config.python_exe + ' ' + shell.which('meson')

I originally used shell.which('meson') here because meson.py was installed into the system prefix. But since it's in the build tools prefix, we know exactly where it is. We should use that path directly.

This also means we don't need find_build_tools(), and we can hard-code paths to both meson and ninja in the class variables.
Comment 96 Nirbheek Chauhan 2018-04-17 12:08:23 UTC
Review of attachment 370596 [details] [review]:

looks good, once the other patches in this bug are merged.
Comment 97 Nirbheek Chauhan 2018-04-17 12:11:38 UTC
Review of attachment 370597 [details] [review]:

Do we still need this since glib doesn't seem to need any special cross properties now?
Comment 98 Xavier Claessens 2018-04-17 13:13:15 UTC
(In reply to Nirbheek Chauhan from comment #95)
> Review of attachment 370594 [details] [review] [review]:
> 
> ::: cerbero/build/build.py
> @@ +399,3 @@
> +            --default-library=%(default-library)s --buildtype=%(buildtype)s
> \
> +            --backend=%(backend)s ..'
> +    default_library = 'shared'
> 
> Do we want to change this? This made sense when we had separate recipes for
> static and shared, and this value would be overriden in the static recipes.

Note we still have glib-networking-static but that's going away when we update to latest glib, I made the patch upstream to build static gio modules. I guess we should set default_library='both' as soon as meson 0.46 gets released. But we cannot always build static, for example glib cannot do static for windows atm. So I think we still need to be able to override default_library from recipes.

> @@ +425,3 @@
> +        # Find Meson
> +        if not self.meson_sh:
> +            self.meson_sh = self.config.python_exe + ' ' +
> shell.which('meson')
> 
> I originally used shell.which('meson') here because meson.py was installed
> into the system prefix. But since it's in the build tools prefix, we know
> exactly where it is. We should use that path directly.
> 
> This also means we don't need find_build_tools(), and we can hard-code paths
> to both meson and ninja in the class variables.

Good point, I'm happy to be able to remove that hack, I'll update patch in a bit.
Comment 99 Xavier Claessens 2018-04-17 13:16:21 UTC
(In reply to Nirbheek Chauhan from comment #97)
> Review of attachment 370597 [details] [review] [review]:
> 
> Do we still need this since glib doesn't seem to need any special cross
> properties now?

Yes we need this for glib, most properties are missing from glib's meson but I already added some and more should be added later. That's bug #795100.
Comment 100 Xavier Claessens 2018-04-17 15:11:45 UTC
Created attachment 371052 [details] [review]
Meson: Add BuildType.Meson to compile recipes

Based on code from Nirbheek Chauhan <nirbheek@centricular.com>
Comment 101 Xavier Claessens 2018-04-17 15:12:23 UTC
Created attachment 371053 [details] [review]
Fetch meson and ninja from tarball instead of git
Comment 102 Nirbheek Chauhan 2018-05-02 15:11:31 UTC
Review of attachment 371052 [details] [review]:

lgtm
Comment 103 Nirbheek Chauhan 2018-05-02 15:11:54 UTC
Review of attachment 371053 [details] [review]:

lgtm
Comment 104 Nirbheek Chauhan 2018-05-02 15:12:40 UTC
Review of attachment 370597 [details] [review]:

lgtm
Comment 105 Nirbheek Chauhan 2018-05-02 15:13:02 UTC
Review of attachment 370596 [details] [review]:

lgtm