GNOME Bugzilla – Bug 788146
Port to meson build system
Last modified: 2020-11-12 14:28:07 UTC
meson is a build system focused on speed an ease of use, which helps speeding up the software development.
Created attachment 360364 [details] [review] Port to meson build system Patch that adds meson build system along autotools. There is a minor issue regarding 'nm-applet.desktop' file, which is copied to both 'applications' and 'xdg/autostart' directories. Although it's working thanks to a workaround, there is an open query about it[0]. Although I tried to test it as much as I could, more testing would be welcome. Any suggestion would also be very appreciated. [0] https://groups.google.com/forum/#!topic/mesonbuild/3iIoYPrN4P0
Created attachment 360365 [details] [review] Remove autotools Some applications are removing autotools to avoid the burden of maintaining two build systems. This patch removes autotools in case it is considered.
Finally I've created a wip branch[0] to ease it's testing. [0] https://git.gnome.org/browse/network-manager-applet/log/?h=wip/inigomartinez/meson
Thanks Iñigo for the contribution. > meson is a build system focused on speed an ease of use, which helps speeding > up the software development. I agree, recursive make with automake is not nice. But we no longer do that, and the Makefile has all dependencies nicely declared. It's quite nice, you can build every target explicitly, and it rebuilds exactly whatever it needs. Also, the single Makefile.am is pretty neat. It's not clear that the meson version is significantly nicer. Is it necessary to distribute the meson.build files accross the subdirectories? I find it better if the entire build is laid out at one place (in the main Makefile.am file), instead of spreading it over various directories. The patch drops src/libnma/libnma.pc.in entirely. How is that supposed to be done? I am not convinced, that this looks better. Line-wise, it merely saves 25 %, although not supporting "src/libnma/libnma.pc.in" (does it?). How does it support distribution tarballs? But of course, faster build-times are a very solid argument. Is meson measurably faster? For network-manager-applet a $ git clean -fdx && time sh -c './autogen.sh && make -j 6' takes ~58 seconds for me (from a tempfs. Actually, the time to build fluctuates quite strongly, it's hard to get a good timing). A subsequent $ time make -j 6 without any changes takes less then a second (ok: that does not qualify as "instantaneously"). Then time sh -c 'touch configure.ac && make -j 6' takes ~22 seconds. Surely, build speed is very important. But with these timings, is meson that much faster to pay off? I am not familiar with meson. How to do a dist-tarball? Can you show a few examples of how to use it? How to run tests? I could google it myself, but I think the commit message should show such examples uses anyway.
what I dislike about our current autotools setup, is that we still use recursive make for the po directory. But that is just because nobody invested the time to fix that. The patch drops m4/compiler_options.m4 and m4/git-sha-record.m4 without functional replacement.
(In reply to Thomas Haller from comment #4) > Thanks Iñigo for the contribution. You are welcome. I'm just trying to be helpful. I'm sorry I didn't answer sooner. Gmail grouped the notification mail into another "conversation" thread and I missed it. I have added some comments below. > > meson is a build system focused on speed an ease of use, which helps speeding > > up the software development. > > I agree, recursive make with automake is not nice. But we no longer do that, > and the Makefile has all dependencies nicely declared. It's quite nice, you > can build every target explicitly, and it rebuilds exactly whatever it needs. I've made the `network-manager-applet` meson port as part of the GNOME packet porting initiative[0]. I have been using autotools for many years and autotools has always been something I ended up working on longer than I wanted. Actually, the fact that meson could be faster is just a side benefit for me. Describing how the build has to be done is much more pleasant in meson. That's, for me, the real benefit of using meson. Although meson is still young, I think that it covers all the needs of many applications. It's also nice that meson's main developers are working very hard and are very open to new features. > Also, the single Makefile.am is pretty neat. It's not clear that the meson > version is significantly nicer. > Is it necessary to distribute the meson.build files accross the > subdirectories? I find it better if the entire build is laid out at one > place (in the main Makefile.am file), instead of spreading it over various > directories. Probably it's not necessary to distribute the `meson.build` file accross the entire source code, though, I've never done it that way and I've never seen any project ported to meson doing it that way. In fact, after porting several GNOME packages to meson, `network-manager-applet` has been the first one which only uses one `Makefile.am` file. > The patch drops src/libnma/libnma.pc.in entirely. How is that supposed to be > done? This happens in the second patch, the one that removes autotools, just in case it's considered. Although the `libnma.pc.in` can still be used by meson using the `configure_file` function, there is also a meson module called Pkgconfig[1] which also generates the file. The `libnma.pc` file is created as follows: pkg.generate( libraries: libnma, version: nma_version, name: lib_name, description: 'NetworkManager UI utilities (libnm version)', filebase: lib_name, subdirs: lib_name, requires: 'libnm', variables: 'exec_prefix=' + nma_prefix, install_dir: join_paths(nma_libdir, 'pkgconfig') ) There are some combinations of meson's objects, strings, etc... A description of the objects used is as follows: - `libnma` is actually the built object that represents the 'libnma.so' shared library, it will be used as the library the user needs to link against. - nma_version is the string which hold the version number of `network-manager-applet`. It has been created with the information described in the `project` function in the main `meson.build` file. It's used to describe the library version. - lib_name is a auxiliary string variable I've created in the `meson.build` where `pkg.generate` is called and `libnma` is build with the `libnma` string. I try to use variables when a string is duplicated so it's less error prone. As you can see, even in the `pkg.generate` function is used several times. It's used to describe the name of the library (name), the base name of the file (filebase) and the subdirectory inside include where header files are located (subdirs). - `nma_prefix` is a string which holds the `prefix` option used for building. It's used to set the extra `exec_prefix` variable which in this case based on the original `pc` file, points to the `prefix` location. - `nma_libdir` is a string which holds the `libdir` option used for building. It's used to set the install directory ($libdir/pkgconfig). > I am not convinced, that this looks better. Line-wise, it merely saves 25 %, > although not supporting "src/libnma/libnma.pc.in" (does it?). How does it > support distribution tarballs? Since version 0.41 of meson, it supports creating distribution tarballs by using the `ninja dist` command. As far as I know, it uses the information from the last `git` commit to create the tarball, that's why there is no need to indicate the files to be packed in the build files. > But of course, faster build-times are a very solid argument. Is meson > measurably faster? For network-manager-applet a > > $ git clean -fdx && time sh -c './autogen.sh && make -j 6' > > takes ~58 seconds for me (from a tempfs. Actually, the time to build > fluctuates quite strongly, it's hard to get a good timing). > A subsequent > > $ time make -j 6 > > without any changes takes less then a second (ok: that does not qualify as > "instantaneously"). Then > > time sh -c 'touch configure.ac && make -j 6' > > takes ~22 seconds. > > > Surely, build speed is very important. But with these timings, is meson that > much faster to pay off? Although I have the feeling that meson is usually faster than autotools, I have never done any solid benchmarks. Recently, after porting `gnome-control-center`[3] and `gnome-online-accounts`[4] which were some ~big~ programs, I also tried to measure the time used to build the full packages from clean trees by using the `time` command, and meson has been able to do a full build in autoools 1/4 of the time. However they I have also tried to measure `network-manager-applet` by using your same commands in my computer, and their meson's correspondant. Here are some raw numbers: $ git clean -fdx && time sh -c './autogen.sh && make -j 6' real 1m10,221s user 2m47,178s sys 0m14,682s $ time sh -c 'touch configure.ac && make -j 6' real 0m30,160s user 1m6,397s sys 0m6,476s $ time sh -c 'touch meson.build && ninja -C _build' real 0m1,788s user 0m1,588s sys 0m0,378s $ time sh -c 'meson _build && ninja -C _build' real 0m25,570s user 1m24,805s sys 0m8,272s I also tried to measure full builds and installs: $ time sh -c './autogen.sh --prefix=/tmp/nma --enable-gtk-doc --enable-introspection=yes --enable-more-warnings --enable-lto --enable-ld-gc --with-appindicator --with-wwan --with-selinux --with-team --with-gcr --with-more-asserts=yes && make install' real 2m9,749s user 1m55,974s sys 0m12,887s $ time sh -c 'meson --prefix=/tmp/nma --libdir=lib -Db_lto=true -Denable-introspection=true -Denable-gtk-doc=true -Dwith-more-asserts=yes _build && ninja -C _build install' real 0m36,627s user 1m43,569s sys 0m9,874s > I am not familiar with meson. How to do a dist-tarball? Can you show a few > examples of how to use it? How to run tests? I could google it myself, but I > think the commit message should show such examples uses anyway. There is a thread from Sébastien Wilmet in `desktop-devel-list`[5] in which he states that he has the same problem with meson. There is a Quick guide[6] in meson's site which helps with the first steps with meson. Although I recommend you to read it in case you decide to test meson, here are some commands you could use: $ meson build (./autogen) Execute it in the source code's root, where the main `meson.build` is. It will parse the main `meson.build` file and all the directories which are pointed by using the `subdir` function and it will create the `build` (or whatever you name it) directory and put all the building information there. I almost forgot to say that meson is designed to build everything in different build directories, without messing with the source code tree, which will stay clean. This also helps when you want to build the package with different parameters. For example, one directory for every architecture, OS, etc... If you want to add different parameters you have to use the '-D' parameter. For example `-Denable-gtk-doc=true` to set the `enable-gtk-doc` parameter to true. You can see the available options by looking at the `meson_options.txt`, which format is described also in the meson's site[7], or by using the following command: $ meson configure build This will show you the build parameters set by meson in the `build` directory. $ ninja -C build (make) It will start compiling the package by using ninja (other tools are also supported), and using the -C parameter you indicate which directory is holding you build parametes. You will see the progress in the screen with the current build target and the number of targets $ ninja -C build install (make install) It will build the whole package, if it's not already build, and will move to install everything that should be installed. $ ninja -C build test If you want to execute unit test, if the package has it (actually `network-manager-applet` has 2), which helps a lot if you use a CI tool. Finally to make a distributable package, which also executes all the unit tests, you can use the following command: $ ninja -C _build dist (make distcheck) I'm sorry for the lenghty post, though I hope it helps. Please don't hesitate to comment or to contact me in case you need have any doubt, comment or anything else :) [0] https://wiki.gnome.org/Initiatives/GnomeGoals/MesonPorting [1] http://mesonbuild.com/Pkgconfig-module.html [2] http://mesonbuild.com/Release-notes-for-0-41-0.html#a-target-for-creating-tarballs [3] https://bugzilla.gnome.org/show_bug.cgi?id=785414#c8 [4] https://bugzilla.gnome.org/show_bug.cgi?id=787634#c3 [5] https://mail.gnome.org/archives/desktop-devel-list/2017-July/msg00000.html [6] http://mesonbuild.com/Quick-guide.html [7] http://mesonbuild.com/Build-options.html
(In reply to Thomas Haller from comment #5) > what I dislike about our current autotools setup, is that we still use > recursive make for the po directory. But that is just because nobody > invested the time to fix that. > > > The patch drops m4/compiler_options.m4 and m4/git-sha-record.m4 without > functional replacement. Regarding, `m4/compiler_options.m4`, I tried to do what it does by using a combination of meson features. meson is able to add a set of compile warning options using the build type[0], which can be `plain`, `release`, `debug` and `debugoptimized`. The meson.build file also adds some of the warning options present in the `m4/compiler_options.m4` if the build is `debug` or `debugoptimized`. meson also has three warning levels, which can also add more warning options. iirc, these warning levels add '-Wall', '-Wextra', '-pedantic`, which are a set of different warning options. The level can be set using the `meson`/`meson configure` commands. There is a "replacement" for the `GIT_SHA_RECORD` function from `m4/git-sha-record.m4` in the main `meson.build`: res = run_command(find_program('git'), 'rev-parse', '--verify', '-q', 'HEAD') config_h.set_quoted('NMA_GIT_SHA', res.stdout().strip()) [0] http://mesonbuild.com/Running-Meson.html#configuring-the-source
> In fact, after porting several GNOME packages to meson, `network-manager- > applet` has been the first one which only uses one `Makefile.am` file. It was recursive make before, and I think that's (very) inferior to non-recursive. Apparently, this is not a popular opinion. But I must admit, I like our non-recursive Makefile.am. To me personally, the only argument that I care at the moment is the build-time (correctness must be a given). I think the greatest downside of make is that it cannot be introspected. It creates the build-artifact one-way, but you cannot ask what individual steps are necessary. But that would be required from an IDE and is the reason why we still use an editor instead of an IDE (ctags/cscope does not understand the way how the project is build). I think meson addresses that too ( http://mesonbuild.com/IDE-integration.html#ide-integration ) and in the long term, that is the strongest selling point to me. > There is a thread from Sébastien Wilmet in `desktop-devel-list`[5] in which he > states that he has the same problem with meson. There is a Quick guide[6] in I was wondering whether the commit message should explain basic usage. Anyway, you explained it here nicely, so that's fine. Thanks. Maybe we should initally add meson support in parallel, and not drop autotools right away.
(In reply to Thomas Haller from comment #8) > > There is a thread from Sébastien Wilmet in `desktop-devel-list`[5] in which he > states that he has the same problem with meson. There is a Quick guide[6] in > > I was wondering whether the commit message should explain basic usage. > Anyway, you explained it here nicely, so that's fine. Thanks. After your first comment, I had been considering adding some introductory commands in the commit message. Although the commit message can be improved, I concluded that writing only some commands wouldn't be the best introduction to what meson does. However, the commit message can include the URI to the meson's Quick Guide. What do you think? > Maybe we should initally add meson support in parallel, and not drop > autotools right away. The patch that drops autotools is just optional. However, if someday meson offers better features then this patch could be applied.
$ git clean -fdx && time sh -c './autogen.sh && make -j 6' takes for me now ~20 seconds compared to $ git clean -fdx && time sh -c 'meson build && ninja -C build' with 3.2 seconds (this is with ccache enabled (on a repeated build), which significantly speeds up both cases). The speedup if amazing. I merge the first patch (and add meson support) as-is. https://git.gnome.org/browse/network-manager-applet/commit/?id=1c1168d4c7a9ed600ea880ed953b9d05b8f04582 Hopefully this gives the patch more exposure, people can use it and we can find issues. Let's drop autotools not until we are sure that the meson build works as we want it. I leave the bug hence open. Thanks Iñigo for the contribution!!
(In reply to Thomas Haller from comment #10) > Hopefully this gives the patch more exposure, people can use it and we can > find issues. > > Let's drop autotools not until we are sure that the meson build works as we > want it. I leave the bug hence open. It think it's a good idea. Please let me know if there is any issue. > Thanks Iñigo for the contribution!! Thank you for reviewing and including this :)
Regarding "Remove autotools" patch[0], if it's finally merged there are steps before/after to be done. One of the steps is related to `gnome-continuous`[1]. Actually the `configure` script is not strictly necessary, as `gnome-continuous` may patch it on the fly[2]. It's up to the maintainer to include it. Another step is related to `jhbuild`. It must be updated to set the build system to meson. This also implies a release with meson build system merged. There is more information in the GNOME wiki[3]. These are also notes for me, and I'm writing them here to avoid any issue with GNOME's infrastructure. I know how to proceed with these changes, so I can take responsibility for making the appropriate changes, but I have to know when the patch is going to be merged. [0] https://bugzilla.gnome.org/attachment.cgi?id=360365 [1] https://mail.gnome.org/archives/desktop-devel-list/2017-April/msg00080.html [2] https://git.gnome.org/browse/gnome-continuous/tree/patches [3] https://wiki.gnome.org/Initiatives/GnomeGoals/MesonPorting
Hi Iñigo, I see that the meson build system support was merged for nma, which I think is nice. I did a test build and the result looked fine compared to an autotools build. What I noticed is, that the build options (meson_options.txt) are named enable-foo. Those enable- or with- prefixes are autotools anachronisms. With meson they are discouraged. -Denable-foo=false just looks strange. Meson upstream also recommends to drop those prefixes. See also a related discussion at https://mail.gnome.org/archives/desktop-devel-list/2017-July/msg00000.html (it's a longer thread) Iñigo, would you consider updating current git master to drop those prefixes?
(In reply to Michael Biebl from comment #13) > Hi Iñigo, > > I see that the meson build system support was merged for nma, which I think > is nice. I did a test build and the result looked fine compared to an > autotools build. Thank you very much for trying/testing the meson port. It's good to know that everything worked properly :) > What I noticed is, that the build options (meson_options.txt) are named > enable-foo. > > Those enable- or with- prefixes are autotools anachronisms. With meson they > are discouraged. -Denable-foo=false just looks strange. Meson upstream also > recommends to drop those prefixes. > > See also a related discussion at > https://mail.gnome.org/archives/desktop-devel-list/2017-July/msg00000.html > (it's a longer thread) > > > Iñigo, would you consider updating current git master to drop those prefixes? I'm aware of the link, and I even tried to help[0]. However the discussion went in a different direction :) I'm also aware of your message[1] and Emmanuelle's response[2]. I believe that meson offers nice improvements over autotools[3]. However, the fact that it's rather new, scares some developers/maintainers[4][5][6]. I try to pave the way for them so I usually follow the pattern used by autotools. Actually, in general I don't have any preferences regarding naming, though I would like to see certain consistency[7]. However, there does not seem to be any agreement yet[2], and until then, I would like to keep the autotool's naming so that the option names don't confuse package developers/maintainers too much. Anyway, I'm not a nma's developer, so the decision is not in my hand. If you still feel that the names should be changed, I can prepare the corresponding patch. Please don't take me wrong as I'm just trying to help. [0] https://mail.gnome.org/archives/desktop-devel-list/2017-July/msg00001.html [1] https://mail.gnome.org/archives/desktop-devel-list/2017-September/msg00054.html [2] https://mail.gnome.org/archives/desktop-devel-list/2017-September/msg00055.html [3] https://bugzilla.gnome.org/show_bug.cgi?id=788146#c6 [4] https://bugzilla.gnome.org/show_bug.cgi?id=783819#c1 [5] https://bugzilla.gnome.org/show_bug.cgi?id=784762#c1 [6] https://bugzilla.gnome.org/show_bug.cgi?id=784561#c5 [7] https://mail.gnome.org/archives/desktop-devel-list/2017-July/msg00009.html
Well, -Denable-foo=false is different then --disable-foo, so I don't see the point in keeping the enable- prefix. It's inconsistent anyway,as you used enable-selinux for e.g. --with-selinux. See also https://mail.gnome.org/archives/desktop-devel-list/2017-September/msg00056.html
(In reply to Michael Biebl from comment #15) > Well, -Denable-foo=false is different then --disable-foo, so I don't see the > point in keeping the enable- prefix. It's inconsistent anyway,as you used > enable-selinux for e.g. --with-selinux. You are right. My following statement is a half-true: > I try to pave the way for them so I usually follow the pattern used by autotools. Actually the pattern used is as follows: - If the option is a boolean, the option is named as `enable-{option}` in meson. - If the option in autotools follows the pattern `--disable-{option}`, its default value is set to false in meson. - If the option is a string, the option is named as `with-{option}` in meson. This approach offers the benefit of knowing the type of the option just with its name. This helps with some support tools[0][1][2], which are also used in `gnome-continuous`[2][3], and I also think that helps identifying options at first sight. > See also > https://mail.gnome.org/archives/desktop-devel-list/2017-September/msg00056. > html I was also aware of that mail. As I said, if you still feel that the names should be changed, I can prepare the corresponding patch, but the decision is not in my hand. For example in `glib-networking`, Michael asked me to rename the options following that pattern[4], so they were renamed[5]. [0] https://github.com/ebassi/graphene/blob/master/configure [1] https://bugzilla.gnome.org/show_bug.cgi?id=789146 [2] https://github.com/ebassi/graphene/pull/105 [3] https://git.gnome.org/browse/gnome-continuous/tree/patches [4] https://bugzilla.gnome.org/show_bug.cgi?id=786639#c3 [5] https://git.gnome.org/browse/glib-networking/tree/meson_options.txt
Michael Biebl, as you may know, Emmanuele Bassi has uploaded today the new meson porting guidelines[0]. I just wanted you to know that this afternoon I have been updating some of the ports I made with those guidelines[1][2], also including `network-manager-applet`[3] (and I still have much to do :(). However, I've been making minor improvements and then I had to leave, so I haven't been able to notify you. You can find the patch which renames the meson options to new names based on the new guidelines here[4]. Still, the patch has not been yet reviewed. I would really appreciate if the patch is tested, because although I try my best, I make mistakes. [0] https://wiki.gnome.org/action/diff/Initiatives/GnomeGoals/MesonPorting?action=diff&rev1=112&rev2=113 [1] https://bugzilla.gnome.org/show_bug.cgi?id=786149 [2] https://bugzilla.gnome.org/show_bug.cgi?id=790177 [3] https://bugzilla.gnome.org/show_bug.cgi?id=790178 [4] https://bugzilla.gnome.org/attachment.cgi?id=363362
(In reply to Michael Biebl from comment #13) > Hi Iñigo, > > I see that the meson build system support was merged for nma, which I think > is nice. I did a test build and the result looked fine compared to an > autotools build. nm-connection-editor is not able to enable any security on WiFi connections. The relevant .ui resources are not linked into the binaries. Resulting in errors like ** (nm-connection-editor:20672): WARNING **: 16:43:42.880: Couldn't load UI builder resource /org/freedesktop/network-manager-applet/ws-wep-key.ui: The resource at “/org/freedesktop/network-manager-applet/ws-wep-key.ui” does not exist ** (nm-connection-editor:20672): WARNING **: 16:43:42.880: Couldn't load UI builder resource /org/freedesktop/network-manager-applet/ws-wep-key.ui: The resource at “/org/freedesktop/network-manager-applet/ws-wep-key.ui” does not exist (and the security drop down only containing 'none' in this case)
In reply to Dominique Leuenberger from comment #18) > nm-connection-editor is not able to enable any security on WiFi connections. > The relevant .ui resources are not linked into the binaries. > > Resulting in errors like > > ** (nm-connection-editor:20672): WARNING **: 16:43:42.880: Couldn't load UI > builder resource /org/freedesktop/network-manager-applet/ws-wep-key.ui: The > resource at “/org/freedesktop/network-manager-applet/ws-wep-key.ui” does not > exist > > ** (nm-connection-editor:20672): WARNING **: 16:43:42.880: Couldn't load UI > builder resource /org/freedesktop/network-manager-applet/ws-wep-key.ui: The > resource at “/org/freedesktop/network-manager-applet/ws-wep-key.ui” does not > exist > > (and the security drop down only containing 'none' in this case) I've been trying to reproduce this issue in my computer without any success. I'm able to go through the different security options. However, since the original port, the meson build file that builds the connection manager has been changed[0] with the following commit message: "meson: Make sure the entire wireless-security static lib is used Otherwise ld will not link in the gresources, which contain no "needed" (as far as ld can determine) symbols." This seems directly related to your problem. Have you tried with the last version? If no, could you please try it? [0] https://git.gnome.org/browse/network-manager-applet/commit/?id=6a0967b1b
bugzilla.gnome.org is being shut down in favor of a GitLab instance. We are closing all old bug reports and feature requests in GNOME Bugzilla which have not seen updates for a long time. If you still use NetworkManager and if you still see this bug / want this feature in a recent and supported version of NetworkManager, then please feel free to report it at https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/ Thank you for creating this report and we are sorry it could not be implemented (workforce and time is unfortunately limited).