GNOME Bugzilla – Bug 690136
application framework
Last modified: 2014-07-17 13:44:55 UTC
Basically, I'd like something like: #!/usr/bin/gjs --application=myapp We could have gjs automatically add $(datadir)/myapp/js to the search path for example. Even better, automatically set up GI_TYPELIB_PATH/LD_LIBRARY_PATH to point to $(pkglibdir)/libs. See the hacks that gnome-documents and gnome-shell have, and realize how we could make this a lot less ugly. There is more we can do once we have this. Namely, it'd be nice to have something declarative (possibly even exactly the same as .desktop files). #!/usr/bin/gjs --application=myapp # Requires: Gtk-3.0 A dead simple key-value format would be a lot easier to parse for dependency scanners and the like than the "imports.gi.versions.Gtk = '3.0';" that we need now. *Even better* than that, we could traverse the dependency chain and do automatic imports: const Gtk = imports.gi.Gtk; const Gdk = imports.gi.Gdk; const Pango = imports.gi.Pango; ... const Gio = imports.gi.Gio; const GLib = imports.gi.GLib; (How many of you have been bitten by forgetting an imports.gi.GLib when you already were using Gio or vice versa? I know it's really damn irritating for me...)
(In reply to comment #0) > Basically, I'd like something like: > > #!/usr/bin/gjs --application=myapp > > We could have gjs automatically add $(datadir)/myapp/js to the search path for > example. Even better, automatically set up GI_TYPELIB_PATH/LD_LIBRARY_PATH to > point to $(pkglibdir)/libs. > > See the hacks that gnome-documents and gnome-shell have, and realize how we > could make this a lot less ugly. > > There is more we can do once we have this. Namely, it'd be nice to have > something declarative (possibly even exactly the same as .desktop files). > > #!/usr/bin/gjs --application=myapp > # Requires: Gtk-3.0 No. This makes it more invalid JavaScript, given that # is an invalid character. But I do wonder if we can have something in configure.ac: GOBJECT_INTROSPECTION_REQUIRE(Gtk-3.0 GLib-2.0) > A dead simple key-value format would be a lot easier to parse for dependency > scanners and the like than the "imports.gi.versions.Gtk = '3.0';" that we need > now. > > *Even better* than that, we could traverse the dependency chain and do > automatic imports: > > const Gtk = imports.gi.Gtk; > const Gdk = imports.gi.Gdk; > const Pango = imports.gi.Pango; > ... > const Gio = imports.gi.Gio; > const GLib = imports.gi.GLib; > > (How many of you have been bitten by forgetting an imports.gi.GLib when you > already were using Gio or vice versa? I know it's really damn irritating for > me...) Fix your code, don't make it have automatic magic "GLib" globals that are sometimes there because your configure.ac said something about it.
Wow, a distinct lack of enthusiasm here...what'd I do wrong? (In reply to comment #1) > > No. This makes it more invalid JavaScript, given that # is an invalid > character. Yeah sorry, I mean: // > But I do wonder if we can have something in configure.ac: > > GOBJECT_INTROSPECTION_REQUIRE(Gtk-3.0 GLib-2.0) What would that do? > Fix your code, don't make it have automatic magic "GLib" globals that are > sometimes there because your configure.ac said something about it. But it's basically pointless tedium. The way the libraries are designed is as a dependent stack, not independent fiefdoms. If you use Gtk+, you are *also* expected to use Gio. In C, if you #include <gtk.h>, you get gio.h too. On the linker side, pkg-config --libs gtk-3.0 will also give you -lgio-2.0. It makes total sense to me to have this expectation mirrored in JavaScript too.
(In reply to comment #2) > Wow, a distinct lack of enthusiasm here...what'd I do wrong? Sorry. I like the first idea, because, yeah, the silliness that goes into launching something is a bit absurd. > (In reply to comment #1) > > > > No. This makes it more invalid JavaScript, given that # is an invalid > > character. > > Yeah sorry, I mean: // I think that gobject-introspection is about my level of "things that should scan comments to function". I don't want to see more of our platform going into magic comments. > > But I do wonder if we can have something in configure.ac: > > > > GOBJECT_INTROSPECTION_REQUIRE(Gtk-3.0 GLib-2.0) > > What would that do? Require the specified typelib file, and fail if you don't have it. Basically PKG_CHECK_MODULES, but for typelibs/girs. > > Fix your code, don't make it have automatic magic "GLib" globals that are > > sometimes there because your configure.ac said something about it. > > But it's basically pointless tedium. The way the libraries are designed is as > a dependent stack, not independent fiefdoms. If you use Gtk+, you are *also* > expected to use Gio. In C, if you #include <gtk.h>, you get gio.h too. On the > linker side, pkg-config --libs gtk-3.0 will also give you -lgio-2.0. > > It makes total sense to me to have this expectation mirrored in JavaScript too. My issue is answering "where do magic names that show up in this scope come from". "Hm, why do I have GLib/Gio?" "Because you imported GTK+" This goes against every other import/module system. Yes, I run into the same problem, and it's a tiny annoying time waster, but "include things you didn't explicitly import" is the wrong solution. Do the GLib/Gio bindings always become const in the current scope? Do they follow the binding of the variable assignment? What about cases like: // To be filled in later. var Gtk = null; function init() { // We can't import at parse time for X reasons var MagicGtk = imports.gi.Gtk; Gtk = MagicGtk; } Does that magically put Gio into local scope? Global scope? Do they var binding? let binding? const binding? Are they "MagicGio" and "MagicGLib"? This is effectively now language design. Following the principle of least surprise, even if it's an annoyance, means that the code is more readable, more consistent, and all these questions just disappear.
Note that I've always been of the opinion that we should follow upstream ECMAScript development, potentially along with things like CommonJS. ECMAScript has not established a module system, but if/when they do, I can't imagine it would be that much different from the CommonJS module proposal. So I think we should be heading in the direction of that, instead.
(In reply to comment #3) > > I think that gobject-introspection is about my level of "things that should > scan comments to function". I don't want to see more of our platform going into > magic comments. Maybe...there's a lot of difference between key-value pairs at the top of the file and what g-ir-scanner does. > > > But I do wonder if we can have something in configure.ac: > > > > > > GOBJECT_INTROSPECTION_REQUIRE(Gtk-3.0 GLib-2.0) > > > > What would that do? > > Require the specified typelib file, and fail if you don't have it. Basically > PKG_CHECK_MODULES, but for typelibs/girs. Theoretically, something could scan the configure.ac and generate dependencies that way, but I think it makes significantly more sense to have the metadata associated with the binaries for runtime dependencies. The configure.ac should be about *build* time dependencies. Basically, look at how rpm/dpkg type systems parse source code for #!/usr/bin/foo and generate "foo" as a package dependency. That's associated with what you install, and I think gjs should work the same way. > My issue is answering "where do magic names that show up in this scope come > from". Sure...docs? > "Hm, why do I have GLib/Gio?" "Because you imported GTK+" > > This goes against every other import/module system. But what would be the *problem* with having Gio? What would cause someone to ask that question? > Do the GLib/Gio bindings always become const in the current scope? Yes. > Do they > follow the binding of the variable assignment? What about cases like: > > // To be filled in later. > var Gtk = null; Illegal. > > function init() { > // We can't import at parse time for X reasons "X reasons"? Note that unlike pygobject, gjs does *not* call gtk_init() when you simply reference imports.gi.Gtk. So what are "X reasons"? > var MagicGtk = imports.gi.Gtk; > Gtk = MagicGtk; > } Yes, this is illegal. > This is effectively now language design. Following the principle of least > surprise, even if it's an annoyance, means that the code is more readable, more > consistent, and all these questions just disappear. I'm not sure about that...I mean, understanding what happens when you type the "foo" part of "imports.gi.Gtk.foo" (i.e. introspection) is an enormous level of magic for your average JS programmer. Anyways, the high level problem to solve is - we need a simpler way for application authors to specify that they require Gtk-3.0. See bug 689654 . We don't have to also at the same time do autoimports.
(In reply to comment #5) > > Do they > > follow the binding of the variable assignment? What about cases like: > > > > // To be filled in later. > > var Gtk = null; > > Illegal. Why and how is naming a variable "Gtk" illegal? Does that mean that the set of reserved words depends on the typelibs I have in /usr/lib64/girepository-1.0/ ? yikes
(In reply to comment #5) > Maybe...there's a lot of difference between key-value pairs at the top of the > file and what g-ir-scanner does. It's a slippery slope either way. > > > > But I do wonder if we can have something in configure.ac: > > > > > > > > GOBJECT_INTROSPECTION_REQUIRE(Gtk-3.0 GLib-2.0) > > > > > > What would that do? > > > > Require the specified typelib file, and fail if you don't have it. Basically > > PKG_CHECK_MODULES, but for typelibs/girs. > > Theoretically, something could scan the configure.ac and generate dependencies > that way, but I think it makes significantly more sense to have the metadata > associated with the binaries for runtime dependencies. The configure.ac should > be about *build* time dependencies. I don't care where we put runtime dependencies, as long as they go somewhere. > But what would be the *problem* with having Gio? What would cause someone to > ask that question? With Gio, it might not be a problem. But automatically importing another random system library (like Gdata or similar because libgd has it as a dependency) might be confusing. Otherwise, it's just this magic global that gets injected into the scope. You might have the reverse problem: you have an unused Gtk import, so you remove it, and stuff that used Gio breaks. The time-waster goes both ways. > Note that unlike pygobject, gjs does *not* call gtk_init() when you simply > reference imports.gi.Gtk. So what are "X reasons"? if (PROFILE) { const Gtk = new ProfilingWrapper(imports.gi.Gtk); } else { const Gtk = imports.gi.Gtk; } or something like that. I can imagine situations where people want to control imports.
(In reply to comment #6) > > Why and how is naming a variable "Gtk" illegal? Does that mean that the set of > reserved words depends on the typelibs I have in /usr/lib64/girepository-1.0/ ? > > yikes No. Only things which you depend on are imported.
(In reply to comment #5) > I'm not sure about that...I mean, understanding what happens when you type the > "foo" part of "imports.gi.Gtk.foo" (i.e. introspection) is an enormous level of > magic for your average JS programmer. But the magic doesn't have to be there. There's no reason it couldn't be just another module implemented in JS or a statically generated module. Ideally, at the JavaScript level, it acts like everything else in the system. In Python, this is even more explicit, as introspection modules are supposed to act like every other module/package that you've ever used for Python. So, the "magic" is only an implementation detail, in the best case.
Ok, so I yesterday it was a very boring day, so I decided to implement a package system/application framework for gjs, similar to the one proposed here. The API boils down to: exec @GJS@ -c 'imports.package.start({ name: "@PACKAGE_NAME@", version: "@PACKAGE_VERSION@", prefix: "@prefix@" })' to start the application, and pkg.require({ 'Gd': '1.0', 'Gdk': '3.0', 'GLib': '2.0', 'GObject': '2.0', 'Gtk': '3.0', 'Lang': '1.0', 'Mainloop': '1.0', 'Params': '1.0' }); to import external dependencies. imports.package.start() does the first half of magic: it sets a 'pkg' global variable pointing to itself (I wanted package, but it's a reserved identifier), and inside that it creates variables for the standard Automake directories, based on the passed prefix. Note that it is done in the standard way: if you need to setup a different libdir or datadir, sorry, this is not for you. The advantage of using prefix only is that configure works, instead of requiring sed in Makefile because autotools are stubborn. Also, package.start() takes care to set up GJS search paths and GI typelib + library path (see 694485 for the required g-i API), in a manner that makes sense both if the application is installed and if the application is run from the source tree (it looks at './src' to discriminate). imports.package.require() (or pkg.require() actually) is the second half of magic. It has a fixed list of "known" JS modules you may need, and each has a $API_VERSION that it checked against, while everything else is considered a typelib. After importing, it sets the name as a property on the global object. The package module is completed by imports.package.dumpRequires(), which dumps a JSON of the last .require() call, and utility functions to setup Format and Gettext. The code can be found at https://github.com/gcampax/gjs/commits/package. It includes also a fork of the Params module from gnome-shell, as I believe it is generally useful to JS applications. An example of usage can be found at https://github.com/gcampax/gtk-js-app --- One passage that I still don't like is that you need to pass the name and prefix to package.start(). Doing more magic, we can guess the name and the prefix from the basename of the launched executable, provided you pass "-a $0" to exec. The disadvantage is that you lose version, which was useful to avoid a generated config.js module while keeping updated about dialogs, and that you become dependent on bash (whereas I wanted to use bash just for the "commodity" script, the desktop file would launch gjs straight)
I still think this is a bad idea. We should lean towards what the rest of the JS world is doing. Most everything, including the new Harmony syntax, is using the CommonJS modules spec: http://wiki.ecmascript.org/doku.php?id=harmony:modules http://wiki.commonjs.org/wiki/Modules/1.1.1 People using GJS for GNOME apps should be able to use the already-existing set of modules using require and exports that exist on other platforms, without having to rewrite them to GJS's custom crazy app system. I think we can move to this in a compatible manner: * Provide a per-module "exports" object which is the same as the module global object ("this"). * Provide a stub "require" function that will do "return imports[name];"
Except that there is no rest of the world: people who use JS use jQuery, dojo, Prototype, mootools... Each of those has its own module system, it's own class system, its own convenience APIs above the (ugh!) platform. Just like we have, with Lang, the overrides, and now Package. Harmony's modules are all nice and everything, but they're not something for today: they require support from the interpreter, because they use a completely new syntax. None of the existing JS interpreters out there support it, which means it will take years before developers get accustomed to it. CommonJS is interesting, but I don't see the advantage over the existing "imports" system - which I'm not proposing to replace! Package would be implementable on top of CommonJS just fine: the only non-JS parts are those dealing with typelib and library paths.
(In reply to comment #12) > Except that there is no rest of the world: people who use JS use jQuery, dojo, > Prototype, mootools... Each of those has its own module system, it's own class > system, its own convenience APIs above the (ugh!) platform. Just like we have, > with Lang, the overrides, and now Package. Prototype is dead. dojo is on life support after funding was cut. I'm going to assume they're both irrelevant, since they won't be able to any "modern JS" which we want to smoothly transition into. MooTools and jQuery are the remainders, and they differ in philosophy. jQuery doesn't have its own module system or class system. It has a "plugin" system in terms of monkey patching some global, "jQuery", or the standard "jQuery Object" prototype. It doesn't have any real respect for namespaces -- it just hopes that nothing collides with anything else. MooTools doesn't have its own module system either, but it has a class framework (which I ripped off for Gjs's Lang, as you know). Instead of building wrappers over the platform, it prefers to just monkey patch standard things like Array.prototype, String.prototype, DOMElement.prototype, etc. I'm mostly talking about Node.JS and other non-browser JS implementations, most of which agree use CommonJS. > Harmony's modules are all nice and everything, but they're not something for > today: they require support from the interpreter, because they use a completely > new syntax. None of the existing JS interpreters out there support it, which > means it will take years before developers get accustomed to it. The whole point of Harmony's modules is that they're CommonJS modules, but built into the engine. Harmony is more about adopting existing practices than doing R&D in a standards language (which ES4 has been widely criticized for) > CommonJS is interesting, but I don't see the advantage over the existing > "imports" system - which I'm not proposing to replace! "Package" is just a metadata format. "Module" is probably what you wanted to talk about. The advantage of this is to allow people to use many JS libraries that exist out there that depend upon CommonJS support, most of them hosted at npm: https://npmjs.org/ > Package would be implementable on top of CommonJS just fine: the only non-JS > parts are those dealing with typelib and library paths. "library paths"? Our current imports syntax is a simple variant of require, so I don't really see any reason why we can't provide CommonJS wrappers and eventually remove support for our custom variant at some point in the future. It's just a mechanical transform: const Foo = imports.foo; => const Foo = require('foo'); const Gtk = imports.gi.Gtk; => const Gtk = require('gi/Gtk'); and the latter one I've never seen used in actual JS code out there.
Ok, sorry, but we are on two different levels here. Basically, what I proposed in comment 10 was a full blown application framework - something to deal with private paths, configure directories, standard monkey-patches (like .format()), etc. Something that would reduce the need for generated config.js modules, shell wrapper scripts and GNOME_FOO_UNINSTALLED_SRCDIR environment variables. Then, because it seemed nice, I added also some convenience API for standard and GI imports. But they are orthogonal really, and if we think that injecting names in window is bad, we can always remove it, so that people would have pkg.require(...) for version dependencies, and require() to access the module object. What I proposed used the module system, obviously, but does not depend on the particular module system. If you want to add window.require() and window.exports in addition to window.imports, that's fine by me, but it's not really an "application framework". Anyway, I detailed my proposal at https://live.gnome.org/GiovanniCampagna/Experiments/Package It shows what it attempts to do and why it is done this way. Also, I pushed wip/package, with the "reference" implementation.
We skipped 3.8, and we skipped 3.10, but I still like to land this feature at some point. To this extent, I updated gtk-js-app and gtk-js-app-generator in github, with a new version of package.js. Changes in this iteration: - pkg.require() doesn't do the magic thing, it just checks that GI repositories exist - support for resources - application launching through dbus and better uninstalled running I believe at this point all comments were addressed, and package.js can go in. Please complain if this is not the case.
Forgot to mention, this is kind of a blocker for announcing gtk-js-app-generator.
(In reply to comment #15) > Changes in this iteration: > > - pkg.require() doesn't do the magic thing, it just checks that GI repositories > exist Yeah, that sounds fine. > I believe at this point all comments were addressed, and package.js can go in. > Please complain if this is not the case. So I feel like this is definitely an improvement, and something gjs app authors should probably use. But I also feel like we could do better. Having to sed a launcher script like https://github.com/gcampax/gtk-js-app/blob/master/src/gtk-js-app.in just sucks =/ It's all kind of tough if we want to keep supporting traditional rpm/dpkg type installation (which we do). We could shoot for the goal where it's all one file, but that feels like overkill. What if we recommend a deployment scheme where all of the app data goes in $(pkglibdir), and the the launcher binary is a symlink /usr/bin/org.gnome.Clocks -> /usr/bin/gjs. Then gjs would check to see if argv[0] is a symlink. If so, then take the suffix as the app name, then look for $(dirname $argv0)../lib/applications/${appname}/main.js or so. There are other deployment tools that do stuff like this. It's a bit hacky, but...so is sed on a shell/js script. I need to study what other projects like the Click Package and webapps do; not sure if you have. It just feels like we could do even better than this in terms of being able to create a main.js (and optionally app-icon.png) and be good to go. Even to the extent of shipping a standard /usr/share/gjs/Makefile-app.am that apps could include.
(In reply to comment #17) > We could shoot for the goal where it's all one file, but that feels like > overkill. I really like the gresource approach. I landed Cosimo's GResource imports patch upstream, and I'd like to see it attempted where we have a simple gjs-launcher binary launcher that does the equivalent of "imports.main.main();", and you just have to supply a main.js in your GResource.
(In reply to comment #18) > (In reply to comment #17) > > We could shoot for the goal where it's all one file, but that feels like > > overkill. > > I really like the gresource approach. I landed Cosimo's GResource imports patch > upstream, and I'd like to see it attempted where we have a simple gjs-launcher > binary launcher that does the equivalent of "imports.main.main();", and you > just have to supply a main.js in your GResource. But gnome-shell still needs to read app .desktop files and icons from the system. Unless we teach it directly about a new framework.
So... I updated the wip/package branch, and integrated GResources. Now package.js would try to load ${name}.src.gresource, and if that's found, it will add that to the gjs search path. It will also try to load ${name}.data.gresource, replacing a manual call to initResources(). After all resources are cool, so why not using them? Note that I considered a shim binary compiled with the resources, but I rejected the idea because I don't want to give away on noarch packages. Also note that while single makefile might sound compelling, as soon as you start adding data resources, service files (DBus, systemd), desktop files, GSettings schemas, gettext files, etc. you realize the need for a complete build system, so my advice is to turn gtk-js-app into the standard template.
Created attachment 268096 [details] [review] Add a package/application convenience layer Introduce a new module, called package, which includes convenience API and a set of simple conventions for packaged applications. The module takes care of setting up private search paths for JS modules (as resources, if possible) and private typelibs, and it optionally does gettext too. So... I've been going back and forth on this one, adding features, removing features... It's hard to determine what's good and what's not, especially as the requirements from gnome-weather's side changed over time. But it's time to put it up for review, and possibly merge it. This is the "hardcoded path" version: inside the parameters it takes prefix and libdir. Arguably, libdir can be derived from prefix, except that every distribution does it differently when it comes to multiarch. The alternative, as Colin suggested, was to look at argv[0] (System.programInvocationName, really), take the dirname of that and base everything off it. This would almost work for full JS apps, because they only need pkgdatadir and datadir (localedir, really) - the problem comes from hybrid C+JS apps, like gnome-weather, because you need a pkgdatadir and a pkglibdir. There's a number of approaches to that, namely: - Stuff everything (or at least the application) in pkglibdir, and be happy with that Downsides: - you get JS files in an arch specific dir - going from pkglibdir to prefix (to get to datadir/localedir) doesn't always work: if you think of $(prefix)/lib64 and $(prefix)/lib/x86_64-linux-gnu, the number of dirname() is different - libgd still installs GIR files in $(pkgdatadir)/gir-2.0, so you still own that directory, even if you're never using anything there - Stuff everything (or at least the application) in $(prefix)/lib/$(PACKAGE_NAME) (libexec the Arch/systemd way) - libgd installs stuff in $(pkgdatadir) - the actual libraries from libgd and friends would be in $(pkglibdir), unless you force install them in $(prefix)/lib, which would break downstream policies and cause hassles to packagers Arguably it's pointless to do multiarch for private libraries in an application, but I'm afraid people would complain anyway - Others I didn't think of?
Created attachment 268519 [details] [review] Add a package/application convenience layer Introduce a new module, called package, which includes convenience API and a set of simple conventions for packaged applications. The module takes care of setting up private search paths for JS modules (as resources, if possible) and private typelibs, and it optionally does gettext too. Ok, so this is the other approach. It's no longer required run sed in the Makefile to substitute @prefix@ and @libdir@ (one still needs to substitute @PACKAGE_VERSION@ if desired, but configure can handle that), JS resources are installed in pkglibdir and data resources in pkgdatadir. The launcher binary becomes a symlink, but this is almost transparent to package.js, and prefix is "written" in the symlink target or in the dbus .service file.
So, one thing I really don't like is the hardcoding of 'src' in _runningFromSource(), and also requiring the CWD to be the directory where 'src' is, rather than being in src/ and running ./myprogram. Some projects don't use 'src' but instead use 'js' or their project name. I also think it's a bit too much magic to change running behavior based on where the binary is, and could screw over beginning developers if they're not too careful. Do you have a simple example for an app (maybe a Twitter feed reader) that would use this?
It's a matter of conventions. We can't support all possible package layouts, so I made some choices - detailed here: https://wiki.gnome.org/GiovanniCampagna/Experiments/Package In practice, this doesn't matter: package.js is not meant for existing JS applications, it's meant for boostrapping new apps. Also, we don't change running behavior based on where the binary is, we change it based on how it's invoked. You run it as './src/app', we assume your testing your app uninstalled, any other way, the app is installed. If it's installed, yes, we change the running path according to the prefix guess mechanism - it was Colin's idea. In any case, installation is handled by autotools, and the beginner developer should not touch any of that stuff, unless he knows what he's doing. Being able to test your app without installing is a big requirement, especially when developing outside of jhbuild, and it's also a requirement for getting proper IDE behavior (press F3 and the app runs, without crashing and complaining).
We went back and forth on this one for several cycles, so I decided to push the last iteration, to avoid skipping another cycle. What I pushed uses resources everywhere applicable, uses conventions but does not impose them (for example gnome-sound-recorder is able to use it while still being installed in /usr/share/gnome-sound-recorder), has all the bad GApplication magic stuff removed (GApplication itself handles that), is generic enough to run multiple apps in one package (gnome-weather uses it for the search provider), and does not do any magic to guess libdir (although it does guess the application name for resources using the installed path) This is already used and validated by gnome-weather and gnome-sound-recorder.