GNOME Bugzilla – Bug 445078
Play sounds through GStreamer
Last modified: 2008-08-20 06:44:46 UTC
I know this has been proposed several times, but no one actually did it. Now I tried to make a dbus-server to load and cache the sounds and allow others to easily play it. Patches for libsounds, gnome-settings-daemon, gnome-session, libgnome and libgnomeui included. Comments welcome :-)
Created attachment 89543 [details] GNOME Sound Effects Daemon including client lib and patches for various GNOME libraries
(See also the previous discussions on this in bug 82340.)
Note: The main idea is from http://live.gnome.org/GStreamer/esd (see the various mailing-list discussions) I didn't integrate it with libnotifiy because this API will also be used for sound playback in GNOME games and I really don't think that's something notify-daemon should do.
Could you mail desktop-devel-list@gnome.org about this? It's going to get more attention there than in bugzilla, and all the right people can look at it. Thanks!
Done. http://mail.gnome.org/archives/desktop-devel-list/2007-June/msg00037.html
Looks very nice, thanks Although it's not clear for me why new library is required. The same can be done in libgnome.
You're right, libgse-glib could be integrated in libgnome, but the long-term goal was to kill libgnome, so I didn't know what to do... By the way, I just found out that someone seems to have made something like this already: http://etudiant.epita.fr/~lureau_m/ds-0.1/
Yeah, this work is really very important and popular. As for libgnome vs libgse personally don't see why another library will be better than libgnome but probably others think differently. It's interesting to find maintainer opinion on this. Advantage of just using libgnome - you don't need to patch control-center and others then. Minor recent news about the same subject - libnotify now support sound event through gstreamer too although sound range is limited. gnome-games completely dropped libgnome sound support, now they play sounds directly through gstreamer. I'm not sure why but this work is committed already.
Of course we would have to patch gnome-settings-daemon etc. to use the new API even if we integrated that into libgnome - because the old gnome_sound API is pretty useless and almost all applications use esd directly, for instance gnome-settings-daemon does to load samples.
I just saw this: http://live.gnome.org/TwoPointSeventeen/Platform (Point "Upcoming Deprecations") It's clearly stated there that libgnome will die sooner or later and therefore I don't think we should add any new functionality to it just to move it out again when libgnome is dead.
As it stands now, this patch doesn't fully deprecate sound-in-libgnome, because it still leaves the sound event lists (eg, $sysconfdir/gnome/sound/events/gnome-2.soundlist) in libgnome, and things still depend on those files to map the event names ("gnome-2/login") to the actual sound files ($prefix/share/sounds/startup3.wav). If the code is going to be in a separate library rather than in libgnome, it needs to include include that stuff too (and presumably move it from gnome_config to gconf).
I haven't really understood this topic yet, what library provides this soundlist-related things? libsounds perhaps? Or libgnome itself? I would really appreciate if you could introduce me to that topic so I can try to implement it if requested.
I don't really know the details of how it works, but I know that, for example, gnome-session does gnome_triggers_do (NULL, NULL, "gnome-2", "login", NULL); and *somehow* inside libgnome, that gets translated to (by default) playing startup3.wav. This is also tied in to the "System Sounds" in gnome-sound-properties (eg, playing a sound when a dialog pops up).
gnome-settings-daemon loads all that sounds, that's why I had to patch it, but I didn't look how it assigns the names to the files, but most likely libsounds does that task.
Quote from IRC (01:41:56) nsh: chpe: what do you think, should we ping kmaaras about sound in libgnome or will he respond soon? (01:42:14) nsh: I wonder who is responsible for this patch review (01:43:24) chpe: nsh: I guess first it needs to be decided if the whole approach is right. reviewing the patches will be easy then (01:44:03) chpe: nsh: I can review the libgnome[ui] parts (01:44:17) nsh: agree, but who will decide :) 01:45 (01:45:01) nsh: looks like it's the case where one should take responsibility for decision ;) (01:46:33) chpe: I guess it might be better to put this demon into an established module instead of adding yet another one... gnome-session or gnome-control-centre/settings-daemon ? (01:47:12) nsh: completely agree (01:50:00) nsh: and where should library go? (01:51:38) chpe: nsh: well, libgnome is the obvious place, but the guy's right in that it's also not the right place since libgnome wants to die... (01:52:09) chpe: nsh: since the 'library' is just 1 gobject, maybe just c-n-p ? (01:52:41) nsh: what's c-n-p? (01:52:48) chpe: nsh: cut and paste :) (01:52:52) chpe: like egg (01:54:11) nsh: well, personally I don't see why we should change something in existing api (01:54:36) nsh: it will be still usable 01:55 (01:55:10) nsh: and implementation of gnome_sound_play is quite small and could be cut-n-pasted for sure (01:55:21) nsh: it's just one dbus method call (01:55:30) chpe: yeah (01:56:51) nsh: so we should just get thos agreement and that's all
I'm not really a friend of c-n-p, but if you would like to do it that way, we should make sure we've really considered every possible use case to minimize API changes in future.
Please, please, no cut and paste :-) It's hard enough to try to remove all the libegg code out there. I think it makes sense to keep all this in libgnome, even if it's dying (well, the daemon could probably live in gnome-settings-daemon - it's not a setting, though...). I'm not quite sure, but in the long term, we'll probably want to see this kind of API in gtk+ or somewhere else in our stack, and adding a new library is probably not the best idea if we keep this in mind. Also, I didn't look at the new API and at the old libgnome API, but is there any reason to not replace the old code with the new and keep the API? cc'ing Marc-André, who thought a lot about this, I believe.
> Also, I didn't look at the new API and at the old libgnome API, but is there > any reason to not replace the old code with the new and keep the API? Yes: As you can see in the patch to libgnome, the old API is unusable for such tasks, as applications still have to use esd directly in order to get the samples and play them - that's what libgnome does in e.g. gnome_trigger_play_sound.
(In reply to comment #18) > > Also, I didn't look at the new API and at the old libgnome API, but is there > > any reason to not replace the old code with the new and keep the API? > Yes: As you can see in the patch to libgnome, the old API is unusable for such > tasks, as applications still have to use esd directly in order to get the > samples and play them - that's what libgnome does in e.g. > gnome_trigger_play_sound. That's obviously not a good reason since there's gnome_sound_play(). The question is why wasn't it used?
(In reply to comment #19) > (In reply to comment #18) > > > Also, I didn't look at the new API and at the old libgnome API, but is there > > > any reason to not replace the old code with the new and keep the API? > > Yes: As you can see in the patch to libgnome, the old API is unusable for such > > tasks, as applications still have to use esd directly in order to get the > > samples and play them - that's what libgnome does in e.g. > > gnome_trigger_play_sound. > > That's obviously not a good reason since there's gnome_sound_play(). The > question is why wasn't it used? > Because esd loads the samples when you use gnome_sound_sample_load() and caches them in memory. gnome_sound_play() loads a sample, plays it and frees it afterwards, it doesn't allow it to be cached and plaied again. That's why libgnome itself doesn't use it for playing event sounds. gnome-settings-daemon loads the samples by gnome_sound_sample_load but libgnome uses esd directly to get their id and play them because gnome-sound doesn't allow this.
Philipp, Lennart (PulseAudio) is also working on an API for sound sample/event/effects(?). My POV, Given that PulseAudio is reaching the graal of audio unification, GNOME should not try to implement its own solution. The APIs Lennart works on are not tight to PA, hence, there will be different flavors (using dbus with whatever daemon implementing the service - as you implemented, or pulseaudio, or directly OSS, for instance) - in a way, its close to DAPI sound proposal you mentioned (comment #7). The sound API will come with libsydney & is called libcanberra (it was initiated during last LAC) The fresh mailing list is there: https://tango.0pointer.de/mailman/listinfo/libsydney-discuss lennart in CC too. (PS: for location of the code, I end up thinking that it should live in the portland/DAPI world, which offers common set of client interfaces but different desktop flavors of the implementation)
OK, then I'll resolve this as INCOMPLETE. Was a good practice though :-)
BTW, a current draft of this API is available here: http://0pointer.de/public/canberra.h Comments welcome.
Sorry Philipp, I don't understand you. The patch is nice and the only problem is layout of components and it seems that everyone almost agree on it. About canberra I don't see how this API will help us. For example we'd like to play any audio file, not just in ogg/wav format. There is no gobject bindings, no gstreamer. There is sound server some systems don't require. Is it really the thing we need?
(In reply to comment #24) > Sorry Philipp, I don't understand you. The patch is nice and the only problem > is layout of components and it seems that everyone almost agree on it. Well, I thought these guys were working on the same thing I did and that they have planned it far longer and it was some sort like accepted - sorry if I got that wrong. Besides that, I just noticed to problems with current implementation (I´ll look if this can be patched): 1. When setting the sound state to GST_STATE_PAUSED on loading, GStreamer opens a connection to ALSA (or whatever) already. I don't really want to have 20 connections to ALSA (or whatever) only for sounds that aren't actually being played. (Perhaps leave it GST_STATE_NULL, but loading problems might not be noticed then.) 2. Takes up CPU even when idle (Perhaps fixing #1 also fixes this?) And of course, comments on how to improve the API or the layout of components is always welcomed.
It would be nice to have a daemon as a part of existing component, say gnome-session. New C API for sound is not required too. It's enough to have dbus daemon api and reimplementation of existing libgnome functions. About problem 2: really reproducable :( No idea if it's a gstreamer issue. From strace it looks like daemon polling unavailable file: [pid 6117] read(16, "\1\0\0\0\220\257qFAj\32\26\1\0\0\0", 16) = 16 [pid 6117] read(16, 0xb68adf54, 16) = -1 EAGAIN (Resource temporarily unavailable) [pid 6117] poll([{fd=16, events=POLLIN|POLLERR|POLLNVAL, revents=POLLIN}], 1, -1) = 1
(In reply to comment #26) > It would be nice to have a daemon as a part of existing component, say > gnome-session. New C API for sound is not required too. It's enough to have > dbus daemon api and reimplementation of existing libgnome functions. It won't live in gnome-session. gnome-session is critical because if it crashes, that's really very bad for the user. The best candidate for an existing component would be gnome-settings-daemon, but this is not really about settings... Anyway, if we decide we want this, we'll find a home :-)
(In reply to comment #26) > New C API for sound is not required too. It's enough to have > dbus daemon api and reimplementation of existing libgnome functions. As I´ve already mentioned several times, the current gnome-sound API is simply not usable. Look at [1]: You can only load samples, but don´t play them. You can only play sounds by file. All modules play sounds by getting the samples and playing them through esd directly or play the file by using the file name. [1] http://developer.gnome.org/doc/API/libgnome/gnome-gnome-sound.html
Philipp: it's just a matter of adding one function, then: gnome_sound_play_sample (int sample_id)
(In reply to comment #29) > Philipp: it's just a matter of adding one function, then: > gnome_sound_play_sample (int sample_id) > It sounded like "The current API is perfect and doesn´t have to be changed" to me... So you want to convert GseClient to some kind of singleton object?
Please also consider that libgnome wants to die...
The point is: we don't want to add yet-another-library unless we're sure that we need another library. And really, I don't think that a simple sound API should live in a separate library. In the long term, it should probably go in gtk+. So, if we don't add a new library and can't go for the long term solution right now, the best short term is to keep the API we have.
I agree that there´s no separate library needed, I´m just not too happy about integrating it with libgnome. But as this sound API is quite small, we should be able to move it to gtk+ or something later on. So although now one will ask me, I agree with integrating it into libgnome.
Well, but what to return on gnome_sound_sample_load? It´s supposed to return the ESD sample ID, but obviously we wouldn´t have something like that any more. Simply return -1?
Fixed the issues with CPU usage and ALSA connections.
Created attachment 90127 [details] [review] Lower CPU usage and don't keep ALSA connections open
(In reply to comment #24) > About canberra I don't see how this API will help us. For example we'd like to > play any audio file, not just in ogg/wav format. There is no gobject bindings, > no gstreamer. There is sound server some systems don't require. Is it really > the thing we need? The available fields which are defined in the current canberra.h spec are merely examples. Of course, instead of defining seperate fields for WAV and OGG files we could just pass a generic file name, where it is the playback code's problem to decode the specified files correctly. OTOH I think explicitly saying that we support only WAV and OGG makes a lot of sense, because it would otherwise make it very difficult to implement backends. The supported formats should be from a very limited set only, to allow simple backend implementations. Much the same way as the fdo icon spec mandates exclusive usage of PNG and SVG for icon files. There are a few reason why I push for libcanberra, and I don't think that the sound effects daemon that has been proposed here would solve all those issues and is thus a real improvement above the current situations. Among them: - l10n of events sounds. i.e. english desktops would use an audio file saying "You've got mail", german desktops however are using "Sie haben Post", and suchlike. Selecting the right audio file to play depending on locale settings should be a job for a library, not for the developer implementing the user application. - a11y for event sounds. People with hearing problems should be notified with a textual string on the screen. Thus each event sounds needs a human readable description attached. Also, event sounds for a11y need to be attached to a specific window to be really useful. - Themeing. Event sounds should be loaded accroding to some theme definition, and not be decided upon by the application. We probably need some kind of "sound-naming-spec", similar to the already existing "icon-naming-spec" from fdo. - Advanced sound server stuff. When a sound server like PulseAudio is running, it is a strong advantage if all kinds of meta information can be attached to a stream. For example, an icon for an event sound should be shown in the volume control screen. A sample needs to be properly named, also for the volume control. If sounds are attached to windows this could allow some nifty tricks like positioning event sounds in space according to the position of the windows on the screen. (i.e. if an event sound is played by an app with a window on the left edge of the screen it should be played primarily on the left speaker.) - A system like this should be cross-desktop. i.e. KDE should preferable use the same API for event sounds like these. In summary: I believe that a good event sounds API should provide considerably more functionality than what has been proposed in the sound effects daemon. libcanberra is my attempt to make a good, easy-to-use API out of this: basically it is just an API to compile a couple of properties into a single event and than fire it. The list of supported properties is extensible. A few have be defined in the header file. For special purposes users are welcome to define their own. Of course, instead of agreeing on canberra.h as common API we could instead agree on some kind of D-Bus API. However, if you take PA as one of the providers of this API: we don't use D-Bus, because we need low-latency and network transparency, hence proxying all this stuff over D-Bus doesn't look like a good idea to me. I guess to D-Bus people every problem looks like D-Bus-problem. ;-) I hope this explains a little the rationale behind my canberra proposal, and also why I don't think we should follow the gnome sound effects daemon path.
This was really just a quick´n´dirty replacement of the gnome_sound API. It isn´t intended to do such great new things, I only made it to hear event sounds until something better and bigger - like libcranberra - is ready, because I thought that it´ll take some time until you finished all that amazingly sounding stuff. And for this job D-Bus is just fine: Low-latency is not required. And I agree, loading sounds shouldn´t be done by something like gnome-settings-daemon, as it is now. But it was the quickest solution I could think of.
I can add some refs and a few comments. (In reply to comment #37) > - l10n of events sounds. i.e. english desktops would use an audio file saying > "You've got mail", Couldn't that be in lookup part of theming spec? Just asking. > - a11y for event sounds. People with hearing problems should be notified with a > textual string on the screen. Ack. See http://lists.freedesktop.org/archives/accessibility/2005-May/000055.html > - Themeing. [...] We probably need some kind of > "sound-naming-spec", similar to the already existing "icon-naming-spec" from > fdo. Patryk Zawadzki did it, it has to be reviewed. Last mail sent to him, no reply: http://lists.freedesktop.org/archives/xdg/2006-November/007092.html > - Advanced sound server stuff. For the sake of the API, probably better to avoid talking about server stuff. Let's just say "advanced~fancy stuff". Don't you think? > In summary: I believe that a good event sounds API should provide considerably > more functionality than what has been proposed in the sound effects daemon. Ack > proxying all this stuff over D-Bus doesn't look like a good idea to me. Yep, C API has to be agreed. Proposing a dbus iface at the same time is just a different manner to express it. It is faster to review, at least for me. :)
(In reply to comment #38) > Low-latency is not required. From the accessibility spec: ,,no later than in 10 milliseconds'' as (NICE TO HAVE)
(In reply to comment #39) > I can add some refs and a few comments. > > (In reply to comment #37) > > - l10n of events sounds. i.e. english desktops would use an audio file saying > > "You've got mail", > > Couldn't that be in lookup part of theming spec? Just asking. Yes, having this in the theming/naming spec would make a lot of sense. > > - Themeing. [...] We probably need some kind of > > "sound-naming-spec", similar to the already existing "icon-naming-spec" from > > fdo. > > Patryk Zawadzki did it, it has to be reviewed. Last mail sent to him, no reply: > http://lists.freedesktop.org/archives/xdg/2006-November/007092.html Awesome. Sounds good. We should get this resurrected eventually. > > - Advanced sound server stuff. > > For the sake of the API, probably better to avoid talking about server stuff. > Let's just say "advanced~fancy stuff". Don't you think? Dunno. You cannot do this "advanced stuff" without a sound server, so why not call it by its name?
(In reply to comment #40) > (In reply to comment #38) > > Low-latency is not required. > > From the accessibility spec: > > ,,no later than in 10 milliseconds'' as (NICE TO HAVE) > IMHO D-Bus is fast enough for this temporary replacement.
What about this? Thanks :-)
(In reply to comment #43) > What about this? Seems like the idea got rejected (I can understand that though).
Everything uses GStreamer, but system sounds. Why?
Then, Isn't there any way for getting system sounds without needing esd or pulseaudio?
Maybe GNOME is going to be silent xD.
You can just apply the patches here if you want them...
That's not really a solution. Think about it: this is something many users want, but the current implementation is so limited that it won't to the job for some. Consider you own a sound card that does hardware mixing etc. Still, if you want sound effects, you will be forced to use esd (or pulse, with esd emulation). Same for dmix users. The solution seems to be done for a long time, but it was not completed because people don't want another library. Well... do users care? I doubt. Also, it was mentioned that this functionality should go to GTK+ at some point. Personally I would expect this in GLib... but it's the same thing, basically. See what GIO did: first it was distributed as "another library" and then it was merged to GLib. Why not do it the same here (libgsound)?
libcanberra is in for 2.24, so I'm closing this.