GNOME Bugzilla – Bug 648010
Better handling of blacklisted plugins: re-scan occasionally, store reason for blacklisting
Last modified: 2018-11-03 12:15:08 UTC
+++ This bug was initially created as a clone of Bug #503675 +++ <__tim> nekohayo, your registry file is full of BLACKLISTED. The problem isn't registry file writing or parsing, but that plugins get blacklisted for some reason <nekohayo> When does blacklisting occur? <__tim> nekohayo, when an error occurs when loading the plugin, usually a missing symbol or library or somesuch <nekohayo> videomixer was indeed on the gst-inspect blacklist <__tim> are you sometimes downgrading versions of core or gst-plugins-base? <nekohayo> no, not that I know of. Downgrading was downright impossible in ubuntu, and I haven't used downgrade in fedora yet <__tim> GStreamer installed in different prefixes? <nekohayo> currently this machine is using the official gstreamer packages from fedora + rpmfusion <nekohayo> I'm wondering why those blacklisted plugins are never "re-scanned" by gstreamer <__tim> that's a fair question I guess. But when would one rescan them? You don't want to do it every time. So once an hour? once a day? once a week? <nekohayo> what if they were rescanned when apps try to access them? like when pitivi tries to use videomixer (on top of your approach of rescanning periodically maybe) <__tim> interesting, maybe one could do something clever there <__tim> ideally we'd store the reason for the blacklisting as well <ocrete> arent they rechecked if the file changes already ? <thaytan> the plugins are rescanned if they change <__tim> but if they don't change ... <thaytan> but if they failed because some underlying lib was broken and then fixed, that's not detected <__tim> right <ocrete> hmm interesting point <thaytan> storing the reason for the blacklisting is probably hard <ocrete> also check if ld.so.cache has changed ? <thaytan> since it's usually just 'failed to g_module_open() because of an unresolved symbol' <ocrete> and ignore blacklists if ld.so.cache has changed (and rescan) <__tim> thaytan, it doesn't mention the symbol? I think it does.. <ds> if the so fails to load, there isn't much overhead trying to load every time <ocrete> that said, doesnt help if one use using LD_LIBRARY_PATH <thaytan> ds: except for forking the scanner process to re-check an otherwise clean registry <ds> oooh, maybe recheck blacklisted plugins whenever the scanner is run, but not run the scanner just for blacklisted plugins. Although, also mark potential crasher blacklist plugins
<__tim> that's a fair question I guess. But when would one rescan them? You don't want to do it every time. So once an hour? once a day? once a week? Store a timestamp or checksum with the blacklist entry and discard the blacklist entry when the file changes.
We already do that, but frequently the reason for blacklisting is underlying: a crash in a library that the plugin uses, or some system error that goes away when the underlying library is upgraded (which doesn't change the timestamp on the GStreamer plugin, of course).
It is a bit arbitrary, but maybe do a re-scan once a day? Basically the first time GStreamer is initiated that day?
If other people think it is a good idea, I could add a --recheck argument to gst-inspect-1.0. Then users could request a recheck of the blacklisted elements when they have updated/fixed libraries.
I recently had a similar issue and the only way I could get GStreamer regenerating the registry was by removing the cached file from disk, even though now I know that simply "touching" the plugin's file after resolving the underlying problem would do the job too. So, I personally think that having a --recheck option in gst-inspect-1.0 could be helpful. The "user" would still need to go into the command line, but a parameter is something that can be found easily from the man page (definitely easier than learning that either you need to remove the registry or touch the plugin)
I think it's a good idea to have such option and it's more convenient than removing the cache file.
I guess this bit me hard over the weekend; is the cache designed to reduce startup statting of the huge number of shared libraries that are installed ? ultimately - it is my expectation that upgrading and installing new packages left & right should refresh the cache; is that so ?
Michael: yes, the purpose of the registry cache is to avoid loading hundreds of plugins on every gst_init(), or when we need to find all elements that can do X. The cache should be refreshed when GStreamer plugins get upgraded, removed or added. The cache won't get refreshed when just the underlying library that a GStreamer plugin depends on gets updated, but the GStreamer plugin doesn't get updated (which shouldn't be a problem, unless ABI changes, in which case the GStreamer plugin needs to be updated as well of course).
It seems those error are mostly from link issue with dependency library. I believe we could add more files to the scanner in order to reload if a dependent library was updated ?
Not sure it's that easy. If we knew which dependencies to watch, adding them to the list is easy enough, but figuring out which libraries caused trouble or should be watched is tricky. We don't want to re-scan every plugin if libc or libgstreamer are updated, for example, but beyond that I don't know how we even get the list of candidate libraries - let alone handle situations like the LD_LIBRARY_PATH being changed and bringing a different library version into play.
Re-scanning everything is not that slow. I wouldn't be opposed to re-scanning the blacklisted entries if LD_LIBRARY_PATH changed or if the ld.so cache changes.
> Re-scanning everything is not that slow. I wouldn't be opposed to > re-scanning the blacklisted entries if LD_LIBRARY_PATH changed or if the > ld.so cache changes. I wouldn't be opposed to either of these, I just don't think it would really help with anything, since when plugins get blacklisted that's usually a packaging/depdendency issue or setup/source-install problem (like an old lib version in /usr/local from ages ago).
In Debian we've recently seen several reports of programs complaining of missing plugins after an upgrade. Removing the ~/.cache/gstreamer-1.0 folder has fixed the problem in these cases. In the most recent report, I asked the submitter to run "gst-inspect-1.0 -b" and it turns out that the libgstcoreelements.so plugin was blacklisted. I assume that something about the upgrade process is causing the plugins to be blacklisted. Reports: https://bugs.debian.org/867574 https://bugs.debian.org/866571 another on #debian-gnome IRC
This should be fixed in Debian now the next time the users update: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=866339#25 That's of course only a workaround at this point (for a packaging bug), and it would be better to solve this specific bug here.
We could check for the direct dependencies *at least*, i.e. those linked with at build time, so we know their name already. For xvimagesink: libgstxvimagesink_la_LIBADD = \ $(top_builddir)/gst-libs/gst/video/libgstvideo-$(GST_API_VERSION).la \ $(GST_BASE_LIBS) \ $(GST_LIBS) \ $(X_LIBS) $(XVIDEO_LIBS) $(XSHM_LIBS) $(LIBM) libgstxvimagesink_la_LDFLAGS = $(GST_PLUGIN_LDFLAGS) -lgobject-2.0 -lglib-2.0 -lX11 -lXv -lXext -lm
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gstreamer/issues/22.