After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 749790 - Make Metadata harvester a process
Make Metadata harvester a process
Status: RESOLVED FIXED
Product: rygel
Classification: Applications
Component: MediaExport plugin
git master
Other Linux
: Normal enhancement
: ---
Assigned To: Jens Georg
rygel-maint
Depends on:
Blocks:
 
 
Reported: 2015-05-24 08:18 UTC by Jens Georg
Modified: 2015-07-04 11:01 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
media-export: Extract meta-data in external process (56.86 KB, patch)
2015-06-22 21:16 UTC, Jens Georg
committed Details | Review
media-export: Re-add album art extraction (5.99 KB, patch)
2015-07-04 11:01 UTC, Jens Georg
none Details | Review
media-export: Listen to meta-data config again (6.48 KB, patch)
2015-07-04 11:01 UTC, Jens Georg
none Details | Review

Description Jens Georg 2015-05-24 08:18:18 UTC
That way, if a broken file crashes GStreamer or libmediaart, we can
 - gracefully revover
 - blacklist such a file
 - give precise information about a problematic file instead of poking in the dark with debug info as we do currently

Issues:
 - That's duplicating Tracker/LMS architecture, but I'm sick of the bugs
 - How to get meta-data to calling process?
   - gst_discoverer_info_to_variant - quite recent, not sure what is serialized
   - Custom text-based format with the necessary meta-data
     - process would do the DLNA guessing and media-art extraction as well
   - direct write into SQLite db
     - potentially problematic, as we might have many writers in upload use-case
Comment 1 Jens Georg 2015-06-22 21:16:31 UTC
Created attachment 305873 [details] [review]
media-export: Extract meta-data in external process

Move GStreamer calls to extract meta-data into own process.
This enables us to recover from crashes due to bugs with broken media files,
knowing exactly which URI caused the crash and (later) blacklist the broken
files.
Comment 2 Jens Georg 2015-06-22 21:21:25 UTC
Comment on attachment 305873 [details] [review]
media-export: Extract meta-data in external process

First step. Has some regressions such as
 - MetaData extraction cannot be disabled
 - Blacklisting of failing uris is missing
 - AlbumArt extraction is missing
 - There is one process per harvesting task.

These will be addresses in the next release.

Features are:
 - Process is restarted if it is killed
 - URI that made the extraction process fail is known and shown to the user
 - Extraction process is run at nice level 19 which should lower system load
   during initial crawl.

Attachment 305873 [details] pushed as 1e1dc09 - media-export: Extract meta-data in external process
Comment 3 Jens Georg 2015-07-04 11:01:15 UTC
Created attachment 306797 [details] [review]
media-export: Re-add album art extraction

Signed-off-by: Jens Georg <mail@jensge.org>
Comment 4 Jens Georg 2015-07-04 11:01:19 UTC
Created attachment 306798 [details] [review]
media-export: Listen to meta-data config again

Signed-off-by: Jens Georg <mail@jensge.org>
Comment 5 Jens Georg 2015-07-04 11:01:44 UTC
More missing features