GNOME Bugzilla – Bug 481075
Full support for ASS/SSA subtitles
Last modified: 2009-03-03 01:26:20 UTC
Gstreamer needs full support for ASS/SSA subtitles. That means: * correct parsing of ASS/SSA styles and events * correct rendering of ASS/SSA subtitles with all effects/transitions etc. Currently only mplayer has an (almost) complete ASS renderer on Linux. Supporting ASS/SSA subtitles in Gstreamer would be a big step towards bringing Linux on the same level with other operating system's multimedia-support.
*** Bug 493029 has been marked as a duplicate of this bug. ***
Support of subtitle of xine: #define FORMAT_MICRODVD 0 #define FORMAT_SUBRIP 1 #define FORMAT_SUBVIEWER 2 #define FORMAT_SAMI 3 #define FORMAT_VPLAYER 4 #define FORMAT_RT 5 #define FORMAT_SSA 6 /* Sub Station Alpha */ #define FORMAT_PJS 7 #define FORMAT_MPSUB 8 #define FORMAT_AQTITLE 9 #define FORMAT_JACOBSUB 10 #define FORMAT_SUBVIEWER2 11 #define FORMAT_SUBRIP09 12 #define FORMAT_MPL2 13 /*Mplayer sub 2 ?*/ Support of format of totem: "srt", "sub", "mpsub", "mdvd", "smi", "txt" References http://www.google.com/codesearch?hl=es&q=+sourceforge.net+%23define+FORMAT_PJS+7+%23define+FORMAT_MPSUB+8+%23define+show:e7tMoeJsPFY:AgQYL-GmyWw:4pAPj0dFQ2M&sa=N&cd=2&ct=rc&cs_p=http://downloads.sourceforge.net/xine/xine-lib-1.1.5.tar.bz2&cs_f=xine-lib-1.1.5/src/libsputext/demux_sputext.c#first http://webcvs.freedesktop.org/gstreamer/gst-plugins-base/gst/subparse/gstsubparse.c?hideattic=0&sortby=rev&view=log
Gstreamer also ignore Vobsub (the sub from orgvobis) i don't now if xine can do this.
gnome-subtitle use "SubLib"(GPL), SubLib support many formats: http://sublib.sourceforge.net/about Subtitle formats: * Adobe Encore DVD * Advanced Sub Station Alpha * AQ Title * DKS Subtitle Format * Karaoke Lyrics LRC * Karaoke Lyrics VKT * MacSUB * MicroDVD * MPlayer * MPlayer 2 * MPSub * Panimator * Phoenix Japanimation Society * Power DivX * Sofni * SubCreator 1.x * SubRip * Sub Station Alpha * SubViewer 1.0 * SubViewer 2.0 * ViPlay Subtitle File Many aplications is necesary subtitle see: http://bugzilla.gnome.org/show_bug.cgi?id=534581 In totem not work many subtitles.
Other subtitle necesary: http://bugzilla.gnome.org/show_bug.cgi?id=343281
As sublib is written in C# it can't be used for GStreamer (in an easy and clean way).
Sebastian, let me pose some questions. Besides calling managed code from unmanaged code, would running an application which pipped the results to gstreamer be unacceptable? Also, is there a way to have this feature as an "alternative" plugin? I think SubLib's features would enrich gstreamer's subtitle handling, as it supports multiple subtitle formats among other features. If not possible in the base platform, I think it would be welcome as an option.
(In reply to comment #7) [...] > I think SubLib's features would enrich gstreamer's subtitle handling, as it > supports multiple subtitle formats among other features. Yes. =), Is many work rewrite sublib (write in C#) to C?
Pedro: It's mostly important that the method of calling C# in a gstreamer element to be sane and maintainable, and bonus points for being reusable. It's unlikely that sublib will be the only C# library to be wrapped in a gstreamer plugin. I assume that Sebastian was skipping to the conclusion that spending the time working on a C# plugin would be more than it is worth.
(In reply to comment #7) > Sebastian, let me pose some questions. > > Besides calling managed code from unmanaged code, would running an application > which pipped the results to gstreamer be unacceptable? Also, is there a way to > have this feature as an "alternative" plugin? From the application side there are many possible implementations for this, two that I'm thinking of right now would be: - The application ships a private element with a very high rank that can handle all kinds of subtitle formats. This would then be autoplugged and on instanciation the application would install some managed callbacks that will use sublib for decoding - The application uses sublib for parsing the subtitles and put them into the pipeline via fdsrc for example The first solution would be much better though ;) > I think SubLib's features would enrich gstreamer's subtitle handling, as it > supports multiple subtitle formats among other features. If not possible in the > base platform, I think it would be welcome as an option. I don't think it will be very hard to add support for all those formats in GStreamer without using sublib. Parsing subtitles is not exactly rocket science, it just needs someone to do it. (In reply to comment #9) > It's mostly important that the method of calling C# in a gstreamer element to > be sane and maintainable, and bonus points for being reusable. It's unlikely > that sublib will be the only C# library to be wrapped in a gstreamer plugin. I > assume that Sebastian was skipping to the conclusion that spending the time > working on a C# plugin would be more than it is worth. Well, a generic gstreamer plugin using sublib could probably be done by using libmono but IMHO it's not worth the effort and the overhead of running a JIT just for parsing subtitles. Enhancing our current subtitle parsers and writing new ones for unsupported formats shouldn't be too hard.
Also, on a side note: SubLib is GPL, which means a plugin depending on it is unlikely to ever make it into gst-plugins-base or gst-plugnis-good. I'd also prefer C implementations in GStreamer. The way the subparse element works is a bit awkward, but nothing forces you to use it if you find it too cumbersome.
(In reply to comment #6) > As sublib is written in C# it can't be used for GStreamer (in an easy and clean > way). And Xine is write in C. See source: http://sourceforge.krugle.com/kse/files/cvs/cvs.sourceforge.net/xine/xine-lib/src/libsputext/demux_sputext.c#3 " [...] 113 #define FORMAT_UNKNOWN -1 114 #define FORMAT_MICRODVD 0 115 #define FORMAT_SUBRIP 1 116 #define FORMAT_SUBVIEWER 2 117 #define FORMAT_SAMI 3 118 #define FORMAT_VPLAYER 4 119 #define FORMAT_RT 5 120 #define FORMAT_SSA 6 /* Sub Station Alpha */ 121 #define FORMAT_PJS 7 122 #define FORMAT_MPSUB 8 123 #define FORMAT_AQTITLE 9 124 #define FORMAT_JACOBSUB 10 125 #define FORMAT_SUBVIEWER2 11 126 #define FORMAT_SUBRIP09 12 127 #define FORMAT_MPL2 13 /*Mplayer sub 2 ?*/ " Sorry, we need better support for people who do not speak English =(
(In reply to comment #12) > Sorry, we need better support for people who do not speak English =( Could you explain what exactly is missing for you? This bug here is about suboptimal ASS/SSA subtitle support, i.e. it works good but the effects and text styles are not used. Is there a subtitle format which is completely unusable for you or not supported at all, but you need it? From the ones that Xine support GStreamer already handles MICRODVD, SUBRIP, SUBVIEWER, SAMI, SSA, MPSUB, SUBVIEWER2, SUBRIP09, MPL2. If one of those doesn't work as expected for you please file a bug. It does not support (yet) VPLAYER, RT, PJS, AQTITLE, JACOBSUB. If you need one of those please file a bug so we can get support added. TBH I never saw a subtitle in those formats though :)
This bug is about adding a complete (or at least more complete) ASS/SSA renderer to gstreamer, like mplayer has one. ASS is very complex subtitle format with support for many formatting options, styles and effects. Other text subtitles don't work "fine" either because of some shortcomings of the textoverlay plugin: 1) textoverlay can only display one line of text at once. Some subtitle formats (including ASS) can display more than one line at a time. In that case the textoverlay plugin would have to handle collision detection, so that the lines won't overlap. 2) It does not offer nice formatting and styling options. At least being able to set text color and outline would be nice. However, for these, a SEPARATE BUG should be opened. This bug here is solely about ASS/SSA rendering. For a start, mplayer has the most complete ASS rendering via libass. It should be possible to use this in gstreamer somehow.
(In reply to comment #13) > (In reply to comment #12) > > Sorry, we need better support for people who do not speak English =( > > Could you explain what exactly is missing for you? This bug here is about > suboptimal ASS/SSA subtitle support, i.e. it works good but the effects and > text styles are not used. Is there a subtitle format which is completely > unusable for you or not supported at all, but you need it? Hello Sebastian In bug 493029 send to LP_bug 159718. In LP_bug 159718 my subtitle: https://bugs.edge.launchpad.net/ubuntu/+source/totem/+bug/159718/comments/2 I download film (Ghost in of Shell) and search subtitle in spanish. Not other subtitle for film (correct sync)
ASA (http://asa.diac24.net/Asa) seems to work with Gstreamer and supports a large part of the ASS/SSA specification (http://asa.diac24.net/Compatibility).
There's also gstlibass (http://sourceforge.net/projects/gstlibass/). Seems to work pretty well: gst-launch videotestsrc ! video/x-raw-rgb,width=640,height=480 ! libass location=test.ass ! ffmpegcolorspace ! xvimagesink
Created attachment 114794 [details] gstlibass fade test
Created attachment 121801 [details] assrender plugin (ASS/SSA renderer via libass) assrender plugin that works much like textoverlay. It has two sink pads: "video_sink" and "text_sink". Receives RGB24 video frames on video_sink and raw (i.e. unparsed) ass events on text_sink. Overlays subtitles directly onto the video. Features: * ASS/SSA parsing and rendering via libass (http://sourceforge.net/projects/libass/) * on-the-fly registering of fonts that are attached to the stream. This currently only works with fonts attached to matroska containers using the latest CVS version of the matroskademux plugin. Limitations: * only RGB colorspace is supported. Needs colorspace conversion for YUV video frames.
I've tried both the libass ad assrender plugins. Seems like assrender is more cpu-intensive than libass - e.g. I am unable to play HD videos with assrender, while they play fine with either mplayer or libass. The pipeline I use for assrender is filesrc location=<filename> ! matroskademux name=v v.subtitle_00 ! queue ! sub. v.video_00 ! queue ! decodebin ! ffmpegcolorspace ! assrender name=sub ! ffmpegcolorspace ! videoscale ! xvimagesink v.audio_00 ! queue ! decodebin ! pulsesink On the other hand assrender is able to use fonts attached to the stream which is a big advantage for me. Is there a way to persuade e.g. totem to use the assrender plugin? It's suboptimal to have to use gst-launcher...
Hi. The additional CPU load when using assrender is because of the necessary two additional colorspace conversions (YUV to RGB, then RGB back to YUV). This is rather suboptimal, so I'm thinking about rewriting assrender so that it generates ARGB frames instead. These could then be converted to AYUV and put on top of the video with videomixer. That would reduce the amount of necessary colorspace conversions to 1 and additionally lower the amount of data that has to be processed. However, assrender somehow needs to know the geometry and FPS of the video stream - otherwise it cannot render anything.
I would like to test assrender. How would I go about compiling it in with gstreamer? I ask this because I have a deep interest in seeing gstreamer being able to handle complex subtitle effects and karaoke.
I've put together a simple build system for it (mostly by copying over from the other libass plugin) and built a rpm for Fedora. You should be able to figure the steps out from source rpm (don't look at the formalities though, I've only quickly edited a rpm for gstlibass...): http://mso.fedorapeople.org/packages/SRPMS/gstassrender-0.1.0-1.fc10.src.rpm If you still have issues, feel free to send me an e-mail, I'll write you more detailed guide ;-)
*** Bug 569096 has been marked as a duplicate of this bug. ***
*** Bug 569095 has been marked as a duplicate of this bug. ***
(In reply to comment #23) > I've put together a simple build system for it (mostly by copying over from the > other libass plugin) and built a rpm for Fedora. You should be able to figure > the steps out from source rpm (don't look at the formalities though, I've only > quickly edited a rpm for gstlibass...): > http://mso.fedorapeople.org/packages/SRPMS/gstassrender-0.1.0-1.fc10.src.rpm > > If you still have issues, feel free to send me an e-mail, I'll write you more > detailed guide ;-) > Nice. You should submit this as a patch to mailing list.
btw, i noticed that gst-launch requires libavcodec52, not libavcodec52-unstripped in order to work. gstreamer kept crashing until i did this.
I've committed the assrender plugin to GIT now, it will be in 0.10.11 of gst-plugins-bad. This won't make it be used automatically by playbin though as playbin will only use subparse for subtitles unfortunately. commit 4a23c7e912a130e89614fd08129873cf690623f7 Author: Sebastian Dröge <sebastian.droege@collabora.co.uk> Date: Thu Feb 5 17:42:37 2009 +0100 Random cleanup and smaller bugfixes Use gst_element_klass_set_details_simple(), install properties with static strings, create pads directly from the static pad templates, directly put the segment into the instance struct, use GST_MSECOND instead of numbers. The PAR is a GstFraction and not a double, use gst_structure_get_fraction(). Simplify setcaps() functions. commit aba72fcce963656c24fb06e3c017b5b911395a38 Author: Sebastian Dröge <sebastian.droege@collabora.co.uk> Date: Thu Feb 5 17:00:30 2009 +0100 Use GST_VIDEO_CAPS_RGB for the template caps commit a0e06b965a3ef372652ae8c0e99babce6b83ff08 Author: Sebastian Dröge <sebastian.droege@collabora.co.uk> Date: Thu Feb 5 16:57:55 2009 +0100 Fix compiler warnings commit 0bc20cb186574ddda39a80acef491e9af7045cfc Author: Benjamin Schmitz <vortex@wolpzone.de> Date: Thu Feb 5 16:54:01 2009 +0100 Add assrender plugin for rendering ASS/SSA subtitles This element has the advantage over subparse's ASS/SSA support that more features are supported, like fading, by using libass. Fixes bug #481075.
Is there a separate bug opened about making playbin(2) use this renderer?
No... I'm currently looking at what is needed and will probably fix it later if it's easy ;)
Would it be possible to extend libass and incorporate new ASS tags from vsfilter 2.39c? The vsfilter source can be grabbed here: http://sourceforge.net/svn/?group_id=205650
Ok, let's close this one now... bug #570753 is for adding support for assrender (and similar subtitle renderers) to playbin(2).
There are some brand new tags for libass that were patched in recently. http://greg.geekmind.org/mplayer/ Courtesy of the fansub community and aegisub developers.
Just a heads up, the coder says that there will be more fixes in the near future.