are you willing to help us out somewhat ?
ie, making a list of all formats, if there are libs out there that
support them, and so on

I do not know about any library but it seems that 
MPlayer's

http://www.mplayerhq.hu/cgi-
bin/cvsweb.cgi/main/subreader.c
(may be some other sources around are 
relevant)

does that. From MPlayer's docs: It supports 10 formats 
(with
autodetection):

MicroDVD
SubRip
SubViewer
Sami
VPlayer
RT
SSA
AQTitle
ACOsub
(Mplayer's 
own) MPsub

I do not know if that source is GPL or LGPL (but sure one of these!).. 
LGPL for the legal use in LGPled GStreamer seems to be necessary but I might be 
wrong...

this is being worked on in gst-sandbox/textoverlay currently

Created attachment 35227 [details] [review]
partial patch

This patch implements:
* matroska plain UTF-8 (S_TEXT/UTF8) subtitle reading
* filler data availability in matroskademux
* subtitle awareness in decodebin/playbasebin
* subtitle plugging in playbin
* deadlock prevention in textoverlay (pango-based, not cairo-based)

With this, I can play matroska files containing subtitles and they show in
Totem. Obviously, stream selection (or disabling) needs work (like for audio)
and other formats (.sub, .ogm, subs other than plain UTF-8 in .mkv) need more
effort, too, but this is a start. The most difficult thing is scheduling, and
that's solved here.

Screeny: http://ronald.bitfreak.net/priv/subs2.jpg

Note to self: we currently _default() all events. That's wrong. We should ignore
most events on text_sink, except the interrupt/eos events. For the video_sink,
we should default them. For source events, we should forward them to video_sink,
not text_sink.

Created attachment 35252 [details] [review]
update

Changes:
* event handling is correct
* .ogm supports subs too

Todo:
* .sub
* loading external subtitle files in totem
* stream selection (and disabling)

Created attachment 35274 [details] [review]
update

This patch adds:
* loading external subs in playbin (file[#sub1[,sub2,...]])
* .sub file support through subparse (from gst-sandbox), updated to use
DURATION and so on

Todo:
* stream selection (playbin)
* seeking (for .sub files; textoverlay/subparse)
* when the text stream is longer than the video stream and video signals eos,
textoverlay will eos without killing the text stream (textoverlay).

Long-term:
* support more subtitle formats in .ogm, .mkv and .sub

Created attachment 35280 [details] [review]
update

Changes:
* seeking with .sub works now
* EOS handling for text/video streams of different length fixed

Todo:
* stream selection (playbin)

I've noticed that seeking in matroska causes subs to disappear. Need to look at
that. In its current state, it's applyable and doesn't seem to cause any big
issues. The lack of stream selection makes it kinda annoying, though,
especially if you don't actually want subtitles.

Longterm:
* support more subtitle formats
* implement font selection in Totem (needs new property in playbin?)

Created attachment 35293 [details] [review]
update

Changes:
* mpsub support added to subparse, and cleaned up subparse a bit.

Created attachment 35610 [details] [review]
additional core patch

Attached patch adds fillers to core. Using events rather than buffers makes the
fillers no part of the actual datastream. This fixes a lot of problems with
elements such as vorbisdec or ogmtextparse, who expect fixed packet order.
Also, most elements will simply forward them, which is what we want. That makes
that we can also use those events for long init_delays, which happens in many
streams and causes hangs in playbin right now (but then sinks/muxers need to be
aware of this event). Putting here for review.

Created attachment 35611 [details] [review]
plugins update

Use filler events.

Created attachment 35618 [details] [review]
pango fix

* lame fix for pango UTF-8 strictness update

Created attachment 35630 [details] [review]
stream selection (gst-plugins)

This additional patch needs to be applied on top of the subtitles patch. It
adds stream selection support. I have a patch for Totem, you can also test it
on the commandline using the current-{audio,video,text} properties (-1=disable,
>=0=stream index) on playbin. Note that this patch is rather invasive and may
thus break stuff. Please test.

The above patch in comment #13 depends on the queue renegotiation patch from
Martin Soto that was reverted. I'll try to resurrect it. It's available in the
CVS logs (e.g.
http://cvs.freedesktop.org/gstreamer/gstreamer/gst/gstqueue.c?r1=1.110&r2=1.112&makepatch=1&diff_format=u).

Created attachment 35632 [details] [review]
totem patch

This patch adds stream selection to the Totem UI.

Created attachment 35660 [details] [review]
additional core fix

this patch needs to be applied to core, on top of 35610. It replaces Martin's
queue patch. It's pretty much hangsafe.

Created attachment 35661 [details] [review]
stream selection update

This patch fixes some oddities and cornercases. Even multi.ogg plays perfectly
fine with this patch applied, although it goes to 100% CPU on EOS and playbin
stops emitting EOS for multi.ogg. I need to fix that.

I think I'm getting close to workingness now.

As for the problem on EOS, it seems to be related to yet another race in core
w.r.t. probes. Here's some logs from valgrind and gdb (you can get it to crash
if you try hard enough):

Program received signal SIGSEGV, Segmentation fault.

+ Trace 54264

$3 = (GstProbeDispatcher *) 0x9184934
(gdb) print *disp
$4 = {active = 65, probes = 0x91bdfa8}
(gdb) print *probe->probes
There is no member named probes.
(gdb) print *disp->probes
$5 = {data = 0x65732064, next = 0x7463656c}
(gdb) print *disp->probes->next
Cannot access memory at address 0x7463656c

==26745== Thread 3:
==26745== Invalid read of size 4
==26745==    at 0x1B9418FE: gst_probe_dispatcher_dispatch (gstprobe.c:222)
==26745==    by 0x1B93C95F: gst_pad_push (gstpad.c:3202)
==26745==    by 0x1C27A581: get_group_schedule_function (gstoptimalscheduler.c:1305)
==26745==    by 0x1C27A037: schedule_group (gstoptimalscheduler.c:1154)
==26745==  Address 0x1C0DD498 is 168 bytes inside a block of size 196 free'd
==26745==    at 0x1B904FA1: free (vg_replace_malloc.c:153)
==26745==    by 0x7BFB023: g_free (in /usr/lib/libglib-2.0.so.0.600.0)
==26745==    by 0x69125A: g_type_free_instance (in
/usr/lib/libgobject-2.0.so.0.600.0)
==26745==    by 0x67CC80: (within /usr/lib/libgobject-2.0.so.0.600.0)

It seems like the realpad itself is dereferenced (and free'ed) in the probe
callback (perhaps the part where we destroy the GstPlayBaseGroup and unref the
GstStreamInfo list?). That makes this very hard to get right...

Found it, the source pad of each preroll element is disposed as the element is
destroyed in group_destroy() from the probe dispatcher callback. From there on,
control is returned to the dispatch function, but it's fully freed already.
That'll take a while to get correct, we could use g_idle_add hacks... I don't
like that, but hey... We're basically destroying an element in its own
context... Bah.

Created attachment 35672 [details] [review]
another selection update

Fix 100% CPU use on multi.ogg EOS. Speed also updated. This has no regressions
as far as I know.

All above checked in. Will require HEAD of everything (gstreamer >= 0.8.8.1,
plugins >= 0.8.7.1, totem > 0.100.0).