In various streams coming from DVDs, I've seen subtitles stored as they are in the DVD itself, which includes absolute positioning/sizing. However, the original size of the video this was authored for is not stored anywhere (that I am aware of). Therefore, when playing those subs on a larger video, subtitles are placed where they are supposed to, instead of being scalred in the same way as the video was.

I'm not sure what's the best fix for this. It's possible to look at a selection of subtitles, assume they're horizontally centered, and placed at the bottom of the screen, then deduce the original video size, then scale/move subs accordingly. However, that's not foolproof, and requires preloading of enough subs to eliminate any oddly placed ones before averaging. I did some code for that for another project, and, while it worked well enough, there was some issue I can't recall at the moment that made this not such a great solution.

The other solution is to just ignore positioning, and always center all subtitles.
This loses the ability to move subs out of potentially useful/interesting part of the video.

Is there no way to figure out the size of the subtitle bitmaps?

The original image dimensions are available in the .idx file (with a corresponding .sub file), but I suppose this info is lost when embedded in an MKV or MP4, right? But don't the subtitle bitmaps themselves have well-defined dimensions?

They do have well defined dimensions.
The issue is that since you do not know the original size of the video (*), you can't scale size/position to make the subs appear as they were on the DVD.
Hence the fallback solution of always centering, which is suboptimal when the DVD was encoded at original resolution (because in that case, the subs position/size are already correct, but there's no foolproof way to tell).
That said, the vast majority of subs seem to be centered anyway, so in most cases it would be acceptable to always center...

(*) maybe in some cases it's possible to recover it, if it's been saved as metadata in the Matroska stream.

Might be fixed by Bug #685282

*** Bug 715136 has been marked as a duplicate of this bug. ***

According to bug #715136 VLC is able to render the subpictures properly, so maybe we can take some ideas from there.

The patch on Bug #685282 handles it properly. There's a case where the subtitles will be cropped out, when they were displayed inside the black borders area on the original medium, and the black borders were cropped when reencoding the video. This case only works when the video sink renders the video to the original size with black borders and handles the GstVideoOverlay rendering.

I'm not sure what VLC does in this case. I think I have an MKV file to test this case, I'll upload it somewhere.

here is sample video:
https://www.dropbox.com/s/0lrcmxj1g8wwhpt/out.mkv

It seems there is another issue besides clipping on this video, the color is 0 when blitting the RLE data so nothing can be seen on screen.

hmm... i didn't understood: "color is 0 when blitting the RLE data so nothing can be seen on screen."

Still an issue in master (1.5.90).

MKV with PGS subs (probably from Blu-Ray): Video is 720p, pgs subs are 1920x1080 video. Almost all of the subs are invisible because they fall outside of the video window, example:

  Begin PRESENTATION_SEGMENT (0x16) packet len 19
  Video width 1920 height 1080 fps code 32  
  Win ID 0 x 686 y 888 w 542 h 117

Implementing GstVideoOverlayComposition support as in bug #685282 is probably going to help with this, we can then use it to resize the video and use the video overlay composition blending path (if we blit and not downstream).

The patches in Bug #685282 should help with that. The code makes all the spu data fit in the video frame: see https://github.com/rawoul/gst-plugins-bad/blob/dvdspu/gst/dvdspu/gstdvdspu.c#L869

As I commented on Bug #685282, I think to render these correctly we need a way to resize the video in dvdspu to add black bars when composing locally in software, or if we're passing the overlay downstream for composition to ensure that it's composed after final resize /  positioning - so that  the subs appear in the black bars where they're supposed to.

Bug #685282 does help though - at least the subs are visible now, instead of clipped off-screen

From bug #685282 by Jan:
> On a real disc, the video would actually be encoded at 1920x1080, so the 
> placement would be a non-issue.

I do remember a random question on IRC and me replying strictly that overlay composition must be inside the video. Sorry, if I didn't get the point here. So from what I now understand, while on DVD there is black bars (so no issues), those are stripped off when put on file. Though the overlay remain placed in what use to be a black bar so they don't hide the content.

I think we will all agree that "adding" black bars by copying the stream is not exactly where we want to go. I have few ideas, they might not all make sense (or be doable in sinks, we need to study that).

1) Do nothing here, assume that sinks will figure-out by calculating the bounding box of the video and the overlay and make sure it's all right by adding black bars (or moving the overlay). Not sure this one is realist, could mean that bars will change in size every time a new text is sent. That means sink need to stop clipping.

2) We are aware of the display size. We can add the display aspect ratio, and with that we'd have complete information to figure-out best adjustement placement for the text. It does not matter if sink scales or not, the goal would be to make sure the text falls inside the display. That means, if the sink keep the aspect ratio, and it's using one of TV standard resolution, the text will endup where we expect it to be (though will vary slightly between screen aspect ratio). That requires sink to clip on display size instead of video size.

3) Add and negotiate a new meta, so signal the sink that black borders are required. Hopefully most sink can do something cheaper, but have a fallback where we copy the frame into a larger frame. This one will give the same placement regardless of the display size. It is I think the most correct, as you get that same placement regardless of your setup. Though is really expensive if you have a sink that does not support overlay composition and also that feature.

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/49.