GNOME Bugzilla – Bug 629165
interlace: Add telecine mode support
Last modified: 2013-08-25 16:48:51 UTC
Created attachment 169853 [details] [review] Add telecine support I have implemented telecine support for the new interlace element. The timestamping is a bit hackish right now as it takes the timestamp of the first buffer and then just increments by a hard-coded duration depending on the pattern chosen. Comments very welcome.
Why not put a videorate in front of it, as I've been doing?
That's a possibility but I still need ballpark timestamps as I believe videorate select the buffer closest to the timestamp desired to push as the buffer at that timestamp. I should reset the base timestamp when a seek is received too.
Do you mean that you want to create a stream with RFF flags set? That would mean making interlace more like videorate, taking whatever timestamps on input, and creating perfect timestamps on output. This makes sense as a direction to go.
It's not for RFF as the output of interlace in telecine mode will be buffers that have all the fields that they should. I would like interlace in telecine mode to output the correct timestamps on buffers, though I'm not sure that this approach I'm using is best. Telecine mode will output more buffers than it takes in. When one uses one input buffer to create more than one output buffer, one can't assign the same timestamp to the buffer and just use videorate later because how will it know which buffer to choose? As such one needs to adjust the timestamp to something sane and I thought why not fix the timestamps to what they probably should be. When I later work on the deinterlace element as a result of field analysis to set buffer flags to indicate to deinterlace and other elements what the buffer state and stream state is, I would like deinterlace to output correct timestamps too. This isn't easy and for telecine requires a pattern lock which requires some context to get right. The comment about seeking was just for if someone wanted to be able to seek in a stream when using this element, also not about RFF. Maybe this isn't so useful but I think it's doable.
I'm going to go a little meta for a second and talk somewhat ramblingly. There are two main concepts: the telecine/interlacing properties of the content, and the telecine/interlacing properties of the encoding. I use the term "encoding" in the loose sense that includes raw video -- film content is "encoded" to raw NTSC video using a telecine process. So, given a base format like NTSC that describes size/framerate, etc., the content can be described as: - progressive, different rate (i.e., 24 fps film content) - progressive, same rate (30 fps progressive video content) - interlaced (60 field per second video content) - content changing over time The encoding for NTSC is fundamentally 60 fields per second, however, in GstBuffers, it has the following possibilities: - consistent 1 field per buffer (ONEFIELD set) - consistent 2 fields per buffer (RFF unset, TFF set or unset consistently) - consistent 2/3 fields per buffer, mainly for 24 fps film (RFF, TFF set appropriately) - varying number of fields per buffer (i.e., random crap) I'm going to try to start using standard nomenclature to refer to these: 24p, 30p, 60i, varying, for the former. 1fpb, 2fpb, 23fpb, vfpb, for the latter. Stuff on a DVD, including titles, would be "varying/60i/vfpb". Sections of film content on a DVD would be "24p/60i/23fpb". HDV in 24p mode can be either "24p/60i/23fpb" or "24p/60i/2fpb" depending on the manufacturer of the camera. Progressive content and encoding (e.g, internet video) would be "30p/30p" or similar. The original goal of invtelecine was to detect the content and convert varying/60i/vfpb to cleaner streams: 24p/24p, 30p/30p, or 60i/60i/2fpb. This is different from deinterlace and interlace, which converts *the content* to a different content format: deinterlace converts 60i/60i/2fpb to 60p/60p, and interlace does the reverse. deinterlace will also convert 24p/60i/2fpb to 24p/60p, but that's not ideal. Another thing we need is rebuffering: 24p/60i/2fpb to and from 24p/60i/23fpb. And then also converting 24p/24p into 24p/60i/(2fpb|23fpb). Also, we need pattern generation for all these modes, and then fix all the plugins to handle 1fpb/2fpb/23fpb/varying gracefully, which I doubt they currently do.
Being able to telecine content in the same element as one can interlace content doesn't inhibit any of what you're discussing. Code can be shared between the interlacing and telecine processes. I understand the difference you're detailing between the non-destructive (inverse) telecine process and (de)interlacing, but I don't think it's harmful to have them all under one roof. I haven't considered all avenues for the * -> plain progressive conversion by any means. For field analysis, considering whether two opposing parity fields are progressive or not is something common to both interlace and telecine detection. As content/encoding can vary within one stream, if we separate the analyses, we may either need to perform the same analyses twice or have less information available. Perhaps some of the information is irrelevant. Contextual information however is probably essential for managing transitions between content/encoding types in an input stream. Unless you're considering that for a generic handle-it-all-and-output-progressive pipeline one must ivtc first and then adaptively deinterlace, making both ivtc and deinterlace pass-through progressive frames, ivtc reconstructs progressive content from telecined encodings and deinterlace just deinterlaces interlaced frames that were passed through ivtc. I'm not really sure what I think is best so I'm going to reread this on Monday. :)
Most reasonable pipelines would use invtelecine ! deinterlace, as I've described them. Some transcoding pipelines would use just invtelecine. In any case, that's a good justification for them being combined into one element with a flag to turn off deinterlacing. (However, the clear choice for a name would be 'deinterlace', and it would be amusing to have a deinterlace=false property on it. Maybe call it inverse-telecine-only=true.) Anyway, back on topic for this bug. Patch looks good. I'd like to add the patterns 1:1 and 2:2 for 50/60p and 25/30p -> 50i/60i, and then 'auto' to make the element work like videorate, pushing out fields based on incoming timestamps. This means the mode property can go away. Haven't decided if 'auto' or '1:1' should be default, please offer comment. I'm also going add an allow-rff=boolean property, indicating whether or not 3-field frames should be sent as RFF or not.
Pushed. commit 351562936c626602116eead687fcb61d5c2d9f94 Author: Robert Swain <robert.swain@collabora.co.uk> Date: Wed Sep 8 15:08:50 2010 +0200 interlace: Add telecine support Leaving bug open so that comments get copied to a doc/random file.
I think the goal should be for the content/encoding transformation elements to apply correct timestamps to the buffers they output.
Any progress here?
Robert, David?
In the meantime, I think I came to the conclusion that Robert did, which is that any interlace or telecine related elements should just retimestamp and rebuffer similar to what an interlace-aware videorate would do. Also in the meantime, I wrote the ivtc element, which is *really* *good* at what it does. It's effectively an interlace-aware videorate.
Dave, if you feel like reviewing / fleshing out http://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/docs/design/part-interlaced-video.txt with any of your above explanations/thoughts/recommendations/mentions of elements, that'd be great.