GNOME Bugzilla – Bug 643847
deinterlace: Add support for deinterlacing using buffer caps/flags
Last modified: 2011-10-29 15:35:39 UTC
When not using the fieldanalysis element immediately upstream of deinterlace, behaviour should remain unchanged. fieldanalysis will set the caps and flags on the buffers such that they can be interpreted and acted upon to produce progressive output. There are two main modes of operation: - Passive pattern locking Passive pattern locking is a non-blocking, low-latency mode of operation that is suitable for close-to-live usage. Initially a telecine stream will be output as variable framerate with naïve timestamp adjustment. With each incoming buffer, an attempt is made to lock onto a pattern. When a lock is obtained, the src pad and output buffer caps will reflect the pattern and timestamps will be accurately interpolated between pattern repeats. This means that initially and at pattern transitions there will be short periods of inaccurate timestamping. - Active pattern locking Active pattern locking is a blocking, high-latency mode of operation that is targeted at use-cases where timestamp accuracy is paramount. Buffers will be queued until enough are present to make a lock. When locked, timestamps will be accurately interpolated between pattern repeats. Orphan fields can be dropped or deinterlaced. If no lock can be obtained, a single field might be pushed through to be deinterlaced. Locking can also be disabled or 'auto' chooses between passive and active locking modes depending on whether upstream is live.
Created attachment 182441 [details] [review] deinterlace: Add support for deinterlacing using buffer caps/flags An early version of this patch, that is heavily dependent on the fieldanalysis element that was pushed into -bad, was discussed in 642671. Sebastian reviewed it here: https://bugzilla.gnome.org/show_bug.cgi?id=642671#c11 The only thing remaining to do is to handle the GAP buffers for real gaps. I will do this when I find time. I could still push GAP buffers to the history and state history and then if they indicate a real GAP, when popping I could push them on and if not, drop them or something. The case where I think they should be dropped is when fieldanalysis has detected them to be unused frames in a telecine sequence. For example, if the stream has AtAb AtBb BtBb, the second frame is not needed as the first and third frames are progressive and contain its fields. fieldanalysis will mark this buffer with the GAP flag to indicate to deinterlace that it should be dropped. Deinterlace needs to receive this buffer for its timing information.
After a discussion with Sebastian, he has suggested that I reuse the RFF flag to indicate these buffers that contain only repeated fields and can be dropped in deinterlace as I don't make use of the RFF flag and using the GAP flag appears to have slightly different semantics and consequences to what I really want to achieve. I will document this case in -base as well.
Created attachment 185316 [details] [review] deinterlace: Add support for deinterlacing using buffer caps/flags Reusing RFF flag as suggested.
Created attachment 185317 [details] [review] fieldanalysis: Use RFF flag to indicate buffers to drop downstream Use of the GAP flag is not really correct here and makes it difficult to handle real GAP buffers in deinterlace. The RFF flag is unused and can be reused with similar semantics - the buffers marked with RFF that are in a telecine state contain only unneeded repeated fields and so can be dropped.
Created attachment 185318 [details] [review] docs: Update interlaced video design document The RFF flag is to be reused for buffers in the telecine state to indicate that the buffer contains only unneeded repeated fields that are present in other buffers and as such this buffer can be dropped.
Comment on attachment 185318 [details] [review] docs: Update interlaced video design document commit e284455272857f9c09315d917b3a49640b24de8b Author: Robert Swain <robert.swain@collabora.co.uk> Date: Wed Apr 6 16:11:02 2011 +0200 docs: Update interlaced video design document The RFF flag is to be reused for buffers in the telecine state to indicate that the buffer contains only unneeded repeated fields that are present in other buffers and as such this buffer can be dropped.
Comment on attachment 185317 [details] [review] fieldanalysis: Use RFF flag to indicate buffers to drop downstream commit 4cd15193bb90da97b6d585942f16281d227dfa95 Author: Robert Swain <robert.swain@collabora.co.uk> Date: Wed Apr 6 15:58:07 2011 +0200 fieldanalysis: Use RFF flag to indicate buffers to drop downstream Use of the GAP flag is not really correct here and makes it difficult to handle real GAP buffers in deinterlace. The RFF flag is unused and can be reused with similar semantics - the buffers marked with RFF that are in a telecine state contain only unneeded repeated fields and so can be dropped.
Created attachment 185398 [details] [review] deinterlace: Add support for deinterlacing using buffer caps/flags Addressed some more review comments and shifted to be committed after .28.
Created attachment 185514 [details] [review] small patch required on top of 185398 for deinterlace Fixes an infinite loop when the pad is flushed.
(In reply to comment #9) > Created an attachment (id=185514) [details] [review] > small patch required on top of 185398 for deinterlace > > Fixes an infinite loop when the pad is flushed. Thanks, good catch. I'll add this to the commit-after-freeze stack.
commit c1b100cf9cbe50dd677fddef0eba68c7a195573f Author: Robert Swain <robert.swain@collabora.co.uk> Date: Mon Nov 8 14:06:15 2010 +0100 deinterlace: Add support for deinterlacing using buffer caps/flags When not using the fieldanalysis element immediately upstream of deinterlace, behaviour should remain unchanged. fieldanalysis will set the caps and flags on the buffers such that they can be interpreted and acted upon to produce progressive output. There are two main modes of operation: - Passive pattern locking Passive pattern locking is a non-blocking, low-latency mode of operation that is suitable for close-to-live usage. Initially a telecine stream will be output as variable framerate with naïve timestamp adjustment. With each incoming buffer, an attempt is made to lock onto a pattern. When a lock is obtained, the src pad and output buffer caps will reflect the pattern and timestamps will be accurately interpolated between pattern repeats. This means that initially and at pattern transitions there will be short periods of inaccurate timestamping. - Active pattern locking Active pattern locking is a blocking, high-latency mode of operation that is targeted at use-cases where timestamp accuracy is paramount. Buffers will be queued until enough are present to make a lock. When locked, timestamps will be accurately interpolated between pattern repeats. Orphan fields can be dropped or deinterlaced. If no lock can be obtained, a single field might be pushed through to be deinterlaced. Locking can also be disabled or 'auto' chooses between passive and active locking modes depending on whether upstream is live.