After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 643847 - deinterlace: Add support for deinterlacing using buffer caps/flags
deinterlace: Add support for deinterlacing using buffer caps/flags
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal enhancement
: 0.10.31
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2011-03-04 07:01 UTC by Robert Swain
Modified: 2011-10-29 15:35 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
deinterlace: Add support for deinterlacing using buffer caps/flags (58.40 KB, patch)
2011-03-04 07:07 UTC, Robert Swain
none Details | Review
deinterlace: Add support for deinterlacing using buffer caps/flags (58.34 KB, patch)
2011-04-06 15:17 UTC, Robert Swain
accepted-commit_after_freeze Details | Review
fieldanalysis: Use RFF flag to indicate buffers to drop downstream (10.09 KB, patch)
2011-04-06 15:19 UTC, Robert Swain
committed Details | Review
docs: Update interlaced video design document (1.80 KB, patch)
2011-04-06 15:20 UTC, Robert Swain
committed Details | Review
deinterlace: Add support for deinterlacing using buffer caps/flags (58.29 KB, patch)
2011-04-07 08:36 UTC, Robert Swain
committed Details | Review
small patch required on top of 185398 for deinterlace (552 bytes, patch)
2011-04-08 10:59 UTC, Jan Schmidt
committed Details | Review

Description Robert Swain 2011-03-04 07:01:24 UTC
When not using the fieldanalysis element immediately upstream of deinterlace,
    behaviour should remain unchanged. fieldanalysis will set the caps and flags on
    the buffers such that they can be interpreted and acted upon to produce
    progressive output.
    
    There are two main modes of operation:
    
    - Passive pattern locking
      Passive pattern locking is a non-blocking, low-latency mode of operation that
      is suitable for close-to-live usage. Initially a telecine stream will be
      output as variable framerate with naïve timestamp adjustment. With each
      incoming buffer, an attempt is made to lock onto a pattern. When a lock is
      obtained, the src pad and output buffer caps will reflect the pattern and
      timestamps will be accurately interpolated between pattern repeats. This
      means that initially and at pattern transitions there will be short periods
      of inaccurate timestamping.
    
    - Active pattern locking
      Active pattern locking is a blocking, high-latency mode of operation that is
      targeted at use-cases where timestamp accuracy is paramount. Buffers will be
      queued until enough are present to make a lock. When locked, timestamps will
      be accurately interpolated between pattern repeats. Orphan fields can be
      dropped or deinterlaced. If no lock can be obtained, a single field might be
      pushed through to be deinterlaced.

    Locking can also be disabled or 'auto' chooses between passive and active
    locking modes depending on whether upstream is live.
Comment 1 Robert Swain 2011-03-04 07:07:50 UTC
Created attachment 182441 [details] [review]
deinterlace: Add support for deinterlacing using buffer caps/flags

An early version of this patch, that is heavily dependent on the fieldanalysis element that was pushed into -bad, was discussed in 642671.

Sebastian reviewed it here: https://bugzilla.gnome.org/show_bug.cgi?id=642671#c11

The only thing remaining to do is to handle the GAP buffers for real gaps. I will do this when I find time. I could still push GAP buffers to the history and state history and then if they indicate a real GAP, when popping I could push them on and if not, drop them or something.

The case where I think they should be dropped is when fieldanalysis has detected them to be unused frames in a telecine sequence. For example, if the stream has AtAb AtBb BtBb, the second frame is not needed as the first and third frames are progressive and contain its fields. fieldanalysis will mark this buffer with the GAP flag to indicate to deinterlace that it should be dropped. Deinterlace needs to receive this buffer for its timing information.
Comment 2 Robert Swain 2011-04-05 13:50:19 UTC
After a discussion with Sebastian, he has suggested that I reuse the RFF flag to indicate these buffers that contain only repeated fields and can be dropped in deinterlace as I don't make use of the RFF flag and using the GAP flag appears to have slightly different semantics and consequences to what I really want to achieve. I will document this case in -base as well.
Comment 3 Robert Swain 2011-04-06 15:17:54 UTC
Created attachment 185316 [details] [review]
deinterlace: Add support for deinterlacing using buffer caps/flags

Reusing RFF flag as suggested.
Comment 4 Robert Swain 2011-04-06 15:19:28 UTC
Created attachment 185317 [details] [review]
fieldanalysis: Use RFF flag to indicate buffers to drop downstream

Use of the GAP flag is not really correct here and makes it difficult to
handle real GAP buffers in deinterlace. The RFF flag is unused and can
be reused with similar semantics - the buffers marked with RFF that are
in a telecine state contain only unneeded repeated fields and so can be
dropped.
Comment 5 Robert Swain 2011-04-06 15:20:40 UTC
Created attachment 185318 [details] [review]
docs: Update interlaced video design document

The RFF flag is to be reused for buffers in the telecine state to
indicate that the buffer contains only unneeded repeated fields that are
present in other buffers and as such this buffer can be dropped.
Comment 6 Robert Swain 2011-04-07 08:23:02 UTC
Comment on attachment 185318 [details] [review]
docs: Update interlaced video design document

commit e284455272857f9c09315d917b3a49640b24de8b
Author: Robert Swain <robert.swain@collabora.co.uk>
Date:   Wed Apr 6 16:11:02 2011 +0200

    docs: Update interlaced video design document
    
    The RFF flag is to be reused for buffers in the telecine state to
    indicate that the buffer contains only unneeded repeated fields that are
    present in other buffers and as such this buffer can be dropped.
Comment 7 Robert Swain 2011-04-07 08:23:48 UTC
Comment on attachment 185317 [details] [review]
fieldanalysis: Use RFF flag to indicate buffers to drop downstream

commit 4cd15193bb90da97b6d585942f16281d227dfa95
Author: Robert Swain <robert.swain@collabora.co.uk>
Date:   Wed Apr 6 15:58:07 2011 +0200

    fieldanalysis: Use RFF flag to indicate buffers to drop downstream
    
    Use of the GAP flag is not really correct here and makes it difficult to
    handle real GAP buffers in deinterlace. The RFF flag is unused and can
    be reused with similar semantics - the buffers marked with RFF that are
    in a telecine state contain only unneeded repeated fields and so can be
    dropped.
Comment 8 Robert Swain 2011-04-07 08:36:41 UTC
Created attachment 185398 [details] [review]
deinterlace: Add support for deinterlacing using buffer caps/flags

Addressed some more review comments and shifted to be committed after .28.
Comment 9 Jan Schmidt 2011-04-08 10:59:31 UTC
Created attachment 185514 [details] [review]
small patch required on top of 185398 for deinterlace

Fixes an infinite loop when the pad is flushed.
Comment 10 Robert Swain 2011-04-10 11:34:52 UTC
(In reply to comment #9)
> Created an attachment (id=185514) [details] [review]
> small patch required on top of 185398 for deinterlace
> 
> Fixes an infinite loop when the pad is flushed.

Thanks, good catch. I'll add this to the commit-after-freeze stack.
Comment 11 Robert Swain 2011-05-19 03:52:57 UTC
commit c1b100cf9cbe50dd677fddef0eba68c7a195573f
Author: Robert Swain <robert.swain@collabora.co.uk>
Date:   Mon Nov 8 14:06:15 2010 +0100

    deinterlace: Add support for deinterlacing using buffer caps/flags
    
    When not using the fieldanalysis element immediately upstream of deinterlace,
    behaviour should remain unchanged. fieldanalysis will set the caps and flags on
    the buffers such that they can be interpreted and acted upon to produce
    progressive output.
    
    There are two main modes of operation:
    
    - Passive pattern locking
      Passive pattern locking is a non-blocking, low-latency mode of operation that
      is suitable for close-to-live usage. Initially a telecine stream will be
      output as variable framerate with naïve timestamp adjustment. With each
      incoming buffer, an attempt is made to lock onto a pattern. When a lock is
      obtained, the src pad and output buffer caps will reflect the pattern and
      timestamps will be accurately interpolated between pattern repeats. This
      means that initially and at pattern transitions there will be short periods
      of inaccurate timestamping.
    
    - Active pattern locking
      Active pattern locking is a blocking, high-latency mode of operation that is
      targeted at use-cases where timestamp accuracy is paramount. Buffers will be
      queued until enough are present to make a lock. When locked, timestamps will
      be accurately interpolated between pattern repeats. Orphan fields can be
      dropped or deinterlaced. If no lock can be obtained, a single field might be
      pushed through to be deinterlaced.
    
    Locking can also be disabled or 'auto' chooses between passive and active
    locking modes depending on whether upstream is live.