GNOME Bugzilla – Bug 767771
qtdemux/jpegdec: Interlaced content detected as progressive
Last modified: 2016-11-01 18:10:57 UTC
When playing back interlaced SD video in either DV or MJPEG the uridecodebin detects the interlace mode as 'mixed' instead of 'interleaved'. Passing mixed video to decklinkvideosink causes a 'not negotiated' error. Using, as suggested, a capssetter element creates a valid work around.
Which demuxer and decoder is used in both cases? This is not a problem in uridecodebin, but something that would have to be implemented in the relevant decoders and/or demuxers. Or maybe mixed support can be added to decklinkvideosink.
The demuxer is qtdemux, the decoder jpegdec.
That would be qtdemux then which could do something, like checking what sample flags it gets in the future and assume interleaved instead of mixed unless something more complicated happens. OTOH it should be fine to support interlaced-mode=mixed in decklinkvideosink. But would have to be researched. Can you open a new bug for decklinkvideosink about this? Also you mentioned DV?
Might be a duplicate of bug #568555, could you make a sample file available?
Created attachment 329993 [details] Test case mjpeg quicktime file This was converted and shortened from a SD anamorphic 720x576i DV50 mov file with ffmpeg.
MP4 does not seem to have anything to signal interlaced content, and jpegdec currently does not put it into the caps either (can it even detect it?). So the problem here is something different than what I first understood. It's not detected as interlaced-mode=mixed but as progressive. Tim, is that what the other bug is about? If I understand that one correctly it is about actual decoding problems with interlaced JPEG?
This is not what the other bug is about. Here there is a single JPEG frame which represents an interleaved interlaced image. Somewhere there needs to be information that this is interlaced, either in the JPEG headers (doubtful, but didn't check) or in qt headers.
Is the interlace information not available in the 'fiel' area of the Quicktime video sample description atom?
Dave, do you see this in the file?
Yes, the information is in the header at 3005820d-3005820e: 300581AF Video (86 bytes) 300581AF Header (8 bytes) 300581AF Size: 112 (0x00000070) 300581B3 Name: jpeg 300581B7 Reserved: 0 (0x0) 300581BD Data reference index: 1 (0x0001) 300581BF Version: 0 (0x0000) 300581C1 Revision level: 0 (0x0000) 300581C3 Vendor: FFMP 300581C7 Temporal quality: 512 (0x00000200) 300581CB Spatial quality: 512 (0x00000200) 300581CF Width: 720 (0x02D0) 300581D1 Height: 576 (0x0240) 300581D3 Horizontal resolution: 4718592 (0x00480000) 300581D7 Vertical resolution: 4718592 (0x00480000) 300581DB Data size: 0 (0x00000000) 300581DF Frame count: 1 (0x0001) 300581E1 Compressor name size: 5 (0x05) 300581E2 Compressor name: mjpeg 300581E7 Padding: (26 bytes) 30058201 Depth: 24 (0x0018) 30058203 Color table ID: 65535 (0xFFFF) 30058205 Field/Frame Information (10 bytes) 30058205 Header (8 bytes) 30058205 Size: 10 (0x0000000A) 30058209 Name: fiel 3005820D fields: 2 (0x02) 3005820E detail: 6 (0x06) 3005820F Pixel Aspect Ratio (16 bytes) 3005820F Header (8 bytes) 3005820F Size: 16 (0x00000010) 30058213 Name: pasp 30058217 hSpacing: 64 (0x00000040) 3005821B vSpacing: 45 (0x0000002D)
This contains a jpeg chunk, instead of the expected jp2h chunk, so I found the definition of that and start to parse it. However, I do not see a fiel chunk in the sample file: 00357d20: 0001 0000 0066 6a70 6567 0000 0000 0000 .....fjpeg...... 00357d30: 0001 0000 0000 4646 4d50 0000 0200 0000 ......FFMP...... 00357d40: 0200 02d0 0240 0048 0000 0048 0000 0000 .....@.H...H.... 00357d50: 0000 0001 056d 6a70 6567 0000 0000 0000 .....mjpeg...... 00357d60: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00357d70: 0000 0000 0018 ffff 0000 0010 7061 7370 ............pasp 00357d80: 0000 0010 0000 000f 0000 0018 7374 7473 ............stts 00357d90: 0000 0000 0000 0001 0000 0064 0000 0001 ...........d.... 00357da0: 0000 0040 7374 7363 0000 0000 0000 0004 ...@stsc........ 00357db0: 0000 0001 0000 0022 0000 0001 0000 0002 ......."........ 00357dc0: 0000 0020 0000 0001 0000 0003 0000 001a ... ............ 00357dd0: 0000 0001 0000 0004 0000 0008 0000 0001 ................ 00357de0: 0000 01a4 7374 737a 0000 0000 0000 0000 ....stsz........ 00357df0: 0000 0064 0000 79d2 0000 7741 0000 780c ...d..y...wA..x. Did you run that header dump command on the file that's attached ?
The metadata dumper seems to have not reliably extracted the interlace data. I will upload a new file which demonstrates the interlace data and checked with a hex dump. Looks like not all mjpeg/quicktime writers support the interlace tags.
There are fiel related patches in #769048 btw
Created attachment 337219 [details] Short Interlaced 720x576i quicktime file
Created attachment 337220 [details] Hex dump screenshot of interlace meta from sample quicktime file
I can test on OSX (1.9.90 binary) and Git Master for Linux. It seems Final Cut Pro 7 does not include this metadata so I assume some files will still be tagged incorrectly.
Created attachment 337306 [details] [review] extract interlaced information from jpeg in qt Note that gst-play-1.0 still displays video that looks not deinterlaced, so something somewhere isn't working as well as it should.
And it seems the jpeg decoder not caring about interlaced on input caps (both jpegdec and avdec_jpeg).
Created attachment 337307 [details] [review] extract interlaced information from jpeg in qt Looks like the actual field is interlace-mode, not interlaced, either this changed or I'm misremembering. This patch makes gst-play-1.0 play as deinterlaced.
The first value is the number of fields per frame (1 or 2). If 2 fields are indicated then the second value describes the field order. From the Quicktime format specification: The following defines the permitted variants: 0 – There is only one field. 1 – T is displayed earliest, T is stored first in the file. 6 – B is displayed earliest, B is stored first in the file. 9 – B is displayed earliest, T is stored first in the file. 14 – T is displayed earliest, B is stored first in the file.
Sorry, I was referring to the field in gstreamer caps, not the field in the jpeg structure. The patch I posted works with your latest sample file (as per the spec excerpt you quoted).
Review of attachment 337307 [details] [review]: ::: gst/isomp4/qtdemux.c @@ +10021,3 @@ + int clen = + *(const guint8 *) ((const guint8 *) jpeg->data + + compressor_offset); Use GstByteReader or similar... also below for >8 bit integers it is not endianness-independent the way you do it, and will also crash on architectures that don't allow unaligned access.
Created attachment 337567 [details] [review] extract interlaced information from jpeg in qt Done, and I also removed parsing of the compressor name, since it was really a way for me to double check I had thee offsets right when coding.
Created attachment 337607 [details] [review] extract interlaced information from jpeg in qt Same, with a link to the bug in the commit message added before tpm sees it!
Comment on attachment 337607 [details] [review] extract interlaced information from jpeg in qt You also should add this fourcc to the qt_node_types array.
commit 5a889647ba202d1fe8d902eef4251ca6ed1283bd Author: Vincent Penquerc'h <vincent.penquerch@collabora.co.uk> Date: Mon Oct 10 13:00:01 2016 +0100 qtdemux: extract interlaced information from jpeg video This information is hidden in a small chunk of data. Format found at https://developer.apple.com/standards/qtff-2001.pdf, page 92, "Video Sample Description", under table 3.1. https://bugzilla.gnome.org/show_bug.cgi?id=767771