After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 697892 - mpegtsdemux: Add support for DigiCipher II stream type.
mpegtsdemux: Add support for DigiCipher II stream type.
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
1.0.6
Other Linux
: Normal enhancement
: 1.1.1
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2013-04-12 15:49 UTC by Greg Rutz
Modified: 2013-07-03 14:59 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Proposed fix (2.82 KB, patch)
2013-04-12 15:49 UTC, Greg Rutz
needs-work Details | Review
DigiCipher II MPEG2 Single Program Transport Stream (786.52 KB, application/octet-stream)
2013-04-15 17:19 UTC, Greg Rutz
  Details
Updated fix (7.58 KB, patch)
2013-04-17 20:55 UTC, Greg Rutz
committed Details | Review

Description Greg Rutz 2013-04-12 15:49:53 UTC
Created attachment 241366 [details] [review]
Proposed fix

In the CableTV industry, may headends push out MPEG2 transport streams compliant with the Motorola DigiCipher II proprietary standard.  tsdemux handles these streams fine with one exception, the MPEG2 video stream_type in the PMT for these streams is 0x80 instead of 0x2.

There is currently a conflicting stream type in the BluRay program stream types (ST_BD_AUDIO_LPCM).  I used the packet size (BluRay is only 192 M2TS packets) to be able to choose between them.
Comment 1 Sebastian Dröge (slomo) 2013-04-15 08:04:50 UTC
Is there a better way to detect if this is a DC2 MPEG TS stream? This does not look very robust
Comment 2 Greg Rutz 2013-04-15 12:26:00 UTC
I'll do some looking around to see if there is another marker in the stream that indicates DCII.
Comment 3 Greg Rutz 2013-04-15 16:18:17 UTC
Here is how both vlc and libav handle stream types in BluRay m2ts and standard MPEG2 TS.  They identify the stream as being BluRay by looking for the "registration_descriptor" in the PMT.  In the case of BluRay, it seems that this descriptor's "format_identifier" field always contains a FOURCC value of "HDMV".  Then, they both come to the same conclusion when interpreting the stream_type field of the PMT for the 0x80 value, but in different orders.

In VLC:
The codec type is first set as MPEG2 video if the stream_type is 0x1, 0x2, or 0x80.  Later, if the PMT registration_descriptor contains the "HDMV" value, the codec type is modified to be the BluRay LPCM audio type.

In libav:
If the PMT registration_descriptor contains the "HDMV" value, the codec type is set to the BluRay LPCM audio type.  If a codec type has not yet been set after executing the HDMV-specific code, the stream_type is checked again for other non-BluRay types, which leads to the proper setting of MPEG2 video for the 0x80 stream_type.

In both libraries, all of the BluRay-specific stream types are handled this way.  We could do the same thing in GStreamer -- handle BluRay stream types separately.  If you all agree, I could put together another patch that tries to solve the problem this way.
Comment 4 Greg Rutz 2013-04-15 17:19:51 UTC
Created attachment 241582 [details]
DigiCipher II MPEG2 Single Program Transport Stream

This is a 3-second DigiCipher II stream (short enough to avoid legal issues) captured from our headend.
Comment 5 Sebastian Dröge (slomo) 2013-04-16 09:07:36 UTC
Sounds sensible, yes. Please put together a patch for this :)
Comment 6 Greg Rutz 2013-04-17 20:55:02 UTC
Created attachment 241777 [details] [review]
Updated fix

Fix updated to handle BluRay stream type options first, then the rest.
Comment 7 Greg Rutz 2013-04-17 20:56:11 UTC
This most recent patch should have obsoleted the previous -- I forgot to check the box.
Comment 8 Sebastian Dröge (slomo) 2013-04-18 07:42:27 UTC
commit 2306d51d9dddffe49c07d87858f57e6b3614effc
Author: Greg Rutz <greg@gsr-tek.com>
Date:   Wed Apr 17 14:45:19 2013 -0600

    tsdemux: Add support for Motorola DigiCipher II MPEG2 video
    
    Since there is a conflict between the DCII stream type and BluRay
    stream types, moved the processing of BluRay-specific stream types
    to the beginning of the function.  Only if a BluRay stream type
    IS NOT found do we proceed to check the rest of the stream type
    identifiers
    
    Previous code was also "sort-of" handling a similar conflict between
    BluRay AC3 audio and standard AC3 audio.  Moved the special case BluRay
    AC3 handling in the main switch statement to the new BluRay-specific
    switch.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=697892
Comment 9 Edward Hervey 2013-07-03 09:20:39 UTC
Greg, that sample you provided doesn't contain either the HDMV or the DCII descriptor. Is that normal ?
Comment 10 Greg Rutz 2013-07-03 14:20:13 UTC
The crux of this bug is that there are several stream type IDs that conflict between BluRay and other standards.  The HDMV registration descriptor will only be present in BluRay streams and thus will not be seen in my stream.  We use the presence of this descriptor to determine whether we will interpret the stream types as a BluRay type or "other".

The presence of DCII video is not indicated by a descriptor, but in the "stream_type" field in the elementary stream loop of the PMT.  If you look at the PMT (pid 0x35), you will see an elementary stream with a stream type of 0x80 (DCII).  If this were standard MPEG2 video, it would be a stream type of 0x02.

If you use the dvbsnoop utility (available on most linux systems), you can run the following command to look at the PMT of my stream:

dvbsnoop -s ts -tssubdecode -if dcii_stream.ts -N 10 0x35
Comment 11 Edward Hervey 2013-07-03 14:40:52 UTC
The problem is that, without any external information (like registration descriptors), we'd have to consider stream_type 0x80 (and 0x81) as "private".

Do you have any other hints/ways to detect that a stream is digicipherII ?

Note: I'm not removing the commit, don't worry. Just investigating some refactoring in tsdemux.
Comment 12 Greg Rutz 2013-07-03 14:59:22 UTC
I don't know of any other way to determine whether a stream was generated with
DCII.  In practice, DCII was differentiated from standard DVB by the SI carried
in out-of-band channels, so the stream alone would not be enough to tell.  In
the GStreamer world, a DCII MPEG2TS looks no different from a standard MPEG2TS
with these exceptions:

a) Video is always MPEG-2 (no I-frames) with stream_type of 0x80
b) Audio is only AC3 (no MPEG1/2 audio)

The "no I-frames" constraint is interesting.  I found the citation here
(http://www.tsreader.com/tsreader/support.html).

However, even with these constraints, there is nothing (at the demux level) we
can use to determine the type.

Here is a good read on DCII:

http://www.tsreader.com/mpeg/#dcii