After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 730435 - mpegts: add support for ATSC tables MGT / ETT / EIT / STT
mpegts: add support for ATSC tables MGT / ETT / EIT / STT
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other All
: Normal enhancement
: 1.3.3
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2014-05-20 11:26 UTC by Thiago Sousa Santos
Modified: 2014-05-29 08:45 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
mpegts: add atsc MGT section parsing (8.04 KB, patch)
2014-05-20 11:28 UTC, Thiago Sousa Santos
needs-work Details | Review
tsbase: parse the mgt and add listed EIT/ETT pids to the known psi (2.44 KB, patch)
2014-05-20 11:28 UTC, Thiago Sousa Santos
none Details | Review
mpegts: atsc: add ETT structures and parsing (9.97 KB, patch)
2014-05-20 11:28 UTC, Thiago Sousa Santos
none Details | Review
examples: mpegts: add function to dump ETT tables (2.30 KB, patch)
2014-05-20 11:28 UTC, Thiago Sousa Santos
none Details | Review
mpegts: atsc: add encoding conversion for UTF-16 ETT strings (1.84 KB, patch)
2014-05-20 11:29 UTC, Thiago Sousa Santos
none Details | Review
mpegts: atsc: add STT table parsing (7.70 KB, patch)
2014-05-23 14:18 UTC, Thiago Sousa Santos
none Details | Review
mpegts: add atsc MGT section parsing (10.01 KB, patch)
2014-05-28 21:29 UTC, Thiago Sousa Santos
committed Details | Review
tsbase: parse the mgt and add listed EIT/ETT pids to the known psi (2.44 KB, patch)
2014-05-28 21:30 UTC, Thiago Sousa Santos
committed Details | Review
mpegts: atsc: add ETT structures and parsing (10.37 KB, patch)
2014-05-28 21:30 UTC, Thiago Sousa Santos
committed Details | Review
examples: mpegts: add function to dump ETT tables (2.31 KB, patch)
2014-05-28 21:30 UTC, Thiago Sousa Santos
committed Details | Review
mpegts: atsc: add encoding conversion for UTF-16 ETT strings (1.84 KB, patch)
2014-05-28 21:30 UTC, Thiago Sousa Santos
committed Details | Review
mpegts: atsc: add STT table parsing (8.08 KB, patch)
2014-05-28 21:30 UTC, Thiago Sousa Santos
committed Details | Review
tests: mpegts: add test for STT parsing (2.49 KB, patch)
2014-05-28 21:31 UTC, Thiago Sousa Santos
committed Details | Review
mpegts: atsc: add atsc's EIT table parsing (18.89 KB, patch)
2014-05-28 21:31 UTC, Thiago Sousa Santos
committed Details | Review
mpegts: atsc: add missing field to ETT table (1.98 KB, patch)
2014-05-28 21:31 UTC, Thiago Sousa Santos
committed Details | Review

Description Thiago Sousa Santos 2014-05-20 11:26:51 UTC
ATSC has the Master Guide Table that contains an association of PIDs to EITs abd also ETTs (extended text table) tables, the ETTs contains internationalized descriptions of the content in the virtual channels or for events.

The table id for ETT should be 0xCC and for EIT (in ATSC) should be 0xCB, the MGT has the table id 0xC7 and is on pid 0x1FFB.

The proposed solution adds:
1) Add MGT/ETT parsing to mpegts
2) Add identification of EIT and ETT tables so they are properly identified by mpegts lib and used in tsbase (tsparse/tsdemux) element
3) Add dumping of ETT table to ts-parse example
Comment 1 Thiago Sousa Santos 2014-05-20 11:28:44 UTC
Created attachment 276842 [details] [review]
mpegts: add atsc MGT section parsing

Add a parsing function for MGT and also detect the EIT tables
for ATSC, the EIT pids are reported inside the MGT and we are still
only relying only on the table id for detecting it. In the future we
would want to also check the pid and compare with whatever the MGT
previously reported to confirm that it is indeed the EIT.
Comment 2 Thiago Sousa Santos 2014-05-20 11:28:49 UTC
Created attachment 276843 [details] [review]
tsbase: parse the mgt and add listed EIT/ETT pids to the known psi

This will make tsbase also parse the EITs and ETTs from ATSC streams that have
their pids reported on the MGT and post to the bus
Comment 3 Thiago Sousa Santos 2014-05-20 11:28:54 UTC
Created attachment 276844 [details] [review]
mpegts: atsc: add ETT structures and parsing

ETT (extended text table) contains ATSC text information with descriptions
of virtual channels and events. The text can be internationalized and also
compressed.
Comment 4 Thiago Sousa Santos 2014-05-20 11:28:59 UTC
Created attachment 276845 [details] [review]
examples: mpegts: add function to dump ETT tables
Comment 5 Thiago Sousa Santos 2014-05-20 11:29:04 UTC
Created attachment 276846 [details] [review]
mpegts: atsc: add encoding conversion for UTF-16 ETT strings
Comment 6 Thiago Sousa Santos 2014-05-23 14:18:07 UTC
Created attachment 277072 [details] [review]
mpegts: atsc: add STT table parsing

Also adding STT (system time table)
Comment 7 Edward Hervey 2014-05-28 06:23:04 UTC
Could you add ts-parser support for MGT ?
Also it looks as though you're leaking all the returned parsed sections/tables
Comment 8 Edward Hervey 2014-05-28 06:35:32 UTC
Review of attachment 276842 [details] [review]:

::: gst-libs/gst/mpegts/gstmpegtssection.c
@@ +1055,3 @@
+    case GST_MTS_TABLE_ID_ATSC_EVENT_INFORMATION:
+      /* FIXME check pids reported on the MGT to confirm expectations */
+      return GST_MPEGTS_SECTION_EIT;

ATSC has a different formatting for EIT than the way it's implement in DVB, you can't re-use that one.

And it made me realize that EIT is a DVB-specific table (and not a generic H.222.0 one ...)
Comment 9 Thiago Sousa Santos 2014-05-28 21:29:51 UTC
Created attachment 277412 [details] [review]
mpegts: add atsc MGT section parsing

Add a parsing function for MGT and also detect the EIT tables
for ATSC, the EIT pids are reported inside the MGT and we are still
only relying only on the table id for detecting it. In the future we
would want to also check the pid and compare with whatever the MGT
previously reported to confirm that it is indeed the EIT.
Comment 10 Thiago Sousa Santos 2014-05-28 21:30:21 UTC
Created attachment 277413 [details] [review]
tsbase: parse the mgt and add listed EIT/ETT pids to the known psi

This will make tsbase also parse the EITs and ETTs from ATSC streams that have
their pids reported on the MGT and post to the bus
Comment 11 Thiago Sousa Santos 2014-05-28 21:30:31 UTC
Created attachment 277414 [details] [review]
mpegts: atsc: add ETT structures and parsing

ETT (extended text table) contains ATSC text information with descriptions
of virtual channels and events. The text can be internationalized and also
compressed.
Comment 12 Thiago Sousa Santos 2014-05-28 21:30:41 UTC
Created attachment 277415 [details] [review]
examples: mpegts: add function to dump ETT tables
Comment 13 Thiago Sousa Santos 2014-05-28 21:30:48 UTC
Created attachment 277416 [details] [review]
mpegts: atsc: add encoding conversion for UTF-16 ETT strings
Comment 14 Thiago Sousa Santos 2014-05-28 21:30:57 UTC
Created attachment 277417 [details] [review]
mpegts: atsc: add STT table parsing

Adds the system time table structure and functions for convenient parsing of
it and for getting the UTC datetime that it represents. Also adds its
information dumping to the ts-parser example
Comment 15 Thiago Sousa Santos 2014-05-28 21:31:12 UTC
Created attachment 277418 [details] [review]
tests: mpegts: add test for STT parsing

Specially for the time to UTC datetime conversion

Example taken from the A65 spec
Comment 16 Thiago Sousa Santos 2014-05-28 21:31:21 UTC
Created attachment 277419 [details] [review]
mpegts: atsc: add atsc's EIT table parsing

ATSC has its own version of the EIT table (DVB also has one).

This patch adds parsing for the ATSC EIT table and also fixed
the section identification to mark it as the ATSC one.

The implementation aws refactored to reuse some common internal
structures from ETT.

Also adds its dumping function to ts-parser example
Comment 17 Thiago Sousa Santos 2014-05-28 21:31:27 UTC
Created attachment 277420 [details] [review]
mpegts: atsc: add missing field to ETT table

Set the subtable_extension as ett_table_id_extension
for ETT tables as it is used by it
Comment 18 Thiago Sousa Santos 2014-05-28 21:32:57 UTC
New versions with requested fixes.
Comment 19 Edward Hervey 2014-05-29 06:30:00 UTC
Not sure why ... but git bz apply is failing for me with those patches.
Comment 20 Edward Hervey 2014-05-29 08:18:24 UTC
My bad, just realized I needed to apply the patches from #730642 first
Comment 21 Edward Hervey 2014-05-29 08:44:38 UTC
yay, commited \o/