All the streams in there are not available anymore it seems. But your attached file has the first frame decoded and then playback stops.

Yes, problem is that the timestamps are all about 18 hours but the newsegment event is wrong:

/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = "event   ******* E (type: 102, GstEventNewsegment, update=(boolean)false, rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, start=(gint64)40676035677777, stop=(gint64)-1, position=(gint64)0;) 0xa07800"
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = "chain   ******* < (  162 bytes, timestamp: 18:37:58.500000000, duration: none, offset: -1, offset_end: -1, flags: 32) 0xa0c700"

More precise, this is what happens while the first buffer is handled:

0:00:18.022240444 27106       0xe2e530 DEBUG            mpegtsdemux gstmpegtsdem
ux.c:1140:gst_mpegts_demux_data_cb:<mpegtsdemux0> setting PTS to (6037065000) ti
me: 18:37:58.500000000 on buffer 0xe54700 first buffer: 1 base_time: 0:00:00.000
000000
[...]
0:00:18.022266774 27106       0xe2e530 DEBUG            mpegtsdemux gstmpegtsdemux.c:864:gst_mpegts_demux_setup_base_pts:<mpegtsdemux0> no base PCR, using last 
PCR 3660843211
[...]
0:00:18.022684425 27106       0xe2e530 DEBUG            mpegtsdemux gstmpegtsdemux.c:910:gst_mpegts_demux_send_new_segment:<mpegtsdemux0> segment PTS to time: 11:17:56.035677777


The PTS of the buffer is somewhere around 18 hours but no base PCR was detected and the last PCR is somewhere around 11 hours, which is then used as segment start time.

Maybe in gst_mpegts_demux_setup_base_pts() the PTS should always be preferred if the PCR is too far away

Seems to work properly with tsdemux.

Fails to work with current master. Both the segment and live stream pre-rolls ... and only first frame is displayed without audio

grmbl, this might require some re-thinking of how PCR/PTS are handled.

Right now if upstream provides timestamped buffers (dvb, hls, ...) we assume (wrongly) that the same clock is used for PCR and PTS.... and not just the same clock "rate".

As a result, we end up with something like the following:
Input buffer timestamp : 0s
First PCR seen : 15 hours
First PTS seen : 19 hours

... and we end up with a first timestamps of 19 - 15 = 4 hours :(

Note that this issue will also happen when working with non-timed input also now that I look at it.


Maybe something smarter would be, if no base PTS has been established to:
* Calculate lowest PTS between all streams
* Figure out PCR when beginning of that buffer was received
* Use that "smallest (PCR vs PTS) diff" as a base pts time

And then report for the latency:
* Difference between smallest and biggest PTS accross all stream pids of a program.

This is in par with a discussion that took place (late at night) in Milan during GStreamer hackfest.

Changing title accordingly

Some more explanations regarding this proposed change in regards to the latency part for live pipelines.

NOTE: While this explanation is for streams that are *NOT* broken like the ones above (very big difference between PCR and PTS/DTS received at the same time), the same technique can be trivially used to support those.

NOTE: For the sake of simplicity, we assume the rate is perfect (remote clock goes at exactly the same speed as the local clock) and no clock estimation is needed. We also assume PCR/PTS/DTS wrapover is taken care of.

NOTE: We use DTS for the various calculation. If no DTS is present in PES header, it is the same as the PTS. If a DTS is present and different from PTS, that reordering/latency will be handled and reported by decoders (if decoding is needed).

NOTE: We use the term running time for both input and output. Buffer timestamps can/might be different from those, but proper use of segment.base will take care of that.

In mpeg-ts, the timing system is based around a full end-to-end "System Target Decoder" (STD), which is a whole system including capture, demuxing, buffering, decoding and presentation.
The reason why in mpeg-ts streams the PTS of the various audio and video streams can be so much higher than the co-located PCR is because it takes into account the maximum latency/buffering needed in demuxing/buffering/decoding/reordering in order for the target buffer to be properly displayed.

In GStreamer, the latency an element reports is the min/max difference in running-time between input and output of that element (and only that element) for the specific stream it is receiving.

Due to that difference, tsdemux needs to report not the end-to-end latency (PTS - PCR received at the same time) but only the min/max latency it introduces itself. This boils down to the interleaving/packetization latency. The other downstream elements (parsers, queue, decoders, ...) will add min/max latency specific to those streams and end up with the ideal capture-to-display latency.

Furthermore, if we do not need to do decoding/display, but instead want to use a different live stream (such as capture ! demuxing ! rtp payloading ! transmission), we do not need to add unneeded latency. Once again, with this breakdown of latency per element we can reach that.



An example input is (somewhat simplified) as follows:

(VBUF(A).(B) => Video buffer number A, sub part B (if B==0 it's the start of a PES and it has a PTS and maybe a DTS (if different from PTS))
(ABUF(A).(B) => Same deal except for audio)

Buffer IN0 (Runningtime INRT0)
Contents
  XXX PCR0 XXX VBUF0.2 ABUF5.0 VBUF0.3 ABUF5.1 XXX ...

...

Buffer IN3 (Runningtime INRT3)
Contents
  VBUF0.42 ABUF5.8 PCR2 XXX VBUF1.0 ABUF6.0 VBUF1.0 ABUF6.1 XXX ...


Notes:
  In IN0:
   * We use PCR0 for rate estimation (against INRT0)
   * we see the start of an audio PES (ABUF5.0).
      * It has a certain DTS (higher than PCR0, say PCR0 + 500)

  In IN3:
   * We use PCR2 for rate estimation (against INRT3)
      * PCR2 is higher than PCR0 (say PCR2 = PCR0 + 50)
   * We see the start of a video PES (VBUF1.0)
      * It has a certain DTS (higher than PCR2, say PCR2 + 300)
   * We get the beginning of a new ABUF, we can output the previous one (ABUF5).


What happens with the above example ?
  On running time 0, we received the beginning of ABUF5, but we can't output it yet (=> introducing latency).
  On running time 3, we received the beginning of VBUF1, but we can't output it yet (=> introducing latency)
  On running time 3, we can output the previously accumulated ABUF5
    ==> What latency do we report and what running-time do we use for that outputted buffer ?


Several options are available:
  1) Correlate exactly input runningtime to PCR, figure out the PCR/runningtime offset and set the output runnning time as DTS + offset.
    This can't be used for several reasons:
    * The DTS - colocated PCR includes the end-to-end latency, we would end up with a big delay
    * The DTS - colocated PCR can be huge on non-standard streams (initial topic of this bug) resulting in huuuuuuuuuuuuge delays in playback
    * The latency reported would have to be negative to have accurate playback (we can't do that).
    ==>> NOT A SOLUTION

  2) Ignore PCR/DTS and use the running-time at which the beginning of a buffer was received.
    * This doesn't work since the various stream packets can be delayed (the beginning of an audio buffer is not received at the same time as the beginning of the video bfufer that should be displayed at the same time).
    ==>> NOT A SOLUTION

  3) An intermediate solution, where we use the lowest DTS seen at any given time as a base DTS/PTS offset against PCR located where the beginning of that PES was seen (proposal explained in previous comment).
    * This keeps the relationship between DTS of various streams (ABUF5 is to be presented 150 time units after VBUF1)
    * By figuring out the smallest/maximum delay between the moment the beginning of a packet was received and the moment it was outputted from a demuxer, we effectively calculate the latency introduced by the demuxer.
    

  With the example values we calculate:

  The minimum offset between DTS and PCR:

    DTS diff for ABUF5 : DTS(ABUF5) - PCR0 = PCR0 + 500 - PCR0 = 500
    DTS diff for VBUF1 : DTS(VBUF1) - PCR2 = PCR2 + 300 - PCR2 = 300
    DTS_PCR_offset = MIN(DTS diff) = 300

  (Note: in the initial bug report, this is the 4h value)

  Running times:
    ABUF5 : DTS(ABUF5) - DTS_PCR_offset - PCR(ABUF5 was received)
          : PCR0 + 500 - 300            - PCR0
          : 150

    VBUF1 : DTS(VBUF1) - DTS_PCR_offset - PCR(VBUF1 was received)
          : PCR2 + 300 - 300            - PCR2
          : 0

  Latency introduced:
    Delay between moment beginnning of PES was received and it was outputted
    ABUF5 : Received at PCR0, outputted at PCR2:
          : PCR2 - PCR0
          : PCR0 + 50 - PCR0
          : 50
    VBUF1 : Not outputted yet, but if it was outputted at PCRX = PCR0 + 200 it would be
          : PCRX - PCR2
          : PCR0 - PCR0 + 200
          : 200
    We store those various min/max values so that we can answer the latency query if/when it arrives. We should have enough observations by the time it arrives for it to be coherent and stable.

    (For those paying really attention, you will notice we have effectively given enough room (with the maximum latency) to downstream so that the video buffer has enough time to be decoded in time).


Limitations:
  The example provided is somewhat idealized, there might be some corner cases where we need to output the buffer of one stream without having received the beginning of a PES on all streams and therefore can't figure out the MIN(DTS diff) accross all streams.
  To mitigate this we could:
    * Set some default minimum DTS diff depending on the nature of the stream (video, audio, subtitle....).
    * Or have a global default MIN(DTS diff)
    * Or delay pushing the buffers until we have seen at least one PES start on each stream (maybe not on the subtitle streams though). Those delayed buffers might end up arriving too late for display in the sinks, but at least would ensure downstream decoders/elements have all the data they need.

  I prefer the last solution fwiw. It could also fit well with the "Fast start dvb" proposal in bug #703884 .

And fixed with latest commits.

commit 2762ead5ef2a16885fbb6918dae9b8df836a361f
Author: Edward Hervey <edward@collabora.com>
Date:   Fri Jul 26 07:54:30 2013 +0200

    mpegtsdemux: New PCR<=>Offset estimation code
    
    This allows:
    * Better duration estimation
    * More accurate PCR location
    * Overall more accurate running-time location and calculation
    
    Location and values of PCR are recorded in groups (PCROffsetGroup)
    with notable PCR/Offset observations in them (when bitrate changed
    for example). PCR and offset are stored as 32bit values to
    reduce memory usage (they are differences against that group's
    first_{pcr|offset}.
    
    Those groups each contain a global PCR offset (pcr_offset) which
    indicates how far in the stream that group is.
    
    Whenever new PCR values are observed, we store them in a sliding
    window estimator (PCROffsetGroupCurrent).
    
    When a reset/wrapover/gap is detected, we close the current group with
    current values and start a new one (the pcr_offset of that new group
    is also calculated).
    
    When a notable change in bitrate is observed (+/- 10%), we record
    new values in the current group. This is a compromise between
    storing all PCR/offset observations and none, while at the same time
    providing better information for running-time<=>offset calculation
    in VBR streams.
    
    Whenever a new non-contiguous group is start (due to seeking for example)
    we re-evaluate the pcr_offset of each groups. This allows detecting as
    quickly as possible PCR wrapover/reset.
    
    When wanting to find the offset of a certain running-time, one can
    iterate the groups by looking at the pcr_offset (which in essence *is*
    the running-time of that group in the overall stream).
    Once a group (or neighbouring groups if the running-time is between two
    groups) is found, once can use the recorded values to find the most
    accurate offset.
    
    Right now this code is only used in pull-mode , but could also
    be activated later on for any seekable stream, like live timeshift
    with queue2.
    
    Future improvements:
    * some heuristics to "compress" the stored values in groups so as to keep
      the memory usage down while still keeping a decent amount of notable
      points.
    * After a seek compare expected and obtained PCR/Offset and if the
      difference is too big, re-calculate position with newly observed
      values and seek to that more accurate position.
    
    Note that this code will *not* provide keyframe-accurate seeking, but
    will allow a much more accurate PCR/running-time/offset location on
    any random stream.
    For past (observed) values it will be as accurate as can be.
    For future values it will be better than the current situation.
    Finally the more you seek, the more accurate your positioning will be.

And it obviously wasn't this bug I wanted to close. Sorry :(

Any progress here?

This requires more work. I'll tackle it once 703884 is done (unless someone else wants to tackle it).

703884 will help having more data (and therefore information) available before the first push.

> And fixed with latest commits.
> 
> commit 2762ead5ef2a16885fbb6918dae9b8df836a361f
> Author: Edward Hervey <edward@collabora.com>
> Date:   Fri Jul 26 07:54:30 2013 +0200
> 
>     mpegtsdemux: New PCR<=>Offset estimation code

This appears to have introduced bug #721035 .

Since the commit above, streams with no PCR cannot be played anymore. Here are two videos from my sample collection with PCR pid set to 0x1fff:

http://absolut.zogzog.org/share/samples/ts/Aladin.ts
http://absolut.zogzog.org/share/samples/ts/youtube-hls.ts

The second one is encoded by Google for Youtube when using HLS

Streams without PCR are not valid AFAIU... but we should still be able to make them do something at least.

This seems like a blocker bug.

Edward? What should we do with this?

I'd say there are two parts to solving this:

1) Figuring out if the stream really has no PCR present (regardless of whether it's specified by a PMT or not). This would need a time-based threshold to decide that (i.e. we've seen 1-2 seconds worth of PTS/DTS, but no PCR on any PID).

2) What do we use instead of PCR if really not present ? Some interpolated values from elementary stream DTS ?

Hi Edward,

1) the stream is served using HLS, which already has some timestamps at the playlist level. Those timestamps are not very precise, but they are precise enough to seek or compute the stream duration. That's why there's no PCR.

2) I have patches to at least allow the stream to start playing when the PCR pid is invalid, in both pull in push mode. I'm attaching them.

Created attachment 303209 [details] [review]
tsdemux: only wait for PCR when the PCR pid is set

Created attachment 303210 [details] [review]
tsdemux: do not abort playback when no PCR were found  during initial scan

We might also consider skipping the scan if the PCR pid is invalid.

There's still an issue though, which was already present before the regression: PTS are completely ignored when no PCR is available, so the video doesn't play at the correct rate.

Demoting this bug to major.

The issue with streams without PCR is here : https://bugzilla.gnome.org/show_bug.cgi?id=752843

Created attachment 329023 [details] [review]
tsdemux: handle pcr_pid == 0x1fff

I've come across the same problem. Playing streams with pcr_pid = 0x1fff works for me with this patch. And they are played at the correct rate.

Right now hlsdemux only sets the timestamp at the beginning because they may not be accurate. So doing more than just syncing the time at the beginning is currently not possible.

From what I've seen and from reading the spec[1], I think we can assume that the timestamps are accurate for version >= 3 and floating point values.
Then base_time and base_pcrtime could be updated whenever a buffer with a valid timestamp arrives.

Does that make sense?


[1] https://tools.ietf.org/html/draft-pantos-http-live-streaming-19#section-4.3.2.1

Review of attachment 329023 [details] [review]:

::: gst/mpegtsdemux/mpegtspacketizer.c
@@ +2190,3 @@
   pcrtable = get_pcr_table (packetizer, pcr_pid);
 
+  if (!GST_CLOCK_TIME_IS_VALID (pcrtable->base_time) && pcr_pid == 0x1fff &&

Using the input buffer time is a good idea to solve this issue.

But this will fail if there's a discontinuity in the input stream. We should set the base_time to GST_CLOCK_TIME_NONE again in mpegts_packetizer_{flush|clear} for this to work properly.

And hlsdemux provides a timestamp if there's a discontinuity so base_time is immediately set again with the new correct value.

Created attachment 329042 [details] [review]
tsdemux: handle pcr_pid == 0x1fff

updated with suggested changes.

Created attachment 331330 [details] [review]
mpegtsdemux: only wait for PCR when PCR pid is set

Streams without PCR make senses in HLS, where the playlist timestamps
can be used to seek or calculate the duration.

Created attachment 331331 [details] [review]
mpegtsdemux: do not abort playback when no PCR were found during initial scan

Seeking will not be possible in those streams but at least the can be
played. Note scanning is only done when tsdemux is configured in pull mode.

Created attachment 331332 [details] [review]
tsdemux: handle pcr_pid == 0x1fff

Streams without PCR make senses in HLS, where the playlist timestamps
can be used to seek or calculate the duration.

Created attachment 331333 [details] [review]
tsdemux: Fix GAP synchronization without a valid PCR PID

For streams that don't have a valid PCR PID, we just use the latest
PTS from any given stream.

Created attachment 331591 [details]
GoPro pipeline error

I testing live streaming from a GoPro Hero 4 camera over wifi. I originally observed the same problems as reported in this thread:

http://gstreamer-devel.966125.n4.nabble.com/Streaming-GoPro-with-GStreamer-1-0-td4674216.html

I then applied the last four patches posted here, and built the latest revision of GStreamer 1.9.1 from Git on Mac OSX 10.10 using Cerbero. The following pipeline 

gst-launch-1.0 -e -v udpsrc port=8554 ! queue ! decodebin ! videoconvert ! videoscale ! autovideosink

now seems to be processing the stream, however it still gives an error (see my last attachment for full log) and does not show the video sink window. 

However, one interesting thing is that if I quit the pipeline by pressing CTR-C while it is still launching (before the "libav :0:: error while decoding MB 6 12, bytestream -xx" lines come up), I can briefly see the live video from the camera for 1 second or two before the pipepline shuts down.

I testing live streaming from a GoPro Hero 4 camera over wifi. I originally observed the same problems as reported in this thread:

http://gstreamer-devel.966125.n4.nabble.com/Streaming-GoPro-with-GStreamer-1-0-td4674216.html

I then applied the last four patches posted here, and built the latest revision of GStreamer 1.9.1 from Git on Mac OSX 10.10 using Cerbero. The following pipeline 

gst-launch-1.0 -e -v udpsrc port=8554 ! queue ! decodebin ! videoconvert ! videoscale ! autovideosink

now seems to be processing the stream, however it still gives an error (see my last attachment for full log) and does not show the video sink window. 

However, one interesting thing is that if I quit the pipeline by pressing CTR-C while it is still launching (before the "libav :0:: error while decoding MB 6 12, bytestream -xx" lines come up), I can briefly see the live video from the camera for 1 second or two before the pipepline shuts down.

For some reason those streams have two video streams (a mpeg2 one with no frames, and a h264 one with the actual stream).

The reason you're seeing nothing at the beginning is because it's on the mpeg2 stream. When you abort the pipeline it switches to the h264 stream (notice how videoconvert receives 720p video all of a sudden).

Hi Edward, thanks for the reply (and sorry for the duplicated comment). 

Is there any way to set the specific video stream I want to demux? I went over some older threads (https://lists.freedesktop.org/archives/gstreamer-devel/2014-May/047689.html, https://lists.freedesktop.org/archives/gstreamer-devel/2015-May/052917.html) and looks like I should be able to do something like this:

gst-launch-1.0 -e -v udpsrc port=8554 ! queue ! decodebin ! autovideosink d.video_<h264pid> ! videoconvert ! videoscale ! autovideosink

where <h264pid> is the PID of the h264 stream. Does it make sense? If so, how can I get the PID for the streams?

Andres

commit 51c5ff45de81e96dbace2a872505274d7de0341c
Author: Edward Hervey <edward@centricular.com>
Date:   Wed Jul 13 11:02:23 2016 +0200

    tsdemux: Fix draining on wrong programs
    
    When draining a program, we might send a newsegment event on the pads
    that are going to be removed (and then the pending data).
    
    In order to do that, calculate_and_push_newsegment() needs to know
    what list of streams it should take into account (instead of blindly
    using the current one).
    
    All callers to calculate_and_push_newsegment() and push_pending_data()
    can now specify the program on which to act (or NULL for the default
    one).

commit e2b98a7721c7c81fd08813d1c575fc4ad81df38e
Author: Edward Hervey <edward@centricular.com>
Date:   Tue Jul 12 15:58:40 2016 +0200

    tsdemux: Fix GAP synchronization without a valid PCR PID
    
    For streams that don't have a valid PCR PID, we just use the latest
    PTS from any given stream.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=608148

commit 36e6590335ba0551f0e1882d58cf24429b2ec066
Author: Michael Olbrich <m.olbrich@pengutronix.de>
Date:   Tue May 31 13:09:43 2016 +0200

    tsdemux: handle pcr_pid == 0x1fff
    
    Streams without PCR make senses in HLS, where the playlist timestamps
    can be used to seek or calculate the duration.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=608148

commit 7a3d1892503370113938b60fee58efe99cde4cde
Author: Arnaud Vrac <avrac@freebox.fr>
Date:   Sun May 10 20:10:49 2015 +0200

    mpegtsdemux: do not abort playback when no PCR were found during initial scan
    
    Seeking will not be possible in those streams but at least the can be
    played. Note scanning is only done when tsdemux is configured in pull mode.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=608148

commit 35a1720de17b0cd6a902898336a0946692dc9e63
Author: Arnaud Vrac <avrac@freebox.fr>
Date:   Sat May 9 15:52:22 2015 +0200

    mpegtsdemux: only wait for PCR when PCR pid is set
    
    Streams without PCR make senses in HLS, where the playlist timestamps
    can be used to seek or calculate the duration.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=608148

I have a test video file（https://drive.google.com/file/d/0B5Qs0jIvnVNTM2xGTlprb3NZTFU/view?usp=sharing）

With ffplay, VLC playback is very smooth,but once with GStreamer there is no sound, and not smoothing.

I guess it should be related to the PCR<=>PTS conversion, but not sure, anyone could help confirm it? Because this file is not just an “exception”, many use ffmpeg to transcode as ts files have the same problem.

Andres Colubri and I are working on streaming a GoPro Hero4 via Gstreamer.
It seems that the problem we are having with Gstreamer is very similar to what Shakin described.

Capturing the GoPro via ffplay generates a very smooth video, as can be seen here: https://youtu.be/5rQiDtO_0so

However, while streaming the GoPro directly via Gstreamer, the video flickers.
As can be seen in the following video: https://youtu.be/L5D26-NUZnY

The artifacts appeared only on the bottom part of the video and mainly when there is a movement (of an object or the camera).

This is the pipeline I used to stream the GoPro:
$ gst-launch-1.0 -e -v --  udpsrc port=8554 ! tsdemux ! queue ! decodebin ! videoconvert ! videoscale ! autovideosink sync=false

Btw I managed to decrease the delay by adding sync=false to the pipeline.

Even though the pipeline above enables to process the video, errors are still being printed. See: https://docs.google.com/document/d/1XOseNnhE_rR16LJOoEsbFo3cEmvdv9D6ClqdOXeCBkY/edit?usp=sharing

We suspect that there is still something that doesn't function correctly.

Do you have any insights regarding how we can fix this? 

Thanks,
Gal and Andres

Gal, does this also happen if you directly write the UDP stream (without the demuxer, etc) to a file, and then play the file with GStreamer and/or ffplay?

Can you try increasing the UDP receive buffer size, e.g. with the buffer-size property on udpsrc or kernel settings? Make sure to select a value with buffer-size that is allowed by your kernel configuration (i.e. not far too high).

And last, can you also reproduce it when using GStreamer on a PCAP capture (e.g. from wireshark)? If so, can you share that capture?

(In reply to shakin@outlook.com from comment #36)
> I have a test video
> file（https://drive.google.com/file/d/0B5Qs0jIvnVNTM2xGTlprb3NZTFU/
> view?usp=sharing）
> 
> With ffplay, VLC playback is very smooth,but once with GStreamer there is no
> sound, and not smoothing.
> 
> I guess it should be related to the PCR<=>PTS conversion, but not sure,
> anyone could help confirm it? Because this file is not just an “exception”,
> many use ffmpeg to transcode as ts files have the same problem.

Add information!
I try to dig mpegtspacketizer.c：mpegts_packetizer_pts_to_ts（）’　code as follows:
--------
GstClockTime
mpegts_packetizer_pts_to_ts (MpegTSPacketizer2 * packetizer,
    GstClockTime pts, guint16 pcr_pid)
{
  GstClockTime res = GST_CLOCK_TIME_NONE;
  MpegTSPCR *pcrtable;
 .....

 
 if (refpcr != G_MAXINT64) {
/*************!!!!Modifid here!!!!*************************/
      res = pts;
//    res =
//        pts - PCRTIME_TO_GSTTIME (refpcr) + PCRTIME_TO_GSTTIME (refpcroffset);
/***********************************************************/    
    }
    else
      GST_WARNING ("No groups, can't calculate timestamp");
  } else
    GST_WARNING ("Not enough information to calculate proper timestamp");

  PACKETIZER_GROUP_UNLOCK (packetizer);

  GST_DEBUG ("Returning timestamp %" GST_TIME_FORMAT " for pts %"
      GST_TIME_FORMAT " pcr_pid:0x%04x", GST_TIME_ARGS (res),
      GST_TIME_ARGS (pts), pcr_pid);
  return res;
}
--------
The above modification is just an attempt not to make a PCR <=> PTS conversion, while using the original PTS,the result is more smoother playback than before.

Of course, this is just an attempt, i read mpegtspacketizer.c but don't get a deep understanding,so the real reason still is not clear.

(In reply to Sebastian Dröge (slomo) from comment #38)
> Gal, does this also happen if you directly write the UDP stream (without the
> demuxer, etc) to a file, and then play the file with GStreamer and/or ffplay?

Hi Sebastian,

I used the following pipeline that generated a file that included only audio (attached is the file - udp_dump.ts):
gst-launch-1.0 -e -v udpsrc port=8554 ! filesink location=udp_dump.ts async=false

We found that the following components are essential in order for the video to be shown from the GoPro:

gst-launch-1.0 -e -v --  udpsrc port=8554 ! tsdemux ! decodebin ! autovideosink

> Can you try increasing the UDP receive buffer size, e.g. with the
> buffer-size property on udpsrc or kernel settings? Make sure to select a
> value with buffer-size that is allowed by your kernel configuration (i.e.
> not far too high).
> 
> And last, can you also reproduce it when using GStreamer on a PCAP capture
> (e.g. from wireshark)? If so, can you share that capture?

I also tested the following while running Wireshark (see attached files), the change in the buffer size did not seem to affect the artifacts.

1. Gstreamer_buffer-size=1000000.pcap was reproduce while using the following pipeline:
gst-launch-1.0 -e -v --  udpsrc port=8554 buffer-size=1000000 ! tsdemux ! queue ! decodebin ! videoconvert ! videoscale ! autovideosink sync=false

2. Gstreamer_buffer-size=192000.pcap was reproduce while using the following pipeline:
gst-launch-1.0 -e -v --  udpsrc port=8554 buffer-size=192000 ! tsdemux ! queue ! decodebin ! videoconvert ! videoscale ! autovideosink sync=false

note: the 192000 was taken from this thread.

3. Wireshark_when_using_GStreamer.pcap as reproduce while using the following pipeline (that does not include "buffer-size"):
gst-launch-1.0 -e -v --  udpsrc port=8554 ! tsdemux ! queue ! decodebin ! videoconvert ! videoscale ! autovideosink sync=false

Thanks,
Gal

Created attachment 333507 [details]
Gstreamer_buffer-size=192000.pcap

Created attachment 333508 [details]
Gstreamer_buffer-size=1000000.pcap

Created attachment 333509 [details]
Wireshark_when_using_GStreamer.pcap

Created attachment 333510 [details]
udp_dump.ts

Edward, 

>> The reason why in mpeg-ts streams the PTS of the various 
>> audio and video streams can be so much higher than the 
>> co-located PCR is because it takes into account the maximum 
>> latency/buffering needed in demuxing/buffering/decoding/reordering 
>> in order for the target buffer to be properly displayed.
Can this be assumed? Shouldn't the buffering at the decoder side by independent of PTS-PCR gap? 
If we do PTS - first_pcr, it would invariablly introduce latency due to timestamp GAP. Assume 1second gap between PTS and PCR for H264 video. This would make the video timestamp to start from 1s leading to latency due to starting timestamp
Does it make sense to do PTS-first_seen_ES_PTS in such a case? first_seen_ES_PTS should be common across all the ES belonging to that program

>> Due to that difference, tsdemux needs to report not 
>> the end-to-end latency (PTS - PCR received at the same time) 
>> but only the min/max latency it introduces itself. This boils 
>> down to the interleaving/packetization latency. The other 
>> downstream elements (parsers, queue, decoders, ...) will add min/max 
>> latency specific to those streams and end up with the ideal 
>> capture-to-display latency
Ideally this says
1. txdemux should report only its latency(processing) and do not report about max interleaving latency of 700ms which might have been introduced due to difference in buffering of ES(Video needing lookahead/Bframes buffering )
2. Parsers should advertise buffering latency, something like vbv_delay or init_cbp_removal_delay
3. Decoder should report B pic reordering delay

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/16.