GNOME Bugzilla – Bug 779696
Videocompositor halts if PTS = none
Last modified: 2018-11-03 14:05:28 UTC
VideoAggregator errors when it receives a frame with PTS = TIME_NONE https://github.com/GStreamer/gst-plugins-bad/blob/08c52c931e5145a569da1efcd53d77e0090ec898/gst-libs/gst/video/gstvideoaggregator.c#L1128 Perhaps it should drop the frame instead and continue? === PS === The bad PTS, in my case, is coming from rtpbasedepayload, I think. I have found a case where it will emit frames out of order and then will emit multiple frames with no PTS, one sync and another non-sync. I don't know the root cause, but here is a log showing the input to gstvideodecoder: 0:00:31.022978000 84684 0x7fd5c808e280 WARN videodecoder gstvideodecoder.c:2767:void gst_video_decoder_prepare_finish_frame(GstVideoDecoder *, GstVideoCodecFrame *, gboolean):<vtdec0> decreasing timestamp (0:00:27.235867490 < 0:00:27.258838694) 0:00:31.055632000 84684 0x7fd5c808e280 DEBUG videodecoder gstvideodecoder.c:2730:void gst_video_decoder_prepare_finish_frame(GstVideoDecoder *, GstVideoCodecFrame *, gboolean):<vtdec0> invalidating PTS 0:00:31.056315000 84684 0x7fd5c808e280 LOG videodecoder gstvideodecoder.c:2416:GstFlowReturn gst_video_decoder_chain(GstPad *, GstObject *, GstBuffer *):<vtdec0> chain PTS 99:99:99.999999999, DTS 0:00:26.953637406 duration 99:99:99.999999999 size 5007 0:00:31.056345000 84684 0x7fd5c808e280 LOG videodecoder gstvideodecoder.c:3374:GstFlowReturn gst_video_decoder_decode_frame(GstVideoDecoder *, GstVideoCodecFrame *):<vtdec0> PTS 99:99:99.999999999, DTS 0:00:26.953637406, dist 300 0:00:31.056840000 84684 0x7fd5c808e280 LOG videodecoder gstvideodecoder.c:2416:GstFlowReturn gst_video_decoder_chain(GstPad *, GstObject *, GstBuffer *):<vtdec0> chain PTS 99:99:99.999999999, DTS 0:00:26.956768313 duration 99:99:99.999999999 size 3546 0:00:31.056870000 84684 0x7fd5c808e280 LOG videodecoder gstvideodecoder.c:3374:GstFlowReturn gst_video_decoder_decode_frame(GstVideoDecoder *, GstVideoCodecFrame *):<vtdec0> PTS 99:99:99.999999999, DTS 0:00:26.956768313, dist 301 0:00:31.057982000 84684 0x7fd5c808e280 LOG videodecoder gstvideodecoder.c:2612:void gst_video_decoder_prepare_finish_frame(GstVideoDecoder *, GstVideoCodecFrame *, gboolean):<vtdec0> finish frame 0x7fd5c632ad00 (#300) sync:1 PTS:99:99:99.999999999 DTS:0:00:26.953637406 0:00:31.058014000 84684 0x7fd5c808e280 WARN videodecoder gstvideodecoder.c:2767:void gst_video_decoder_prepare_finish_frame(GstVideoDecoder *, GstVideoCodecFrame *, gboolean):<vtdec0> decreasing timestamp (0:00:26.953637406 < 0:00:27.258838694) 0:00:31.058135000 84684 0x7fd5c808e280 WARN videodecoder gstvideodecoder.c:2767:void gst_video_decoder_prepare_finish_frame(GstVideoDecoder *, GstVideoCodecFrame *, gboolean):<vtdec0> decreasing timestamp (0:00:27.257010699 < 0:00:27.258838694) 0:00:31.058940000 84684 0x7fd5c808e280 LOG videodecoder gstvideodecoder.c:2612:void gst_video_decoder_prepare_finish_frame(GstVideoDecoder *, GstVideoCodecFrame *, gboolean):<vtdec0> finish frame 0x7fd5c6372a20 (#301) sync:0 PTS:99:99:99.999999999 DTS:0:00:26.956768313 0:00:31.058987000 84684 0x7fd5c808e280 DEBUG videodecoder gstvideodecoder.c:3174:GstFlowReturn gst_video_decoder_clip_and_push_buf(GstVideoDecoder *, GstBuffer *):<vtdec0> pushing buffer 0x7fd5c63722b0 of size 115200, PTS 99:99:99.999999999, dur 99:99:99.999999999 0:00:31.059808000 84684 0x7fd5c808e280 WARN videodecoder gstvideodecoder.c:2767:void gst_video_decoder_prepare_finish_frame(GstVideoDecoder *, GstVideoCodecFrame *, gboolean):<vtdec0> decreasing timestamp (0:00:27.169113426 < 0:00:27.258838694) 0:00:31.061337000 84684 0x7fd5c808e280 WARN videodecoder gstvideodecoder.c:2767:void gst_video_decoder_prepare_finish_frame(GstVideoDecoder *, GstVideoCodecFrame *, gboolean):<vtdec0> decreasing timestamp (0:00:27.127792885 < 0:00:27.258838694)
The biggest question is *why* it receives a buffer without PTS. It really shouldn't. And just dropping it is not ideal because you might then wait forever without anything happening (if all buffers have no PTS).
gstvideodecoder.c tries to populate PTS if there isn't one, setting it to DTS if its a sync frame. But it wont do anything if it's a non-sync frame. If you're asking why rtpbasedepayload is producing a pts=none (I don't think it's the client, but i'm not 100% sure) it's probably some complicated case.
I believe the root cause is: gstbaseparser sets pts = next_pts for each frame, next_pts = TIME_NONE and then next_pts = pts if prev_pts != pts that's almost always ok, but sometimes the prev_pts == pts and next_pts is invalid, setting the pts to TIME_NONE. Patch attached
Created attachment 347492 [details] [review] Never set pts to TIME_NONE if you can avoid it
No, that doesn't seem valid. When you're parsing a video stream, receiving a new buffer with PTS = None doesn't mean it has the same timestamp as whatever you received last - it means you don't know the PTS.
Here are some logs indicating a bad next_pts overwriting a "good" pts: gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 501942001 and next pts 501942001 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 535195675 and next pts 535195675 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 568393657 and next pts 568393657 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 601475594 and next pts 601475594 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 738069646 and next pts 738069646 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 771377429 and next pts 771377429 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 804628617 and next pts 804628617 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 837769186 and next pts 837769186 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 870602270 and next pts 870602270 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 902843131 and next pts 902843131 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 934552643 and next pts 934552643 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 965078179 and next pts 965078179 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 993999635 and next pts 993999635 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1022294711 and next pts 1022294711 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1048467481 and next pts 1048467481 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1072964300 and next pts 1072964300 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1096290794 and next pts 1096290794 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1116381992 and next pts 1116381992 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1132809266 and next pts 1132809266 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1146795892 and next pts 1146795892 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1156561816 and next pts 1156561816 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1164412026 and next pts 1164412026 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1172541275 and next pts 1172541275 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1178266406 and next pts 1178266406 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1179784228 and next pts 1179784228 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1179784228 and next pts 18446744073709551615 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1179784228 and next pts 18446744073709551615 gstbaseparse.c:3222:gst_base_parse_chain:<h264parse0> Existing pts 1179784228 and next pts 18446744073709551615 An alternative solution is: if (parse->priv->next_pts != GST_CLOCK_TIME_NONE) GST_BUFFER_PTS (tmpbuf) = parse->priv->next_pts; I don't exactly know why there are four identical PTS' in a row, but at this point, setting them to none seems to make them worse.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/529.