After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 796862 - Videomixer fails with assert if its sink pad offset is set to negative value and the absolute value of offset is greater than pipeline running time
Videomixer fails with assert if its sink pad offset is set to negative value ...
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
1.14.2
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2018-07-24 11:50 UTC by Yury Mukhitov
Modified: 2018-11-03 15:31 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Test application pipeline I’m running (48.95 KB, image/png)
2018-07-24 16:12 UTC, Yury Mukhitov
Details

Description Yury Mukhitov 2018-07-24 11:50:03 UTC
Expected behavior: Setting pad offset to negative value usually results in media stream being "shifted forward" by the amout of offset.

Observed behavior: If setting pad offset to negative value which absolute value is greater than the pipeline running time videomixer fails with assert.

Pipeline structure:
$ gst-launch-1.0 videotestsrc ! videomixer ! ximagesink

How to reproduce:
Have pipeline running for N seconds then set videomixer's pad offset to -(N+1) seconds (e.g. to skip some video frames).

I have an example application which reproduces this problem.

Regards,
Yury

=============================
GStreamer console output:
=============================
ERROR:videomixer2.c:961:gst_videomixer2_fill_queues: assertion failed: (start_time != -1 && end_time != -1)

=============================
Application backtrace:
=============================
  • #0 raise
  • #1 abort
  • #2 g_assertion_message
  • #3 g_assertion_message_expr
  • #4 gst_videomixer2_fill_queues
    at videomixer2.c line 961
  • #5 gst_videomixer2_collected
    at videomixer2.c line 1285
  • #6 gst_collect_pads_check_collected
    at gstcollectpads.c line 1368
  • #7 gst_collect_pads_chain
    at gstcollectpads.c line 2216
  • #8 gst_pad_chain_data_unchecked
    at gstpad.c line 4205
  • #9 gst_pad_push_data
    at gstpad.c line 4457
  • #10 gst_pad_push
    at gstpad.c line 4576
  • #11 gst_base_src_loop
    at gstbasesrc.c line 2854
  • #12 gst_task_func
    at gsttask.c line 334
  • #13 default_func
    at gsttaskpool.c line 68
  • #14 g_thread_pool_thread_proxy
  • #15 g_thread_proxy
  • #16 start_thread
  • #17 clone

Comment 1 Yury Mukhitov 2018-07-24 11:53:00 UTC
An offset is being applied to videomixer's sink pad.
Comment 2 Tim-Philipp Müller 2018-07-24 12:00:51 UTC
Could you also try with the "compositor" element instead, it replaces videomixer.
Comment 3 Yury Mukhitov 2018-07-24 16:12:10 UTC
Created attachment 373147 [details]
Test application pipeline I’m running
Comment 4 Yury Mukhitov 2018-07-24 16:13:13 UTC
I’ve drop-in replaced ‘videomixer’ with ‘compositor’. The problem persists.
Please see test application pipeline I’m running (testapp_pipeline.png, attachment 373147 [details]).

=============================
GStreamer console output:
=============================
ERROR:gstvideoaggregator.c:1192:gst_video_aggregator_fill_queues: assertion failed: (start_time != -1 && end_time != -1)

=============================
Application backtrace:
=============================
  • #0 raise
  • #1 abort
  • #2 g_assertion_message
  • #3 g_assertion_message_expr
  • #4 gst_video_aggregator_fill_queues
    at gstvideoaggregator.c line 1192
  • #5 gst_video_aggregator_aggregate
    at gstvideoaggregator.c line 1532
  • #6 gst_aggregator_aggregate_func
    at gstaggregator.c line 1127
  • #7 gst_task_func
    at gsttask.c line 332
  • #8 default_func
    at gsttaskpool.c line 69
  • #9 g_thread_pool_thread_proxy
  • #10 g_thread_proxy
  • #11 start_thread
  • #12 clone

Comment 5 Yury Mukhitov 2018-07-24 16:26:40 UTC
More information on what I’m trying to achieve.

The application is processing several video streams in parallel (reading VP8 video from rtp stream files). All streams get decoded then routed to ‘videomixer’ (or ‘compositor’ as Tim-Philipp Müller suggested) to create single video stream with grid-like layout, each video in separate grid cell (position). At some point at pipeline playback there is application-level (out-of-band) ‘pause’ event of arbitrary duration. This event signifies that some portion of media has to be skipped.

To implement the pause I update all ‘videomixer’ (or ‘compositor’) sink pads offset to negative value abs()-equal to the ‘pause’ duration. This (from my understanding) commands ‘videomixer’ to discard some video frames treating those being ‘late’.

If pause happens to occur early in pipeline playback and pause duration is longer than current pipeline running time then ‘videomixer’ (and ‘compositor’) element does an assert as described above.

Can you please hint me if the approach ‘pause’ handling I’m implementing is a reasonable one?

(The ‘videotestsrc->‘videomixer’ (compositor)->’ximagesink’ pipeline above is a simplified pipeline to demonstrate the problem I face.)
Comment 6 Yury Mukhitov 2018-07-25 14:19:35 UTC
>> if the approach ‘pause’ handling I’m implementing is a reasonable one

In other words, is this a proper (correct) use of videomixer's pad offset? Are there any limitations on sink pad offset values (both positive and negative) besides (1) possibility of int64_t overflow and (2) application-specific validity of buffer timesetamp values?
Comment 7 Sebastian Dröge (slomo) 2018-07-26 10:53:18 UTC
I didn't look at the details here, but whatever you set as pad offset should never ever cause assertions anywhere. If anything it should cleanly error out, but here it should actually just work if I understand correctly what you're doing.

What should happen is that all values before the pipeline running time are simply dropped/clipped (they're in the past after all).
Comment 8 GStreamer system administrator 2018-11-03 15:31:57 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/492.