GNOME Bugzilla – Bug 749517
aggregator: need support for sparse streams
Last modified: 2015-11-05 16:16:15 UTC
gstaggregator won't start until all pads have a buffer which is a shame really since it can handle all sorts of events (pads added/removed, format changes, ...). I quickly hacked gst_aggregator_check_pads_ready() to make it return true if at least one pad has a buffer and was happy to see it start in a use case where it previously was not. I can give more details about the use case/pipeline. My use case has 2 branches feeding into a GstCompositor element. Both branches do not start at the same time but I would like the pipeline to start anyways. With the hack mentioned above the pipeline will start in any case. But depending on which branch starts first, I get good or bad results. I think it is due to clock or latency being improperly handled. I am quite new to gstreamer development so I am seeking guidance/hints/help on how to improve gstaggregator so it can start whenever one pad has a buffer.
The reason for the waiting is that otherwise it depends on the OS's scheduler if frames of different streams are properly overlayed together or not. If your inputs are live inputs, this shouldn't be a problem because of the live handling in aggregator (not available in 1.4, only git master). If your inputs are not live inputs, you should make them produce data immediately and if e.g. the second one should be overlayed much later you would control that with the timestamps (or more precise with the running time).
What are the two input streams you have?
A related discussion can be found here : http://gstreamer-devel.966125.n4.nabble.com/videomixer-with-videotestsrc-and-udpsrc-td4671726.html
This is a minimalistic and testable version of the target pipeline: compositor name=mixer sink_1::ypos=50 ! videoconvert ! timeoverlay shaded-background=true auto-resize=false ! autovideosink videotestsrc ! video/x-raw, width=320, height=240, framerate=20/1 ! mixer. udpsrc port=9000 ! identity dump=false ! textrender halignment=left line-alignment=left ! video/x-raw, width=320, height=120 ! videorate drop-only=true ! video/x-raw, framerate=10/1 ! mixer. The goal is to overlay generated gfx (here some text) over a live video source (here a videotstsrc). The gfx generator (here a textrender) consumes data buffers and turns them into a video buffer. To send pango text packets to the udpsrc use this command line : echo -n '<span color="blue">Hello</span>' | nc -4u -w1 127.0.0.1 9000 As stated, the pipeline will not start (i.e. display the live source) until a first packet is sent to the udpsrc.
It is going to work better if you set is-live=true on videotestsrc, to make it a real live source. However the problem here is that aggregator has no real support for sparse streams yet, like subtitle/text streams.
Apparently compositor (in 1.7.0) now starts whenever it receives a first buffer from any branch.
PS: Some docs might be a bit obsolote. in gstaggregator.c the following seems to be not true anymore : * When data is queued on all pads, tha aggregate vmethod is called. in gstvideoaggregator.c the following depends on what the derived classes do : * VideoAggregator can accept AYUV, ARGB and BGRA video streams. For each of the requested * sink pads it will compare the incoming geometry and framerate to define the * output parameters. Indeed output video frames will have the geometry of the * biggest incoming video stream and the framerate of the fastest incoming one. in compositor.c more formats are handled : * Compositor can accept AYUV, ARGB and BGRA video streams. For each of the requested * sink pads it will compare the incoming geometry and framerate to define the * output parameters. Indeed output video frames will have the geometry of the * biggest incoming video stream and the framerate of the fastest incoming one.
Can you file a separate bug ? Some correction, that's 1.6, and there is 3 start modes.
Created obsolete doc issue : https://bugzilla.gnome.org/show_bug.cgi?id=757642