GNOME Bugzilla – Bug 727180
videomixer: Unexpected behaviour when scaling after the mixer
Last modified: 2014-08-25 07:37:31 UTC
GStreamer 1.2.3 With gstreamer 1.0 and the videomixer plugin the scaling doesn't seem to be negotiated correctly. e.g. Works as expected (no videomixer element): gst-launch-1.0 -v ximagesrc ! videoscale ! video/x-raw,width=600,height=300 ! ximagesink Add videomixer element now it doesn't scale to the width/height: gst-launch-1.0 -v ximagesrc ! videomixer name=mix ! videoscale ! video/x-raw,width=600,height=300 ! ximagesink This works in the expected way if I use gstreamer-0.1 gst-launch-0.10 -vv ximagesrc ! videomixer name=mix ! videoscale ! video/x-raw-rgb,width=600,height=300 ! ximagesink
Can you provide more details on what you are trying to do? The videomixer behavior seems to make sense to me. Maybe if you give the bigger picture of your use case we can understand what is the real issue.
What details would you like me to provide? I am trying to scale the output of the videomixer element to 600x300. Without the mixer element in the middle, it works as intended. With the video mixer element in the pipeline, the image is being cropped instead. It's for the insight-recorder project https://github.com/insight-recorder/insight-recorder/
I think the behaviour is correct. videomixer always generates output with the size that is requested via caps downstream of videomixer. And into the size it places the input videos (of which you can modify the locations inside the output video). If you want scaling you either need to do it before videomixer, or explicitly after videomixer: ... ! videomixer ! "video/x-raw,width=300,height=150" ! videoscale ! "video/x-raw,width=600,height=300" ! ...
You said "or explicitly after the videomixer" which is what I'm doing: > Add videomixer element now it doesn't scale to the width/height: > gst-launch-1.0 -v ximagesrc ! videomixer name=mix ! videoscale ! video/x-raw,width=600,height=300 ! ximagesink (The above produces the cropped output) I'm fairly confused now, would you be able to provide a pipeline that has two src elements into the videomixer; v4l2src and ximagesink with the composited output scaled to 600 by 300.
See the pipeline part I wrote in the last line of comment 3 :) That forces videomixer to output 300x150 and the videoscale then scales that up.
Well, I'm really sorry to insist. Maybe I haven't been clear enough. I don't want to resize the surfaces before composition. I'd like to compose at full resolution, and then scale the composed surface. I may be able to change how the software works, but I believe that gstreamer not being about to do both is a bug. By doing both I mean: 1/ compose a full resolution, then scale 2/ scale the input surface, then compose What you're proposing is 2/. How would I go about doing 1/?
For 1/, what is the "full resolution"? What I described allows you to compose at a resolution of your choice and then scale the composition.
If you mean with full resolution the maximum of all input streams as layed out on the composited surface, then you'll currently have to negotiate that. I think if downstream allows everything, then videomixer tries to use that currently. If downstream has a specific preference (like in the above cases), it will use that instead. Maybe videomixer should always use this maximum if it is supported by downstream, even if it has a different preference.
(In reply to comment #8) > If you mean with full resolution the maximum of all input streams as layed out > on the composited surface, then you'll currently have to negotiate that. The documentation says: "[The] output video frames will have the geometry of the biggest incoming video stream". So I think that's well defined and indeed what I expected. > I think if downstream allows everything, then videomixer tries to use that > currently. If downstream has a specific preference (like in the above cases), > it will use that instead. Right, so if downstream accepts everything, we'll have the maximum of all input stream as output size. That works. Now when we put a videoscale + capsfilter after videomixer, that's what this bug is about. The expected behaviour is for videomixer to work with the maximum size of all input stream, and then have videoscale take that buffer and scale it down. Currently, the resulting buffer is a crop of the output of videomixer. > Maybe videomixer should always use this maximum if it is supported by > downstream, even if it has a different preference. That may be the "bug" indeed. Yes, I think videomixer should always use that maximum for its sink pad. I did a quick test to force that behaviour and the results is what I expected (and how it worked in 0.10): Here, ximagesrc outputs 3926x1440 frames. Forcing the output caps of videomixer does the trick. I believe it should be the default behaviour (ie using that maximum size as output caps). gst-launch-1.0 -vv ximagesrc ! videomixer name=mix ! video/x-raw,width=3926,height=1440 ! videoscale ! video/x-raw,width=400,height=300 ! ximagesink
commit ce1d4d9f21e0681f3c6025ffb27b1ae625329b11 Author: Sebastian Dröge <sebastian@centricular.com> Date: Thu Aug 14 16:36:44 2014 +0300 videomixer: Use the best width/height/etc if downstream can handle that Before it was always using whatever downstream preferred, while the code and documentation claimed something different. https://bugzilla.gnome.org/show_bug.cgi?id=727180