GNOME Bugzilla – Bug 777475
compositor : Issue in computing position when pixel-aspect-ratio is not 1/1
Last modified: 2017-01-19 11:24:01 UTC
Compositor element seems to stretch/compress the video based on pixel-aspect-ratio, and then use xpos and ypos on the stretched/compressed video size. This looks wrong to me and like a bug. Can someone confirm pls? For example, if video is 100x100 par of 2/1, and I want to position another 100x100 adjacent to it, second 100x100 and xpos=100 and ypos=0, this second video gets positioned from the center of first video and resultant video is 300x100 Pipeline: gst-launch-1.0 videotestsrc pattern=0 ! video/x-raw,width=100,height=100,pixel-aspect-ratio=2/1 ! comp. videotestsrc pattern=18 ! video/x-raw,width=100,height=100,pixel-aspect-ratio=2/1 ! comp. compositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=100 sink_1::ypos=0 ! videoconvert ! autovideosink -v No composition also stretches the video gst-launch-1.0 videotestsrc pattern=0 ! video/x-raw,width=100,height=100,pixel-aspect-ratio=2/1 ! compositor name=comp sink_0::xpos=0 sink_0::ypos=0 ! videoconvert ! autovideosink -v Setting pixel aspect ratio downstream to 1/1 doesnt seem to help either
pixel-aspect-ratio=2/1 by definition is a stretched video and the lack of a pixel-aspect-ratio on the output caps (resulting in a default of 1/1), the second pipeline is behaving as expected, i.e. 100x100 at a PAR of 2/1 is transformed into 200x100 at a PAR of 1/1. It's the exact same pipeline/functionality as: videotestsrc ! video/x-raw,pixel-aspect-ratio=2/1 ! videoscale ! autovideosink As for positioning in the first pipeline, the positioning happens in relation to the output position of which the input has been scaled according to the width/height/PAR into the output width/height/PAR. In the first pipeline example, this manifests in the following way. 1. output PAR is 1/1 2. the output size for the first stream is calculated from the width/height and PAR to be 200x100 in the output canvas which is placed at 0,0. 3. the output size for the second stream is calculated from the width/height and PAR to be 200x100 in the output canvas which is placed at 100,0. Thus output size of 300x100 and the streams overlapping. This is calculated here: https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst/compositor/compositor.c#n217 and the same process is undertaken by any video sink that deals with non 1/1 PAR. As such, I don't think there is a bug here.
Thank you for the review. Sorry if this sounds naive. But should not the position be re-calculated based on stretched width and height i.e., the position too should be computed as follows actual_pos_x = pos_x * par_n / par_d; actual_pos_y = pos_y * par_n / par_d; And use actual_pos_x and actual_pos_y for all pixel based operations? The reasoning behind the change is that, a user may not be aware of pixel-aspect-ratio which typically comes in stream header. But the user would be more aware of video resolution. It could be difficult for the user to adjust pos_x and pos_y based on incoming stream's par. If the incoming stream keeps changing its par dynamically(sometimes seen during content to ad transition in TS streams), this will make it even more difficult Thanks
The position is based on the outgoing PAR, which will not change. You position the inputs somewhere in the output frames based on the coordinate system of the output frame.
Thanks for the feedback. I understand that the position is based on output par Should the user know incoming PAR well in advance so that he can set output_par = input_par, in order the position to be as per coordinate system? Is this is a bug or by design Pipeline1 : Positions wrongly gst-launch-1.0 videotestsrc pattern=0 ! video/x-raw,width=100,height=100,pixel-aspect-ratio=2/1 ! comp. videotestsrc pattern=18 ! video/x-raw,width=100,height=100,pixel-aspect-ratio=2/1 ! comp. compositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=100 sink_1::ypos=0 ! video/x-raw,pixel-aspect-ratio=1/1 ! videoconvert ! autovideosink Pipeline2 : Positions perfectly gst-launch-1.0 videotestsrc pattern=0 ! video/x-raw,width=100,height=100,pixel-aspect-ratio=2/1 ! comp. videotestsrc pattern=18 ! video/x-raw,width=100,height=100,pixel-aspect-ratio=2/1 ! comp. compositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=100 sink_1::ypos=0 ! video/x-raw,pixel-aspect-ratio=2/1 ! videoconvert ! autovideosink For pipeline2 to work, I need to force output_par same as input_par. I may not know input-par well in advance. And if input_par changes on the fly? Then output par too needs to be forced dynamically
The output caps (and thus the par) can't change dynamically after configured once. You as the user have to know what you want to configure there, or let it automatically configure something if you don't care (but you have to care if you want to position things, you also need to know the resolution for example then!). I think this is all by design and makes sense as is.
It is by design then! Thank you very much for the confirmation Summarizing for future reference 1. Know the resolution(obviously) and input_par in advance for the position to be accurate 2. If input par changes dynamically, dynamically change posx and posy property of the pad, for the coordinate position to remain unchanged Thanks