GNOME Bugzilla – Bug 639506
xvimagesink: take into account XvQueryImageAttributes width/height parameters
Last modified: 2018-05-04 12:43:48 UTC
Created attachment 178305 [details] Correct frame from mplayer (buffer 640x480 for 636x470 image, frame00064.raw) and incorrect from totem (claimed as if the buffer was 640x480, but indeed is 636x470, frame00377.raw) According to XV driver spec (can be found at http://www.xfree86.org/current/DESIGN16.html), function QueryImageAttributes allows driver to specify how the buffer should be organized for the images in later PutImage calls. Eg. if video is of 636x470 size and the driver sets width and height in QueryImageAttributes to 640x480, the driver should put the image in such a format in PutImage (parameter buf). It works in mplayer/smplayer, but does not work in totem. Totem says (parameters width and height of PutImage) that the buffer is of 640x480, but the image has width 636 and the buffer also 636. Id of the format in QueryImageAttributes: 0x30323449 i.e. I420 according to FourCC.org (8 bit Y and 2x2 sub-sampled U and V planes). I am attaching frames from mplayer and totem for the same movie with the same QueryImageAttributes width=640 and height=480. In totem, the video is skewed, however mplayer handles the attributes correctly. Frames can be opened in gimp in raw mode and specifying appropriate width and height - the frame from totem is skewed for 640x480 and looks correctly only when width is set to 636.
When I do not alter the width and height in QueryImageAttributes, Totem does not want to play the video and displays message box: "Failed to create output image buffer of 636x470".
I don't know how I'm supposed to open the file, or what I'm supposed to do with them. Totem itself never calls XVQueryImageAttributes() or anything like that. Punting to GStreamer.
When I start playing the movie with totem, totem uses the XV driver and calls the XV QueryImageAttributes. I am developing a XV driver for linux (for NComputing's device - it does not matter here) and I see the calls in debug log (Xorg.XXX.log). Maybe totem does not call it directly but the effect is as I described. Regarding the files they can be opened in such a way: * open Gimp * File - Open... * choose mask: All files * select file eg frame00377.raw * choose file type: Image data Raw * the image is displayed * choose image type: RGB (image will be displayed as gray and trippled as Y will be treated as RGB) * select width 640 and height 480 (image is skewed) * only when width is 636, image is correct what proves that the buffer is not aligned to 640 I need to enforce the width to be multiplication of 16 (due to 420 2:2 subsampling and DCT size in JPEG what gives 2 * 8), but it looks like it is not possible only with totem.
You'll probably have the same problem running: gst-launch playbin2 uri=file:///path/to/file which would point to a problem with the xvimagesink in GStreamer, rather than Totem.
The code that uses Xv is here: http://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/sys/xvimage
Yes, the problem is the same with gst-launch. After investigation of the example frame of 636x470 forced to be placed in buffer 640x480, I see: Y: image: 636*470 (298920), buffer 636*470 (should be 640*480) U: image: offset 298920 - 318*235, buffer 320*235 (should be 320*240) V: image: offset 374120 - 318*235, buffer 320*235 (should be 320*240) (easy to see if you select "Indexed" in Gimp instead of "RGB"). I'll look into the code why it is so happening, however gstreamer makes many calls (first for 1x1 with driver answer 16x16, and later 636x470 with answer 640x480): XVQueryImageAttributes enter (id=808596553 [1x1]) XVQueryImageAttributes enter (id=808596553 [636x470]) XVQueryImageAttributes enter (id=808596553 [636x470]) XVQueryImageAttributes enter (id=808596553 [636x470]) XVQueryImageAttributes enter (id=808596553 [640x480]) XVQueryImageAttributes enter (id=808596553 [640x480]) XVPutImage ([636x470]/[640x480](0,0)=>[636x470](303,551)) // img/buf XVQueryImageAttributes enter (id=808596553 [636x470]) XVQueryImageAttributes enter (id=808596553 [640x480]) XVPutImage ([636x470]/[640x480](0,0)=>[636x470](303,551)) XVQueryImageAttributes enter (id=808596553 [636x470]) XVQueryImageAttributes enter (id=808596553 [636x470]) XVQueryImageAttributes enter (id=808596553 [636x470]) XVQueryImageAttributes enter (id=808596553 [636x470]) XVQueryImageAttributes enter (id=808596553 [640x480]) XVPutImage ([636x470]/[640x480](0,0)=>[636x470](303,551)) XVQueryImageAttributes enter (id=808596553 [640x480]) XVPutImage ([636x470]/[640x480](0,0)=>[636x470](303,551)) ... no more changes later - i.e. 1 QueryImageAttributes and 1 PutImage
GStreamer doesn't use XVQueryImageAttributes() for historical reasons that are mostly no longer relevant. Back in the day, only one driver implemented XVQueryImageAttributes(), and did so incorrectly. It was noted at the time that all existing hardware and drivers expected the same memory layouts, which not coincidentally were exactly the same layouts that DirectShow used. So we standardized on that layout, and it's still part of the ABI today. In 0.11, we're planning to remove the layout restriction. For now, it would probably be best to make the driver handle the standard strides, even if it means copying the data. 99% of the time, it won't matter, since almost all video comes in a width divisible by 16, although the height often isn't. (And it's only the small sizes that are problematic.)
Thank you for explanation - I have tested QueryImageAttributes with standard strides and finally I got the same layout for gstreamer and mplayer. I found that QueryImageAttributes is called by XvCreateImage in libxv - so it is called indirectly. Gstreamer also does not ignore values in QueryImageAttributes, as when I do not change the width/height, pithes (YUV) and offsets in XVQueryImageAttributes, I get from gstreamer/totem an error "Failed to create output buffer" i.e. in case they are not aligned to standard strides like 640x470, 318x235, 318x235 (instead of 640x470, 320x235, 320x235, which work). Nevertheless, I'll implement it with standard strides and instead of copying the image data I'll probably truncate the movie width.
It looks like it is possible to make JPEG compression of YUV image with width not divisible by 16, without copying or truncating the data - so it is not a big problem.
I'm marking this as WONTFIX, but really that means "We're going to fix this in 0.11".
While working on an X driver under similar circumstances, I hit this too. As of gstreamer-plugins-base-1.0.7 the info returned by QueryImageAttributes is still not honoured. For now I will tweak the app to use more standard resolutions to avoid the alignment issues I was trying to solve.
So let's reopen that and fix it in 1.0
There was some refactoring of the image allocation, but I'm not sure if it fixed this issue. Has anyone had a chance to test with a recent GStreamer version?
Closing this bug report as no further information has been provided. Please feel free to reopen this bug report if you can provide the information that was asked for in a previous comment. Thanks!
(In reply to David Schleef from comment #7) > GStreamer doesn't use XVQueryImageAttributes() for historical reasons that > are mostly no longer relevant. Just a little context, Glamor XV driver, which is relatively recent, have bugs in the QueryImageAttributes implementation that was still being workaround by this in GStreamer. Not using this call though, have help hiding X11 bugs for years. There was a bug report recently, saying that resolution 905x720 didn't work (it's probably still broken on your PC), and it's quire related.