GNOME Bugzilla – Bug 748163
crop handling in gl plugins (gleffect, glimagesink, glshader, .....)
Last modified: 2018-11-03 11:37:04 UTC
Is there a plan to support video cropping in gl plugins? Although this can be done when you uploading the pixel data into texture, there is still cases that you can use some extension to bind a buffer to the texture, such as eglimage, dma-bufer or some private extensions. If decoder needs padding or alignment when output video into this buffer, the gl plugins need to handle it correctly. I tried to support it, but have below question: 1. How to get the video buffer width and height? I can get crop info from GstVideoCropMeta, but how can I get the buffer width and height? We can get real video width/height from GstVideoInfo, but can't get the buffer widht/height after padding/aligned. One way I found is like below for I420 and NV12 video: videometa = gst_buffer_get_video_meta (buf); bufw = videometa->stride[0]; bufh = videometa->offset[1] / bufw; Is there a better way to get this info? 2. Supporting crop handling only need to recalculate the texture coordinate, but I found every gl plugin has their own vertices definition, this means I need to modify everywhere to support this. Is there a plan to make it into a common module?
(In reply to Jian Li from comment #0) > Is there a plan to support video cropping in gl plugins? > Although this can be done when you uploading the pixel data into texture, > there is still cases that you can use some extension to bind a buffer to the > texture, such as eglimage, dma-bufer or some private extensions. If decoder > needs padding or alignment when output video into this buffer, the gl > plugins need to handle it correctly. GstVideoAlignment is more suited to any data alignment constraints which GLMemory already supports in master. > I tried to support it, but have below question: > > 1. How to get the video buffer width and height? > > I can get crop info from GstVideoCropMeta, but how can I get the buffer > width and height? We can get real video width/height from GstVideoInfo, but > can't get the buffer widht/height after padding/aligned. One way I found is > like below for I420 and NV12 video: > > videometa = gst_buffer_get_video_meta (buf); > bufw = videometa->stride[0]; > bufh = videometa->offset[1] / bufw; > > Is there a better way to get this info? Currently, no. IIRC the display video width/height is in the caps/video meta. The GstVideoCropMeta instead is subtractive of the data size. The data width for each plane is equivalent to the stride. The plane data height is a little more tricky to compute directly. In practice though one only needs the plane display height of the frame. With the crop meta, one is given a crop rectangle defined by a point and two lengths and so the data height isn't needed. One just uses crop->height rows of pixels. In the GL case, caps size == data size, there's no padding/alignment for the textures within GL. It's only when the textures are in system memory that padding/alignment is taken into account. > 2. Supporting crop handling only need to recalculate the texture coordinate, > but I found every gl plugin has their own vertices definition, this means I > need to modify everywhere to support this. > Is there a plan to make it into a common module? My plan for this was to wrap VAO's in a new object and have a new memory type based off glbasebuffer to contain the required vertex data.
For crop meta, the caps in the CAPS event contain the display size *after* cropping. The caps inside the ALLOCATION query contain the "memory size", i.e. *before* cropping.
In addition, the old bugs related to having a previous pool configured with the cropped caps are gone with the removal of buffer pool caching that is undergo. I believe it would be nice to implement this properly somewhere.
*** Bug 766572 has been marked as a duplicate of this bug. ***
*** Bug 766932 has been marked as a duplicate of this bug. ***
Hi Matthew, Is there any plan to support video crop handle in gl plugins? For example, we can convert video crop meta into a new defined gl video crop meta. Accurately, this gl video crop meta is a vertex array contain a recalculated texture coordinate or just a vao. Bind this meta into glmemory buffer, and all gl plugin will load this meta to GPU.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/182.