After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 748163 - crop handling in gl plugins (gleffect, glimagesink, glshader, .....)
crop handling in gl plugins (gleffect, glimagesink, glshader, .....)
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
1.4.5
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
: 766572 766932 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2015-04-20 05:52 UTC by Jian Li
Modified: 2018-11-03 11:37 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Jian Li 2015-04-20 05:52:18 UTC
Is there a plan to support video cropping in gl plugins? 
Although this can be done when you uploading the pixel data into texture, there is still cases that you can use some extension to bind a buffer to the texture, such as eglimage, dma-bufer or some private extensions. If decoder needs padding or alignment when output video into this buffer, the gl plugins need to handle it correctly.

I tried to support it, but have below question:

1. How to get the video buffer width and height?

I can get crop info from GstVideoCropMeta, but how can I get the buffer width and height? We can get real video width/height from GstVideoInfo, but can't get the buffer widht/height after padding/aligned. One way I found is like below for I420 and NV12 video: 

videometa = gst_buffer_get_video_meta (buf);
bufw = videometa->stride[0];
bufh = videometa->offset[1] / bufw;

Is there a better way to get this info?

2. Supporting crop handling only need to recalculate the texture coordinate, but I found every gl plugin has their own vertices definition, this means I need to modify everywhere to support this.
Is there a plan to make it into a common module?
Comment 1 Matthew Waters (ystreet00) 2015-07-07 08:14:14 UTC
(In reply to Jian Li from comment #0)
> Is there a plan to support video cropping in gl plugins? 
> Although this can be done when you uploading the pixel data into texture,
> there is still cases that you can use some extension to bind a buffer to the
> texture, such as eglimage, dma-bufer or some private extensions. If decoder
> needs padding or alignment when output video into this buffer, the gl
> plugins need to handle it correctly.

GstVideoAlignment is more suited to any data alignment constraints which GLMemory already supports in master.

> I tried to support it, but have below question:
> 
> 1. How to get the video buffer width and height?
> 
> I can get crop info from GstVideoCropMeta, but how can I get the buffer
> width and height? We can get real video width/height from GstVideoInfo, but
> can't get the buffer widht/height after padding/aligned. One way I found is
> like below for I420 and NV12 video: 
> 
> videometa = gst_buffer_get_video_meta (buf);
> bufw = videometa->stride[0];
> bufh = videometa->offset[1] / bufw;
> 
> Is there a better way to get this info?

Currently, no.  IIRC the display video width/height is in the caps/video meta.  The GstVideoCropMeta instead is subtractive of the data size.  The data width for each plane is equivalent to the stride.  The plane data height is a little more tricky to compute directly.  In practice though one only needs the plane display height of the frame.  With the crop meta, one is given a crop rectangle defined by a point and two lengths and so the data height isn't needed.  One just uses crop->height rows of pixels.

In the GL case, caps size == data size, there's no padding/alignment for the textures within GL.  It's only when the textures are in system memory that padding/alignment is taken into account.

> 2. Supporting crop handling only need to recalculate the texture coordinate,
> but I found every gl plugin has their own vertices definition, this means I
> need to modify everywhere to support this.
> Is there a plan to make it into a common module?

My plan for this was to wrap VAO's in a new object and have a new memory type based off glbasebuffer to contain the required vertex data.
Comment 2 Sebastian Dröge (slomo) 2015-07-07 10:26:24 UTC
For crop meta, the caps in the CAPS event contain the display size *after* cropping. The caps inside the ALLOCATION query contain the "memory size", i.e. *before* cropping.
Comment 3 Nicolas Dufresne (ndufresne) 2015-07-07 12:39:42 UTC
In addition, the old bugs related to having a previous pool configured with the cropped caps are gone with the removal of buffer pool caching that is undergo. I believe it would be nice to implement this properly somewhere.
Comment 4 Matthew Waters (ystreet00) 2016-05-17 18:04:17 UTC
*** Bug 766572 has been marked as a duplicate of this bug. ***
Comment 5 Sebastian Dröge (slomo) 2016-05-27 07:57:39 UTC
*** Bug 766932 has been marked as a duplicate of this bug. ***
Comment 6 Haihua Hu 2016-05-27 07:59:47 UTC
Hi Matthew,

Is there any plan to support video crop handle in gl plugins?

For example, we can convert video crop meta into a new defined gl video crop meta. Accurately, this gl video crop meta is a vertex array contain a recalculated texture coordinate or just a vao. Bind this meta into glmemory buffer, and all gl plugin will load this meta to GPU.
Comment 7 GStreamer system administrator 2018-11-03 11:37:04 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/182.