After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 732281 - new element: inteldmabufupload
new element: inteldmabufupload
Status: RESOLVED WONTFIX
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2014-06-26 12:52 UTC by George Kiagiadakis
Modified: 2018-05-05 14:38 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
inteldrm: Add a new 'inteldmabufupload' element (28.37 KB, patch)
2015-09-23 08:36 UTC, Lubosz Sarnecki
none Details | Review
fix with tee (1.40 KB, patch)
2015-09-23 09:29 UTC, Julien Isorce
none Details | Review

Description George Kiagiadakis 2014-06-26 12:52:19 UTC
As part of my work implementing some experimental dmabuf support for waylandsink, I have implemented an element that can upload video buffers to dmabuf buffers allocated from the Intel DRM/PRIME stack.

The reason for creating this element was to test dmabuf support in waylandsink, simply because I don't have any other hardware that can produce dmabuf buffers and that I could use with GStreamer.

The element is based on GstBaseTransform. It provides a custom buffer pool, which allocates buffer objects using libdrm_intel and exports them to dmabuf objects. The GstMemory of those buffers is allocated from the GstDmabufAllocator. This buffer pool is provided upstream, so either upstream uses it and we have a zero-copy upload, or upstream doesn't use it, in which case this element will copy the contents of the upstream buffers into these dmabuf buffers.

The src caps are transformed to have a "memory:dmabuf" caps feature.
    
Example usage:
  gst-launch-1.0 videotestsrc ! inteldmabufupload ! waylandsink

The code can be found in this branch:
http://cgit.collabora.com/git/user/gkiagia/gst-plugins-bad.git/log/?h=inteldmabufupload

Note that this element will obviously only work on systems with Intel GPUs and it also requires booting a recent linux kernel with the drm.rnodes=1 option on the kernel command line.

I am submitting this element because I feel it might seem useful to other developers for testing as well. I don't know of any use case where it would be useful for end users. Also note that waylandsink support for dmabuf is not going to be merged at this point, because it is not standarized in wayland yet.
Comment 1 Sebastian Dröge (slomo) 2014-06-26 16:49:00 UTC
Why is this not done transparently inside Wayland with the driver and a generic interface for this kind of stuff? Why would the app need to know what driver and hardware is used by Wayland?
That seems a bit backwards :)
Comment 2 Nicolas Dufresne (ndufresne) 2014-06-28 16:26:26 UTC
I'm also a bit unhappy that this memory:dmabuf feature haven't been discussed/proposed anywhere else (like on the ML). Negotation of DMABuf is currently undefined, we need to agree all together instead of doing our own. One problem with this approach is that dmabuf can often be mapped to userspace, hence can also have the default system memory feature. Will wayland sink correctly handle that ?
Comment 3 George Kiagiadakis 2014-07-01 10:00:58 UTC
(In reply to comment #2)
> I'm also a bit unhappy that this memory:dmabuf feature haven't been
> discussed/proposed anywhere else (like on the ML). 

I thought this was pretty much standard, i.e. using "memory:ALLOCATOR_NAME" to negotiate buffers with memory coming only from a specific allocator.

> Negotation of DMABuf is
> currently undefined, we need to agree all together instead of doing our own.
> One problem with this approach is that dmabuf can often be mapped to userspace,
> hence can also have the default system memory feature. Will wayland sink
> correctly handle that ?

waylandsink can handle 3 things at the moment (in my dmabuf branch):

1) buffers with memory from its own allocator (which allocates memory on tmpfs files and shares them with the compositor using wl_shm), in which case it just informs the compositor that there is new content there

2) buffers with memory from the dmabuf allocator, in which case it just passes the file descriptor to the compositor

3) any other kind of mappable buffers, in which case it copies contents into new buffers from its own allocator and uses wl_shm again.

The fundamental problem that led me to use "memory:dmabuf" to separate it from other kinds of buffers is that the compositor may accept different formats when dmabuf is used and when wl_shm is used. So the caps need a way to express that wl_shm accepts for example only ARGB and xRGB, but the sink will also accept dmabuf with other formats.

In any case, I know this is a very primitive way of negotiating dmabuf because there are also other restrictions (hardware-specific restrictions), which GStreamer cannot negotiate. wl_dmabuf (or whatever it is going to be called) is very far from being standarized and it needs work from various sides in order to achieve automatic negotiation. At the moment, what we have is just a proof of concept.

You can read various comments about this in the relevant wayland-devel thread:
http://lists.freedesktop.org/archives/wayland-devel/2014-June/015362.html


Now regarding inteldmabufupload, as I said in the description, it's just a testing element. It obviously is a bad idea to have the application know about the driver being used. There is also no generic interface for allocating dmabuf, because dmabuf is in its early stages and it is tied with specific hardware at the moment. It's a proof of concept from all sides. There are plans, though, for a central allocator in the kernel (see the above thread).

Also, this code is not inside waylandsink, because it has nothing to do with wayland. It could be used with another video sink that knows how to import dmabuf buffers.


PS: I am happy if this element just dies. I only submitted it in case somebody finds it useful, but I don't expect it to be merged.
Comment 4 Julien Isorce 2015-09-23 08:25:40 UTC
Just a note, I found this inteldmabufupload really useful for testing/debugging. Latest version I think is there: https://git.collabora.com/cgit/user/lubosz/gst-plugins-bad.git/commit/?h=dmabuf&id=21432bb0036dd2b0bba8454a4a1f026701fcd939

It can be rebased smoothly with current master. But I think it should be renamed to drmdmabufupload. Indeed it can be used with nouveau, radeon etc.. with a few modifications. So adding some generic stuffs inside + a property to select which backend to use. Depending you are using libdrm_intel.h, libdrm_nouveau.h etc..
Comment 5 Lubosz Sarnecki 2015-09-23 08:36:55 UTC
Created attachment 311927 [details] [review]
inteldrm: Add a new 'inteldmabufupload' element

Rebased to current master. Tested with the patches from Bug #743345.
Comment 6 Julien Isorce 2015-09-23 09:29:11 UTC
Created attachment 311931 [details] [review]
fix with tee

I do not really remember how to reproduce the problem but I think it was by adding a tee just before interdmabufupload :)
I think the problem was that ->transform_ip was called instead of ->transform.
I'll probably double check at some point but I wanted to keep the patch around.
Comment 7 Nicolas Dufresne (ndufresne) 2015-09-23 13:55:21 UTC
(In reply to Julien Isorce from comment #6)
> Created attachment 311931 [details] [review] [review]
> fix with tee
> 
> I do not really remember how to reproduce the problem but I think it was by
> adding a tee just before interdmabufupload :)
> I think the problem was that ->transform_ip was called instead of
> ->transform.
> I'll probably double check at some point but I wanted to keep the patch
> around.

It's nice, but it does not represent so well the reality. Dump Buffer are vmalloc buffer, in real life, DMABUF are very often backed by CMA or Scatter Gather allocator. The is vivi driver which do a better job here, as it supports all of this along with many RGB and YUV variants. That being said, it's a interesting toy for testing the basic.
Comment 8 Julien Isorce 2015-09-23 14:04:24 UTC
(In reply to Nicolas Dufresne (stormer) from comment #7)
> It's nice, but it does not represent so well the reality. Dump Buffer are
> vmalloc buffer, in real life, DMABUF are very often backed by CMA or Scatter
> Gather allocator. The is vivi driver which do a better job here, as it
> supports all of this along with many RGB and YUV variants. That being said,
> it's a interesting toy for testing the basic.

I thought there was actually no dump buffer in this element. What am I missing ?
Comment 9 Nicolas Dufresne (ndufresne) 2015-09-23 14:14:52 UTC
I might be confused, this one call drm_intel_bo_alloc_tiled() and then prime it. Looks Intel specific to me of course ;-P It is also limited to single planar (single FD) DMABUF (and RGB only, that is hardcoded this way, not sure if there is a way to probe that ?)
Comment 10 Nicolas Dufresne (ndufresne) 2017-05-29 18:16:13 UTC
Does this element still have some use ? Or does having a gldownload that produce DMABuf superseed this ?

My impression would be that this would become equivalent:

  ...! glupload ! gldownload ! video/x-raw,(memory:DMABuf) ! ...

see bug #776927
Comment 11 George Kiagiadakis 2018-05-05 14:38:48 UTC
Yes, I think there is no use for this element. It was meant for testing, and we have better ways of testing dmabuf nowadays.