GNOME Bugzilla – Bug 732281
new element: inteldmabufupload
Last modified: 2018-05-05 14:38:48 UTC
As part of my work implementing some experimental dmabuf support for waylandsink, I have implemented an element that can upload video buffers to dmabuf buffers allocated from the Intel DRM/PRIME stack. The reason for creating this element was to test dmabuf support in waylandsink, simply because I don't have any other hardware that can produce dmabuf buffers and that I could use with GStreamer. The element is based on GstBaseTransform. It provides a custom buffer pool, which allocates buffer objects using libdrm_intel and exports them to dmabuf objects. The GstMemory of those buffers is allocated from the GstDmabufAllocator. This buffer pool is provided upstream, so either upstream uses it and we have a zero-copy upload, or upstream doesn't use it, in which case this element will copy the contents of the upstream buffers into these dmabuf buffers. The src caps are transformed to have a "memory:dmabuf" caps feature. Example usage: gst-launch-1.0 videotestsrc ! inteldmabufupload ! waylandsink The code can be found in this branch: http://cgit.collabora.com/git/user/gkiagia/gst-plugins-bad.git/log/?h=inteldmabufupload Note that this element will obviously only work on systems with Intel GPUs and it also requires booting a recent linux kernel with the drm.rnodes=1 option on the kernel command line. I am submitting this element because I feel it might seem useful to other developers for testing as well. I don't know of any use case where it would be useful for end users. Also note that waylandsink support for dmabuf is not going to be merged at this point, because it is not standarized in wayland yet.
Why is this not done transparently inside Wayland with the driver and a generic interface for this kind of stuff? Why would the app need to know what driver and hardware is used by Wayland? That seems a bit backwards :)
I'm also a bit unhappy that this memory:dmabuf feature haven't been discussed/proposed anywhere else (like on the ML). Negotation of DMABuf is currently undefined, we need to agree all together instead of doing our own. One problem with this approach is that dmabuf can often be mapped to userspace, hence can also have the default system memory feature. Will wayland sink correctly handle that ?
(In reply to comment #2) > I'm also a bit unhappy that this memory:dmabuf feature haven't been > discussed/proposed anywhere else (like on the ML). I thought this was pretty much standard, i.e. using "memory:ALLOCATOR_NAME" to negotiate buffers with memory coming only from a specific allocator. > Negotation of DMABuf is > currently undefined, we need to agree all together instead of doing our own. > One problem with this approach is that dmabuf can often be mapped to userspace, > hence can also have the default system memory feature. Will wayland sink > correctly handle that ? waylandsink can handle 3 things at the moment (in my dmabuf branch): 1) buffers with memory from its own allocator (which allocates memory on tmpfs files and shares them with the compositor using wl_shm), in which case it just informs the compositor that there is new content there 2) buffers with memory from the dmabuf allocator, in which case it just passes the file descriptor to the compositor 3) any other kind of mappable buffers, in which case it copies contents into new buffers from its own allocator and uses wl_shm again. The fundamental problem that led me to use "memory:dmabuf" to separate it from other kinds of buffers is that the compositor may accept different formats when dmabuf is used and when wl_shm is used. So the caps need a way to express that wl_shm accepts for example only ARGB and xRGB, but the sink will also accept dmabuf with other formats. In any case, I know this is a very primitive way of negotiating dmabuf because there are also other restrictions (hardware-specific restrictions), which GStreamer cannot negotiate. wl_dmabuf (or whatever it is going to be called) is very far from being standarized and it needs work from various sides in order to achieve automatic negotiation. At the moment, what we have is just a proof of concept. You can read various comments about this in the relevant wayland-devel thread: http://lists.freedesktop.org/archives/wayland-devel/2014-June/015362.html Now regarding inteldmabufupload, as I said in the description, it's just a testing element. It obviously is a bad idea to have the application know about the driver being used. There is also no generic interface for allocating dmabuf, because dmabuf is in its early stages and it is tied with specific hardware at the moment. It's a proof of concept from all sides. There are plans, though, for a central allocator in the kernel (see the above thread). Also, this code is not inside waylandsink, because it has nothing to do with wayland. It could be used with another video sink that knows how to import dmabuf buffers. PS: I am happy if this element just dies. I only submitted it in case somebody finds it useful, but I don't expect it to be merged.
Just a note, I found this inteldmabufupload really useful for testing/debugging. Latest version I think is there: https://git.collabora.com/cgit/user/lubosz/gst-plugins-bad.git/commit/?h=dmabuf&id=21432bb0036dd2b0bba8454a4a1f026701fcd939 It can be rebased smoothly with current master. But I think it should be renamed to drmdmabufupload. Indeed it can be used with nouveau, radeon etc.. with a few modifications. So adding some generic stuffs inside + a property to select which backend to use. Depending you are using libdrm_intel.h, libdrm_nouveau.h etc..
Created attachment 311927 [details] [review] inteldrm: Add a new 'inteldmabufupload' element Rebased to current master. Tested with the patches from Bug #743345.
Created attachment 311931 [details] [review] fix with tee I do not really remember how to reproduce the problem but I think it was by adding a tee just before interdmabufupload :) I think the problem was that ->transform_ip was called instead of ->transform. I'll probably double check at some point but I wanted to keep the patch around.
(In reply to Julien Isorce from comment #6) > Created attachment 311931 [details] [review] [review] > fix with tee > > I do not really remember how to reproduce the problem but I think it was by > adding a tee just before interdmabufupload :) > I think the problem was that ->transform_ip was called instead of > ->transform. > I'll probably double check at some point but I wanted to keep the patch > around. It's nice, but it does not represent so well the reality. Dump Buffer are vmalloc buffer, in real life, DMABUF are very often backed by CMA or Scatter Gather allocator. The is vivi driver which do a better job here, as it supports all of this along with many RGB and YUV variants. That being said, it's a interesting toy for testing the basic.
(In reply to Nicolas Dufresne (stormer) from comment #7) > It's nice, but it does not represent so well the reality. Dump Buffer are > vmalloc buffer, in real life, DMABUF are very often backed by CMA or Scatter > Gather allocator. The is vivi driver which do a better job here, as it > supports all of this along with many RGB and YUV variants. That being said, > it's a interesting toy for testing the basic. I thought there was actually no dump buffer in this element. What am I missing ?
I might be confused, this one call drm_intel_bo_alloc_tiled() and then prime it. Looks Intel specific to me of course ;-P It is also limited to single planar (single FD) DMABUF (and RGB only, that is hardcoded this way, not sure if there is a way to probe that ?)
Does this element still have some use ? Or does having a gldownload that produce DMABuf superseed this ? My impression would be that this would become equivalent: ...! glupload ! gldownload ! video/x-raw,(memory:DMABuf) ! ... see bug #776927
Yes, I think there is no use for this element. It was meant for testing, and we have better ways of testing dmabuf nowadays.