After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 782774 - kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other Linux
: Normal enhancement
: 1.12.2
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2017-05-18 09:57 UTC by Guillaume Desmottes
Modified: 2017-07-07 19:50 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries (1.78 KB, patch)
2017-05-18 09:57 UTC, Guillaume Desmottes
none Details | Review
kmssink: Factor out copying to dump buffer (1.87 KB, patch)
2017-07-05 21:16 UTC, Nicolas Dufresne (ndufresne)
needs-work Details | Review
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries (1.92 KB, patch)
2017-07-05 21:16 UTC, Nicolas Dufresne (ndufresne)
none Details | Review
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries (2.33 KB, patch)
2017-07-06 13:32 UTC, Nicolas Dufresne (ndufresne)
none Details | Review
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries (2.40 KB, patch)
2017-07-06 13:35 UTC, Nicolas Dufresne (ndufresne)
none Details | Review
kmssink: Factor out copying to dump buffer (1.99 KB, patch)
2017-07-06 21:25 UTC, Nicolas Dufresne (ndufresne)
committed Details | Review
kmssink: Don't leak GEM primed from DMABuf (1.32 KB, patch)
2017-07-06 21:25 UTC, Nicolas Dufresne (ndufresne)
committed Details | Review
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries (3.15 KB, patch)
2017-07-06 21:25 UTC, Nicolas Dufresne (ndufresne)
committed Details | Review
kmssink: Move kmsmem cache code higher (1.77 KB, patch)
2017-07-07 17:39 UTC, Nicolas Dufresne (ndufresne)
committed Details | Review
kmssink: Track cached kmsmem and clear them on drain (3.38 KB, patch)
2017-07-07 17:39 UTC, Nicolas Dufresne (ndufresne)
committed Details | Review

Description Guillaume Desmottes 2017-05-18 09:57:07 UTC
.
Comment 1 Guillaume Desmottes 2017-05-18 09:57:22 UTC
Created attachment 352077 [details] [review]
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries

kmssink keeps a reference on the last rendered buffer. This reference
should be released on DRAIN and ALLOCATION queries so all upstream
buffers can be returned to the pool if needed.
Comment 2 Víctor Manuel Jáquez Leal 2017-05-19 16:04:18 UTC
Talking with Nicolas about the patch, this issue seems to be trickier: 

We have two cases: 1\ the buffer is an internal kms buffer and 2\ it is an imported dmabuf

In the case 1\ as is an internal buffer, the kmssink, AFAIK, doesn't have to reclaim its own buffers, so it is not oblige to attend the DRAIN and ALLOCATION queries.

In the case of 2\ as is a buffer for upstream, so kmssink has to attend those events. But not just forget about the last buffer, since it might keep being rendered by DRM. To avoid a possible glitch an option would be to copy the dmabuf onto a dumb buffer, render it, and keep it as the last buffer.
Comment 3 Randy Li (ayaka) 2017-06-21 10:07:49 UTC
I have the problem in the case 2. But I wonder whether copy it is necessary. Just find a way to increase the reference of drm gem object or fd is enough.
Comment 4 Nicolas Dufresne (ndufresne) 2017-06-21 17:15:07 UTC
In omxvideodec case, this won't be sufficient to renegotiate.
Comment 5 Nicolas Dufresne (ndufresne) 2017-07-05 21:16:53 UTC
Created attachment 354965 [details] [review]
kmssink: Factor out copying to dump buffer
Comment 6 Nicolas Dufresne (ndufresne) 2017-07-05 21:16:58 UTC
Created attachment 354966 [details] [review]
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries

kmssink keeps a reference on the last rendered buffer. If this buffer
refers to an upstream buffer, it should be should be released on DRAIN
and ALLOCATION queries so all upstream buffers can be returned to the
pool if needed. As the buffr may be used for scanout, we copy this
buffer into a dumb buffer prior to let it go.
Comment 7 Nicolas Dufresne (ndufresne) 2017-07-05 21:18:38 UTC
This has been tested to not cause regressions, but I still need to write a proper test. Guillaume, could you also test this on your platform ?
Comment 8 Guillaume Desmottes 2017-07-06 06:28:19 UTC
Review of attachment 354966 [details] [review]:

typo in commit message: "buffr".

This patch doesn't seem to actually copy anything?
Comment 9 Nicolas Dufresne (ndufresne) 2017-07-06 13:28:46 UTC
Review of attachment 354966 [details] [review]:

Oops, missing a git add before attaching.
Comment 10 Nicolas Dufresne (ndufresne) 2017-07-06 13:32:04 UTC
Created attachment 355014 [details] [review]
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries

kmssink keeps a reference on the last rendered buffer. If this buffer
refers to an upstream buffer, it should be should be released on DRAIN
and ALLOCATION queries so all upstream buffers can be returned to the
pool if needed. As the buffr may be used for scanout, we copy this
buffer into a dumb buffer prior to let it go.
Comment 11 Nicolas Dufresne (ndufresne) 2017-07-06 13:35:28 UTC
Created attachment 355015 [details] [review]
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries

kmssink keeps a reference on the last rendered buffer. If this buffer
refers to an upstream buffer, it should be should be released on DRAIN
and ALLOCATION queries so all upstream buffers can be returned to the
pool if needed. As the buffer may be used for scanout, we copy this
buffer into a dumb buffer prior to let it go.

Based on patch from Guillaume Desmottes <guillaume.desmottes@collabora.com>
Comment 12 Nicolas Dufresne (ndufresne) 2017-07-06 13:37:57 UTC
Sorry, forgot the typo. If you test resolution change, you'll notice that with previous drain, you can observe a render glitch, and it's now clean again as we don't clear that last_buffer when it's owned by kmssink. I could not test the DMABuf case yet.
Comment 13 Víctor Manuel Jáquez Leal 2017-07-06 14:03:28 UTC
Review of attachment 354965 [details] [review]:

Looks good to me. As nitpick I would add, in the commit log, that this a previous step for the drain logic.
Comment 14 Víctor Manuel Jáquez Leal 2017-07-06 14:03:47 UTC
Review of attachment 355015 [details] [review]:

it looks good to me.
Comment 15 Nicolas Dufresne (ndufresne) 2017-07-06 15:27:23 UTC
Review of attachment 355015 [details] [review]:

I resurrected some ancient resolution change test for v4l2src (which I know is pretty picky about drain implementation), unfortunately it raised issues, apparently the video info is out of sync, as a side effect, we can't map last_buffer anymore. That makes the copy to fail, the buffer to be kept and finally v4l2src return EBUSY -> not-negotiated.
Comment 16 Nicolas Dufresne (ndufresne) 2017-07-06 15:28:03 UTC
Review of attachment 354965 [details] [review]:

I'll add a description explaining why we factor this out.
Comment 17 Nicolas Dufresne (ndufresne) 2017-07-06 21:25:09 UTC
Created attachment 355048 [details] [review]
kmssink: Factor out copying to dump buffer

This will be used to copy any upstream memory in order to return it on
resolution change, allocation query or drain query.
Comment 18 Nicolas Dufresne (ndufresne) 2017-07-06 21:25:20 UTC
Created attachment 355049 [details] [review]
kmssink: Don't leak GEM primed from DMABuf

This otherwise breaks DMABuf reclaiming. This is not visible from
userspace, but inside the kernel, the DRM driver will hold a ref to the
DMABuf object. With a V4L2 driver allocating those DMABuf, it then
prevent changing the resolution and re-allocation new buffers.
Comment 19 Nicolas Dufresne (ndufresne) 2017-07-06 21:25:27 UTC
Created attachment 355050 [details] [review]
kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries

kmssink keeps a reference on the last rendered buffer. If this buffer
refers to an upstream buffer, it should be should be released on DRAIN
and ALLOCATION queries so all upstream buffers can be returned to the
pool if needed. As the buffer may be used for scanout, we copy this
buffer into a dumb buffer prior to let it go.

Based on patch from Guillaume Desmottes <guillaume.desmottes@collabora.com>
Comment 20 Nicolas Dufresne (ndufresne) 2017-07-06 21:31:41 UTC
New patchset, I update the commit on the "kmssink: Factor out copying to dump buffer". I also had to add a bug fix, as the GEM handle were leaked. These GEM are holding ref to the DMABuf object (kernel side). For the main patch, I now properly use show_frame() to update the image, I just realized that there was some state that would not get updated otherwise.

Yet, with v4l2src ! kmssink with Intel DRM it still does not work (get ebusy from the v4l2 driver). This is caused by the DRM driver holding a ref for an arbitrary amount of time after all userspace reference being dropped and despite a non-dmabuf backed FB being set for scanout. I could of course work around with sleeps, but we'll need to figure-out something better. I have been told that Atomic DRM API would make this easier, I haven't read that yet.
Comment 21 Víctor Manuel Jáquez Leal 2017-07-06 22:26:32 UTC
Review of attachment 355049 [details] [review]:

this one looks worthy to backport to 1.12 and 1.10
Comment 22 Nicolas Dufresne (ndufresne) 2017-07-07 17:39:14 UTC
Created attachment 355107 [details] [review]
kmssink: Move kmsmem cache code higher

This will be needed as this API will be used elsewhere to clear
the cache.
Comment 23 Nicolas Dufresne (ndufresne) 2017-07-07 17:39:18 UTC
Created attachment 355108 [details] [review]
kmssink: Track cached kmsmem and clear them on drain

In this patch we keep track of the cached kmsmem in a way
that we can clear the cache during the drain process. This
release the framebuffer before waiting for the next vblank,
hence add support for DRM driver (like Intel one) that release
the associated DMABuf reference asynchronously.
Comment 24 Nicolas Dufresne (ndufresne) 2017-07-07 18:37:15 UTC
Attachment 355048 [details] pushed as 6e3fe4e - kmssink: Factor out copying to dump buffer
Attachment 355049 [details] pushed as cfadd5a - kmssink: Don't leak GEM primed from DMABuf
Attachment 355050 [details] pushed as b8c5a4c - kmssink: drop last rendered buffer on ALLOCATION and DRAIN queries
Attachment 355107 [details] pushed as 98b91d0 - kmssink: Move kmsmem cache code higher
Attachment 355108 [details] pushed as 0a0bc8a - kmssink: Track cached kmsmem and clear them on drain
Comment 25 Nicolas Dufresne (ndufresne) 2017-07-07 19:50:31 UTC
Backported the GEM leak fix to 1.12.

commit e1c02e097d671c454e5e3afdad3c96b1eedebaf5
Author: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Date:   Thu Jul 6 17:20:56 2017 -0400

    kmssink: Don't leak GEM primed from DMABuf