After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 792034 - implement dmabuf-import negotiation with upstream
implement dmabuf-import negotiation with upstream
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gstreamer-vaapi
1.12.2
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
: 794247 (view as bug list)
Depends on: 789476 793270 793271
Blocks:
 
 
Reported: 2017-12-29 02:13 UTC by Lim Siew Hoon
Modified: 2018-11-03 15:52 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
hack for icamerasrc to enable the dmabuf import with vaapi (1.63 KB, patch)
2018-01-03 03:24 UTC, renwei.wu
none Details | Review
plugins: remove dmabuf-import hack (3.19 KB, patch)
2018-01-12 18:09 UTC, Víctor Manuel Jáquez Leal
committed Details | Review
vaapivideobufferpool: dynamically build options (3.30 KB, patch)
2018-02-07 18:25 UTC, Víctor Manuel Jáquez Leal
none Details | Review
vaapivideobufferpool: add dmabuf pool option (1.96 KB, patch)
2018-02-07 18:25 UTC, Víctor Manuel Jáquez Leal
none Details | Review
plugins: propose a second pool with dmabuf capabilites (4.07 KB, patch)
2018-02-07 18:25 UTC, Víctor Manuel Jáquez Leal
none Details | Review
vaapivideobufferpool: dynamically build options (3.30 KB, patch)
2018-02-08 09:10 UTC, Víctor Manuel Jáquez Leal
none Details | Review
vaapivideobufferpool: handle dmabuf pool option (3.63 KB, patch)
2018-02-08 09:10 UTC, Víctor Manuel Jáquez Leal
none Details | Review
plugins: propose a second pool with dmabuf capabilites (4.92 KB, patch)
2018-02-08 09:10 UTC, Víctor Manuel Jáquez Leal
none Details | Review
v4l2src and vaapi logs (79.52 KB, application/octet-stream)
2018-03-29 11:26 UTC, deshui
  Details

Description Lim Siew Hoon 2017-12-29 02:13:28 UTC
The dmabuf import previous working with gst-vaapi 1.8.3 version.
Aware that in gst-vaapi 1.12.2 version got a lot of design have been changing and no longer working with icamerasrc plugin in 1.12.2 version.

Previously we added the workaround to get dmabuf import with icamerasrc plugin working with gst-vaapi. 

Now to get it properly working, icamerasrc plugin developer will working with gst-vaapi developer what needs to done and how to handle it.
Comment 1 sreerenj 2017-12-29 18:29:58 UTC
We should remove the super hack in gstremaer-vaapi which is to support the dmabuf export from vaapi element to upstream (eg: gst-launch-1.0 v4l2src io-mode=dmabuf-import ! vaapipostproc format=nv12 ! fakesink):
https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst/vaapi/gstvaapipluginbase.c#n425

We had some discussion regarding the same:
https://bugzilla.gnome.org/show_bug.cgi?id=765435#c7

Not sure whether v4l2src already implemented it, I guess not.

BTW, IIUC regression is not because of gstreamer-vaapi code, but because of icamerasrc changes, right?
Comment 2 Nicolas Dufresne (ndufresne) 2017-12-29 22:35:15 UTC
The hack is no-op now since dmabuf export is default in V4L2 now. If icamsrc do the same, then it will work for both, I hope dmabuf caps feature won't be needed.

Can you put a link to some icamsrc code, it's not clear where this is hosted, if it's open source, what hardware is available with that can/ISP, and which drivers (kernel and user space) are needed.
Comment 3 sreerenj 2017-12-30 01:17:52 UTC
(In reply to Nicolas Dufresne (stormer) from comment #2)
> The hack is no-op now since dmabuf export is default in V4L2 now. If icamsrc
> do the same, then it will work for both, I hope dmabuf caps feature won't be
> needed.
> 

The dmabuf export from source element can work without any hack (both v4l2 and icamerasrc) in gstreamer-vaapi.

But IIUC, they want dmabuf-import and I don't think it can work without having the hack in gstreamer-vaapi (I guess it is the same situation for v4l2src too).

> Can you put a link to some icamsrc code, it's not clear where this is
> hosted, if it's open source, what hardware is available with that can/ISP,
> and which drivers (kernel and user space) are needed.

Here is the repo: https://github.com/intel/icamerasrc
I told them to comment here, hopefully, we will get more comments soon :)
Comment 4 Nicolas Dufresne (ndufresne) 2017-12-30 01:58:22 UTC
Ok, indeed automatic importation from vaapi dmabuf to the camera is not automatically supported. It is also broken in v4l2src, so if there is a hack to enable it, please remove right away, I'll remove it if it is still there when I'm back in Jan (unless I have a patch that adds the missing validation).

What is complicated is that you need to tell the driver the strides and offsets of the allocated images. And as there is currently no kernel API to help with negotiating the allocation between two drivers, these need to match on first try. Drivers like UVC, that could support arbitrary alignement simply don't implement it, and enforce a certain alignment. V4L2 API is not very friendly as it merge the format and alignment ll together.
Comment 5 sreerenj 2017-12-30 02:07:20 UTC
(In reply to Nicolas Dufresne (stormer) from comment #4)
> Ok, indeed automatic importation from vaapi dmabuf to the camera is not
> automatically supported. It is also broken in v4l2src, so if there is a hack
> to enable it, please remove right away, I'll remove it if it is still there
> when I'm back in Jan (unless I have a patch that adds the missing
> validation).

Does it mean to remove the "dmabuf-import" option completely from v4l2src (+ the corresponding code from gstreamer-vaapi)?

> 
> What is complicated is that you need to tell the driver the strides and
> offsets of the allocated images. And as there is currently no kernel API to
> help with negotiating the allocation between two drivers, these need to
> match on first try. Drivers like UVC, that could support arbitrary
> alignement simply don't implement it, and enforce a certain alignment. V4L2
> API is not very friendly as it merge the format and alignment ll together.
Comment 6 Nicolas Dufresne (ndufresne) 2017-12-30 21:04:05 UTC
(In reply to sreerenj from comment #5)
> (In reply to Nicolas Dufresne (stormer) from comment #4)
> Does it mean to remove the "dmabuf-import" option completely from v4l2src (+

That I can't, it's in the API now.

> the corresponding code from gstreamer-vaapi)?

That's the code I'll remove as it's unsafe (or at least make sure it's not used without user knowledge of the risk). The fix in the source will cause pipeline error in many cases, instead of crash or corrupted image, it's not much better.
Comment 7 sreerenj 2017-12-31 03:28:40 UTC
(In reply to Nicolas Dufresne (stormer) from comment #6)
> (In reply to sreerenj from comment #5)
> > (In reply to Nicolas Dufresne (stormer) from comment #4)
> > Does it mean to remove the "dmabuf-import" option completely from v4l2src (+
> 
> That I can't, it's in the API now.
> 
> > the corresponding code from gstreamer-vaapi)?
> 
> That's the code I'll remove as it's unsafe (or at least make sure it's not
> used without user knowledge of the risk). The fix in the source will cause
> pipeline error in many cases, instead of crash or corrupted image, it's not
> much better.

Tha hack in gstreamer-vaapi is for checking whether user explicitly enabled upstream element's "io-mode" to "dmabuf-import". Yes, it a super hack, which was added a long time ago when there were no dmabuf negotiation support in GStreamer and inherited all the way till 1.12.x.

 
BTW, good to see "dmabuf-export" as default in v4l2src. Great!
So I am fine with removing the hack for v4l2 from gstreamer-vaapi, but please keep it for "icamerasrc", hopefully we can add some other fix than the nasty hack.
Comment 8 renwei.wu 2018-01-03 03:24:59 UTC
Created attachment 366233 [details] [review]
hack for icamerasrc to enable the dmabuf import with vaapi
Comment 9 renwei.wu 2018-01-03 03:26:48 UTC
(In reply to sreerenj from comment #1)
> We should remove the super hack in gstremaer-vaapi which is to support the
> dmabuf export from vaapi element to upstream (eg: gst-launch-1.0 v4l2src
> io-mode=dmabuf-import ! vaapipostproc format=nv12 ! fakesink):
> https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst/vaapi/
> gstvaapipluginbase.c#n425
> 
> We had some discussion regarding the same:
> https://bugzilla.gnome.org/show_bug.cgi?id=765435#c7
> 
> Not sure whether v4l2src already implemented it, I guess not.
> 
> BTW, IIUC regression is not because of gstreamer-vaapi code, but because of
> icamerasrc changes, right?

(In reply to sreerenj from comment #1)
> We should remove the super hack in gstremaer-vaapi which is to support the
> dmabuf export from vaapi element to upstream (eg: gst-launch-1.0 v4l2src
> io-mode=dmabuf-import ! vaapipostproc format=nv12 ! fakesink):
> https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst/vaapi/
> gstvaapipluginbase.c#n425
> 
> We had some discussion regarding the same:
> https://bugzilla.gnome.org/show_bug.cgi?id=765435#c7
> 
> Not sure whether v4l2src already implemented it, I guess not.
> 
> BTW, IIUC regression is not because of gstreamer-vaapi code, but because of
> icamerasrc changes, right?

(In reply to sreerenj from comment #1)
> We should remove the super hack in gstremaer-vaapi which is to support the
> dmabuf export from vaapi element to upstream (eg: gst-launch-1.0 v4l2src
> io-mode=dmabuf-import ! vaapipostproc format=nv12 ! fakesink):
> https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst/vaapi/
> gstvaapipluginbase.c#n425
> 
> We had some discussion regarding the same:
> https://bugzilla.gnome.org/show_bug.cgi?id=765435#c7
> 
> Not sure whether v4l2src already implemented it, I guess not.
> 
> BTW, IIUC regression is not because of gstreamer-vaapi code, but because of
> icamerasrc changes, right?

Can't simply say it caused by icamerasrc change. The icamerasrc is only validated on certain version of gstreamer like 1.8.3. For about one year, the gstreamer version stays on this version. And we do the upgrade gstreamer version to 1.12.2, the issue is captured.
Go back to check the history, we found 
1 code refactoring in icamerasrc to support multi streams and icamerasrc is not inherited from PUSHSRC.
2 vaapi remove the option GST_BUFFER_POOL_OPTION_DMABUF_MEMORY.

Now it becomes the allocator and pool is empty when using the dmabuf import.

Since the memory:dmabuf is still in development. I suggest to still add the hack in vappi for icamerasrc to enable the dmabuf import.

When the memory:dmabuf filter is enabled, the hack can be removed throughly.

A hack of hack patch is attached.
Comment 10 Lim Siew Hoon 2018-01-10 03:47:09 UTC
Hi,
Any update on this?
Comment 11 Víctor Manuel Jáquez Leal 2018-01-10 15:15:27 UTC
Sorry for the delay.

I have a different proposal. I haven't tried it, but it is an idea:

Instead of using that hack, the gstreamer-vaapi elements will add, in the propose_allocation() method's query, two allocators: the normal one for VAAPI surfaces and another dmabuf-based.

Thus, when the icamerasrc processes the offered buffer pool, and dmabuf-import is enabled, will look for the dmabuf-base allocator in the query, and will set that allocator in the buffer pool configuration.

what do you think?
Comment 12 Víctor Manuel Jáquez Leal 2018-01-10 16:21:03 UTC
said that, that would go to master. For 1.12, dunno, perhaps merge that code out-of-tree, but not in upstream.
Comment 13 sreerenj 2018-01-10 19:00:56 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #11)
> Sorry for the delay.
> 
> I have a different proposal. I haven't tried it, but it is an idea:
> 
> Instead of using that hack, the gstreamer-vaapi elements will add, in the
> propose_allocation() method's query, two allocators: the normal one for
> VAAPI surfaces and another dmabuf-based.
> 
> Thus, when the icamerasrc processes the offered buffer pool, and
> dmabuf-import is enabled, will look for the dmabuf-base allocator in the
> query, and will set that allocator in the buffer pool configuration.
> 
> what do you think?

That could be the only way to do it :),
Two allocators or two separate pools?
Comment 14 sreerenj 2018-01-10 19:07:27 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #12)
> said that, that would go to master. For 1.12, dunno, perhaps merge that code
> out-of-tree, but not in upstream.

Yup, not in 1.12 for sure.
Comment 15 Lim Siew Hoon 2018-01-11 07:29:20 UTC
(In reply to sreerenj from comment #14)
> (In reply to Víctor Manuel Jáquez Leal from comment #12)
> > said that, that would go to master. For 1.12, dunno, perhaps merge that code
> > out-of-tree, but not in upstream.
> 
> Yup, not in 1.12 for sure.

Currently, we are using it based on Gstreamer framework 1.12.2 version and Gstreamer-vaapi 1.12.2 version. And also we are using libva v1.8-branch.
There is no plan for us to move to libva 2.0. But, I think we will be ok to have the code patch can apply to 1.12 version and put on recipes side.
Comment 16 Víctor Manuel Jáquez Leal 2018-01-11 11:22:56 UTC
(In reply to Lim Siew Hoon from comment #15)
> (In reply to sreerenj from comment #14)
> > (In reply to Víctor Manuel Jáquez Leal from comment #12)
> > > said that, that would go to master. For 1.12, dunno, perhaps merge that code
> > > out-of-tree, but not in upstream.
> > 
> > Yup, not in 1.12 for sure.
> 
> Currently, we are using it based on Gstreamer framework 1.12.2 version and
> Gstreamer-vaapi 1.12.2 version. And also we are using libva v1.8-branch.
> There is no plan for us to move to libva 2.0. But, I think we will be ok to
> have the code patch can apply to 1.12 version and put on recipes side.

Does this mean that you need something else beside the posted patch? (attachment 366233 [details] [review])
Comment 17 Víctor Manuel Jáquez Leal 2018-01-12 18:09:34 UTC
Created attachment 366741 [details] [review]
plugins: remove dmabuf-import hack

Remove the hack to check if an upstream element has enabled the
property io-mode enabled as dmabuf-import.
Comment 18 Nicolas Dufresne (ndufresne) 2018-01-12 18:12:37 UTC
Review of attachment 366741 [details] [review]:

Thanks. This removes a bad friend.
Comment 19 Víctor Manuel Jáquez Leal 2018-01-16 18:35:42 UTC
Comment on attachment 366741 [details] [review]
plugins: remove dmabuf-import hack

Attachment 366741 [details] pushed as 8688e81 - plugins: remove dmabuf-import hack
Comment 20 Víctor Manuel Jáquez Leal 2018-01-16 18:37:46 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #19)
> Comment on attachment 366741 [details] [review] [review]
> plugins: remove dmabuf-import hack
> 
> Attachment 366741 [details] pushed as 8688e81 - plugins: remove
> dmabuf-import hack

This was merged in master, not in branch 1.12
Comment 21 Víctor Manuel Jáquez Leal 2018-02-07 18:25:46 UTC
Created attachment 368085 [details] [review]
vaapivideobufferpool: dynamically build options

Instead of return a static list of options, the array should be
constructed dinamycally with the options processed by the
configuration.
Comment 22 Víctor Manuel Jáquez Leal 2018-02-07 18:25:52 UTC
Created attachment 368086 [details] [review]
vaapivideobufferpool: add dmabuf pool option
Comment 23 Víctor Manuel Jáquez Leal 2018-02-07 18:25:57 UTC
Created attachment 368087 [details] [review]
plugins: propose a second pool with dmabuf capabilites

If the caps are raw of have the Dmabuf feature, a second pool is
proposed upstream with a dmabuf allocator and dmabuf
capabilities. Also the allocator is appended in the query.
Comment 24 Lim Siew Hoon 2018-02-08 07:18:45 UTC
Ok, noted. We will start porting into gst-vaapi 1.12.2 version and verify with icamerasrc plugin side once icamerasrc plugin rework is done.
Comment 25 Víctor Manuel Jáquez Leal 2018-02-08 09:10:42 UTC
Created attachment 368133 [details] [review]
vaapivideobufferpool: dynamically build options

Instead of return a static list of options, the array should be
constructed dinamycally with the options processed by the
configuration.
Comment 26 Víctor Manuel Jáquez Leal 2018-02-08 09:10:48 UTC
Created attachment 368134 [details] [review]
vaapivideobufferpool: handle dmabuf pool option
Comment 27 Víctor Manuel Jáquez Leal 2018-02-08 09:10:54 UTC
Created attachment 368135 [details] [review]
plugins: propose a second pool with dmabuf capabilites

If the caps are raw or have the DMAbuf feature, a second pool is
proposed upstream with a DMAbuf allocator and capabilities.

It is the upstream obligation to look for the appropiate pool in
decide_allocation().

The DMABuf allocator is not appended in the query because it is not
certain if it has a custom alloc function or not.
Comment 28 Lim Siew Hoon 2018-02-12 10:44:30 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #22)
> Created attachment 368086 [details] [review] [review]
> vaapivideobufferpool: add dmabuf pool option

I can't find this macro in gst-plugins-base master branch for GST_BUFFER_POOL_OPTION_VIDEO_DMABUF.
Comment 29 Víctor Manuel Jáquez Leal 2018-02-12 11:04:09 UTC
(In reply to Lim Siew Hoon from comment #28)
> (In reply to Víctor Manuel Jáquez Leal from comment #22)
> > Created attachment 368086 [details] [review] [review] [review]
> > vaapivideobufferpool: add dmabuf pool option
> 
> I can't find this macro in gst-plugins-base master branch for
> GST_BUFFER_POOL_OPTION_VIDEO_DMABUF.

You need to apply the patch in https://bugzilla.gnome.org/show_bug.cgi?id=793270

It is under discussion and it may change in the future.
Comment 30 Nicolas Dufresne (ndufresne) 2018-03-12 19:47:19 UTC
*** Bug 794247 has been marked as a duplicate of this bug. ***
Comment 31 deshui 2018-03-13 08:06:01 UTC
i notice with all above patches, the sinkpad_allocator for plugin always be video memory, not dma-buf memory.
to use the dma-buf import feature, icamsrc have to check the mem type, since the memory type is video memory, not dma-buf memory, icamsrc plugin can't work will.
so, would you like to tell me how to make the sinkpad_allocator be dma-buf memory?
Comment 32 renwei.wu 2018-03-19 08:21:03 UTC
Any feekback to deshui's question?
Comment 33 Nicolas Dufresne (ndufresne) 2018-03-19 13:28:09 UTC
(In reply to renwei.wu from comment #32)
> Any feekback to deshui's question?

It was broadcasted on multiple channels, also bugzilla is not a forum. The answer is that icamsrc must negotiate memory:DMABuf caps feature if it cannot always produce DMAbuf (like v4l2src does).
Comment 34 Víctor Manuel Jáquez Leal 2018-03-23 17:00:10 UTC
(In reply to deshui from comment #31)
> i notice with all above patches, the sinkpad_allocator for plugin always be
> video memory, not dma-buf memory.
> to use the dma-buf import feature, icamsrc have to check the mem type, since
> the memory type is video memory, not dma-buf memory, icamsrc plugin can't
> work will.
> so, would you like to tell me how to make the sinkpad_allocator be dma-buf
> memory?


Since there must two pools, it doesn't matter if plugin->sinkpad_allocator is vaapi memory based because:

Two pools are offered upstream in propose_allocation(): one produces buffers with vaapi memory and another with dmabuf memory.

The first one, will set the sinkpad_allocator and sinkpad_buffer_pool in gstvaapipluginbase.  But the second one is not hold by gstvaapipluginbase, it is only offered and forgotten. Upstream element could hold it and use it, otherwise it is unrrefed.

If the dmabuf-memory-based pool is held by upstream element (camerasrc should be  do this in this case), the source element will use that pool to produce the output buffers and later they will be imported by vaapi elements as they were imported before.

Read the patches proposed for v4l2src in bug #793271 for how I proposed to handle this. The logic, basically is, if the io-mode property in v4l2src in dmabuf-import, then check for all the offered pool in the query and if it is there one with DMABUF property, it is hold and use.
Comment 35 deshui 2018-03-29 06:18:05 UTC
after apply the patches from bug #793271 and bug #793274 on master branch
other format works well except format NV12 WITH command
gst-launch-1.0  v4l2src  io-mode=5 num-buffers=100 ! video/x-raw,format=NV12,width=1280,height=720 ! vaapipostproc ! filesink location=1.raw 
so, gstreamer-vappi seems not support NV12 now?
by the way, vappisink works well but filesink failed
Comment 36 deshui 2018-03-29 06:19:00 UTC
(In reply to deshui from comment #35)
> after apply the patches from bug #793271 and bug #793274 on master branch
> other format works well except format NV12 WITH command
> gst-launch-1.0  v4l2src  io-mode=5 num-buffers=100 !
> video/x-raw,format=NV12,width=1280,height=720 ! vaapipostproc ! filesink
> location=1.raw 
> so, gstreamer-vappi seems not support NV12 now?
> by the way, vappisink works well but filesink failed

sorry bug #793274 should be bug #792034
Comment 37 Víctor Manuel Jáquez Leal 2018-03-29 08:21:50 UTC
(In reply to deshui from comment #35)
> after apply the patches from bug #793271 and bug #793274 on master branch
> other format works well except format NV12 WITH command
> gst-launch-1.0  v4l2src  io-mode=5 num-buffers=100 !
> video/x-raw,format=NV12,width=1280,height=720 ! vaapipostproc ! filesink
> location=1.raw 
> so, gstreamer-vappi seems not support NV12 now?

That's because my (and I assume yours) v4l2src device doesn't handle NV12

0:00:00.207682288  5580      0x1e652d0 DEBUG                v4l2src gstv4l2src.c:504:gst_v4l2src_negotiate:<v4l2src0> caps of src: video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)10/1; video/x-raw, format=(string)YUY2, width=(int)960, height=(int)540, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)15/1; video/x-raw, format=(string)YUY2, width=(int)848, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1; video/x-raw, format=(string)YUY2, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; video/x-raw, format=(string)YUY2, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; video/x-raw, format=(string)YUY2, width=(int)424, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; video/x-raw, format=(string)YUY2, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; video/x-raw, format=(string)YUY2, width=(int)320, height=(int)180, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)960, height=(int)540, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)848, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)424, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)320, height=(int)180, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.211450330  5580      0x1e652d0 DEBUG                v4l2src gstv4l2src.c:512:gst_v4l2src_negotiate:<v4l2src0> caps of peer: video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string){ progressive, interleaved, mixed }
0:00:00.211518706  5580      0x1e652d0 DEBUG                v4l2src gstv4l2src.c:518:gst_v4l2src_negotiate:<v4l2src0> intersect: EMPTY
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.

> by the way, vappisink works well but filesink failed

fails how?
Comment 38 deshui 2018-03-29 08:46:14 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #37)
> (In reply to deshui from comment #35)
> > after apply the patches from bug #793271 and bug #793274 on master branch
> > other format works well except format NV12 WITH command
> > gst-launch-1.0  v4l2src  io-mode=5 num-buffers=100 !
> > video/x-raw,format=NV12,width=1280,height=720 ! vaapipostproc ! filesink
> > location=1.raw 
> > so, gstreamer-vappi seems not support NV12 now?
> 
> That's because my (and I assume yours) v4l2src device doesn't handle NV12
> 
> 0:00:00.207682288  5580      0x1e652d0 DEBUG                v4l2src
> gstv4l2src.c:504:gst_v4l2src_negotiate:<v4l2src0> caps of src: video/x-raw,
> format=(string)YUY2, width=(int)1280, height=(int)720,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)10/1; video/x-raw,
> format=(string)YUY2, width=(int)960, height=(int)540,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)15/1; video/x-raw,
> format=(string)YUY2, width=(int)848, height=(int)480,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1; video/x-raw,
> format=(string)YUY2, width=(int)640, height=(int)480,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> video/x-raw, format=(string)YUY2, width=(int)640, height=(int)360,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> video/x-raw, format=(string)YUY2, width=(int)424, height=(int)240,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> video/x-raw, format=(string)YUY2, width=(int)352, height=(int)288,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> video/x-raw, format=(string)YUY2, width=(int)320, height=(int)180,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> image/jpeg, width=(int)1280, height=(int)720,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> image/jpeg, width=(int)960, height=(int)540,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> image/jpeg, width=(int)848, height=(int)480,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> image/jpeg, width=(int)640, height=(int)480,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> image/jpeg, width=(int)640, height=(int)360,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> image/jpeg, width=(int)424, height=(int)240,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> image/jpeg, width=(int)352, height=(int)288,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> image/jpeg, width=(int)320, height=(int)240,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
> image/jpeg, width=(int)320, height=(int)180,
> pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> 0:00:00.211450330  5580      0x1e652d0 DEBUG                v4l2src
> gstv4l2src.c:512:gst_v4l2src_negotiate:<v4l2src0> caps of peer: video/x-raw,
> format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)[
> 0/1, 2147483647/1 ], interlace-mode=(string){ progressive, interleaved,
> mixed }
> 0:00:00.211518706  5580      0x1e652d0 DEBUG                v4l2src
> gstv4l2src.c:518:gst_v4l2src_negotiate:<v4l2src0> intersect: EMPTY
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal
> data stream error.
> 
> > by the way, vappisink works well but filesink failed
> 
> fails how?

I use vivid, which should support NV12
I can find "video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720" in v4l2src gstv4l2src.c:504:gst_v4l2src_negotiate:

and the error log is:

(gst-launch-1.0:745): GStreamer-CRITICAL **: gst_memory_unmap: assertion 'mem != NULL' failed
0:00:00.445999797   745 0x564d5250b540 DEBUG                v4l2src gstv4l2src.c:913:gst_v4l2src_create:<v4l2src0> ts: 0:01:08.072851000 now 0:01:08.073164703 delay 0:00:00.000313703
0:00:00.446087104   745 0x564d5250b540 INFO                 v4l2src gstv4l2src.c:949:gst_v4l2src_create:<v4l2src0> sync to 0:00:00.133333332 out ts 0:00:00.090602346
0:00:00.446215700   745 0x564d5250b540 ERROR       vaapivideomemory gstvaapivideomemory.c:278:map_vaapi_memory: failed to make image current
Comment 39 Víctor Manuel Jáquez Leal 2018-03-29 08:58:45 UTC
please upload a complete log vaapi*:5,v4l2*:5
Comment 40 deshui 2018-03-29 11:26:53 UTC
Created attachment 370288 [details]
v4l2src and vaapi logs
Comment 41 deshui 2018-03-29 11:27:43 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #39)
> please upload a complete log vaapi*:5,v4l2*:5

please refer to the attachment
Comment 42 Víctor Manuel Jáquez Leal 2018-03-29 11:43:22 UTC
That failure is when using filesink?

I'm lost now. If I understand correctly, there are two different issues

1\ NV12 doesn't work (it is not clear to me why and how in your case)
2\ when using filesink, you got the error in attachment 370288 [details], 
  2.1\ but using vaapisink, the pipeline works

Am I right?
Comment 43 deshui 2018-04-02 00:41:19 UTC
NV12 + filesink doesn't work
NV12 + vaapisink works
other format + filesink works
Comment 44 GStreamer system administrator 2018-11-03 15:52:54 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/81.