After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 784372 - videorate: support frame-by-frame for multiview-mode
videorate: support frame-by-frame for multiview-mode
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
git master
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2017-06-30 07:38 UTC by Hyunjun Ko
Modified: 2018-11-03 11:57 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Hyunjun Ko 2017-06-30 07:38:59 UTC
Recently I've worked for MVC encoding using vaapi.
Basically it's been successful with static pipeline, but I found one issue when I test with encodebin.

My test pipeline is as the following:

> gst-launch-1.0 gltestsrc ! glviewconvert input-mode-override=mono output-mode-override=frame-by-frame ! glcolorconvert ! gldownload ! encodebin profile="video/mpegts, systemstream=true, packetsize=188:video/x-h264" ! filesink location=mvc_va.ts

I thought it's working fine but I realized it's not.
That is videorate drops one of two frames every time by comparing timestamp of each buffer, which is what videorate should be doing normally.

So I'm asking this question.
Do we need to implement something in videorate to support multiview-mode, especially for frame-by-frame?
Or is there another solution for doing this?
Comment 1 Sebastian Dröge (slomo) 2017-07-03 06:00:58 UTC
frame-by-frame mode needs to be handled differently somehow. The assumption right now is that video/x-raw always contains one frame for one time per buffer, but in your case you have two frames in two buffers following after each other for the same time.

These probably either need to be combined into a single buffer, or different caps might have to be used.
Comment 2 Hyunjun Ko 2017-07-04 04:05:20 UTC
(In reply to Sebastian Dröge (slomo) from comment #1)
> frame-by-frame mode needs to be handled differently somehow. The assumption
> right now is that video/x-raw always contains one frame for one time per
> buffer, but in your case you have two frames in two buffers following after
> each other for the same time.
> 
> These probably either need to be combined into a single buffer, or different
> caps might have to be used.

Currently, multiview-mode is set on the caps, and I think it can be used in videorate to handle this case. (Until GstVideoMultiviewMeta is enabled maybe?)
Comment 3 Jan Schmidt 2017-07-04 05:45:26 UTC
videorate would have to track each view separately, yeah.

'separated' mode - where all views are attached as an array of GstMemory on a single buffer should work, but 'frame-by-frame' needs explicit support. The idea for that, btw, has always been to add a multiview meta as Sebastian mentioned. The current method of 'set the flag on the first buffer of the view bundle' isn't perfect since it means a buffer can't be easily identified on its own - you have to be able to count how long it's been since the buffer with the flag.
Comment 4 GStreamer system administrator 2018-11-03 11:57:25 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/361.