GNOME Bugzilla – Bug 784372
videorate: support frame-by-frame for multiview-mode
Last modified: 2018-11-03 11:57:25 UTC
Recently I've worked for MVC encoding using vaapi. Basically it's been successful with static pipeline, but I found one issue when I test with encodebin. My test pipeline is as the following: > gst-launch-1.0 gltestsrc ! glviewconvert input-mode-override=mono output-mode-override=frame-by-frame ! glcolorconvert ! gldownload ! encodebin profile="video/mpegts, systemstream=true, packetsize=188:video/x-h264" ! filesink location=mvc_va.ts I thought it's working fine but I realized it's not. That is videorate drops one of two frames every time by comparing timestamp of each buffer, which is what videorate should be doing normally. So I'm asking this question. Do we need to implement something in videorate to support multiview-mode, especially for frame-by-frame? Or is there another solution for doing this?
frame-by-frame mode needs to be handled differently somehow. The assumption right now is that video/x-raw always contains one frame for one time per buffer, but in your case you have two frames in two buffers following after each other for the same time. These probably either need to be combined into a single buffer, or different caps might have to be used.
(In reply to Sebastian Dröge (slomo) from comment #1) > frame-by-frame mode needs to be handled differently somehow. The assumption > right now is that video/x-raw always contains one frame for one time per > buffer, but in your case you have two frames in two buffers following after > each other for the same time. > > These probably either need to be combined into a single buffer, or different > caps might have to be used. Currently, multiview-mode is set on the caps, and I think it can be used in videorate to handle this case. (Until GstVideoMultiviewMeta is enabled maybe?)
videorate would have to track each view separately, yeah. 'separated' mode - where all views are attached as an array of GstMemory on a single buffer should work, but 'frame-by-frame' needs explicit support. The idea for that, btw, has always been to add a multiview meta as Sebastian mentioned. The current method of 'set the flag on the first buffer of the view bundle' isn't perfect since it means a buffer can't be easily identified on its own - you have to be able to count how long it's been since the buffer with the flag.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/361.