GNOME Bugzilla – Bug 753592
v4l2: implement TimeCode meta
Last modified: 2018-11-03 15:03:09 UTC
Currently, we have no way to retrieve the original timestamp of the v4l2buffer in downstream element due to the addition of the delay between the time at which buffer has been timestamped and the processing by v4l2src element. The original v4l2 timestamp could be useful for application or elements which needs the exact acquisition timestamp to perform some processing which involve external timestamped values. For instance here is my current use case: I have an element which takes raw frames, normally from a v4l2src, and perform video stabilization using data from an IMU (inertial measurement unit). These data are timestamped during acquisition, and are queried with timestamp. To perform accurate video stabilization, I shall use the frame acquisition timestamp. So I propose to add V4L2 metadata to buffer pushed by v4l2src containing the v4l2buffer timestamp. These metadata could be extended as well to contains other v4l2 specific fields. This changes is very specific to V4L2 and maybe overkill, I mean adding a meta just for a timestamp (currently). It may also be very specific to my use case. However, I didn't find an other way to retrieve this info, nor generic metadata which can handle this. Don't hesitate to comment P.S.: Could be linked with https://bugzilla.gnome.org/show_bug.cgi?id=753562
Created attachment 309217 [details] [review] v4l2: introduce GstV4l2Meta metadata Implementation of the proposal. Add a new library 'libgstv4l2' defining v4l2meta implementation. Also makes the changes in v4l2 plugins to add metadata if requested by downstream element.
See also #753562 which wants to embed another field in a GstMeta.
*** Bug 753562 has been marked as a duplicate of this bug. ***
I've asked to make this a duplicate, because they want to address the same problem. My main concert is exposing an V4L2 specific API here. I think we could consider adding a GstStructure based GstElementMeta, so we can have custom per element meta, without having to create a library for it. Also, GstStructure is easy to extend without breaking API.
(In reply to Nicolas Dufresne (stormer) from comment #4) > I've asked to make this a duplicate, because they want to address the same > problem. My main concert is exposing an V4L2 specific API here. I think we > could consider adding a GstStructure based GstElementMeta, so we can have > custom per element meta, without having to create a library for it. Also, > GstStructure is easy to extend without breaking API. It could be a way to solve the issue, but in this case, we may need to define when to implement a custom GstMeta or just use the GstElementMeta. I write a simple implementation of GstElementMeta as proposed and I currently see two problems: - what do to when a copy transform occurs and we just copy a region ? Should GstElementMeta contains only global metadata. ie not dependant of the region of the buffer. It could be partial answer to when use GstElementMeta or not. - how peer elements can know what will contains these meta before receiving one buffer, at negotiation time for exemple ?
Created attachment 309567 [details] [review] introduce GstElementMeta metadata (Proof of concept) A proof of concept of a GstElementMeta.
(In reply to Aurélien Zanelli from comment #6) > Created attachment 309567 [details] [review] [review] > introduce GstElementMeta metadata (Proof of concept) > > A proof of concept of a GstElementMeta. I forgot to precise. It apply on gstreamer-core.
Created attachment 309569 [details] [review] v4l2: use GstElementMeta to carry v4l2 timestamp (proof of concept) Exemple of usage of GstElementMeta with v4l2 timestamp.
Do we still need this ?
(In reply to Nicolas Dufresne (stormer) from comment #9) > Do we still need this ? On my side, yes. :) Also I think a structure based GstElementMeta is better for such metadata: additional timestamp, timecode... Do you think GstElementMeta could be implemented/integrated ? If so, I could rebase/continue work on that.
I'm just book keeping the bugs, I must admit I have little time to give to this. The GstElementMeta approach benefit is that we don't need yet another public Library.
There is a specific meta for the time code now, we could implement that. It only partly resolve the requested feature I guess, but as most party have abandoned by now, it should be good enough.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/211.