After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 793683 - gstclockoverlay: Add ability to specify which time to be displayed by GstClockOverlay
gstclockoverlay: Add ability to specify which time to be displayed by GstCloc...
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
1.10.x
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2018-02-21 05:50 UTC by Abhinav
Modified: 2018-11-03 12:03 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Patch for adding ability to specify time to be displayed by GstClockOverlay (7.38 KB, patch)
2018-02-21 05:50 UTC, Abhinav
none Details | Review

Description Abhinav 2018-02-21 05:50:43 UTC
Created attachment 368688 [details] [review]
Patch for adding ability to specify time to be displayed by GstClockOverlay

GstClockOverlay presently displays buffer arrival time as default. In certain scenarios, users would like to display buffer's timestamp, rather than buffer arrival time - to avoid any gaps caused by latency between frame source and clockoverlay element in pipeline. 

I propose to add property called "time-mode", where users should be able to specify what time to show. So other than current default, there can be another option for time-mode say "buffer-time", to indicate that clockoverlay will be using buffer's time for display. 

Example element configuration can look like following : 

clockoverlay halignment=right valignment=bottom time-format="%Y-%m-%d %H:%M:%S" time-mode=1

Where time-mode=1 represents "buffer-time".

Attached is the implementation for the same.

Note: GstTimeOverlay uses buffer time, but that doesn't have capability to display in clock format. Since clockoverlay is used to display text in clock format, it was found more suitable for said requirement.
Comment 1 Nicolas Dufresne (ndufresne) 2018-02-21 16:01:19 UTC
Review of attachment 368688 [details] [review]:

Weird, how is this different from using timeoverlay with the already implemented time-mode ?
Comment 2 Abhinav 2018-02-23 00:39:24 UTC
Sorry for late response. AFAIK, timeoverlay displays monotonic clock value in numeric format and doesn't have capability to display in clock time format(e.g. date-time format). For e.g. in timeoverlay you don't have option to specify "time-format". Based on my limited understanding, this is the primary difference between timeoverlay and clockoverlay - otherwise they look almost the same.
Comment 3 Nicolas Dufresne (ndufresne) 2018-02-23 01:03:09 UTC
clockoverlay is meant to be the "wall clock", what you see on your watch. Jitter on the processing will affect the value. Note that the formating is same as strftime(), so no fraction of a second.

timeoverlay is meant to print information that travels with the buffer. The speed of processing should have no effect on the numbers. Many of the strftime() formatter makes no sense.

If I understand well, what you would like is the ability to format the output of timeoverlay. While your patch changes the intent of the plugin itself. Using strftime() to format this is a bad fit. Right now it simply uses the fixed debug formatter found in GStreamer:
  
  "%" GST_TIME_FORMAT, GST_TIME_ARGS (time)

But writing a similar formatter, but that supports day, hours, minutes, seconds, fraction (currently millisecond) and enabled it as property to timeoverlay would in my opinion make a lot more sense.
Comment 4 Abhinav 2018-02-23 01:24:26 UTC
Well - thanks for explaining the intent and difference between clockoverlay and timeoverlay. While, what you explained makes sense - as A novice it was hard for me(and probably others) to get this interpretation correct by looking at plugin names "timeoverlay" vs "clockoverlay".

Anyways, we used clockoverlay and found that it is not same as "Wall Clock" value of stream which is being recorded and there is a difference (more than few seconds) in our pipeline. We realized that it is because clockoverlay uses "current time" and has no direct relation with "Wall Clock" of stream capture time. 

I am not sure, whats the use of such wall clock for end users, if this wall clock doesn't have any predictable relation with stream time. Then we realized that there is a need for user to specify "What time he wants to display in wall clock format" ? Is it the wall clock time of buffer arrival at clockoverlay ? Is it the wall clock time of recording ? or any other. IMHO, most users would be interested in Wall Clock which corresponds to buffer.

IMHO, clockoverlay needs a property for users to be able to specify which "Wall Clock" time they are interested in ? For many users, if wall clock has no relation with buffer, it may not be useful enough.
Comment 5 Nicolas Dufresne (ndufresne) 2018-02-23 03:08:19 UTC
(In reply to Abhinav from comment #4)
> Well - thanks for explaining the intent and difference between clockoverlay
> and timeoverlay. While, what you explained makes sense - as A novice it was
> hard for me(and probably others) to get this interpretation correct by
> looking at plugin names "timeoverlay" vs "clockoverlay".
> 
> Anyways, we used clockoverlay and found that it is not same as "Wall Clock"
> value of stream which is being recorded and there is a difference (more than
> few seconds) in our pipeline. We realized that it is because clockoverlay
> uses "current time" and has no direct relation with "Wall Clock" of stream
> capture time. 

It renders the wall clock time of when the buffer passed through clockoverlay, so depending on your pipeline it can be way in the past when it reaches your display.

> 
> I am not sure, whats the use of such wall clock for end users, if this wall
> clock doesn't have any predictable relation with stream time. Then we
> realized that there is a need for user to specify "What time he wants to
> display in wall clock format" ? Is it the wall clock time of buffer arrival
> at clockoverlay ? Is it the wall clock time of recording ? or any other.

It's what I just said before, the users decides the when by placing the clockoverlay at specific place in it's pipeline. If you have a live capture, and want something close to the capture time, place it right next to your capture source.

> IMHO, most users would be interested in Wall Clock which corresponds to
> buffer.

I'm not sure what you want exactly. In GStreamer, there is no relationship between buffer time and wall clock. There is though a relation ship between buffer timestamp / segment and the pipeline clock. The pipeline clock can be the system monotonic clock (which is not a wall clock) or an clock derived from audio ring buffer, which has event less relationship with a wall block.

timeoverlay currently don't have a mode that display the pipeline clock of a buffer, this is running-time + base_time, though it's not a useful value, the running-time is much nicers, it's common to all synchronized buffers and starts at 0.

> 
> IMHO, clockoverlay needs a property for users to be able to specify which
> "Wall Clock" time they are interested in ? For many users, if wall clock has
> no relation with buffer, it may not be useful enough.

I'm sorry, I don't understand what you mean by "which wallclock", do you mean Time Zone support ? I have never had a clock on my wall that runs at different rate, so it's hard to think of a different wall clock (outside timezone of course, which could be fine addition).

That being said, documentation update are always welcome.
Comment 6 Abhinav 2018-02-23 04:42:18 UTC
Hi Nicolas, 

Thanks for quick comment and guidance. 

> It's what I just said before, the users decides the when by placing the
> clockoverlay at specific place in it's pipeline. If you have a live capture,
> and want something close to the capture time, place it right next to your
> capture source.
> 
> > IMHO, most users would be interested in Wall Clock which corresponds to
> > buffer.
> 
> I'm not sure what you want exactly. 

Sorry for not being able to clearly articulate our requirement earlier. 
What we want is to display "wall clock" at the time of video capture (with sub second precision), in the format as already supported by clokoverlay.

Even if we place clockoverlay just next to video capture source (e.g. v4l2src) in our pipeline, we observed gap of few seconds as compared to wall clock time of recorded video. There can be several reasons (e.g. buffer cached at source and so on..), So placing clockoverlay just next to capture source is not enough to achieve desired precision. 

Any suggestions for meeting above requirement is welcome.
Comment 7 Nicolas Dufresne (ndufresne) 2018-02-24 13:11:46 UTC
(In reply to Abhinav from comment #6)
> Hi Nicolas, 
> 
> Thanks for quick comment and guidance. 
> 
> > It's what I just said before, the users decides the when by placing the
> > clockoverlay at specific place in it's pipeline. If you have a live capture,
> > and want something close to the capture time, place it right next to your
> > capture source.
> > 
> > > IMHO, most users would be interested in Wall Clock which corresponds to
> > > buffer.
> > 
> > I'm not sure what you want exactly. 
> 
> Sorry for not being able to clearly articulate our requirement earlier. 
> What we want is to display "wall clock" at the time of video capture (with
> sub second precision), in the format as already supported by clokoverlay.

Maybe you should have used capture-time then to describe your mode. There is an issue though, there is no sub-second precision in strftime(). I still think timeoverlay would be a better place for this.

> 
> Even if we place clockoverlay just next to video capture source (e.g.
> v4l2src) in our pipeline, we observed gap of few seconds as compared to wall
> clock time of recorded video. There can be several reasons (e.g. buffer
> cached at source and so on..), So placing clockoverlay just next to capture
> source is not enough to achieve desired precision. 

Obviously clockoverlay is not going to match the HW capture timestamp. Though, few seconds tell me that you omitted to properly describe what you are trying to test. I believe if you could enable time codes for you stream, implementing timecode support and overlaying both running time and time code using timeoverlay could do a much better job.
Comment 8 GStreamer system administrator 2018-11-03 12:03:36 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/420.