GNOME Bugzilla – Bug 763522
gst_element_query_position doesn't always return the right frame number
Last modified: 2018-11-03 13:47:51 UTC
Created attachment 323743 [details] Log amc*:6 This might not be a bug, I don't know. It seems that on some rare occasions, gst_element_query_position returns the real frame number+1 or the real frame number-1. My pipeline on Android 6.0.1 with gstreamer-1.7.90 is: filesrc location=path ! queue ! decodebin ! queue ! identity ! glimagesink On the identity element, I have a pad callback: GstPad *pad = gst_element_get_static_pad(identity, "sink"); gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_BUFFER, (GstPadProbeCallback)cb_have_data_0, data, NULL); in which I do: frameNbInc++; gst_element_query_position(pipeline, GST_FORMAT_DEFAULT, &frameNb); GST_DEBUG("%lld %lld %lld", frameNb, frameNbInc, frameNb - frameNbInc); The debug log of amc*6 is attached. On line 53 and 138, I have frameNb - frameNbInc == 1 while the delta between an internal incremental counter frameNbInc and the frame number returned by gst_element_query_position should be 2 and is 2 almost all the time. The delta of -1 always happens after the "Queueing buffer", for instance lines 41 to 52, or lines 123 to 137. Sometimes, the delta is +1 like on line 264. It seems that there was a race condition between line 263 and 264. Is this a bug? Or how can I be sure to have gst_element_query_position to return the equivalent of an incremental number with the condition that the system is not late, aka. everything runs smoothly like in this trace.
Ideally the sink should be the one replying to this but depending on the elements involved (and I believe it is rare to see an element that supports the DEFAULT format position reporting) a few things might happen: 1) The sink can't do the position in DEFAULT format and tries to do some magical conversion of its current time to buffers and it might not be perfectly accurate 2) The sink will ask upstream for the position, so the answer will be ahead of the position in the sink. Suppose the sink is rendering buffer number 5, upstream the decoder/demuxer are, at least, dealing with buffer number 6 or greater. I think this would explain this difference you see. What exactly are you trying to do? Perhaps there is a better way.
Thank you for taking a look at this unusual problem. I have some meta data specific for each frame and I need to use such meta data when each frame has been decoded and is about to be rendered. The association of the metadata needs to be precise at the very frame. As you might have guessed the metadata is based on frame number. Why frame number and not time-stamp? Because I was not seeing consistent and reliable time-stamp across multiple playbacks on different platforms. With frame number, it was completely consistent.
Two options here, we could implement the DEFAULT format position query in GstVideoSink by just counting frames... or you could count frames in your application :) In both cases, QoS needs to be taken into account as frames could be dropped because of performance reasons.
I'm actually counting the frames in my application. But if I seek, I need to reset my counter and it's a little bit challenging to make sure that I have the right number. I have been very confused about DEFAULT. I do remember that default didn't return the frame number in libav software decoding - it returns 0 all the time if I remember well. Hardware decoding is returning a number like I showed in my log.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/360.