Not all frames make it to the sink, some frames are dropped by the decoders or by the conversion elements. Maybe that explains your observation.

Maybe also the decoder that drops a frame does not correctly post a QOS message. What decoder is in use here?

The code to calculate total frames and number of dropped frames in the sink is very simple, I would be surprised if there was a bug there.

Why are elements which drop frames not reporting that?

Also, what about the inflated numbers in my second example? According to those messages it should be a ~80fps video while it is in fact 25fps.

(In reply to comment #2)
> Why are elements which drop frames not reporting that?

Because they have a bug.

> 
> Also, what about the inflated numbers in my second example? According to those
> messages it should be a ~80fps video while it is in fact 25fps.

Where do you see that?

(In reply to comment #3)
> (In reply to comment #2)
> > Why are elements which drop frames not reporting that?
> 
> Because they have a bug.

I guess it is tricky to know which ones if they are not reporting it. Since I have two different media types where I saw this, does it mean buggy elements are likely to be common elements, or that different media demuxers/decoders could be buggy just as well?

> > 
> > Also, what about the inflated numbers in my second example? According to those
> > messages it should be a ~80fps video while it is in fact 25fps.
> 
> Where do you see that?

In my second example above, which is output from a PyGst test program. I can attach that program here, but test stream is to big at 35Mb.

> > Because they have a bug.
> 
> I guess it is tricky to know which ones if they are not reporting it. Since I
> have two different media types where I saw this, does it mean buggy elements
> are likely to be common elements, or that different media demuxers/decoders
> could be buggy just as well?

All sinks and basetransform derived elements should work correctly. I know theoradec does it correctly too. I know ffmpeg does not do it correctly and I assume most other video decoders are wrong as well.

Additionally, dropped seems to refer to frames dropped specifically because of being too late. This counter will not account for frames dropped for other reasons, such as waiting for a keyframe, decoding error, etc.

I'm going to look at that issue now.

More findings:

The deinterlacer outputs (on some videos) twice as many frames as it gets in input. The sink (and any element downstream of the deinterlacer, when actually deinterlacing) will see the larger amount. This is seen on the c4-sync.ts sample, where deinterlacing causes twice as many frames to reach the sink.

Additionally, the deinterlacer can switch on and off based on whether input interlacing changes, which can, on some inputs, cary actual frame rate seen by the sink in a non linear way.

Last, on the c4-sync.ts sample, ffmpeg reports the file to be 27.02 fps, but when looking at the timestamps and the duration, the real fps is about 23, making a direct comparison of number of frames against framerate*duration error prone.

(In reply to comment #7)
> More findings:
> 
> The deinterlacer outputs (on some videos) twice as many frames as it gets in
> input. The sink (and any element downstream of the deinterlacer, when actually
> deinterlacing) will see the larger amount. This is seen on the c4-sync.ts
> sample, where deinterlacing causes twice as many frames to reach the sink.
>
> Additionally, the deinterlacer can switch on and off based on whether input
> interlacing changes, which can, on some inputs, cary actual frame rate seen by
> the sink in a non linear way.
> 
> Last, on the c4-sync.ts sample, ffmpeg reports the file to be 27.02 fps, but
> when looking at the timestamps and the duration, the real fps is about 23,
> making a direct comparison of number of frames against framerate*duration error
> prone.

Noted - so that approach goes out the window. I guess that makes it more important to make everyone who drops frames actually report it via a QOS message.

I'll try to capture a stream which demonstrates this.

Thanks, but I don't need a new sample at the moment, I can see the issue here.

The deinterlacer was not sending QoS messages when dropping late frames.

https://bugzilla.gnome.org/show_bug.cgi?id=657941

I'm going through others elements used by playbin2 to check if others don't.

Done for ffmpegdec too.

Done for mpeg2dec too, that's the last element that was missing it in those used in a playbin2 pipeline playing a MPEG TS file.

Looks like we can close this. If any others are missing, please file bugs against the decoder elements that don't implement this properly, thanks!