Created attachment 71957 [details] [review]
implement qos-queries and reporting

Created attachment 71958 [details]
plot data files generated by qos reporting

Since the QoS mechanism is pretty generic and not bound to a particular medium, maybe the QoS statistics querying should be medium-independent as well? Put differently: should we use something more generic in place of 'frames' here?

@tim: good point, what about dropped/rendered-buffers.

Beside we only do QoS or video right now and don't drop audio in general, so I am not sure. For me this is quite useful as it is right now.

My position hasn't really changed since our IRC conversation about this:

 Sep 01 12:11:38 <ensonic>	__tim: thanks for your comment on the qos-reporting patch, what about "buffers" instead of "frames"?
 Sep 01 12:12:35 <__tim>	'buffer' isn't really meaningful if the buffers aren't of uniform size/frequency, is it?
 Sep 01 12:12:55 <__tim>	depends what you want to measure I suppose
 Sep 01 12:14:00 <wtay>	yeah, too specific
 Sep 01 12:17:05 <ensonic>	wtay: any idea?
 Sep 01 12:17:26 <wtay>	 was thinking about an interface on the videosink

I was just idly commenting though, you should probably talk to Wim ...

After thinking more about this the term 'frames' is quite generic. The frame is the atomic quantity of media that goes through a pipeline. For video this one field (e.g. full picture), for audio a frame is one sample value for each channel.

Audiosinks can drop some frames of a buffer in the sink too. This they could also keep stats and report them. The report as such could maybe include the media type (e.g. audio/x-raw-int), so that the report can group the output by major madia type (audio/video).

Looks like a good idea IMHO.

To undermine that 'frames' is not that far-fetched, in gst-libs/gst/audio/audio.h we use the term frames in the context of audio.

TODO:
__tim> 
small nitpick unrelated to the API: please use a GArray for the gst-launch 
reporting, or at the very least g_slist_prepend() and then reverse the list before writing out the report, 's a bit more efficient: )

MikeS>
clearly moving reset_qos is wrong; you need to provide some other way to do what you want for that

-> maybe I should split, resetting qos data and stats

I haven't commented on this bug so far, because other people seemed to be voicing my objections for me.

I'm against committing this for the current release for the following reasons:

  * It doesn't represent critical functionality or a regression - it's an enhancement and it's one where the specifics are still contentious.

  * I don't like the idea of adding our first multimedia-specific terminology to the core. So far, we've kept the core (almost) completely video/audio agnostic and  I like that separation - it speaks to me about keeping a good abstraction. There must be a more general way to express the idea. Other people have suggested counting dropped buffers. Another idea would be to count the dropped vs kept percentage in some GstFormat - TIME, DEFAULT (samples or frames depending on the raw type), BUFFERS and BYTES are the way we normally express our units of processed-content, and I think they can apply here too.

  * I don't like the idea of dropping a qos.data file in gst-launch's current directory - the command line options should include a way of redirecting that.

I'm sorry - I understand that it's frustrating not being able to get traction and comments on a patch for such a long time, but it's also API I think we can get consensus on before it goes in.

I am fine with postponing this for later (you've got valid points in your comment).

* regaring the term 'frames'. Bytes does not make sense imho. Only percent would make sense imho (there is GST_FORMAT_PERCENT). Its maybe even better, because then one can formulate some test criteria that warn if a test-case drops more that a certain percentage of frames. The modification for percent would be simple - I can return
  percent_dropped=percent_dropped/(frames_rendered+frames_dropped);
  (percent_dropped=0.0 if (frames_rendered+frames_dropped)==0)
The value would be a double.

* qos.data file. Right, I can turn '-r' intp '-r <datafile>'. If only '-r' is given, I can just print the sumary table. or should I better add '--qos-data <datafile>' and if that is not given write no datafile.

Jan, its not so much the frustration, it more that this can be quite useful if it can be savely used accross gstreamer, but right now I am reluctant to not have too many local changes. It would be nice to have this in 0.10.14 in a solid way.

I changed the code to add another option: --report-file to specify the data-file-path.

One question: what do you think about extending the query structure-format so that it no just returns the percentage of dropped frames, but optionally also an array/slist of GST_CLOCK_TIME values. Then I could improve the plotting, so that one can see which element dropps when.

I thought about something like that too, and my first thought was that it's that same situation as wanting to find out the current position at each point in the pipeline, for example. 

Our usual solution for that is to zip through querying each element and gathering the results - a utility function "gst_bin_query_all" or something that constructs a GList of element references and their response to the query, plus a cleanup function, seems like a more general solution to the problem to me... although I can see others might not think it elegant :)

Created attachment 89941 [details] [review]
implement qos-queries and reporting

add switch for filename

Created attachment 89991 [details]
mockup of plot with additional data

The image shows a fake, that has vertial bars there to indicate when a frame was dropped. I am not sure wheter the element should keep a drop list (needs memory) or it sends a drop message to the bus (only needs memory if the app is collecting the info, but causes more communication overhead).

By the time I came to the conclusion that we probably don't want to add permanent API for such meassurements. I wrote a preload library over the last days which can grab simmilar data, without the need to mess with the code.

http://www.buzztard.org/files/gst-preload-0.1.tar.gz

I wonder it there would be interest to put that as (uninstalled) lib into core?

Every time you use LD_PRELOAD, God kills a kitten.

Poor kittens, but despite of alternatives, s/gst-preload/gst-tracelib/ -> http://cgit.freedesktop.org/~ensonic/gst-tracelib/ - 0.2 in preparation.

Some commits went in for bug #322947. With that one can listen for dropped frame on the bus and do stats on EOS for how many frames where dropped and by whom.

I'll close this now due to lack of interest.

There's GST_MESSAGE_QOS now...

(In reply to comment #21)
> There's GST_MESSAGE_QOS now...

... that no one posts :/ Maybe a good starting point would be basesinks.