GNOME Bugzilla – Bug 117692
[docs/mime] docs/random/mimetypes needs updating
Last modified: 2007-11-15 21:42:06 UTC
The current documentation leaves out 2 important issues: 1) boundaries 2) media-specific offset 1) boundaries There are implicit assumptions in nearly all plugins about where you may split a stream into buffers. Video often does 1 frame/buffer, raw audio does channelcount samples per buffer and so on. 2) media-specific offset GstFormat and GstBuffer specify a "media specific" offset/format. For every mimetype there needs to be documentation on what they mean. Apart from that, it needs to get out of docs/random and into the official documentation.
I agree. For point 1, we used to use a property 'framed' or 'parsed' for this (more names were used, they all meant the same and there was no common convention...). If this was set to TRUE, then the buffer contained parsed stuff, e.g. an X amount of raw audio samples (aligned by channels * samplesize), or one frame per buffer (video, mp3). We should probably move this back, agree on a common name and make each plugin use it. The rule is then that parsed=TRUE means bordered by whatever is documented in the docs. parsed=FALSE means stream data, no alignment whatsoever. As for point 2, I don't know how that should work either. I don't think it's media-specific, but mediatype-specific. E.g. audio samples (I think mp3 uses samples here, too), video frames. As for point 3: yes.
Doesn't the MIME type itself describe whether the stream info is in encoded/decoded form ? Also, Wingo started adding a 'buffer-frames' property to float audio---the number of audio frames in a buffer (a frame is a set of samples, one from each channel). This is for synchronization with JACK networks, but it's a pretty good idea anyway. We could add the same property to int audio, seems like this would take care of data boundaries in the raw audio formats. The others, though ... :) As for integrating the document into the official docs, should the mimetypes file go in the App Developer's Manual or the Plugin Writer's Guide ? I'd vote for the PWG, it already has a brief description of common MIME types, and application developers shouldn't have to deal too much with internally used MIME types like audio/x-raw-int.
PWG. As for the boundaries, it's not intended as decoded/encoded. It's intended as a "are we parsed into single frames or not", Imagine MPEG, where one MPEG packet could contain exactly one frame (unlikely), or could contain part of the previous frame and part of the next frame (ugh!). Obviously, this will work for decoders/formats that expect this (fortunately, most do), but not for the ones that don't (example: AVI). Same goes for others, like mp3. Matroska, too, uses this, they call it 'lacing', which can be turned on and off to someone's liking. As for frames-per-buffer, there's some convenience macro's in gst-plugins/gst-libs/gst/audio/audio.h for that, I don't think it should be part of the caps, since you assume that each buffer will contain the same amount of data, which is by definition not always true in GStreamer. I would tend to disagree with adding it...
The mime types need to be documented in the API docs or be a seperate document. IMO the PWG is a HOWTO-type document, not a reference.
I guess this belongs to me. :).
For what it's worth, I've documented the mimetypes et al in the PWG. The .xml file can simply be used elsewhere if we want to make it a full document on its own, Benjamin does have sort of a point there. The original points of this bug still need work...
Ronald, ping?
Problem with putting it into gtk-doc is that it will be scattered. I agree that each element would need to document the caps there. Maybe we can improve the introspection mechanism here a bit. But the overview that ronald has put into pwg is definitely useful. I recently updated it, by running the media-test suite and postprocessing the log to get a list of uniq caps.
I think that every caps that is used by GStreamer should be documented in reference documentation. This is as much API as all the other structs, functions, etc that get exported in header files. And I'd want to find it when looking for audio/vorbis in devhelp.
As an app dev., I'd expect to find it: - in the PWG or ADM (which is done now, good job Stefan!) - per caps for each element in their respective docs (I think the caps templates are listed there, so that's done also) - as Benjamin said, in the gtk-doc ref. This can probably be generated in the same way as the list in the PWG/ADM (i.e. it can be shared from the same help "source file") Once that's done and someone checked it to be correct, this bug is done and can be closed. You probably want to add a note on private caps (e.g. what ffmpeg uses if we don't have a gst-caps for it), and also say that that's not recommended and maybe even make a start on adding capses for all the ffmpeg formats.
Talking to myself. To get caps info into plugin docs, we need to extend common/gst-xmlinspect.py and common/plugins.xsl. I just added to FIXME comments there for now.
2007-06-25 Stefan Kost <ensonic@users.sf.net> * gst-xmlinspect.py: * plugins.xsl: Also extract element caps for plugin-docs. Fixes parts of #117692. Can I close the bug?