Created attachment 26040 [details] [review]
GStreamer Encoding Profiles patch (for gst-plugins)

This patch adds a GStreamer Encoding Profile interface + library backend to
GStreamer. This is an element interface with one single function, i.e. to
retrieve a list of default (provided) profiles from the element. Next to that,
it allows setting user-specified profiles. A profile, like in Thomas' initial
proposal, consist of a combination of (GObject) properties. The lib provides
some wrapper functions to edit/add/remove profiles, set profiles (which is a
set_property() wrapper), and some more.

Created attachment 26041 [details]
Profile Editor

This is a Profile Editor based on the backend in the first patch. It is a
GNOME-2.4 UI (didn't update it to gtk-2.4 yet; it uses the deprecated
GtkOptionMenu widget) based on a Glade XML file.

[ adding PATCH keyword ]

Some notes on the above patch:
* I'm not happy with the XML code. Someone with knowledge should give me some
advise on how to save profiles to disk, where & such (specifically, how do I
avoid duplicate entries in system/user registries [should I?], do I want to poll
profiles or just open the file once and be done with it, ...).
* I currently only support elements which implement the interface, though there
is no real reason for this limitation. I can remove it, so do we want that? The
interface then only exists for elements providing default profiles.

aren't media profiles in gnome since gnome 2.6/gstreamer 0.8 ?

They're not enough for video yet.

Marking previous patch as rejected, it doesn't solve the issues the way we want
to. I've been discussing this issue with Edward multiple times recently, I'll
put a log here when we've got a proposal that we think solves the problem
adequately.

Created attachment 38831 [details]
New proposal

Attached is a new proposal from me and edward, which we think suits our needs,
is a good, viable and long-term replacement for GNOME audio profiles and is
both fluent enough to allow for more complex cases as well as still simple to
actually be a useful helper for our goal.

I got some questions on how to handle the difference between video/audio
profiles. This is implicit. Every profile will consist of a muxer (or not) with
request/always pad templates. Based on this and the caps, we already know the
amount of required and supported audio/video links. The subprofile linked at
front (which is 1-to-1) is then unimportant. I'll write a utility function for that.

Here's how elements should work:
avimux:
* one always video pad
* a request pad template for audio (0, n)
oggmux:
* one request serial pad template (0, n)
wavparse:
* one always audio pad
mplex:
* a request video pad template, with one always available (created on _init, not
released during _release_request_pad()) - (1, n)
* request audio/subtitle pads (0, n)
matroskamux:
* request (0, n) video/audio pads

Then, the utility function simply goes through all pad templates on the bin,
iterates pad templates and sees whether the 'application requires/provides
input' matches a template. If, after iterating, one or more always pad templates
(or existing pads) are unlinked, or one did not match, then the application does
not want to use this profile. Else it does.

Example 1, gnome-sound-recorder (provides one audio):
avimux: audiopad matches, but always videopad unlinked - no display
matroskamux: audiopad matches, no always pads unlinked - display
mplex: audiopad matches, but one existing requestpad unlinked - no display
wavparse: audiopad matches, no pads left - display
[etc.]

Example 2, Cupid (provides one video, one audio):
avimux: videopad matches, audiopad matches, none left - display
matroskamux: same
mplex: same
wavparse: audiopad matches, video doesn't match any pad - no display

The logic is not that much code, and if we provide a utility function,
applications won't even notice.

Here's some first tries at user interfaces for profile editing (using my
knowledge of my previous failed attempt). This has nothing to do with
applications, they will just have a selectionbox.

#1 - main window
http://ronald.bitfreak.net/priv/profile-main.jpg
Main window, list of profiles, create new, delete, edit, simple as that.

#2 - edit or create profile
http://ronald.bitfreak.net/priv/profile-edit.jpg
It has a name, and the thing below is a gnomecanvas... I want to draw something
similar to gst-editor, but a lot simpler... basically stacked subprofiles
(vertically) linked in order (horizontally). Each subprofile is clickable to
edit/delete, and add makes a new one.

#3 - edit or create a sub-profile
http://ronald.bitfreak.net/priv/profile-sub-edit.jpg
Also see the text. Name, list of other subprofiles for linking at input (all
checkbuttons), same for output; in between a list of components (elements or
filters) which, in that order, make up the subprofile.
Element edit/add is done using the gst-editor element selection widget.
Edit/create filter is done by clicking an element, then it'll ask for
source/sink-side (for add), and then picks the template caps (or the
intersection, if the element is linked) and provide that as options, and you'll
be able to select properties like in the ascii mockup in the text.

Assigning to Mr. Hervey.

If combined with the preset iface I propose in bug #396779 then the profile only needs to deal with which elements get linked in which order and which (named) preset shall be used on each element.

Any progress on this?

The preset iface solves the problem, where a couple of fixed settings make sense. For media encoding it means we limit the user to select between e.g. quality={poor, average, high}.

Is that good enough? The we should implement presets for encoders and close the bug.
 
Or do we want an interface that allows to configure the quality from 0% to 100% on elements:
* jpegenc would translate that directly to jpeg quality
* audio/video encoders would map that to a good bitrate-range (depending on resolution?)

The iface could have the following methods:
_set_quality(gfloat quality);
_get_quality();
_get_description();
The latter would return a textual description suitable for showing in a ui (e.g. resulting bitrate).

As a complete outsider, I disagree with that. Limiting yourself to such an uninteresting subset of codec features will ensure that nobody will use it or take it seriously.

Take a look at win32/MacOSX graphical encoders. They go from the ground up to provide control of all details. what kind of block do you use for quantization? 4x4? 4x8? Some codecs only support one, some multiple, etc. Quality of output is very different in each case. Global Motion Compensation or only per-block? Look at all options that high-quality MPEG-4 encoders provide. Look at h.264. A float that describes "quality" is just insufficient.

What you want is layers, and your design needs to address this. At the basis, you want control of all individual codec options, with selection boxes. I'm sure GObject properties could be used for this, but you need flags that tell you which ones affect codec quality.

On top of that, you could use a series of "presets" (in a different manner of the word) that combine multiple codecs/filters plus pre-set properties to describe a subset of a pipeline encoding a media.  Users could take a global approach and select "high quality ogg/vorbis" or "low quality 8-bit PCM audio", as in the current profiles in gnome-media. Or they could specify their own and add GMC, 16x16 block encoding etc. in h/264 with AC3 audio at a 192bps bitrate, 5-channels etc.

Ronald, you got my a bit wrong. I am aware of all those options. My pov is that they are overwhelming for most users. For expert users we can just generate a UI that simply offers all parameters in a UI. And we can provide snapshots of parameter-set as presets to be used as a starting point.

I am aware that sometimes the reality is so complex that trying to simplyfy things is doomed to fail. Still I think its worth trying to come up with something simple that provides limited control as a interface for normal people. Its not perfect, it does not offer full control, but they can achive what they want.

/me takes a look at the export settings in Final Cut

- frame rate

- key frames: automatic/every N/all

- frame reordering: yes/no

- data rate: automatic/restrict to N kbits/sec

- optimized for: download...

- quality (slider): least/low/medium/high/best

- encoding: best-quality(multipass)/faster encode(single-pass)

Do you take Final Cut seriously?

(Also, as a codec writer, no user needs to touch block size settings.  The encoder already knows the best setting.)

final Cut is one of the many, David. Other software, particularly Win32 freeware (hey, I'm a student) does offer those options. I was just trying to give an extreme example to counter the point of the "float quality".

In the end, use my comment as you wish, I'm not saying you need to allow the user to set block size, I'm saying that a "float quality" is not sufficient, not for anyone, and your design should keep that in mind.

sorry for the noise. :).

Some design work on http://gstreamer.freedesktop.org/wiki/ZeroPointEleven/EncoderConsensus

See also pitivi bug 593466 for some detailed description of the problems this causes with Theora and possibly other codecs.

Changing bug description to make more generic.

From IRC conversation:
> ds: my preference is to have common properties for all encoders:
> ds: bit rate, key frame interval, profile, level, rate control, quality

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/1.