Sounds like a good idea IMHO, are you planning to work on this?

Right now no.
I think it would be easier for those who have worked on the videomixer filter.
Moreover I am not sure to have the skill for that.

Would be much appreciated for the vdpau plugins :).

If ever anybody tackles this one (before me)... do *NOT* clone the videomixer code, and especially not the caching which videomixer is doing (in addition to the caching which is already done by gstcollectpads).

It's an awesome idea all the same :)

Does that even make sense?
I mean, if there was one video mixer that actually worked, why would anybody need another one?

We need one for :
* opengl
* vdpau
* whatever-format
* cairo (hello Benjamin)
* the currently handled formats in CPU
* the not-currently handled formats in CPU

And I'll repeat myself, but the collectpads/buffer handling part of current videomixer IS made of crack. It should be slimmed down before anyone else starts copy-pasting that code (hello Benjamin again :) )

I only copy-pasted that code so I have something to show off with, using cairotestsrc ! cairoxsink doesn't really give you the desired effect. :p
I only port elements I need for demos or testing specific (mis)features.

And no, I don't think it's a good idea to have a base class and extend it for every video format, even though it's probably a lot of fun to do so.

I'd rather have a generic video handling library that supports all formats out of the box and can - if the need arises - be extended to support new formats at which point all elements that use this library support the new format without any changes.

Otherwise plugging something like OpenGL, vdpau or cairo support into applications like totem or Pitivi is a non-trivial task, in particular because you have to do all the bridging between formats manually.
You'll get to answer all those hard questions like what mixer to use in "glsrc ! mixer ! tee ! xsink  vdpausrc ! mixer0. tee0. ! rawsink" - do you use the X mixer, the GL, the vdpau mixer or the raw mixer?

Copy-pasting videomixer to do cairomix was perfectly understandable. was just nagging you there :)

Why isn't a videomixer-that-does-it-all-and-brews-coffee-at-7am a good idea ? Because then we'd all of a sudden break the atomicity of the elements and link that one element against all those external libraries + duplicate helper code present in other plugins (gl, vdpau, cairo,...)

One idea would be to:
* rename the current videomixer to <somethingelse>videomixer
* Have a new meta-element for videomixing that will pick/convert in a smart fashion [1] which BaseVideoMixer subclass to use based on input/output caps.

We end up with:
* all pipelines/apps using one single videomixer element without caring what it's feeding it (apart obviously from knowing that it's some kind of video format).
* plugins for new systems (i.e. vdpau or cairo) providing new mixing implementations which can be used without rebuilding the widely used videomixer

[1] : No, I have no idea what smartness to use... but using the videomixer implementation which works in the configured output format would be a good start.

With 1.x, a different strategy should be used:

One single videomixer element
* Use a composition/overlay GstMeta
* If downstream can handle it, videomixer just sets the values and forwards
* If not default to cpu mixing.

Didn't we have another open bug about this meta ?

What would this videomixer element send downstream as the buffer that contains the overlays attached? A zero-sized buffer?

I baseclassed videomixer this week-end, and reimplemented videomixer and gstvideomeasure_ssim using it, which incidentally makes it be ported to 1.0 :)

The stuff works with these two cases, but I would need more time and opinions as to how the API should look like, what exactly we should do with that code (where will it go ? -bad ?)

I also implemented it on top of my branch which implements videoconversion in videomixer, so there's that.

The API for now looks like :

GstBasemixerPad*               (*create_new_pad)      (GstBasemixer *basemixer, GstPadTemplate *templ, const gchar* name, const GstCaps *caps);

gboolean                      (*modify_src_pad_info) (GstBasemixer *basemixer, GstVideoInfo *info);

GstFlowReturn                  (*mix_frames)          (GstBasemixer *basemixer, GstVideoFrame *outframe);

The names are not very well chosen, especially modify_src_pad_info which gets called during src pad negotiation, allowing videomixer2 to change the width and height according to the offsets (xpos ypos), and in ssim it lets ssim know the output size and select another format (GRAY8)

Something will have to be done with the sinkpads negotiation mechanism, not sure what exactly for now.

Edward, is your (2009) comment about caching being disposable crack still valid today ? For now, behaviour and code seems quite fine to me but I might miss something.

I'll be on IRC tomorrow so we can discuss all that at length, I hope you don't mind if I don't push the code right now as I'd like to finish cleaning it a little.

As said during the weekend already I don't think videomeasure-ssim implemented as a "videomixer" makes sense, it's a different synchronization problem that is solved there and ssim only really makes sense for two streams.

It makes sense for n streams actually and I will implement that, I wanna be able to compare multiple videos to a reference one, and doing so in one pass will be way more efficient.

That's independent of this bug here, let's not fill it with distracting information and let's talk on IRC later when you're there :)

*** This bug has been marked as a duplicate of bug 731918 ***