After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 651806 - Implement Video compositing and Audio Mixing
Implement Video compositing and Audio Mixing
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-editing-services
unspecified
Other Linux
: High enhancement
: 1.2.0
Assigned To: vladimir.rudoy
GStreamer Maintainers
Depends on: 678558
Blocks: 632319
 
 
Reported: 2011-06-03 19:03 UTC by Thibault Saunier
Modified: 2013-09-24 16:03 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Thibault Saunier 2011-06-03 19:03:54 UTC
Until now, there is no way to mix audios/videos from various layers. We need to get something with a similar behavior as what now have in pitivi.

The solution in pitivi is:
 * Have mixer elements smartVideomixerBin and SmartAdderBin (in pitivi/elements/mixer.py)

 * All tracks add this mixer in a gnloperation with a priority of 0 and which is expandable so it is in the whole duration of the track so any trackobject added to track is plugged on the mixer in pitivi/timeline/track.py

This result in:
    + layers (audio/video from different layer) are mixed when the alpha channel of objects on higher layers are < 100

We also currently have an optimization that take advantage of the alpha passthrough capabilities of the videomixer, we should try to keep it.

If we want to reproduce something similar, we re-implement those custom GstElement-s and modify ges/ges-track.c to handle it.

In GES, we should use the new video mixer: videomixer2

Also I guess this is something we might want to handle directly in GnlComposition, passing a videomixer or adder to it?
Comment 1 Luis de Bethencourt 2011-06-06 15:26:02 UTC
Looking into it/Working on it :)
Comment 2 Jean-François Fortin Tam 2012-03-29 02:24:06 UTC
Currently, this is one of the key regressions in Pitivi. As nobody is working on it at the moment, unassigning.
Comment 3 Thibault Saunier 2013-08-16 08:50:07 UTC
This bug has been fix between

commit 9e0632e353cfbd2301f8d74a7398c4276371a5a3
Author: Mathieu Duponchelle <mathieu.duponchelle@epitech.eu>
Date:   Mon Jun 3 23:02:15 2013 +0200

ges: Add a framepositionner element used in ges-smart-mixer and ges-uri-source

It adds metadata on the buffers and the mixer parses them.
This is done because we want to keep positionning properties
and set them on the dynamic mixer pad.


and ...

commit 7e25aaeccc6334343ea5b179a3347d32444b5b8c
Author: Thibault Saunier <thibault.saunier@collabora.com>
Date:   Sat Mar 30 19:02:52 2013 +0100

    track: Implement infrastructure for mixing