GNOME Bugzilla – Bug 733727
glimagesink GstVideoOverlay not working on Windows OS
Last modified: 2016-11-10 14:51:00 UTC
I am embedding the gstreamer output inside my application. However glimagesink will not use the window handle I am passing it with gst_video_overlay_set_window_handle. If I use the d3dvideosink then video is shown ok inside my application.
Could you provide a minimal example that reproduces the pb ? Have you tried the example http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/gtk/gtkvideooverlay ?
I can confirm bug. - OS: Windows 8.1 - Language: c# (using gstreamer-sharp) - My implemententation is confirm to you provided link Would my c# help?
> Would my c# help? Sorry, i mean: Would my c# example code help? BTW: If I replace 'glimagesink' with 'd3dvideosink' gst_video_overlay_set_window_handle works as expected. Beside the fact d3dvideosink seems to have other problems :-(
Yes, having a testcase for reproducing it would definitely help
Also please report another bug about the d3dvideosink problems :)
Created attachment 292842 [details] c# BugProject
Okay, i have stripped down my code to the essential parts and into a single c# winform. Additional my complete bugdemo project as single zip (without gstreamer-binaries). If you use it, you only have to take Step 4 -----------------8<------------------------------------------------------------- To use it: - Step 1: Create a Winform Application within Visual Studio or Sharp Develop - Step 2: Add the following Assemblys to your Projekt: glib-sharp.dll + gstreamer-sharp.dll - Step 3: Replace your MainForm Code with mine (attention: not the namespace) - Step 4: Change Settings: - private const string filepathvideo MUST be your video - private const videosinktype cfgVideosinkType SHOULD be set to videosinktype.glimagesink OR videosinktype.d3dvideosink (Depends on what you want to test) - Change Environment Path within Method InitializeEnvironment() to your specific setup -----------------8<------------------------------------------------------------- /* * Created by SharpDevelop. * User: marc * Date: 16.12.2014 * Time: 14:55 * */ using System; using System.Collections.Generic; using System.Drawing; using System.Windows.Forms; using System.Threading; using System.Security.Permissions; using Gst; using GLib; namespace gstreamerbug { /// <summary> /// Winform Gstreamer for Bugreporting /// </summary> public partial class MainForm : Form, IDisposable { // CHANGE the following line private const string filepathvideo = @"d:\t2.mp4"; // CHANGE the following line private const videosinktype cfgVideosinkType = videosinktype.glimagesink; // CHANGE Environment Path within Method InitializeEnvironment() public Control videodisplay; public Button startVideoButton; public Button pauseVideoButton; public Label initializingMessage; private System.Threading.Thread mainGlibThread; private GLib.MainLoop mainLoop; private Gst.Pipeline currentPipeline = null; private Gst.Base.BaseSrc filesrc; private Gst.Bin decodebin; private Gst.Element timeoverlay; private Gst.Element videobalance; private Gst.Video.VideoSink glimagesink; private Gst.Base.BaseTransform audioconvert; private Gst.Element volume; private Gst.Bin autoaudiosink; private ulong handle; public MainForm() { // Nothing bug-relevant happens here. InitializeComponent(); // Initialize Controls videodisplay = new Control() { BackColor=Color.Black }; startVideoButton = new Button() { Text="Start", AutoSize=true}; pauseVideoButton = new Button() { Text="Pause", AutoSize=true}; initializingMessage = new Label() { Text="Initializing... please wait", AutoSize=false, TextAlign= ContentAlignment.MiddleCenter, Dock= DockStyle.Fill}; // Add Controls to Form this.SuspendLayout(); this.Controls.Add(initializingMessage); this.Controls.Add(videodisplay); this.Controls.Add(startVideoButton); this.Controls.Add(pauseVideoButton); this.ResumeLayout(); SubscribeButtonClicks(); ResizeControls(); initializeEnvironment(); } protected override void OnShown(EventArgs e) { // Let Form paint Controls before long idle time of gstreamer initialisation System.Windows.Forms.Application.DoEvents(); // Assign Handle to prevent Cross-Threading Access handle = (ulong) videodisplay.Handle; // Initialize Gstreamer InitializeGstreamer(); // Initialize Mainloop InitializeMainloop(); // Build Pipeline BuildPipeline(); // Finaly hide initialisation message initializingMessage.Visible=false; base.OnShown(e); } #region Resizing Controls protected override void OnResize(EventArgs e) { ResizeControls(); base.OnResize(e); } private void ResizeControls() { // Terrible, i know. But: No real-world application, just for bugreporting int w = this.ClientSize.Width; int h = this.ClientSize.Height; int displayH = h - (startVideoButton.Height * 2); int buttonT = displayH + (startVideoButton.Height / 2); if(displayH < 0) { displayH = 0; } if(videodisplay.Top != 0) { videodisplay.Top=0; } if(videodisplay.Left != 0) { videodisplay.Left=0; } if(videodisplay.Width != w) { videodisplay.Width=w; } if(videodisplay.Height != displayH) { videodisplay.Height = displayH; }; if(startVideoButton.Top != buttonT) { startVideoButton.Top = buttonT; } if(startVideoButton.Left != 0) { startVideoButton.Left = 0; } if(pauseVideoButton.Top != buttonT) { pauseVideoButton.Top = buttonT;} if(pauseVideoButton.Left != startVideoButton.Width) { pauseVideoButton.Left = startVideoButton.Width; } } #endregion #region Initializing Gstreamer private void initializeEnvironment() { string path = System.IO.Path.Combine(AppDomain.CurrentDomain.BaseDirectory, @"lib\gstreamer\w32\"); string pluginpath = System.IO.Path.Combine(AppDomain.CurrentDomain.BaseDirectory, @"lib\gstreamer\w32\lib\gstreamer-1.0\"); string registry = System.IO.Path.Combine(path, "registry.bin"); Environment.SetEnvironmentVariable("PATH", Environment.GetEnvironmentVariable("PATH") + ";"+path); Environment.SetEnvironmentVariable("GST_PLUGIN_PATH", pluginpath +";"+Environment.GetEnvironmentVariable("GST_PLUGIN_PATH")); Environment.SetEnvironmentVariable("GST_PLUGIN_SYSTEM_PATH_1_0", pluginpath +";"+Environment.GetEnvironmentVariable("GST_PLUGIN_SYSTEM_PATH_1_0")); Environment.SetEnvironmentVariable("GST_PLUGIN_SYSTEM_PATH", pluginpath +";"+Environment.GetEnvironmentVariable("GST_PLUGIN_SYSTEM_PATH")); Environment.SetEnvironmentVariable("GST_DEBUG","*:4"); Environment.SetEnvironmentVariable("GST_DEBUG_FILE",System.IO.Path.Combine(path, "gstreamer.log")); Environment.SetEnvironmentVariable("GST_REGISTRY", registry); } private void InitializeGstreamer() { // gst_init Gst.Application.Init(); // initialize objectmanager. otherwise bus subscription will fail GtkSharp.GstreamerSharp.ObjectManager.Initialize(); } private void InitializeMainloop() { mainLoop = new GLib.MainLoop(); mainGlibThread = new System.Threading.Thread(mainLoop.Run); mainGlibThread.Start(); } #endregion #region Build Pipeline private void BuildPipeline() { /* * gst-launch-1.0.exe -v filesrc location=d:\t.avi ! decodebin name="dec" ! timeoverlay ! glimagesink dec. ! audioconvert ! autoaudiosink */ filesrc = (Gst.Base.BaseSrc) Gst.ElementFactory.Make("filesrc","filesrc"); decodebin = (Gst.Bin) Gst.ElementFactory.Make("decodebin","decodebin"); timeoverlay = Gst.ElementFactory.Make("timeoverlay", "timeoverlay"); videobalance = Gst.ElementFactory.Make("videobalance", "videobalance"); switch(cfgVideosinkType) { case videosinktype.glimagesink: glimagesink = (Gst.Video.VideoSink) Gst.ElementFactory.Make("glimagesink","glimagesink"); break; case videosinktype.d3dvideosink: glimagesink = (Gst.Video.VideoSink) Gst.ElementFactory.Make("d3dvideosink","d3dvideosink"); break; } audioconvert = (Gst.Base.BaseTransform) Gst.ElementFactory.Make("audioconvert","audioconvert"); volume = (Gst.Element) Gst.ElementFactory.Make("volume", "volume"); autoaudiosink = (Gst.Bin) Gst.ElementFactory.Make("autoaudiosink", "autoaudiosink"); currentPipeline = new Gst.Pipeline("pipeline"); currentPipeline.Add(filesrc, decodebin, timeoverlay, videobalance, glimagesink, audioconvert, volume, autoaudiosink); filesrc["location"] = filepathvideo; decodebin.PadAdded += new PadAddedHandler(decodebin_PadAdded); bool b1 = filesrc.Link(decodebin); // decodebin Link yet not possible bool b2 = timeoverlay.Link(videobalance); bool b4 = videobalance.Link(glimagesink); bool b5 = audioconvert.Link(volume); bool b6 = volume.Link(autoaudiosink); SubscribeBusMessage(); SubscribeBusSyncMessage(); TryPause(); } /// <summary> /// /// </summary> /// <param name="o"></param> /// <param name="args"></param> /// <remarks> /// decodebin has no static src pads (see gst-inspect decodebin) /// decodebin has "sometimes-pad" /// </remarks> void decodebin_PadAdded(object o, PadAddedArgs args) { Gst.Element src = (Element)o; Gst.GhostPad newPad = (Gst.GhostPad) args.NewPad; if(newPad.Target.ParentElement is Gst.Audio.AudioDecoder) { var sinkpad = audioconvert.GetStaticPad("sink"); var newlink = newPad.Link(sinkpad); } if(newPad.Target.ParentElement is Gst.Video.VideoDecoder) { // Das funktioniert: bool b1 = decodebin.Link(timeoverlay); // Das funktioniert (seltsamerweise) nicht // var p = timeoverlay.GetStaticPad("sink"); // var r = newPad.Link(p); } } /// <summary> /// /// </summary> /// <remarks> /// Indeed the application needs to set its Window identifier at the right time to avoid internal Window creation /// from the video sink element. To solve this issue a GstMessage is posted on the bus to inform the application /// that it should set the Window identifier immediately. /// /// API: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-gstvideooverlay.html /// </remarks> /// <param name="o"></param> /// <param name="args"></param> private void bus_SyncMessage(object o, SyncMessageArgs args) { // Convenience function to check if the given message is a "prepare-window-handle" message from a GstVideoOverlay. if(Gst.Video.Global.IsVideoOverlayPrepareWindowHandleMessage(args.Message)) { Element src = (Gst.Element) args.Message.Src; #if DEBUG System.Diagnostics.Debug.WriteLine("Message 'prepare-window-handle' received by: "+src.Name+" "+src.ToString()); #endif if(src != null && (src is Gst.Video.VideoSink | src is Gst.Bin)) { // Try to set Aspect Ratio try { src["force-aspect-ratio"] = true; } catch (PropertyNotFoundException) {} // Try to set Overlay try { Gst.Video.VideoOverlayAdapter overlay_ = new Gst.Video.VideoOverlayAdapter(src.Handle); overlay_.WindowHandle = handle; overlay_.HandleEvents(true); } catch(Exception ex) { } } } } #endregion private void startVideo_Click(object sender, EventArgs e) { TryPlay(); } private void pauseVideoButton_Click(object sender, EventArgs e) { TryPause(); } private bool TryPlay() { bool retval = false; if(currentPipeline != null) { StateChangeReturn s = currentPipeline.SetState(Gst.State.Playing); if(s == StateChangeReturn.Success) { retval = true; } retval = true; } return retval; } public bool TryPause() { bool retval = false; if(currentPipeline != null) { currentPipeline.SetState(Gst.State.Paused); } return retval; } private void HandleMessage (object o, MessageArgs args) { var msg = args.Message; switch (msg.Type) { case MessageType.Error: // GLib.GException err; string debug; msg.ParseError (out err, out debug); if(debug == null) { debug = "none"; } System.Diagnostics.Debug.WriteLine ("Error received from element {0}: {1}", msg.Src, err.Message); System.Diagnostics.Debug.WriteLine ("Debugging information: "+ debug); break; } args.RetVal = true; } private void SubscribeButtonClicks() { startVideoButton.Click += new EventHandler(startVideo_Click); pauseVideoButton.Click += new EventHandler(pauseVideoButton_Click); } private void UnsubscribeButtonClicks() { startVideoButton.Click -= new EventHandler(startVideo_Click); pauseVideoButton.Click -= new EventHandler(pauseVideoButton_Click); } private void SubscribeBusMessage() { Bus bus = currentPipeline.Bus; bus.AddSignalWatch (); bus.Message += HandleMessage; } private void UnsubscribeBusMessage() { Bus bus = currentPipeline.Bus; bus.Message -= HandleMessage; bus.RemoveSignalWatch(); } private void SubscribeBusSyncMessage() { Bus bus = currentPipeline.Bus; bus.EnableSyncMessageEmission(); bus.SyncMessage += new SyncMessageHandler(bus_SyncMessage); } private void UnsubscribeBusSyncMessage() { Bus bus = currentPipeline.Bus; bus.SyncMessage -= new SyncMessageHandler(bus_SyncMessage); bus.DisableSyncMessageEmission(); } [SecurityPermissionAttribute(SecurityAction.Demand, ControlThread = true)] private void KillThatCat() { if(mainGlibThread != null) { mainGlibThread.Abort(); } } public new void Dispose() { UnsubscribeButtonClicks(); UnsubscribeBusSyncMessage(); UnsubscribeBusMessage(); // easy, but not correct if(currentPipeline != null) { currentPipeline.Dispose(); } if(mainGlibThread != null) { mainGlibThread.Abort(); } if(mainGlibThread != null) { // Die Glib.Mainloop kann sehr penetrant darin sein sich nicht beenden zu wollen KillThatCat(); } base.Dispose(); } } internal enum videosinktype { glimagesink, d3dvideosink } }
BTW: In my original Code the mainloop is wrapped, therefore the current IDisposable Implementation is not correct and does not work correct. The mainloop is still running and you have manually stop/kill the assembly. Sorry.
(In reply to comment #5) > Also please report another bug about the d3dvideosink problems :) Done: https://bugzilla.gnome.org/show_bug.cgi?id=741608
(In reply to comment #4) > Yes, having a testcase for reproducing it would definitely help Done. - Posted as plaintext c# code - Uploaded as c# Project
Can i provide any additional info?
I had a go at looking at this. I couldn't get your sample to work for me so I created my own in C++. Thing is, everything worked as expected with my example. Maybe something in the c# wrappers? Regardless I can say that this definitely works so I'm going to close this.