After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 733727 - glimagesink GstVideoOverlay not working on Windows OS
glimagesink GstVideoOverlay not working on Windows OS
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
1.4.0
Other Windows
: Normal major
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2014-07-25 10:23 UTC by Steven
Modified: 2016-11-10 14:51 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
c# BugProject (419.73 KB, application/zip)
2014-12-16 16:02 UTC, Marc Krämer
Details

Description Steven 2014-07-25 10:23:01 UTC
I am embedding the gstreamer output inside my application.

However glimagesink will not use the window handle I am passing it with gst_video_overlay_set_window_handle.

If I use the d3dvideosink then video is shown ok inside my application.
Comment 1 Julien Isorce 2014-10-30 18:32:02 UTC
Could you provide a minimal example that reproduces the pb ? Have you tried the example http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/gtk/gtkvideooverlay ?
Comment 2 Marc Krämer 2014-12-16 13:24:29 UTC
I can confirm bug.

- OS: Windows 8.1
- Language: c# (using gstreamer-sharp)
- My implemententation is confirm to you provided link

Would my c# help?
Comment 3 Marc Krämer 2014-12-16 13:29:27 UTC
> Would my c# help?

Sorry, i mean: Would my c# example code help?

BTW: If I replace 'glimagesink' with 'd3dvideosink' gst_video_overlay_set_window_handle works as expected. Beside the fact d3dvideosink seems to have other problems :-(
Comment 4 Sebastian Dröge (slomo) 2014-12-16 13:30:52 UTC
Yes, having a testcase for reproducing it would definitely help
Comment 5 Sebastian Dröge (slomo) 2014-12-16 13:31:31 UTC
Also please report another bug about the d3dvideosink problems :)
Comment 6 Marc Krämer 2014-12-16 16:02:00 UTC
Created attachment 292842 [details]
c# BugProject
Comment 7 Marc Krämer 2014-12-16 16:02:19 UTC
Okay, i have stripped down my code to the essential parts and into a single c# winform.

Additional my complete bugdemo project as single zip (without gstreamer-binaries). If you use it, you only have to take Step 4


-----------------8<-------------------------------------------------------------

To use it:

- Step 1: Create a Winform Application within Visual Studio or Sharp Develop
- Step 2: Add the following Assemblys to your Projekt: glib-sharp.dll + gstreamer-sharp.dll
- Step 3: Replace your MainForm Code with mine (attention: not the namespace)
- Step 4: Change Settings:
  - private const string filepathvideo MUST be your video
  - private const videosinktype cfgVideosinkType SHOULD be set to videosinktype.glimagesink OR videosinktype.d3dvideosink (Depends on what you want to test)
  - Change Environment Path within Method InitializeEnvironment() to your specific setup
-----------------8<-------------------------------------------------------------

/*
 * Created by SharpDevelop.
 * User: marc
 * Date: 16.12.2014
 * Time: 14:55
 * 
 */
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Windows.Forms;
using System.Threading;
using System.Security.Permissions;
using Gst;
using GLib;

namespace gstreamerbug
{
	/// <summary>
	/// Winform Gstreamer for Bugreporting
	/// </summary>
	public partial class MainForm : Form, IDisposable
	{
		//	CHANGE the following line
		private const string filepathvideo = @"d:\t2.mp4";
		//	CHANGE the following line
		private const videosinktype cfgVideosinkType = videosinktype.glimagesink;
		//	CHANGE Environment Path within Method InitializeEnvironment()
		
		public Control videodisplay;
		public Button startVideoButton;
		public Button pauseVideoButton;
		public Label initializingMessage;
		
		private System.Threading.Thread mainGlibThread;
		private GLib.MainLoop mainLoop;
		
		private Gst.Pipeline currentPipeline = null;
		private Gst.Base.BaseSrc filesrc;
		private Gst.Bin decodebin;
		private Gst.Element timeoverlay;
		private Gst.Element videobalance;
		private Gst.Video.VideoSink glimagesink;
		private Gst.Base.BaseTransform audioconvert;
		private Gst.Element volume;
		private Gst.Bin autoaudiosink;
		
		private ulong handle;
		
		public MainForm()
		{
			//	Nothing bug-relevant happens here.
			InitializeComponent();
			
			//	Initialize Controls
			videodisplay = new Control() { BackColor=Color.Black };
			startVideoButton = new Button() { Text="Start", AutoSize=true};
			pauseVideoButton = new Button() { Text="Pause", AutoSize=true};
			initializingMessage = new Label() { Text="Initializing... please wait", AutoSize=false, TextAlign= ContentAlignment.MiddleCenter, Dock= DockStyle.Fill};
			
			//	Add Controls to Form
			this.SuspendLayout();
			this.Controls.Add(initializingMessage);
			this.Controls.Add(videodisplay);
			this.Controls.Add(startVideoButton);
			this.Controls.Add(pauseVideoButton);
			this.ResumeLayout();
			
			SubscribeButtonClicks();
			
			ResizeControls();
			
			initializeEnvironment();
		}
		
		protected override void OnShown(EventArgs e)
		{
			//	Let Form paint Controls before long idle time of gstreamer initialisation
			System.Windows.Forms.Application.DoEvents();
			//	Assign Handle to prevent Cross-Threading Access
			handle = (ulong) videodisplay.Handle;
			//	Initialize Gstreamer
			InitializeGstreamer();
			//	Initialize Mainloop
			InitializeMainloop();
			//	Build Pipeline
			BuildPipeline();
			
			//	Finaly hide initialisation message
			initializingMessage.Visible=false;
			
			base.OnShown(e);
		}
		
		
		
		#region Resizing Controls
		
		protected override void OnResize(EventArgs e)
		{
			ResizeControls();
			base.OnResize(e);
		}
		
		private void ResizeControls()
		{
			//	Terrible, i know. But: No real-world application, just for bugreporting
			
			int w = this.ClientSize.Width;
			int h = this.ClientSize.Height;
			
			int displayH = h - (startVideoButton.Height * 2);
			int buttonT = displayH + (startVideoButton.Height / 2);
			if(displayH < 0) { displayH = 0; }
			
			if(videodisplay.Top != 0) { videodisplay.Top=0; }
			if(videodisplay.Left != 0) { videodisplay.Left=0; }
			if(videodisplay.Width != w) { videodisplay.Width=w; }
			if(videodisplay.Height != displayH) { videodisplay.Height = displayH; };
			
			if(startVideoButton.Top != buttonT) { startVideoButton.Top = buttonT; }
			if(startVideoButton.Left != 0) { startVideoButton.Left = 0; }
			if(pauseVideoButton.Top != buttonT) { pauseVideoButton.Top = buttonT;}
			if(pauseVideoButton.Left != startVideoButton.Width) { pauseVideoButton.Left = startVideoButton.Width; }
		}
		
		#endregion
		
		#region Initializing Gstreamer
		private void initializeEnvironment()
		{
			string path = System.IO.Path.Combine(AppDomain.CurrentDomain.BaseDirectory, @"lib\gstreamer\w32\");
			string pluginpath = System.IO.Path.Combine(AppDomain.CurrentDomain.BaseDirectory, @"lib\gstreamer\w32\lib\gstreamer-1.0\");
			string registry = System.IO.Path.Combine(path, "registry.bin");
			
			Environment.SetEnvironmentVariable("PATH", Environment.GetEnvironmentVariable("PATH") + ";"+path);
			Environment.SetEnvironmentVariable("GST_PLUGIN_PATH", pluginpath +";"+Environment.GetEnvironmentVariable("GST_PLUGIN_PATH"));
			Environment.SetEnvironmentVariable("GST_PLUGIN_SYSTEM_PATH_1_0", pluginpath +";"+Environment.GetEnvironmentVariable("GST_PLUGIN_SYSTEM_PATH_1_0"));
			Environment.SetEnvironmentVariable("GST_PLUGIN_SYSTEM_PATH", pluginpath +";"+Environment.GetEnvironmentVariable("GST_PLUGIN_SYSTEM_PATH"));
			Environment.SetEnvironmentVariable("GST_DEBUG","*:4");
			Environment.SetEnvironmentVariable("GST_DEBUG_FILE",System.IO.Path.Combine(path, "gstreamer.log"));
			Environment.SetEnvironmentVariable("GST_REGISTRY", registry);
		}
		
		private void InitializeGstreamer()
		{
			//	gst_init
			Gst.Application.Init();
			//	initialize objectmanager. otherwise bus subscription will fail
			GtkSharp.GstreamerSharp.ObjectManager.Initialize();
		}
		
		private void InitializeMainloop()
		{
			mainLoop = new GLib.MainLoop();
			mainGlibThread = new System.Threading.Thread(mainLoop.Run);
			mainGlibThread.Start();
		}
		#endregion
		
		#region Build Pipeline
		private void BuildPipeline()
		{
			/*
			 * gst-launch-1.0.exe -v filesrc location=d:\t.avi  ! decodebin name="dec" ! timeoverlay ! glimagesink  dec. ! audioconvert ! autoaudiosink
			 */
			
			filesrc = (Gst.Base.BaseSrc) Gst.ElementFactory.Make("filesrc","filesrc");
			decodebin = (Gst.Bin) Gst.ElementFactory.Make("decodebin","decodebin");
			
			timeoverlay = Gst.ElementFactory.Make("timeoverlay", "timeoverlay");
			videobalance = Gst.ElementFactory.Make("videobalance", "videobalance");
			switch(cfgVideosinkType)
			{
				case videosinktype.glimagesink:
					glimagesink = (Gst.Video.VideoSink) Gst.ElementFactory.Make("glimagesink","glimagesink");
					break;
				case videosinktype.d3dvideosink:
					glimagesink = (Gst.Video.VideoSink) Gst.ElementFactory.Make("d3dvideosink","d3dvideosink");
					break;
			}
			
			audioconvert = (Gst.Base.BaseTransform) Gst.ElementFactory.Make("audioconvert","audioconvert");
			volume = (Gst.Element) Gst.ElementFactory.Make("volume", "volume");
			autoaudiosink = (Gst.Bin) Gst.ElementFactory.Make("autoaudiosink", "autoaudiosink");
			
			currentPipeline = new Gst.Pipeline("pipeline");
			
			currentPipeline.Add(filesrc, decodebin, timeoverlay, videobalance, glimagesink, audioconvert, volume, autoaudiosink);
			
			filesrc["location"] = filepathvideo;
			
			decodebin.PadAdded += new PadAddedHandler(decodebin_PadAdded);
			
			bool b1 = filesrc.Link(decodebin);
			//	decodebin Link yet not possible
			bool b2 = timeoverlay.Link(videobalance);
			bool b4 = videobalance.Link(glimagesink);
			bool b5 = audioconvert.Link(volume);
			bool b6 = volume.Link(autoaudiosink);
			
			SubscribeBusMessage();
			SubscribeBusSyncMessage();
			
			TryPause();
		}
		
		/// <summary>
		/// 
		/// </summary>
		/// <param name="o"></param>
		/// <param name="args"></param>
		/// <remarks>
		/// decodebin has no static src pads (see gst-inspect decodebin)
		/// decodebin has "sometimes-pad"
		/// </remarks>
		void decodebin_PadAdded(object o, PadAddedArgs args)
		{
			Gst.Element src = (Element)o;
			Gst.GhostPad newPad = (Gst.GhostPad) args.NewPad;
			
			if(newPad.Target.ParentElement is Gst.Audio.AudioDecoder)
			{
				var sinkpad = audioconvert.GetStaticPad("sink");
				var newlink = newPad.Link(sinkpad);
			}
			if(newPad.Target.ParentElement is Gst.Video.VideoDecoder)
			{
				//	Das funktioniert:
				bool b1 = decodebin.Link(timeoverlay);
				//	Das funktioniert (seltsamerweise) nicht
				//	var p = timeoverlay.GetStaticPad("sink");
				//	var r = newPad.Link(p);
			}
		}
		
		/// <summary>
		/// 
		/// </summary>
		/// <remarks>
		/// Indeed the application needs to set its Window identifier at the right time to avoid internal Window creation
		/// from the video sink element. To solve this issue a GstMessage is posted on the bus to inform the application
		/// that it should set the Window identifier immediately.
		/// 
		/// API: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-gstvideooverlay.html
		/// </remarks>
		/// <param name="o"></param>
		/// <param name="args"></param>
		private void bus_SyncMessage(object o, SyncMessageArgs args)
		{
			//	Convenience function to check if the given message is a "prepare-window-handle" message from a GstVideoOverlay.
			if(Gst.Video.Global.IsVideoOverlayPrepareWindowHandleMessage(args.Message))
			{
				Element src = (Gst.Element) args.Message.Src;
				
				#if DEBUG
				System.Diagnostics.Debug.WriteLine("Message 'prepare-window-handle' received by: "+src.Name+" "+src.ToString());
				#endif
				
				if(src != null && (src is Gst.Video.VideoSink | src is Gst.Bin))
				{
					//	Try to set Aspect Ratio
					try
					{
						src["force-aspect-ratio"] = true;
					}
					catch (PropertyNotFoundException) {}
					
					//	Try to set Overlay
					try
					{
						Gst.Video.VideoOverlayAdapter overlay_ = new Gst.Video.VideoOverlayAdapter(src.Handle);
						overlay_.WindowHandle = handle;
						overlay_.HandleEvents(true);
					}
					catch(Exception ex) { }
				}
			}
		}
		#endregion

		private void startVideo_Click(object sender, EventArgs e)
		{
			TryPlay();
		}
		
		private void pauseVideoButton_Click(object sender, EventArgs e)
		{
			TryPause();
		}
		
		private bool TryPlay()
		{
			bool retval = false;
			if(currentPipeline != null)
			{
				StateChangeReturn s = currentPipeline.SetState(Gst.State.Playing);
				if(s == StateChangeReturn.Success) { retval = true; }
				retval = true;
			}
			return retval;
		}
		
		public bool TryPause()
		{
			bool retval = false;
			if(currentPipeline != null)
			{
				currentPipeline.SetState(Gst.State.Paused);
			}
			return retval;
		}
		
		private void HandleMessage (object o, MessageArgs args)
		{
			var msg = args.Message;
			switch (msg.Type)
			{
				case MessageType.Error:
					//
					GLib.GException err;
					string debug;
					msg.ParseError (out err, out debug);
					if(debug == null) { debug = "none"; }
					System.Diagnostics.Debug.WriteLine ("Error received from element {0}: {1}", msg.Src, err.Message);
					System.Diagnostics.Debug.WriteLine ("Debugging information: "+ debug);
					break;
			}
			args.RetVal = true;
		}
		
		private void SubscribeButtonClicks()
		{
			startVideoButton.Click += new EventHandler(startVideo_Click);
			pauseVideoButton.Click += new EventHandler(pauseVideoButton_Click);
		}
		private void UnsubscribeButtonClicks()
		{
			startVideoButton.Click -= new EventHandler(startVideo_Click);
			pauseVideoButton.Click -= new EventHandler(pauseVideoButton_Click);
		}
		private void SubscribeBusMessage()
		{
			Bus bus = currentPipeline.Bus;
			bus.AddSignalWatch ();
			bus.Message += HandleMessage;
		}
		private void UnsubscribeBusMessage()
		{
			Bus bus = currentPipeline.Bus;
			bus.Message -= HandleMessage;
			bus.RemoveSignalWatch();
		}
		private void SubscribeBusSyncMessage()
		{
			Bus bus = currentPipeline.Bus;
			bus.EnableSyncMessageEmission();
			bus.SyncMessage += new SyncMessageHandler(bus_SyncMessage);
		}
		private void UnsubscribeBusSyncMessage()
		{
			Bus bus = currentPipeline.Bus;
			bus.SyncMessage -= new SyncMessageHandler(bus_SyncMessage);
			bus.DisableSyncMessageEmission();
		}
		
		[SecurityPermissionAttribute(SecurityAction.Demand, ControlThread = true)]
		private void KillThatCat()
		{
			if(mainGlibThread != null)
			{
				mainGlibThread.Abort();
			}
		}
		
		public new void Dispose()
		{
			
			UnsubscribeButtonClicks();
			UnsubscribeBusSyncMessage();
			UnsubscribeBusMessage();
			//	easy, but not correct
			if(currentPipeline != null) { currentPipeline.Dispose(); }	
			if(mainGlibThread != null) { mainGlibThread.Abort(); }
			if(mainGlibThread != null)
			{
				//	Die Glib.Mainloop kann sehr penetrant darin sein sich nicht beenden zu wollen
				KillThatCat();
			}
			base.Dispose();
		}
	}
	
	internal enum videosinktype
	{
		glimagesink,
		d3dvideosink
	}
}
Comment 8 Marc Krämer 2014-12-16 16:30:52 UTC
BTW: In my original Code the mainloop is wrapped, therefore the current IDisposable Implementation is not correct and does not work correct. The mainloop is still running and you have manually stop/kill the assembly. Sorry.
Comment 9 Marc Krämer 2014-12-17 12:56:02 UTC
(In reply to comment #5)
> Also please report another bug about the d3dvideosink problems :)

Done: https://bugzilla.gnome.org/show_bug.cgi?id=741608
Comment 10 Marc Krämer 2014-12-17 12:58:33 UTC
(In reply to comment #4)
> Yes, having a testcase for reproducing it would definitely help

Done. 
- Posted as plaintext c# code
- Uploaded as c# Project
Comment 11 Marc Krämer 2015-01-10 21:13:57 UTC
Can i provide any additional info?
Comment 12 Matthew Waters (ystreet00) 2016-11-10 14:51:00 UTC
I had a go at looking at this.  I couldn't get your sample to work for me so I created my own in C++.  Thing is, everything worked as expected with my example.  Maybe something in the c# wrappers?

Regardless I can say that this definitely works so I'm going to close this.