After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 707523 - d3dvideosink memleak
d3dvideosink memleak
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
1.x
Other Windows
: Normal critical
: NONE
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2013-09-04 23:59 UTC by Nicola
Modified: 2015-02-27 17:32 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
test app for gstreamer 1.0 (12.26 KB, application/x-bzip)
2013-09-04 23:59 UTC, Nicola
  Details
test app for gstreamer-sdk 2013.6 (10.95 KB, application/x-bzip)
2013-09-05 00:00 UTC, Nicola
  Details
test app for d3dvideosink memleak in MFC (11.34 KB, application/x-zip-compressed)
2013-09-16 13:31 UTC, Tomas Gazovic
  Details
Fix d3dvideosink memleak (395 bytes, patch)
2013-09-18 14:15 UTC, Tomas Gazovic
none Details | Review
patch for 0.10 (3.10 KB, patch)
2013-09-20 21:41 UTC, Nicola
needs-work Details | Review
patch for 1.0 (1.31 KB, patch)
2013-09-30 14:57 UTC, Tomas Gazovic
needs-work Details | Review
backtrace for d3dvideo sink crash (2.14 KB, text/plain)
2014-02-25 17:20 UTC, Steven
  Details
libgstd3dvideosink.dll (390.54 KB, application/x-msdos-program)
2014-07-14 08:54 UTC, Sebastian Dröge (slomo)
  Details
my gstreamer code (8.53 KB, text/plain)
2014-07-15 15:54 UTC, Steven
  Details

Description Nicola 2013-09-04 23:59:37 UTC
Created attachment 254136 [details]
test app for gstreamer 1.0

d3dvideosink has huge memleak if an application start more than one pipeline,

attached a sample app that start 4 pipelines (videotestsrc ! d3dvideosink) and do stop/play every 10 seconds.

changing d3dvideosink to fakesink fix the leak.

Please run the test app and observe the memory usage, the problem happen with both gstreamer-1.0 and with sdk-2013.6, please note that this is a regression, d3dvideosink included in ossbuild works fine

for convenience the compiled apps can be downloaded from the following links (it requires visual studio 2010 c++ runtime, the rest is included):

http://195.250.34.59/temp/TestMultiViewers_1.0.tar.bz2
http://195.250.34.59/temp/TestMultiViewers_sdk0.10.tar.bz2

please note that the problem does not happen if the app start only one pipeline (really weired)
Comment 1 Nicola 2013-09-05 00:00:34 UTC
Created attachment 254137 [details]
test app for gstreamer-sdk 2013.6
Comment 2 Tomas Gazovic 2013-09-16 13:31:14 UTC
Created attachment 255025 [details]
test app for d3dvideosink memleak in MFC

I detect memory leaks on d3dvideosink too. 
This bug aapears when pipeline is restarting.

Also there is similar problem with d3dvideosink.
This is my pipeline: appsrc ! wavescope ! videoconvert ! videobox ! d3dvideosink

d3dvideosink start memory leaking when it is overlayed in window. Overlay must be set by function gst_video_overlay_set_window_handle.
This is necessary and properties of videobox must be set frequently (e.g. in appsrc push_data).
Comment 3 Tomas Gazovic 2013-09-18 14:15:26 UTC
Created attachment 255220 [details] [review]
Fix d3dvideosink memleak

I have tried to fix this resource leaks in d3dvideosink.
This is my solution.

I'm sorry, but I don't know how to download sources from git repository.

I made a diff file between d3dvideosink.c ver 1.0.10 and my solution.
Comment 4 Nicola 2013-09-20 17:23:42 UTC
I can confirm that the patch solve the major leakeage in my test apps too (applied to gstreamer-sdk)
Comment 5 Andoni Morales 2013-09-20 17:31:43 UTC
Can you please do a diff -u ?
Comment 6 Nicola 2013-09-20 17:42:52 UTC
I'm a bit busy now, later or tomorrow I'll sent a git patch against gst-sdk and 1.x, basically the problem is here:

http://cgit.freedesktop.org/gstreamer-sdk/gst-plugins-bad/tree/sys/d3dvideosink/d3dvideosink.c?h=sdk-0.10.23#n2354

if you have more than a viewer the code at line 2356-2358 is not reached, so these lines:

GST_D3DVIDEOSINK_D3D_DEVICE_LOCK (sink);
gst_d3dvideosink_release_d3d_device (sink);
GST_D3DVIDEOSINK_D3D_DEVICE_UNLOCK (sink);

need to be after:

shared.element_count--;

and however before the goto

if you have only one viewer the problem does not happen (shared.element_count is never >0)
Comment 7 Nicola 2013-09-20 21:41:44 UTC
Created attachment 255448 [details] [review]
patch for 0.10
Comment 8 Sebastian Dröge (slomo) 2013-09-23 10:17:41 UTC
Comment on attachment 255448 [details] [review]
patch for 0.10

Please provide a patch against git master
Comment 9 Nicola 2013-09-23 10:48:51 UTC
the patch works for both 0.10 (sdk) and 1.0, d3dvideosink in git master seems quite different from 1.0 and I never tested with this version, I'll do some test as soon as windows binary for 1.2 are available for windows
Comment 10 Nicola 2013-09-26 09:38:37 UTC
I recompiled my testapp (1.0 version) against 1.2, seems d3dvideosink in 1.2 has major breakages, basically I have an app that start 4 pipeline like this:

videotestsrc ! d3dvideosink

(tested videotestsrc ! queue ! d3dvideosink with no success)

and restart them every 15 seconds, in 1.2 the app seems to deadlock on the first start, changing d3dvideosink with fakesink solve the issue
Comment 11 Tomas Gazovic 2013-09-30 14:57:10 UTC
Created attachment 256092 [details] [review]
patch for 1.0

Patch was created with git diff command.
Comment 12 Sebastian Dröge (slomo) 2013-10-04 10:49:32 UTC
(In reply to comment #10)
> I recompiled my testapp (1.0 version) against 1.2, seems d3dvideosink in 1.2
> has major breakages, basically I have an app that start 4 pipeline like this:
> 
> videotestsrc ! d3dvideosink
> 
> (tested videotestsrc ! queue ! d3dvideosink with no success)
> 
> and restart them every 15 seconds, in 1.2 the app seems to deadlock on the
> first start, changing d3dvideosink with fakesink solve the issue

Can you get a backtrace for the deadlocks? I tried starting two in my VM here but the second sink only shows black
Comment 13 Sebastian Dröge (slomo) 2013-10-04 10:50:15 UTC
Comment on attachment 256092 [details] [review]
patch for 1.0

Also needs to be updated to apply cleanly to 1.2, which is the current stable release
Comment 14 Steven 2014-02-24 20:55:07 UTC
I am also seeing the memory leak issue with 1.0 and 0.1 builds of gstreamer when using the d3dvideosink.
Also I cannot get video to play due to crashing on startup when using version 1.2 with a pipeline containing queues.
Comment 15 Nicolas Dufresne (ndufresne) 2014-02-25 16:14:36 UTC
(In reply to comment #14)
> I am also seeing the memory leak issue with 1.0 and 0.1 builds of gstreamer
> when using the d3dvideosink.
> Also I cannot get video to play due to crashing on startup when using version
> 1.2 with a pipeline containing queues.

Would it be possible to provide a backtrace ? (note that you need msys gdb for that).
Comment 16 Steven 2014-02-25 16:40:53 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > I am also seeing the memory leak issue with 1.0 and 0.1 builds of gstreamer
> > when using the d3dvideosink.
> > Also I cannot get video to play due to crashing on startup when using version
> > 1.2 with a pipeline containing queues.
> 
> Would it be possible to provide a backtrace ? (note that you need msys gdb for
> that).

I will try to obtain a backtrace for you. While I do that here are some example pipelines which should enable you to recreate the crash yourself.

gst-launch-1.0.exe videotestsrc ! videoconvert ! d3dvideosink WORKS OK

gst-launch-1.0.exe videotestsrc ! videoconvert ! tee name=t t. ! queue ! d3dvideosink WORKS OK

gst-launch-1.0.exe videotestsrc ! videoconvert ! tee name=t t. ! queue ! d3dvideosink CRASHES EVERYTIME

My setup:
Visual Studio 2008 on Windows 7 x64
gstreamer 1.2.3 installed with "complete" option

Ive tried 1.2.0 1.2.1 1.2.2 and they all crash the same x32 and x64 versions of gstreamer.

N.B. Im wondering if this issue should be raised as a new bug? The crashing problem is different from the original mem-leak bug.
Comment 17 Steven 2014-02-25 16:43:05 UTC
sorry the example that crashes everytime should be as follows:

gst-launch-1.0.exe videotestsrc ! videoconvert ! tee name=t t. ! queue ! d3dvideosink t. ! queue ! d3dvideosink
Comment 18 Steven 2014-02-25 17:20:25 UTC
Created attachment 270296 [details]
backtrace for d3dvideo sink crash
Comment 19 Steven 2014-07-14 07:42:25 UTC
Just tried this with the new 1.3.91 release and it still crashes.

Looks like the tee element is causing the crash with this version.

This command CRASHES:
gst-launch-1.0.exe videotestsrc ! videoconvert ! tee name=t t. ! queue ! d3dvideosink t. ! queue ! d3dvideosink

This command WORKS:
gst-launch-1.0.exe videotestsrc ! videoconvert ! queue ! d3dvideosink
Comment 20 Sebastian Dröge (slomo) 2014-07-14 08:54:49 UTC
Created attachment 280633 [details]
libgstd3dvideosink.dll

Does this fix it? It initializes the D3D devices with D3DCREATE_MULTITHREADED.

It's a 32 bit dll though, tell me if you need the 64 bit version. If it still crashes, can you get a backtrace of all threads with this version?
Comment 21 Steven 2014-07-15 06:39:10 UTC
Thanks a lot for the attached dll, 32bit is fine for my needs.

I can confirm that the example commands, given in comment 19, run successfully with your dll.

However I am getting the following output on the command-line:

(gst-launch-1.0:10872): GLib-GObject-WARNING **: cannot register existing type 'GstD3DVideoSink'

(gst-launch-1.0:10872): GLib-GObject-CRITICAL **: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed

(gst-launch-1.0:10872): GLib-GObject-CRITICAL **: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed

(gst-launch-1.0:10872): GLib-CRITICAL **: g_once_init_leave: assertion 'result != 0' failed

(gst-launch-1.0:10872): GStreamer-CRITICAL **: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed
Comment 22 Sebastian Dröge (slomo) 2014-07-15 11:31:08 UTC
Thanks, I pushed that change and it will be included in 1.4.0. The other problem seems to be something wrong with your installation though.


However how is this all related to the original memory leak that this bug report is about?
Comment 23 Steven 2014-07-15 15:54:50 UTC
Created attachment 280734 [details]
my gstreamer code

Now that the cause of the crashing has been identified, I dont think it is related to this tickets original memory leak issue.

However I just checked and I am still seeing the original memory leak issue when closing a pipeline and starting another pipeline. This has been reduced from ~30mb to ~10mb each time now.

I have attached my gstreamer code as I may be simply doing something wrong.
The .cc and .h files have been combined.

Thanks.
Comment 24 Sebastian Dröge (slomo) 2014-07-15 16:11:28 UTC
You can't restart gst_parse_launch() pipelines like that reliably. You'll have to create a new one after EOS, etc. Just setting them to READY or NULL, and then starting again is not enough. This is actually a bug in gst_parse_launch() though, with manually built pipelines exactly that is possible.
Not sure if that is related to your problem though... but do you still have leaks when creating a new pipeline each time (and unreffing the old one after setting it to NULL)?


For the decreased memory leakage... I fixed a huge memory leak in d3dvideosink some days ago. That's probably the difference.
Comment 25 Steven 2014-07-15 16:29:44 UTC
Could you please expand on your suggestions for changes to my code. I think I am already doing what you recommend.

Currently on GST_MESSAGE_EOS my cleanup function is called.

cleanup:
1)set the bus sync handler to NULL and unref the bus.
2)set pipleline to GST_STATE_NULL and unref the pipleline

Another pipeline is again built with gst_parse_launch in setupVideo
Comment 26 Sebastian Dröge (slomo) 2014-07-15 17:07:12 UTC
Oh then you're actually doing already what I meant. Means there's a memory leak somewhere else.

Is it proportional to the number of frames that were displayed or always the same number? And is it increasing with the same amount after every run?

How do you measure the memory leak?
Comment 27 Nicola 2014-07-15 21:25:37 UTC
not tested, but looking at the code I think the memory leak is here:

http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/d3dvideosink/d3dhelpers.c#n2304

if you have more than one instance (an app with multiple d3dvideosink)

klass->d3d.refs >= 1 is true

and the d3d object is not released,

sorry I cannot test soon if I'm right, 

Sebiastian it make sense to you?
Comment 28 Sebastian Dröge (slomo) 2014-07-16 06:51:07 UTC
Yes, that memory is "leaked"... however it is only allocated once per process so shouldn't show up anywhere really
Comment 29 Steven 2014-07-16 08:53:36 UTC
(In reply to comment #26)
> Oh then you're actually doing already what I meant. Means there's a memory leak
> somewhere else.
> 
> Is it proportional to the number of frames that were displayed or always the
> same number? And is it increasing with the same amount after every run?
> 
> How do you measure the memory leak?

The size of the leak does not seem to be linked to the number of frames played.
I left my stream playing for a random amount of time before triggering a GST_MESSAGE_EOS which then cleans up and re-creates my pipeline etc.

I am using windows task manager to monitor the total memory used by my app, I understand that this is not the most accurate method. But the leak is repeatable.
The memory used by my app was as follows 165 -> 174 -> 184 -> 194mb

Also the rtsp video stream im viewing is encoded using h264.
Comment 30 Nicola 2014-07-16 09:03:31 UTC
this is exactly the same leak reported for 0.10 and 1.0.x (and fixed with the attached patchs). When you set the pipeline to null d3d instance is not released, on 0.10 and 1.0.x happen only if you have more than one d3dvideosink element in you application (two or more pipeline that use one d3dvideosink element or a single pipeline with multiple d3dvideosink elements)
Comment 31 Steven 2014-07-16 09:11:34 UTC
(In reply to comment #30)
> this is exactly the same leak reported for 0.10 and 1.0.x (and fixed with the
> attached patchs). When you set the pipeline to null d3d instance is not
> released, on 0.10 and 1.0.x happen only if you have more than one d3dvideosink
> element in you application (two or more pipeline that use one d3dvideosink
> element or a single pipeline with multiple d3dvideosink elements)

I am using a single pipeline with 4 d3dvideosink elements.

Will the suggested patches be in 1.3.92 / 1.4.0 ?
Comment 32 Nicola 2014-07-16 09:26:37 UTC
the patch apply on 0.10 and 1.0.x, must be ported to gstreamer >= 1.2.0
Comment 33 Nicola 2014-07-16 09:40:30 UTC
maybe, for test purpose you could try to modify your parse_launch line to use only one d3dvideosink, 

if you see no leak with a single d3dvideosink element is the same problem initially reported
Comment 34 Steven 2014-07-16 09:55:59 UTC
(In reply to comment #33)
> maybe, for test purpose you could try to modify your parse_launch line to use
> only one d3dvideosink, 
> 
> if you see no leak with a single d3dvideosink element is the same problem
> initially reported

I have tested my app using your suggested approach. My original and test pipelines are given below. The memory leak is exactly the same in both cases.

Original Pipeline:
uridecodebin name=vehiclebin uri=rtsp://127.0.0.1:8554/vehicle.sdp !
tee name=tp
tp. ! queue ! videocrop top=0   left=10  right=376 bottom=290 ! autovideosink name=rear
tp. ! queue ! videocrop top=0   left=400 right=0   bottom=290 ! autovideosink name=forward
tp. ! queue ! videocrop top=290 left=10  right=376 bottom=6   ! autovideosink name=left
tp. ! queue ! videocrop top=290 left=398 right=0   bottom=6   ! autovideosink name=right

Test Pipeline:
uridecodebin name=vehiclebin uri=rtsp://127.0.0.1:8554/vehicle.sdp ! autovideosink name=forward

---

On a side note d3dvideosink is no longer stretching the video to fill the window / control. The scaling method now maintains aspect ratio. Can I change this back myself with an option or is it in the code?
Comment 35 Sebastian Dröge (slomo) 2014-07-16 12:33:24 UTC
(In reply to comment #34)
> 
> On a side note d3dvideosink is no longer stretching the video to fill the
> window / control. The scaling method now maintains aspect ratio. Can I change
> this back myself with an option or is it in the code?

The force-aspect-ratio property can be used to modify that. It's TRUE by default.
Comment 36 Sebastian Dröge (slomo) 2014-07-16 12:34:42 UTC
So the problem is that the D3D device is not properly released if multiple are used? I can try to port the patches to 1.4, not sure when I have time for that though. Hopefully before 1.4.0
Comment 37 Steven 2014-07-16 12:54:09 UTC
(In reply to comment #35)
> (In reply to comment #34)
> > 
> > On a side note d3dvideosink is no longer stretching the video to fill the
> > window / control. The scaling method now maintains aspect ratio. Can I change
> > this back myself with an option or is it in the code?
> 
> The force-aspect-ratio property can be used to modify that. It's TRUE by
> default.

Thank you, I will set it to FALSE when creating my pipeline.
Comment 38 Sebastian Dröge (slomo) 2014-07-16 14:29:14 UTC
I looked at the patches now and I don't see what their point is, or how they would map to the current code.

Would be useful to know what exactly is leaked here, I didn't find any leak of D3D resources by reading the code.
Comment 39 voskater15 2014-07-18 19:04:03 UTC
I've noticed a huge leak in the d3d resources specially on Windows 7, I can only give facts, not code (yet, I haven't revised d3dvideosink plugin).

I say that on Windows 7 because my project works like a charm on Windows 8.1 Pro, it does not leaking memory with that OS.

Well the internals of my projects are implemented in this way:

Plugins version-> 1.2.4 (libstd3dvideosink has all the fixes commented in bugzilla and the plugins-bad cgit site, and some improvements by myself).

Pipeline -> Playbin plugin
Video-Sink -> d3dvideosink (libgstd3dvideosink.dll)

I never dispose my pipeline, only when I receive an error in the bus pipeline, so what I do when I want to load a new stream is to change the state of the pipeline to GST_STATE_READY, so the video-sink (d3d) goes to GST_STATE_NULL when the state transition finishes.

I recompiled libgstd3dvideosink few weeks ago with the MULTITHREAD flag (I commented that to you Sebastian remember?) and also updated to Directx9 Ex version (this gains 50% performance, so guys ...what are you waiting for? hehe).

Well this are the facts that I have noticed on both Windows OS.

On Windows 7 x64:

If I put 3 .mp4 videos in a row within 1 minute the memory increases a lot and if I stop the pipeline or change to audio streams, RAM stays uncleaned. After stoping the pipeline I attach a debugger to see what dependencies stays on memory and I can see 4 nvd3dum.dll. On others Windows 7 I can see 4 d3d9.dll.

On Windows 8.1 Pro x64:

Same as above, RAM incerases a little bit, but It disposes itself I leave the video playing either I stop the pipeline or play audio streams.

Question is:

What's the difference between memory management and DX resources management on Windows 7 and Win 8?

Well my business depends on this so I can help like last time, I'll test a little bit but if I see that I cannot fix it then I'll look at the plugin.
Comment 40 Nicola 2015-02-27 17:16:34 UTC
it seems fixed in git master
Comment 41 Tim-Philipp Müller 2015-02-27 17:32:45 UTC
Great, thanks for re-testing and letting us know!