GNOME Bugzilla – Bug 707523
d3dvideosink memleak
Last modified: 2015-02-27 17:32:45 UTC
Created attachment 254136 [details] test app for gstreamer 1.0 d3dvideosink has huge memleak if an application start more than one pipeline, attached a sample app that start 4 pipelines (videotestsrc ! d3dvideosink) and do stop/play every 10 seconds. changing d3dvideosink to fakesink fix the leak. Please run the test app and observe the memory usage, the problem happen with both gstreamer-1.0 and with sdk-2013.6, please note that this is a regression, d3dvideosink included in ossbuild works fine for convenience the compiled apps can be downloaded from the following links (it requires visual studio 2010 c++ runtime, the rest is included): http://195.250.34.59/temp/TestMultiViewers_1.0.tar.bz2 http://195.250.34.59/temp/TestMultiViewers_sdk0.10.tar.bz2 please note that the problem does not happen if the app start only one pipeline (really weired)
Created attachment 254137 [details] test app for gstreamer-sdk 2013.6
Created attachment 255025 [details] test app for d3dvideosink memleak in MFC I detect memory leaks on d3dvideosink too. This bug aapears when pipeline is restarting. Also there is similar problem with d3dvideosink. This is my pipeline: appsrc ! wavescope ! videoconvert ! videobox ! d3dvideosink d3dvideosink start memory leaking when it is overlayed in window. Overlay must be set by function gst_video_overlay_set_window_handle. This is necessary and properties of videobox must be set frequently (e.g. in appsrc push_data).
Created attachment 255220 [details] [review] Fix d3dvideosink memleak I have tried to fix this resource leaks in d3dvideosink. This is my solution. I'm sorry, but I don't know how to download sources from git repository. I made a diff file between d3dvideosink.c ver 1.0.10 and my solution.
I can confirm that the patch solve the major leakeage in my test apps too (applied to gstreamer-sdk)
Can you please do a diff -u ?
I'm a bit busy now, later or tomorrow I'll sent a git patch against gst-sdk and 1.x, basically the problem is here: http://cgit.freedesktop.org/gstreamer-sdk/gst-plugins-bad/tree/sys/d3dvideosink/d3dvideosink.c?h=sdk-0.10.23#n2354 if you have more than a viewer the code at line 2356-2358 is not reached, so these lines: GST_D3DVIDEOSINK_D3D_DEVICE_LOCK (sink); gst_d3dvideosink_release_d3d_device (sink); GST_D3DVIDEOSINK_D3D_DEVICE_UNLOCK (sink); need to be after: shared.element_count--; and however before the goto if you have only one viewer the problem does not happen (shared.element_count is never >0)
Created attachment 255448 [details] [review] patch for 0.10
Comment on attachment 255448 [details] [review] patch for 0.10 Please provide a patch against git master
the patch works for both 0.10 (sdk) and 1.0, d3dvideosink in git master seems quite different from 1.0 and I never tested with this version, I'll do some test as soon as windows binary for 1.2 are available for windows
I recompiled my testapp (1.0 version) against 1.2, seems d3dvideosink in 1.2 has major breakages, basically I have an app that start 4 pipeline like this: videotestsrc ! d3dvideosink (tested videotestsrc ! queue ! d3dvideosink with no success) and restart them every 15 seconds, in 1.2 the app seems to deadlock on the first start, changing d3dvideosink with fakesink solve the issue
Created attachment 256092 [details] [review] patch for 1.0 Patch was created with git diff command.
(In reply to comment #10) > I recompiled my testapp (1.0 version) against 1.2, seems d3dvideosink in 1.2 > has major breakages, basically I have an app that start 4 pipeline like this: > > videotestsrc ! d3dvideosink > > (tested videotestsrc ! queue ! d3dvideosink with no success) > > and restart them every 15 seconds, in 1.2 the app seems to deadlock on the > first start, changing d3dvideosink with fakesink solve the issue Can you get a backtrace for the deadlocks? I tried starting two in my VM here but the second sink only shows black
Comment on attachment 256092 [details] [review] patch for 1.0 Also needs to be updated to apply cleanly to 1.2, which is the current stable release
I am also seeing the memory leak issue with 1.0 and 0.1 builds of gstreamer when using the d3dvideosink. Also I cannot get video to play due to crashing on startup when using version 1.2 with a pipeline containing queues.
(In reply to comment #14) > I am also seeing the memory leak issue with 1.0 and 0.1 builds of gstreamer > when using the d3dvideosink. > Also I cannot get video to play due to crashing on startup when using version > 1.2 with a pipeline containing queues. Would it be possible to provide a backtrace ? (note that you need msys gdb for that).
(In reply to comment #15) > (In reply to comment #14) > > I am also seeing the memory leak issue with 1.0 and 0.1 builds of gstreamer > > when using the d3dvideosink. > > Also I cannot get video to play due to crashing on startup when using version > > 1.2 with a pipeline containing queues. > > Would it be possible to provide a backtrace ? (note that you need msys gdb for > that). I will try to obtain a backtrace for you. While I do that here are some example pipelines which should enable you to recreate the crash yourself. gst-launch-1.0.exe videotestsrc ! videoconvert ! d3dvideosink WORKS OK gst-launch-1.0.exe videotestsrc ! videoconvert ! tee name=t t. ! queue ! d3dvideosink WORKS OK gst-launch-1.0.exe videotestsrc ! videoconvert ! tee name=t t. ! queue ! d3dvideosink CRASHES EVERYTIME My setup: Visual Studio 2008 on Windows 7 x64 gstreamer 1.2.3 installed with "complete" option Ive tried 1.2.0 1.2.1 1.2.2 and they all crash the same x32 and x64 versions of gstreamer. N.B. Im wondering if this issue should be raised as a new bug? The crashing problem is different from the original mem-leak bug.
sorry the example that crashes everytime should be as follows: gst-launch-1.0.exe videotestsrc ! videoconvert ! tee name=t t. ! queue ! d3dvideosink t. ! queue ! d3dvideosink
Created attachment 270296 [details] backtrace for d3dvideo sink crash
Just tried this with the new 1.3.91 release and it still crashes. Looks like the tee element is causing the crash with this version. This command CRASHES: gst-launch-1.0.exe videotestsrc ! videoconvert ! tee name=t t. ! queue ! d3dvideosink t. ! queue ! d3dvideosink This command WORKS: gst-launch-1.0.exe videotestsrc ! videoconvert ! queue ! d3dvideosink
Created attachment 280633 [details] libgstd3dvideosink.dll Does this fix it? It initializes the D3D devices with D3DCREATE_MULTITHREADED. It's a 32 bit dll though, tell me if you need the 64 bit version. If it still crashes, can you get a backtrace of all threads with this version?
Thanks a lot for the attached dll, 32bit is fine for my needs. I can confirm that the example commands, given in comment 19, run successfully with your dll. However I am getting the following output on the command-line: (gst-launch-1.0:10872): GLib-GObject-WARNING **: cannot register existing type 'GstD3DVideoSink' (gst-launch-1.0:10872): GLib-GObject-CRITICAL **: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed (gst-launch-1.0:10872): GLib-GObject-CRITICAL **: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed (gst-launch-1.0:10872): GLib-CRITICAL **: g_once_init_leave: assertion 'result != 0' failed (gst-launch-1.0:10872): GStreamer-CRITICAL **: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed
Thanks, I pushed that change and it will be included in 1.4.0. The other problem seems to be something wrong with your installation though. However how is this all related to the original memory leak that this bug report is about?
Created attachment 280734 [details] my gstreamer code Now that the cause of the crashing has been identified, I dont think it is related to this tickets original memory leak issue. However I just checked and I am still seeing the original memory leak issue when closing a pipeline and starting another pipeline. This has been reduced from ~30mb to ~10mb each time now. I have attached my gstreamer code as I may be simply doing something wrong. The .cc and .h files have been combined. Thanks.
You can't restart gst_parse_launch() pipelines like that reliably. You'll have to create a new one after EOS, etc. Just setting them to READY or NULL, and then starting again is not enough. This is actually a bug in gst_parse_launch() though, with manually built pipelines exactly that is possible. Not sure if that is related to your problem though... but do you still have leaks when creating a new pipeline each time (and unreffing the old one after setting it to NULL)? For the decreased memory leakage... I fixed a huge memory leak in d3dvideosink some days ago. That's probably the difference.
Could you please expand on your suggestions for changes to my code. I think I am already doing what you recommend. Currently on GST_MESSAGE_EOS my cleanup function is called. cleanup: 1)set the bus sync handler to NULL and unref the bus. 2)set pipleline to GST_STATE_NULL and unref the pipleline Another pipeline is again built with gst_parse_launch in setupVideo
Oh then you're actually doing already what I meant. Means there's a memory leak somewhere else. Is it proportional to the number of frames that were displayed or always the same number? And is it increasing with the same amount after every run? How do you measure the memory leak?
not tested, but looking at the code I think the memory leak is here: http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/d3dvideosink/d3dhelpers.c#n2304 if you have more than one instance (an app with multiple d3dvideosink) klass->d3d.refs >= 1 is true and the d3d object is not released, sorry I cannot test soon if I'm right, Sebiastian it make sense to you?
Yes, that memory is "leaked"... however it is only allocated once per process so shouldn't show up anywhere really
(In reply to comment #26) > Oh then you're actually doing already what I meant. Means there's a memory leak > somewhere else. > > Is it proportional to the number of frames that were displayed or always the > same number? And is it increasing with the same amount after every run? > > How do you measure the memory leak? The size of the leak does not seem to be linked to the number of frames played. I left my stream playing for a random amount of time before triggering a GST_MESSAGE_EOS which then cleans up and re-creates my pipeline etc. I am using windows task manager to monitor the total memory used by my app, I understand that this is not the most accurate method. But the leak is repeatable. The memory used by my app was as follows 165 -> 174 -> 184 -> 194mb Also the rtsp video stream im viewing is encoded using h264.
this is exactly the same leak reported for 0.10 and 1.0.x (and fixed with the attached patchs). When you set the pipeline to null d3d instance is not released, on 0.10 and 1.0.x happen only if you have more than one d3dvideosink element in you application (two or more pipeline that use one d3dvideosink element or a single pipeline with multiple d3dvideosink elements)
(In reply to comment #30) > this is exactly the same leak reported for 0.10 and 1.0.x (and fixed with the > attached patchs). When you set the pipeline to null d3d instance is not > released, on 0.10 and 1.0.x happen only if you have more than one d3dvideosink > element in you application (two or more pipeline that use one d3dvideosink > element or a single pipeline with multiple d3dvideosink elements) I am using a single pipeline with 4 d3dvideosink elements. Will the suggested patches be in 1.3.92 / 1.4.0 ?
the patch apply on 0.10 and 1.0.x, must be ported to gstreamer >= 1.2.0
maybe, for test purpose you could try to modify your parse_launch line to use only one d3dvideosink, if you see no leak with a single d3dvideosink element is the same problem initially reported
(In reply to comment #33) > maybe, for test purpose you could try to modify your parse_launch line to use > only one d3dvideosink, > > if you see no leak with a single d3dvideosink element is the same problem > initially reported I have tested my app using your suggested approach. My original and test pipelines are given below. The memory leak is exactly the same in both cases. Original Pipeline: uridecodebin name=vehiclebin uri=rtsp://127.0.0.1:8554/vehicle.sdp ! tee name=tp tp. ! queue ! videocrop top=0 left=10 right=376 bottom=290 ! autovideosink name=rear tp. ! queue ! videocrop top=0 left=400 right=0 bottom=290 ! autovideosink name=forward tp. ! queue ! videocrop top=290 left=10 right=376 bottom=6 ! autovideosink name=left tp. ! queue ! videocrop top=290 left=398 right=0 bottom=6 ! autovideosink name=right Test Pipeline: uridecodebin name=vehiclebin uri=rtsp://127.0.0.1:8554/vehicle.sdp ! autovideosink name=forward --- On a side note d3dvideosink is no longer stretching the video to fill the window / control. The scaling method now maintains aspect ratio. Can I change this back myself with an option or is it in the code?
(In reply to comment #34) > > On a side note d3dvideosink is no longer stretching the video to fill the > window / control. The scaling method now maintains aspect ratio. Can I change > this back myself with an option or is it in the code? The force-aspect-ratio property can be used to modify that. It's TRUE by default.
So the problem is that the D3D device is not properly released if multiple are used? I can try to port the patches to 1.4, not sure when I have time for that though. Hopefully before 1.4.0
(In reply to comment #35) > (In reply to comment #34) > > > > On a side note d3dvideosink is no longer stretching the video to fill the > > window / control. The scaling method now maintains aspect ratio. Can I change > > this back myself with an option or is it in the code? > > The force-aspect-ratio property can be used to modify that. It's TRUE by > default. Thank you, I will set it to FALSE when creating my pipeline.
I looked at the patches now and I don't see what their point is, or how they would map to the current code. Would be useful to know what exactly is leaked here, I didn't find any leak of D3D resources by reading the code.
I've noticed a huge leak in the d3d resources specially on Windows 7, I can only give facts, not code (yet, I haven't revised d3dvideosink plugin). I say that on Windows 7 because my project works like a charm on Windows 8.1 Pro, it does not leaking memory with that OS. Well the internals of my projects are implemented in this way: Plugins version-> 1.2.4 (libstd3dvideosink has all the fixes commented in bugzilla and the plugins-bad cgit site, and some improvements by myself). Pipeline -> Playbin plugin Video-Sink -> d3dvideosink (libgstd3dvideosink.dll) I never dispose my pipeline, only when I receive an error in the bus pipeline, so what I do when I want to load a new stream is to change the state of the pipeline to GST_STATE_READY, so the video-sink (d3d) goes to GST_STATE_NULL when the state transition finishes. I recompiled libgstd3dvideosink few weeks ago with the MULTITHREAD flag (I commented that to you Sebastian remember?) and also updated to Directx9 Ex version (this gains 50% performance, so guys ...what are you waiting for? hehe). Well this are the facts that I have noticed on both Windows OS. On Windows 7 x64: If I put 3 .mp4 videos in a row within 1 minute the memory increases a lot and if I stop the pipeline or change to audio streams, RAM stays uncleaned. After stoping the pipeline I attach a debugger to see what dependencies stays on memory and I can see 4 nvd3dum.dll. On others Windows 7 I can see 4 d3d9.dll. On Windows 8.1 Pro x64: Same as above, RAM incerases a little bit, but It disposes itself I leave the video playing either I stop the pipeline or play audio streams. Question is: What's the difference between memory management and DX resources management on Windows 7 and Win 8? Well my business depends on this so I can help like last time, I'll test a little bit but if I see that I cannot fix it then I'll look at the plugin.
it seems fixed in git master
Great, thanks for re-testing and letting us know!