GNOME Bugzilla – Bug 766135
Memory leak in gnome-settings-daemom ubuntu 16.04
Last modified: 2019-03-20 11:35:18 UTC
Created attachment 327469 [details] htop - processes Hi :) After installinng kubuntu, I switched to gnome-desktop (don't know if relevant and if can help). However after some time, all gnome animations become slower and slower. I did not use my laptop more than 4 hours in a row so I don't know if related to all suspend, but anyway as you can see in the screenshot, there is a nice mem leak. distrib: ubuntu 16.04 Waiting for a fix, any solution to restart it without having to log off / login again ? Thanks !
Which exact version of gnome-settings-daemon is that?
3.18.2 according `apt-get cache`. And from the package sources as well: https://launchpad.net/ubuntu/xenial/+source/gnome-settings-daemon
You should test with 3.18.3. If you see the same problem with that version, try disabling the plugins one-by-one with dconf-editor, under /org/gnome/settings-daemon/plugins in the UI. The "active" setting is the one you want to disable. You'd disable one, reboot, test, and if it still shows the problem, re-enable it, and reboot again. Or you can run gnome-settings-daemon under valgrind, but that's more involved.
Oh, and you can start with disabling the "updates" plugin, which is an Ubuntu only change.
I am now monitoring this process. Nothing to do with the suspend state. It just grows up little by little with a normal use (firefox/thunderbird/terminal). I will try with a more recent version of 3.18.x as soon as I find the corresponding package.
My system slows down actually for sure when it tries to turnoff my external monitor May 17 00:20:21 hatchery /usr/lib/gdm3/gdm-x-session[1483]: (II) intel(0): switch to mode 3840x2160@30.0 on HDMI1 using pipe 0, position (1920, 0), rotation normal, reflection none May 17 00:20:21 hatchery /usr/lib/gdm3/gdm-x-session[1483]: (II) intel(0): switch to mode 1920x1080@60.0 on eDP1 using pipe 1, position (0, 0), rotation normal, reflection none May 17 00:20:22 hatchery /usr/lib/gdm3/gdm-x-session[1483]: (--) intel(0): HDMI max TMDS frequency 300000KHz May 17 00:20:49 hatchery /usr/lib/gdm3/gdm-x-session[1483]: (II) intel(0): resizing framebuffer to 1920x1080 May 17 00:20:49 hatchery /usr/lib/gdm3/gdm-x-session[1483]: (--) intel(0): HDMI max TMDS frequency 300000KHz May 17 00:20:49 hatchery /usr/lib/gdm3/gdm-x-session[1483]: (--) intel(0): HDMI max TMDS frequency 300000KHz May 17 00:20:49 hatchery /usr/lib/gdm3/gdm-x-session[1483]: (II) intel(0): resizing framebuffer to 5760x2160 (http://pastebin.com/SHnJgt1T)
the memory usage usage of gnome-settings-daemon is directly linked to those events
Created attachment 336728 [details] Screenshot of memleak
Created attachment 336729 [details] xrandr-mich
I have the same memory leak problem on Arch Linux. Here are some facts I've noticed about it: * The leak only happens when I have external monitors connected to the laptop * The leak does appear to start when I leave the laptop and the monitors shut down after inactivity, as already pointed out by Damien GERARD * The only way I found to overcome this, is to logoff/login back, which is not a really good solution, I've tried using the --replace switch, but it only restarts the gnome-settings-daemon under my user - although it leaks too, the one that really swallows all of my memory is the on running under the gdm user Some info on my machine: $ pacman -Q gnome-settings-daemon gnome-settings-daemon 3.20.1+8+g40bf4fd-1 xrandr output is attached, explanation of monitors: HDMI1 ==> Physical HDMI port on my laptop, screen number 2 HDMI2 ==> Mini DisplayPort on laptop ==> HDMI adapter, screen number 3 eDP1 ==> Internal display (Turned off when I use external monitors) Screenshot of leak is attached. Please, if I need to supply more data, or help debug this, tell me what to do, I'd really like to help solve this issue, it is very annoying.
It was an external monitor in my case as well, connected in hdmi.
I decided to dig a bit deeper into the problem, and now when the daemon got to around 2.7GB in memory, I used gdb to make a coredump $ pgrep -f gnome-settings-daemon -u gdm 942 # gdb -p 942 (gdb) generate-core-file Saved corefile core.2071 (gdb) quit $ ls -alh core.942 -rw-r--r-- 1 root root 2.7G Oct 2 13:11 core.942 Then I got all the strings from it, and took out the ones that appear a lot of times $ strings core.942 | sort | uniq -c | sed -r 's/\s+([0-9]+)\s(.+)/\1,"\2"/' | sort -nr -t ',' -k 1,2 | head -n 42 601012,"uaua{sv})" 173958,"7`qV" 171232,"underscanning" 171028,"primary" 171027,"serial" 171027,"backlight" 171026,"width-mm" 171026,"vendor" 171026,"supports-underscanning" 171026,"product" 171026,"presentation" 171026,"min-backlight-step" 171026,"height-mm" 171026,"display-name" 171026,"connector-type" 168774,"edid" 125272,"RD24L04140564" 116072,"RD24L" 115499,"auaua{sv})" 113038,"HDMIA" 110869," " 110227,"q8-@X,E" 107324,"DJNC4JA000171" 105642,"i2352Vh" 61452,"DBus" 58477,"4p0 5" 58010,"6RGW0" 57994,"eDP37" 57993,"HDMIA45" 57993,"Built-in display" 57993,"0x13f8" 57993,"0x00000000" 57991,"RIS 23"" 57989,"LQ156Z1" 57988," " 57914,"iuaua{sv})" 55058,"HDMIA52" 53200,"n_idle_cb" 52306," @1 " 52280," n(U" 52247," " 52222,"AOC Intl 23"" As you can see, there a some strings that appear enormous amount of times, the most notable of them are the ones related to my monitors: RD24L i2352Vh AOC Intl 23 Those, among other strings are parts of my monitors' model numbers - so it seems this has something to do with the monitor profiles, but here I'm not really sure, I'm not that of an expert on neither Gnome or it's monitor control routines. Hope somebody here can help us with that.
I can't reproduce this problem here, but you might want to try and run gnome-settings-daemon under valgrind with the "--timed-exit" option, for it to exit cleanly. At the very least, gather a log and see whether some events occur regularly that could help pinpoint the culprit. Other question might be whether you have screens color calibrated. As requested in comment 3 and 4, you should also try to disable some of the plugins, one each time. Don't forget to restart the daemon in between.
Thanks for the tips on debugging the problem, here is what I got so far: 1. I've ran the replace gsd replace command again to look at its output (when I did it before I ran it win nohup), and indeed saw a lot of strings related to the color profile plugin, seems that there is a problem with the profile of one of my monitors, as you guessed. For some reason, 'colormgr get-devices' sees only one of my monitors, and is seems there is no profile attached to it, and therefore it is not calibrated. To really calibrate it I understood I need a Windows/OSX installation to copy the ICC from, which I currently don't have, so I've just disabled the color plugin for now. The repetitive logs of the daemon itself are in the attachment. 2. I've ran the daemon with the replace command using the following valgrind command: $ valgrind --log-file=gsd.valgrind.log --leak-check=full --show-leak-kinds=all /usr/lib/gnome-settings-daemon/gnome-settings-daemon --replace I ran this overnight, and I forgot the '--timed-exit' option you suggested, but it still generate a log, though for some reason it seems that it got overwritten when it reached 11M in size. The log is attached too. This command was ran while the color plugin was still on. 3. Once I disabled the color plugin, it seems that the problem was partially gone, but now it just takes more time for it to eat up memory. Now also, the gsd daemon that was leaking under my user, is leaking a lot less, but the on under the 'gdm' user is still leaking massivly (Why are there two of them?). To tell you the truth, it's kind of hard to tell for sure if disabling it did something to the leak, as numbers are still growing. I'm going to try to install Windows on some flash drive to get an ICC profile, and see if that helps, that'll take some time. Do you have any suggestions on how to give the daemon another run under valgrind with better options? Also, the daemon that runs under the gdm user seems to be hard to debug, as the replace command replaces the one running under my user. Any insight on that?
Created attachment 337180 [details] debug logs
(In reply to Michael Sandler from comment #14) <snip> > 2. I've ran the daemon with the replace command using the following valgrind > command: > $ valgrind --log-file=gsd.valgrind.log --leak-check=full > --show-leak-kinds=all /usr/lib/gnome-settings-daemon/gnome-settings-daemon > --replace > I ran this overnight, and I forgot the '--timed-exit' option you suggested, With the timed-exit, you won't need to run it overnight. Adding the "--track-origins=yes" argument would help get better information. > but it still generate a log, though for some reason it seems that it got > overwritten when it reached 11M in size. Weird. > The log is attached too. > This command was ran while the color plugin was still on. > > 3. Once I disabled the color plugin, it seems that the problem was partially > gone, but now it just takes more time for it to eat up memory. > Now also, the gsd daemon that was leaking under my user, is leaking a lot > less, but the on under the 'gdm' user is still leaking massivly (Why are > there two of them?). There might be 2 problems in that log. The error path would be leaking memory, and colord isn't able to detect the device, in a loop. The log you attached shows that it went up to 4 gigs of memory allocated, in one night. I guess that if, with the color plugin disabled, you don't get to that amount, then this is where the problem lies. > Also, the daemon that runs under the gdm user seems to be hard to debug, as > the replace command replaces the one running under my user. Any insight on > that? If you can reproduce the problem in your session, best not bother trying to debug the gdm one yet. Additional question, what's the output of "xrandr -q" on that system?
What seems to be happening is that the hardware is outputting a *lot* of primary-changed events. Each time this happens g-s-d queries colord for the icc profile as when the primary changes there's a high chance of a different crtc being used which means setting a new gamma table. As it's happening so frequently, I think it's the case that we're just stacking up the async events faster than they can be processed.
I'm hitting the same issue. Here's the most suspicious part of valgrind output (gsd was running for few minutes), sorry for missing debug symbols: ==3347== 3,555,523 (15,920 direct, 3,539,603 indirect) bytes in 398 blocks are definitely lost in loss record 13,820 of 13,820 ==3347== at 0x4C29BBE: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==3347== by 0x68C7B98: g_malloc (in /usr/lib/libglib-2.0.so.0.5000.1) ==3347== by 0x68E00D2: g_slice_alloc (in /usr/lib/libglib-2.0.so.0.5000.1) ==3347== by 0x68FE28D: ??? (in /usr/lib/libglib-2.0.so.0.5000.1) ==3347== by 0x68FAF23: g_variant_builder_end (in /usr/lib/libglib-2.0.so.0.5000.1) ==3347== by 0x6352FEE: ??? (in /usr/lib/libgio-2.0.so.0.5000.1) ==3347== by 0x6352DA1: ??? (in /usr/lib/libgio-2.0.so.0.5000.1) ==3347== by 0x6355003: g_dbus_message_new_from_blob (in /usr/lib/libgio-2.0.so.0.5000.1) ==3347== by 0x635F28C: ??? (in /usr/lib/libgio-2.0.so.0.5000.1) ==3347== by 0x631C322: ??? (in /usr/lib/libgio-2.0.so.0.5000.1) ==3347== by 0x631C358: ??? (in /usr/lib/libgio-2.0.so.0.5000.1) ==3347== by 0x68C2439: g_main_context_dispatch (in /usr/lib/libglib-2.0.so.0.5000.1)
Btw, for me one that leaks seems to be network manager: strings core.826 | sort | uniq -c | sort -nr -t ',' -k 1,2 | head -n 42 23767 org.freedesktop.NetworkManager 11894 org.freedesktop.DBus.Properties 11865 :1.3 11744 org.freedesktop.NetworkManager.AccessPoint 6916 DBus 5910 PropertiesChanged 5865 flags 5864 hw-address 5863 mode 5860 wpa-flags 5860 strength 5860 ssid 5860 rsn-flags 5860 max-bitrate 5860 last-seen 5860 frequency 5812 AccessPoint' 5114 tworkManager' 4688 desktop.DBus
If you're using Fedora 25, you might want to try to upgrade gnome-session and gnome-settings-daemon to 3.23 versions, available at: http://koji.fedoraproject.org/koji/packageinfo?packageID=315 and: http://koji.fedoraproject.org/koji/packageinfo?packageID=5615 The plugins in that version are stand-alone, which should make root-causing easier.
(In reply to Bastien Nocera from comment #20) > If you're using Fedora 25, you might want to try to upgrade gnome-session > and gnome-settings-daemon to 3.23 versions, available at: > http://koji.fedoraproject.org/koji/packageinfo?packageID=315 > and: > http://koji.fedoraproject.org/koji/packageinfo?packageID=5615 > > The plugins in that version are stand-alone, which should make root-causing > easier. I'm using archlinux. However I believe that leaking plugin is 'sharing', I disabled it using dconf-editor and constant leak is gone. But there's another - gsd leaks few megs each time when I plug/unplug external monitor.
Only thing I can advise would be to use gnome-settings-daemon and gnome-session 3.23.2, so you have each plugin split off into its own daemon, and debug those individually. The previous code is just impossible to debug properly with so many moving parts.
(In reply to Bastien Nocera from comment #22) > Only thing I can advise would be to use gnome-settings-daemon and > gnome-session 3.23.2, so you have each plugin split off into its own daemon, > and debug those individually. The previous code is just impossible to debug > properly with so many moving parts. Is gnome-3.22 compatible with gsd-3.23 and gnome-session-3.23? I don't want to compile whole gnome.
(In reply to Vasily Khoruzhick from comment #23) > (In reply to Bastien Nocera from comment #22) > > Only thing I can advise would be to use gnome-settings-daemon and > > gnome-session 3.23.2, so you have each plugin split off into its own daemon, > > and debug those individually. The previous code is just impossible to debug > > properly with so many moving parts. > > Is gnome-3.22 compatible with gsd-3.23 and gnome-session-3.23? I don't want > to compile whole gnome. As long as you have both gsd-3.23 and gnome-session-3.23, you should be fine, yes.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/gnome-settings-daemon/issues/298.