GNOME Bugzilla – Bug 688337
simple compile-time-selectable profiling points
Last modified: 2018-05-24 14:50:27 UTC
A number of GNOME modules have access()-based "profiling points" which can be used in conjunction with strace to get an idea of how long an application spends, for instance, blocking on I/O. While working on Bug #687362 I've found it useful to add some of these to GLib and GIO. I realise that in general, GNOME developers are more in favour of systemtap - perhaps because many GNOME developers use Fedora, which has a kernel with the utrace patchset necessary to make userland systemtap points useful...
Created attachment 228984 [details] [review] Add basic infrastructure for profiling points Loosely based on what's in Folks, which was inspired by <http://people.gnome.org/~federico/news-2006-03.html#timeline-tools>. Yes, I know systemtap exists - but user-space systemtap tracepoints require an unmerged kernel patch, whereas combining access() with strace works anywhere. It's access() because that's: a syscall, making it visible to strace; harmless, since it's read-only and has no side-effects; and classed as a filename by strace, which means it displays the argument in full, rather than truncating at 32 bytes. Via some slightly ugly cpp, gprofileprivate.h can be used from either GLib itself (in which case it's an intra-library call), or GObject/GIO (in which case it's a GLIB_PRIVATE_CALL). These profiling points are by no means zero-overhead - they are themselves I/O, after all, and they contain a malloc/free - but they can give a general picture of where the time is going, and how long annotated functions take.
Created attachment 228985 [details] [review] Emit profiling points around dispatch(), and for long dispatch() --- Requires Bug #588139.
Created attachment 228986 [details] [review] g_find_program_in_path: refactor to have a single exit path
Created attachment 228987 [details] [review] g_find_program_in_path: emit profiling messages
Created attachment 228988 [details] [review] g_file_get_contents: emit profiling messages
Created attachment 228989 [details] [review] g_file_set_contents: emit profiling messages
Created attachment 228990 [details] [review] Add some profiling to blocking D-Bus calls
Created attachment 228991 [details] [review] Add profiling points in gstdio functions
Created attachment 228992 [details] [review] g_spawn_sync: add profiling points
Created attachment 228993 [details] [review] g_settings_sync: add profiling points
Created attachment 228994 [details] [review] GLocalFileInputStream: emit profiling messages
Created attachment 228995 [details] [review] GLocalFileOutputStream: emit profiling messages
Created attachment 228996 [details] [review] GLocalFileInfo: emit profiling messages
Created attachment 228997 [details] [review] GLocalFile: emit profiling messages
It isn't necessarily useful or important to have all of those profiling points: they're a first attempt at annotating a useful proportion of blocking calls, to get a general picture of what's going on when gnome-shell freezes for a moment. They could probably benefit from some sort of bitmask to determine which ones are emitted. I've deliberately made their interface such that they can log anything (e.g. useful context like filenames), and could equally well be any combination of: an access() trace point, a systemtap trace point, a valgrind VALGRIND_PRINTF, a write to stderr, or whatever.
The access() thing is quite a hack, but a useful improvement probably would be having a static string like /nonexistent/glibprofiling/ at the head of the filename, because on Linux at least you'll end up with exactly one /nonexistent in the negative dentry cache instead of polluting it with junk. Anyways in the big picture, I think we should wait a little bit and see how the systemtap/perf stuff works out. Note for example: http://gnu.wildebeest.org/blog/mjw/2012/05/24/pull-user-space-probe-instrumentation/ Which is available in upstream Linux 3.5 (shipped in Ubuntu 12.10, Fedora 17+, but not Debian Wheezy which is 3.2).
(In reply to Colin Walters from comment #16) > Anyways in the big picture, I think we should wait a little bit and see how > the systemtap/perf stuff works out. > > Note for example: > > http://gnu.wildebeest.org/blog/mjw/2012/05/24/pull-user-space-probe- > instrumentation/ > > Which is available in upstream Linux 3.5 (shipped in Ubuntu 12.10, Fedora > 17+, but not Debian Wheezy which is 3.2). SystemTap upstream seem to think it's reasonably easy to use SystemTap on Debian: https://www.sourceware.org/systemtap/wiki/SystemtapOnDebian smcv potentially disagrees, but thinks he should probably take another look at SystemTap on Debian again at some point and re-evaluate how much technical debt there is (if any/if much). So personally I think we can standardise on SystemTap rather than a series of ad-hoc debugging solutions (like as proposed here, but there are others). Not because they're bad, but because it seems a bit silly for everyone to reinvent the wheel each time they encounter an issue which needs debugging like this. Close this?
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/glib/issues/630.