I'm copying an email I sent to gnome-doc-devel-list:

I've have been looking at fixing the startup time for yelp when
man page support is enabled (as it is by default in Ubuntu).

During my research I found one major bottleneck in when yelp
starts up and there are a large number of man pages to index.
(In my case 40,503 - needless to say I haven't reinstalled
in quite awhile).

The bottleneck occurs in the yelp_doc_info_new function, in a
call to gnome_vfs_make_uri_from_input_with_dirs().  Everytime
this function is called, a system call is made to access the
current working directory and then an attempt to the access
the file.  If you do an "strace -e file" on yelp HEAD, you
will system calls like the following:

getcwd("/extra/cvs/gnome2/yelp-head5", 4096) = 29
access("/extra/cvs/gnome2/yelp-head5/man:/usr/share/man/man1/msgcomm.1.gz",
F_OK) = -1 ENOENT (No such file or directory)

More disk access occurs in the convert_man_uri function, due
to a g_file_test() call:

stat64("/usr/share/man/man1/msgcomm.1.gz", {st_mode=S_IFREG|0644, st_size=1648,
...}) = 0

Since we are reading all the man files from the disk anyway,
there's no reason to check for relative paths, and to see if
the file exists; we know it does - therefore, I made a change to the API
for the following functions:

-static gchar *      convert_man_uri    (gchar   *uri);
+static gchar *      convert_man_uri    (gchar   *uri, gboolean trust_uri);

-yelp_doc_info_new (const gchar *uri)
+yelp_doc_info_new (const gchar *uri, gboolean trust_uri)

-yelp_doc_info_get (const gchar *uri)
+yelp_doc_info_get (const gchar *uri, gboolean trust_uri)

Basically, if we know the uri we are passing exists and is an
absolute uri, we set trust_uri = TRUE; this in turn bypasses
the call to gnome_vfs_make_uri_from_input_with_dirs() with a
call to gnome_vfs_make_uri_from_input(), which doesn't
suffer the performance issues of getting the cwd and checking
for the existence of a relative path.  The g_file_test()
in convert_man_uri is also bypassed.

In addition to this simple change, I've been working on a
cache patch, which on first run creates an index of man pages
in ~/.gnome2/yelp.d/manindex.xml.  Any subsequent running
of yelp will use this file instead of scanning the disk.  In
conjunction with the API changes above, on reboot yelp comes
up pretty quickly (knocked about 8 seconds off from my tests)

The disadvantage with the cache patch is that it doesn't
check the mtime of the directories to see if any new manpages
have been added, and recreate the index.  However, I designed
the patch with this in mind, so it shouldn't be too hard to
add this functionality.

I've attached three patches to this email.

[1] yelp-head-timing.patch
[2] yelp-trust-timing.patch
[3] yelp-mancache-timing.patch

[1] simply adds a function to print timing information on
how long process_mandir_pending takes.

[2] adds the API changes discussed above and the timing print stuff

[3] is the full blown cache, API changes, and timing print stuff

I've included the results from my tests below.  I've performed a
reboot before each test, and than ran yelp twice and recorded the
results.  The cache + API + timing patch is definitely the best
since on reboot it doesn't have to touch the disk to create the
TOC for the man pages.

[1] yelp-head-timing.patch
[2] yelp-trust-timing.patch
[3] yelp-mancache-timing.patch (no cache at ~/.gnome2/yelp.d/manindex.xml)
[4] yelp-mancache-timing.patch (cache present at ~/.gnome2/yelp.d/manindex.xml)

Actual timings after a reboot:
[1] 1st run: elapsed: sec=9, usec=117386
    2nd run: elapsed: sec=0, usec=805525

[2] 1st run: elapsed: sec=1, usec=278251
    2nd run: elapsed: sec=0, usec=652525

[3] 1st run: elapsed: sec=1, usec=497169  (cache file gets created during this run)
    2nd run: elapsed: sec=0, usec=664619  (cache file exists for this run)

[4] 1st run: elapsed: sec=0, usec=766423
    2nd run: elapsed: sec=0, usec=644318


All patches are against yelp HEAD.


Again sorry for the length.  Any comments/testing are greatly
appreciated.