Looking at the insane library sizes of broadband users, I have a rather
small one (~1300 files). Nevertheless, after storing it once, rhythmbox
takes extremely long to load it again at the next start.

The splash screen displays and the first icon becomes dark, then the
program seemingly freezes with no disk activity and full CPU utilisation.

It is killable using a normal SIGTERM, however, even if it doesn't actually
freeze or crash, a loading time of more than two minutes - that's how long
I waited before killing it - for as little a library as mine on a 700 MHz
Pentium with plenty of RAM is inacceptable. Loading a smaller, 150-file
subfolder of my collection "only" takes four seconds.