When rhythmbox changes track, its internal volume level changes, rapidly heading towards zero. I start with a comfortable volume level in rhythmbox of 10%. After one track, it changes to 1%, and then after another track it's at 0%. It doesn't happen absolutely every time, but pretty close. I have the gconf key /apps/rhythmbox/use_replaygain set to false, which may make a difference (rhythmbox was hanging frequently when I had it on)

Its internal volume level scale seems weird too: changing it also affects the application volume in pulseaudio, but not in any way I would expect:

Adjusting the volume in Rhythmbox goes as follows:
Rhythmbox volumes (percentages)
muted, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98, Full volume
corresponding to Pulseaudio volumes (percentages)
0, 62, 69, 73, 76, 78, 80, 81, 82, 83, 84, 85, 86, 87, 88, 88, 89, 90, 90, 91, 91, 92, 92, 93, 93, 93, 94, 94, 94, 95, 95, 95, 96, 96, 96, 97, 97, 97, 97, 98, 98, 98, 98, 99, 99, 99, 99, 99, 100, 100, 100
and alsamixer Master volume levels (percentages)
0, 29, 42, 48, 55, 58, 61, 65, 68, 71, 71, 74, 74, 77, 77, 81, 81, 81, 84, 84, 84, 84, 87, 87, 87, 87, 87, 90, 90, 90, 90, 90, 94, 94, 94, 94, 94, 94, 97, 97, 97, 97, 97, 97, 97, 100, 100, 100, 100, 100, 100, 100, 100.

Adjusting the application volume in pavucontrol gives the following:
50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100
corresponds to Rythmbox internal volume
1...1
and alsamixer master volume
3, 13, 26, 32, 42, 52, 61, 71, 81, 90, 100