When you close the main window, rhythmbox should exit.  "close to tray" makes absolutely no sense in gnome-shell.  What version of rhythmbox is this?  (0.13.3, package from fedora 15, git master?)

rhythmbox 2.90.1~20110329-0ubuntu1~build1

Closing to tray makes perfect sense: I get an icon in the bottom-right and clicking rhythmbox there gives a bubble letting me play/pause next/previous and showing current playing song, withing the need to have the full window open.

What output do you get from 'gconftool-2 -R /apps/rhythmbox/plugins/status-icon'?

$ gconftool-2 -R /apps/rhythmbox/plugins/status-icon
 status-icon-mode = 3
 active = true
 mouse-wheel-mode = 0
 hidden = false
 notification-mode = 1
 window-visible = false

Does the window close properly if you set status-icon-mode to 2?

I've changed that value and restarted rhythmbox, nothing changed. when closing window, rb is still running (it should exit with mode 2, right?), and sometimes I can't close the window.

What happens if you disable the plugin?  (you also won't get notifications or anything in the message tray)

What output do you get from 'rhythmbox -D shell' when this happens?

Here is what I get, I've removed the spam about music progress

(12:48:29) [0xcfd040] [paned_size_allocate_cb] rb-shell.c:2972: paned position 213
(12:48:29) [0xcfd040] [paned_size_allocate_cb] rb-shell.c:2973: right_paned position 400
(12:48:29) [0xcfd040] [paned_size_allocate_cb] rb-shell.c:2972: paned position 213
(12:48:29) [0xcfd040] [paned_size_allocate_cb] rb-shell.c:2973: right_paned position 400
(12:48:33) [0xcfd040] [paned_size_allocate_cb] rb-shell.c:2972: paned position 213
(12:48:33) [0xcfd040] [paned_size_allocate_cb] rb-shell.c:2973: right_paned position 400
(12:48:33) [0xcfd040] [sidebar_paned_size_allocate_cb] rb-shell.c:2984: sidebar paned position 300
(12:48:33) [0xcfd040] [paned_size_allocate_cb] rb-shell.c:2972: paned position 213
(12:48:33) [0xcfd040] [paned_size_allocate_cb] rb-shell.c:2973: right_paned position 400


Note that when not playing music, closing the main window seems to exit rhythmbox, in that case my multimedia key play/pause does not work anymore.

Not sure where that package comes from, but I suspect it includes this patch:
http://bazaar.launchpad.net/~ubuntu-desktop/rhythmbox/ubuntu/view/head:/debian/patches/22_hide_on_quit.patch

which does more or less what you're talking about.

please reopen if it turns out that patch isn't responsible.

For your information, I have reported it in Ubuntu's bug system: https://bugs.launchpad.net/ubuntu/+source/rhythmbox/+bug/780747