Recently I noticed, that I (subconsciously) avoid clicking on windows when
clicking on another workspace in the workspace switcher, even if that means
that I have to click on a line of just one pixel height. 
The reason for this is, that it irritates me when I acciddentally focus a
window that was in the background, when I just want to switch the
workspace. I rarely use the taskbar, so it kind of throws me off, when for
example some windows are suddenly covered by a larger window that was
behind them. It also confuses me when I switch to another workspace and the
situation is different from what I expected.
Maybe a less irritating solution would be, that windows are only focused
(after clicking them in the workspace switcher) if they are located on the
already active workspace. Generally I can't quite see the usefullness of
using the workspace switcher to focus windows though.