I experimented a very annoying delay when doing a play-pause-play command sequence when playing from rtsp (rtsp-server). 
I see the server immediately restart sending packets but the client start showing frames after 5-10 sec.
I verified also that  if I comment in gstrtspjitterbuffer.c in gst_rtp_jitter_buffer_clear_pt_map these 2 instructions:
  rtp_jitter_buffer_reset_skew (priv->jbuf);
  priv->next_seqnum = -1;
the client restart playing is instantaneously.
So, how to fix this issue? 
I find out that the gst_rtp_jitter_buffer_clear_pt_map call is caused by gst_rtspsrc_configure_caps (src, segment); called in gst_rtspsrc_play
Should gst_rtspsrc avoid to call gst_rtspsrc_configure_caps when not strictly needed?
Or should jitterbuffer check in gst_rtp_jitter_buffer_clear_pt_map if reset all? (I tried this solution, apparently seems to work, but it is not very smart...)
Or could you suggest a better way?
Thank you very much