I have spotted similar behaviour as in Bug 556963 in latest gst-ffmpeg release.

I am playing some raw mpeg transport stream (recorded from dvb-t transmission) and after few minutes video becomes jerky (sound is still playing good). Also I have been trying to use ffdec_h264 to decode dvb-t live stream. I have spotted that when I am playing such stream and do something that lags PC, then after a while I begin too see sideshow (one frame every second). The video never catches up again. I have tried older gst-ffmpeg releases with no success.

For playing saved raw mpegts stream I have been using this pipeline:

gst-launch-0.10 -m filesrc location=17.01.1100.ts ! mpegtsdemux name=demux0
demux0. ! queue max-size-bytes=0 max-size-buffers=0 max-sizeime=0 ! ffdec_h264
! xvimagesink demux0. ! queue max-size-bytes=0 max-size-buffers=0
max-size-time=0 ! flump3dec ! pulsesink

For playing dvb-t stream I have been using this pipeline:

gst-launch-0.10 -m dvbsrc pids=201:202:204 adapter=0 ! mpegtsdemux name=demux0
demux0. ! queue max-size-bytes=0 max-size-buffers=0 max-size-time=0 !
ffdec_h264 ! xvimagesink demux0. ! queue max-size-bytes=0
max-size-buffers=0 max-size-time=0 ! flump3dec ! pulsesink

Please look at Bug 556963 for more details. Also I can provide You with this raw mpeg ts if You want, but few minutes of this media would be very big since 1 sec is about 3MB (I can do some pid filtering in that case).