GNOME Bugzilla – Bug 703330
Video losing sync with audio when toggling play/pause (RPI)
Last modified: 2013-07-03 07:19:34 UTC
Created attachment 248069 [details] modified gst-omx/examples/testegl.c with Makefile Hi, I was using testegl.c from gst-omx/examples and I've noticed that after a couple of play/pause toggles, sound is played late (or video early). I'm using it on Raspberry PI. Then, I started to investigate a bit, and I've found that it is not strictly binded to testegl.c as eglglessink from gst-plugins-bad have same issue. So, it looks like gst-omx problem (or at least, that looks like one to me). I've attached modified gst-omx/examples/testegl.c file with Makefile, so you can compile it as standalone program and confirm this bug. In that file, comparing to original git testegl.c, I've simplified rendering (removed cubes) and changed player pipeline to gain more control over it (using playbin I could not get sound playing at all). To reproduce bug, you can run program and toggle play->pause->play->pause several times and then you will see how it lost sync. You need to use h264 video and AAC audio file (I've found this one good [1] because it is easy to check delay on start). As I said earlier, I could reproduce this bug with eglglessink too, so that could give good starting point. If anyone need more information, please let me know. Thanks -- [1] http://download2.dvdloc8.com/trailers/divxdigest/simpsons_movie_1080p_trailer.zip
Created attachment 248070 [details] Simple example using eglglessink I've added another example, this time sink is eglglessink and it is much simpler - but with same bug when toggling play-pause.
Created attachment 248071 [details] Simple example using eglglessink and playbin This example is showing that alsasink is not working with playbin (at least not working for me).
Created attachment 248083 [details] [review] Patch for enabling audio in testegl example
Patch is good for enabling audio for playbin, but bug remains for me.
That would mean that either the alsasink or the ALSA driver is broken, or that OpenMAX does not pass through our timestamps the way it should.
That is what I think too. Can anyone confirm this bug?
Does it also happen if you use avdec_h264 for the video decoding for example? I can't test here right now
I'm having hard time finding or transcoding video that would play with avdec_h264 on RPI. I've tried on PC and it works normally, but still we can't know if it is about alsa or omx on RPI.
I've managed to transcode video to super-small size and bitrate and play it with avdec_h264 decoder. Still, after a couple play-pause toggles, audio goes out of sync. So, it looks like this is not a bug in gst-omx, but in Alsa. It would be great if somebody else could try this to confirm that alsasink or alsa drivers have bug, or this is just my local issue. Thanks
Is it possible that problem is with eglglessink here?
After I tried literally every possible combination, I think that this bug is related to alsasink. Should I close this one and reopen new one there, or this can be transfered to proper place?
So you tried with a different video sink than eglglessink (ximagesink) and avdec_h264 at the same time too, and it was still a problem? In that case we should just transfer this bug to gst-plugins-base against alsa. And it would help if you could get gstreamer debug logs with all the alsa/audio specific categories, maybe there's something interesting in there. Something like GST_DEBUG=alsa*:6,audio*:6
No, I haven't tried ximagesink yet, but I will do that later today. Anyways, avdec_h264 and omxh264dec give same results. I've tried to debug alsa but there is nothing interesting, it looks like everything is fine. But, I've done some more testing and by changing "buffer-time" property, it is possible to reduce or even eliminate losing synchronization. But, there rises another problem - crackling sound. So, basically we have 2 options for pause-play toggling now: to get sound delays or get sound crackling. I've oppened issue here [1] and got explanation why there is crackling when "buffer-time" is set too low. So, I would suggest that we wait a bit until I try xvimagesink and then we will have more information about source of the problem. -- [1] https://github.com/raspberrypi/linux/issues/320
I've finally tried ximagesink. With avdec_h264 and omxh264dec there is same result as with eglglessink. So, now we can say that this is definitely not a video-related issue here, as every combination of decoders and sinks is giving same result. From this point of view, this looks like alsasink loosing of synchronization. I hope that it isn't something more complicated. So, I suggest moving this bug to gst-plugins-base, against alsasink. I will attach log with GST_DEBUG:audio*:6,alsa*:6 when using avdec_h264 and ximagesink. I don't see anything strange there, but I'm far from understanding it.
Created attachment 248266 [details] Log of GST_DEBUG=audio*:6,alsa*:6 with avdec_h264 and ximagesink
Could you check with other, non-GStreamer software if it shows this behaviour too? It could also be the ALSA driver.
Good point Sebastian. I've found mplayer and tried with: mplayer -vo x11 -ao alsa video.mp4 and I get same bug there: audio is late after a couple play-pause toggles. So, I guess we can close this bug and continue to investigate on Raspberry pi to find source of the problem. I have bug opened here in case you are interested: https://github.com/raspberrypi/linux/issues/320