GNOME Bugzilla – Bug 768473
playback is not playing smoothly after selecting .sub or .srt subtitle to the playing file
Last modified: 2016-07-06 07:10:36 UTC
Frame drops seen when there is subtitle to display, if we select some subtitle file for some other video. For ex, For some video songs and selecting some movie subtitle, whenever we have subtitle to display, then we are observing the frame drops. Following is the pipeline we are using: gst-launch-1.0 textoverlay name=overlay ! imxeglvivsink filesrc location=<i/p file> ! aiurdemux name=demux demux. ! queue ! imxvpudec ! overlay.video_sink demux. ! queue ! aacparse ! beepdec ! audioconvert ! volume volume=1 ! audioresample ! alsasink filesrc location=<i/p file> ! "application/x-subtitle" ! queue ! subparse ! overlay.text_sink It looks like pipeline is taking around 40-50% of CPU at the time of frame drop. If we dont have subtitle to display, then playback is smooth and CPU usage is also not more than 10%. We are testing on nxp's imxsabrelite dual core board. Can you please help us debug this issue?
Without subtitles, this pipeline should be zerocopy between decoder and sink. With subtitles, they are going to be overlayed in software and that's going to make a big difference performance-wise. Ideally imxeglvivsink would properly implement GstVideoOverlayComposition support, then it should still run zerocopy and do the subtitle rendering in hardware. Or glimagesink could be used instead. Overall this does not seem like a bug but deficiencies in the elements on your target hardware.
40-50% CPU is more for video playback right? If i don't like text_sink of overlay element, then everything is fine. Can you please let us know what exactly you meant by "deficiencies in the elements on your target hardware"? What can we do to fix this issue.
See above, you either need to implement GstVideoOverlayComposition (and you need GStreamer 1.6 or better 1.8 for this to work properly), or use glimagesink (which needs special integration work to be done to work with imx6, and also 1.6 or 1.8). There's nothing that can be done on our side here, it's something that has to be solved in the imx6 elements. The problem is that because of the textoverlay element, whenever there are subtitles the video frames have to be handled by the CPU and downloaded from the GPU and uploaded again. That's slow.