GNOME Bugzilla – Bug 797262
qtdemux: Corrupted CTTS makes file unplayable but can be worked around
Last modified: 2018-11-01 14:04:49 UTC
We're using the MSVS version of gstreamer in Win10 to develop a video player. Everything works ok except that seek operation can't be successfully used on some MP4(H.264+MP3) files(it can be tested in such like gstreamer test program: https://gstreamer.freedesktop.org/documentation/tutorials/basic/playback-speed.html). We used VLC and some other video players to play with these MP4 files and they all can seek the file. This confused us a lot(I mean if no other player can play with these files, we'll not care this issue). So we tried with some old version of gstreamer to test with these video files. And after a lot test, we find: - gstreamer 1.4.5 or earlier version can seek these files smoothly; - gstreamer 1.5.x or 1.6.0 version can seek these files, but not smoothly(some additional wait time exists after we change the seek position); - gstreamer 1.6.x or later version can't seek these files. We're sorry to say we have no idea on this issue because we're not familiar with the source code of gstreamer library. So we hope someone here can help us. Could you please check with this issue to see if there's something wrong with gstreamer lib or maybe these movies files? You can download some sample video files here: https://www.dropbox.com/s/baho67c9zjvugv6/record-20180907_100435-00064.mp4?dl=0 https://www.dropbox.com/s/iv5grnorw2sv48s/record-20180907_100435-00065.mp4?dl=0 Thank you in advance!
I can confirm that here. Seeking just stops playback, and seeking to later positions (after 11 minutes of the first one) in the file is playing back the audio but the video still is stuck.
git bisect gives us: f4ca3c255adc3707daaad9cac24c95483669a04c is the first bad commit commit f4ca3c255adc3707daaad9cac24c95483669a04c Author: Jan Schmidt <jan@centricular.com> Date: Thu Dec 4 17:25:55 2014 +1100 qtdemux: More fixes for reverse playback When seeking or finding the previous keyframe, do comparisons against targets and segments using composition time to correctly decide which sample times match.
(In reply to Sebastian Dröge (slomo) from comment #1) > I can confirm that here. Seeking just stops playback, and seeking to later > positions (after 11 minutes of the first one) in the file is playing back > the audio but the video still is stuck. Really? I can't reproduce that - seeking seems to work fine with git master.
Exciting! I was using 1.14, and there it does not work. Indeed works with git master. So something has fixed it in the meantime.
So does it mean in the next relase(suppose in 1.14.5 release) the bug will be solved?
This bug has been set as blocker for the next stable release, we know a fix exist, we need to find which patches need backport from master to 1.14 branch. Feel free to help if you have time.
There were quite many changes to qtdemux in git master, so the fix might be tricky to backport. In any case, someone first has to "git bisect" to find the actual fix :)
(In reply to Nicolas Dufresne (ndufresne) from comment #6) > This bug has been set as blocker for the next stable release, we know a fix > exist, we need to find which patches need backport from master to 1.14 > branch. Feel free to help if you have time. Hi Nicolas, I don't know if I can help. Does it mean we should check each commit from 1.14.3 to the latest master commit? And before I can do some test check, I need to setup the compile evironment for gstreamer to compile each commit one by one? Sorry, I don't have much experiences on compiling gstreamer libraries. I think I need some instructions before I can do some help.
(In reply to Ricky Gong from comment #8) > (In reply to Nicolas Dufresne (ndufresne) from comment #6) > > This bug has been set as blocker for the next stable release, we know a fix > > exist, we need to find which patches need backport from master to 1.14 > > branch. Feel free to help if you have time. > > Hi Nicolas, > > I don't know if I can help. Does it mean we should check each commit from > 1.14.3 to the latest master commit? And before I can do some test check, I > need to setup the compile evironment for gstreamer to compile each commit > one by one? > > Sorry, I don't have much experiences on compiling gstreamer libraries. I > think I need some instructions before I can do some help. https://cgit.freedesktop.org/gstreamer/gst-build/ is a pretty easy way to set up an uninstalled development environment for GStreamer, just follow the README :)
(In reply to Mathieu Duponchelle from comment #9) > (In reply to Ricky Gong from comment #8) > > (In reply to Nicolas Dufresne (ndufresne) from comment #6) > > > This bug has been set as blocker for the next stable release, we know a fix > > > exist, we need to find which patches need backport from master to 1.14 > > > branch. Feel free to help if you have time. > > > > Hi Nicolas, > > > > I don't know if I can help. Does it mean we should check each commit from > > 1.14.3 to the latest master commit? And before I can do some test check, I > > need to setup the compile evironment for gstreamer to compile each commit > > one by one? > > > > Sorry, I don't have much experiences on compiling gstreamer libraries. I > > think I need some instructions before I can do some help. > > https://cgit.freedesktop.org/gstreamer/gst-build/ is a pretty easy way to > set up an uninstalled development environment for GStreamer, just follow the > README :) Hi Nicolas, Sorry for a late reply. I'm using win10 OS with mingw32 and it seems it can't easily establish a development environment for GStreamer. It failed in "meson build" step with the following error: Sanity check compile stderr: C:\MinGW\bin/ld.exe: crt2.o: No such file: No such file or directory collect2: ld returned 1 exit status So finally I followed the following web page and successfully built gstreamer 1.14.3 from source code: https://gstreamer.freedesktop.org/documentation/installing/building-from-source-using-cerbero.html Gstreamer code is fetched from here: git ls-remote git://anongit.freedesktop.org/gstreamer/cerbero So can I use this git repository to go to next steps? And what's the next steps should I do to do some help?
(In reply to Ricky Gong from comment #10) > Sorry for a late reply. I'm using win10 OS with mingw32 and it seems it > can't easily establish a development environment for GStreamer. It failed in > "meson build" step with the following error: > Sanity check compile stderr: > C:\MinGW\bin/ld.exe: crt2.o: No such file: No such file or directory > collect2: ld returned 1 exit status Annoying, note that on Windows you should be able to build with gst-build and MSVC if that helps :)
(In reply to Mathieu Duponchelle from comment #11) > (In reply to Ricky Gong from comment #10) > > Sorry for a late reply. I'm using win10 OS with mingw32 and it seems it > > can't easily establish a development environment for GStreamer. It failed in > > "meson build" step with the following error: > > Sanity check compile stderr: > > C:\MinGW\bin/ld.exe: crt2.o: No such file: No such file or directory > > collect2: ld returned 1 exit status > > Annoying, note that on Windows you should be able to build with gst-build > and MSVC if that helps :) Yes, you're right. I used MSVC console enviroment to build with gst-build and things changed. But unfortunately an error still blocks the building in the progress, which shows "ERROR: Multiple producers for Ninja target "c:\python37-32\python.exe". Please rename your targets.". I attached the meson-log.txt full log at: https://www.dropbox.com/s/xur24fi8n4c565a/meson-log.txt?dl=0.
(In reply to Ricky Gong from comment #12) > Yes, you're right. I used MSVC console enviroment to build with gst-build > and things changed. But unfortunately an error still blocks the building in > the progress, which shows "ERROR: Multiple producers for Ninja target > "c:\python37-32\python.exe". Please rename your targets.". I attached the > meson-log.txt full log at: > https://www.dropbox.com/s/xur24fi8n4c565a/meson-log.txt?dl=0. You can fix this by moving your gst-build project under c:\, not d:\
(In reply to Seungha Yang from comment #13) > (In reply to Ricky Gong from comment #12) > > Yes, you're right. I used MSVC console enviroment to build with gst-build > > and things changed. But unfortunately an error still blocks the building in > > the progress, which shows "ERROR: Multiple producers for Ninja target > > "c:\python37-32\python.exe". Please rename your targets.". I attached the > > meson-log.txt full log at: > > https://www.dropbox.com/s/xur24fi8n4c565a/meson-log.txt?dl=0. > > You can fix this by moving your gst-build project under c:\, not d:\ Thanks! "ninja -C build/" finished with some code page error problems in some files of glib compilation that was solved by storing these files into "Encoding in UTF-8" using Notepad++(Originally they are stored in "Encoding in UTF-8 without BOM" that caused compilation error report). Now everything seems OK except that I'm confused with how to use the compiled output files. I mean the target include files, binary files and folders are seperated from each other. It's quite different from the installation packge. How can I compile and test with a program using these output packages? And then I can start to help.
The file has empty edit list in audio track, and weird segment time might cause issue. 374 0:00:27.280379000 11328 000001C1B1238480 LOG qtdemux qtdemux.c:1263:gst_qtdemux_find_segment:<qtdemux0:video_0> finding segment for 0:00:00.000000000 375 0:00:27.280406000 11328 000001C1B1238480 LOG qtdemux qtdemux.c:1271:gst_qtdemux_find_segment:<qtdemux0:video_0> looking at segment 0:00:00.000000000-0:14:59.498666666 376 0:00:27.280429000 11328 000001C1B1238480 LOG qtdemux qtdemux.c:5036:gst_qtdemux_activate_segment:<qtdemux0:video_0> activate segment 0, offset 0:00:00.000000000 377 0:00:27.280459000 11328 000001C1B1238480 LOG qtdemux qtdemux.c:4886:gst_qtdemux_stream_segment_get_boundaries:<qtdemux0:video_0> seg_time 0:00:00.000000000 378 0:00:27.280487000 11328 000001C1B1238480 DEBUG qtdemux qtdemux.c:4976:gst_qtdemux_stream_update_segment:<qtdemux0:video_0> new segment 0 from 0:00:00.000000000 to 0:14:59.498666666 , time 0:00:00.000000000 379 0:00:27.280527000 11328 000001C1B1238480 DEBUG qtdemux qtdemux.c:4997:gst_qtdemux_stream_update_segment:<qtdemux0:video_0> New segment: time segment start=55:15:31.533833333, offse t=0:00:00.000000000, stop=55:30:31.032499999, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 55:15:31.533833333, duration 99:99:99.9999 99999 ... 454 0:00:27.284263000 11328 000001C1B1238480 LOG qtdemux qtdemux.c:5901:gst_qtdemux_decorate_and_push_buffer:<qtdemux0> Pushing buffer with dts 0:00:00.009333333, pts 55:15:31.526500 000, duration 0:00:00.016666667 on pad video_0 ... 469 0:00:27.284752000 11328 000001C1B1238480 LOG qtdemux qtdemux.c:5901:gst_qtdemux_decorate_and_push_buffer:<qtdemux0> Pushing buffer with dts 0:00:00.026000000, pts 0:14:59.4820000 00, duration 0:00:00.016666666 on pad video_0
(In reply to Ricky Gong from comment #14) > Now everything seems OK except that I'm confused with how to use the > compiled output files. I mean the target include files, binary files and > folders are seperated from each other. It's quite different from the > installation packge. How can I compile and test with a program using these > output packages? And then I can start to help. Once gst-build has successfully compiled everything, you can enter an uninstalled environment with `ninja -C build uninstalled` . In that environment, a set of influential environment variables is set (such as PATH, PKG_CONFIG_PATH and more), which allow you to build programs against the various build products. The easiest way to do so is to write a meson build definition for your program, and build it in the environment. Note that once in the environment, you should already see that the built gstreamer tools are in your PATH (where gst-inspect-1.0), and inspecting plugins (eg gst-inspect-1.0 identity) should show you a Filename located in your build directory, for example on Linux: gst-inspect-1.0 identity [...] Filename /home/meh/devel/gst-build/build/subprojects/gstreamer/plugins/elements/libgstcoreelements.so [...]
(In reply to Mathieu Duponchelle from comment #16) > (In reply to Ricky Gong from comment #14) > > Now everything seems OK except that I'm confused with how to use the > > compiled output files. I mean the target include files, binary files and > > folders are seperated from each other. It's quite different from the > > installation packge. How can I compile and test with a program using these > > output packages? And then I can start to help. > > Once gst-build has successfully compiled everything, you can enter an > uninstalled environment with `ninja -C build uninstalled` . In that > environment, a set of influential environment variables is set (such as > PATH, PKG_CONFIG_PATH and more), which allow you to build programs against > the various build products. The easiest way to do so is to write a meson > build definition for your program, and build it in the environment. > > Note that once in the environment, you should already see that the built > gstreamer tools are in your PATH (where gst-inspect-1.0), and inspecting > plugins (eg gst-inspect-1.0 identity) should show you a Filename located in > your build directory, for example on Linux: > > gst-inspect-1.0 identity > [...] > Filename > /home/meh/devel/gst-build/build/subprojects/gstreamer/plugins/elements/ > libgstcoreelements.so > [...] Thank you for your reply! I've tried with `ninja -C build uninstalled` and it returned "Could not execute command "c:/gst-build/gst-uninstalled.py": [WinError 193] %1" such like error(Maybe it's because I'm using Windows OS). So I tried to execute the command - "c:/gst-build/gst-uninstalled.py" directly. It returns without error and I can use "gst-inspect-1.0 identity" to inspect elements. It returns: "Filename C:\gst-build\build\subprojects\gstreamer\plugins\elements\gstcoreelements.dll" But now when I played with a MP4 video file using gst-play-1.0, it responsed me "WARNING No decoder available for type 'video/x-h264'", "no suitable plugins found: Missing decoder: MPEG-4 AAC" or "no suitable plugins found: Missing decoder: MPEG-1 Layer 3 (MP3)". I suppose it's because there's no related docoder elements there. Does it mean I missed some elements in compilation or just the uninstalled environment with `ninja -C build uninstalled` not fully performed?
I've check with the build folder and I think it's because no gst-libav library compiled there. So gst-play-1.0 can't find suitable codec plugins to play with a H.264+AAC or MP3 video file. Is there a way I can do with to compile gst-libav library into the build?
OK. After several ten hours of check on the master branch, I finnally on the way to do some help work. I found that 850e678813f11b439994d434f2800bfbbb140864 commit (the second commit just after 1.14.0 tag) in gst-plugins-good solved the seeking problem: Author: Sebastian Dröge <sebastian@centricular.com> 2018-03-05 18:48:15 Committer: Sebastian Dröge <sebastian@centricular.com> 2018-03-20 18:08:28 Tags: 6 Parent: ed2ccb1a60f6756b5c59df40460dc2dd0b7f7f37 (rtp: Fix compilation with non-C99 compilers) Child: 589019d8f5072d4470ea2ed76cfffa75f45e9426 (udpsrc: optimize GstUdpSrc object for cache performance) Branch: remotes/origin/master Follows: 7 Precedes: 5 qtdemux: Fix seeking on streams with frame reordering The samples table is sorted by DTS, not PTS. As such we can only get the correct result when using a binary search on it, if we search for the DTS. Also if we only ever search for the frame, where the following frame is the first one with a PTS after the search position, we will generally stop searching too early if frames are reordered. In forwards playback this is not really a problem (after the decoder reordered the frames, clipping is happening), in reverse playback it means that we can output one or more frames too few as we stop too early and the decoder would never receive it. https://bugzilla.gnome.org/show_bug.cgi?id=782118
But unfortunately I want to say there's still some problems on playing some files. The following two files also can be played with VLC and some other media players, but they can't be played in gstreamer even though I've used the latest master branch: https://www.dropbox.com/s/1niy4ztsenihbkn/record-20181003_180325-00446.mp4?dl=0 https://www.dropbox.com/s/8hoaddlk7bhe4c5/record-20180925_115055-00415.mp4?dl=0 They can't be play with gstreamer at the start of the video and seek to other positions also can't be done.
I can confirm that those don't work in GIT master. Did they work in 1.5 or at some other known time (when?)?
No, I've tried 1.4.5, 1.8.3, 1.14.3 release version and also latest master version with these video files. None of them can play the files.
(In reply to Sebastian Dröge (slomo) from comment #21) > I can confirm that those don't work in GIT master. Did they work in 1.5 or > at some other known time (when?)? Sorry to bother, but are there any solutions or plans that we can solve the problem to play with these two files?
Someone would have to find the time to look into this in more detail. It should obviously be fixed at some point, if the files play in other software they should also be made to play in GStreamer.
(In reply to Sebastian Dröge (slomo) from comment #24) > Someone would have to find the time to look into this in more detail. It > should obviously be fixed at some point, if the files play in other software > they should also be made to play in GStreamer. The ctts atoms of newly reported files seems to be corrupted. But, we can play them with ffplay because ffmpeg just ignores the corrupted ctts box. Anyway, I can play the files with some (might be ugly) additional sanity check of ctts atom.
Created attachment 374125 [details] [review] qtdemux: Ignore corrupted CTTS box If ctts (CompositionOffsetBox) has larger sample_offset (offset between PTS and DTS) than (2 * duration) of the stream, assume the ctts box to be corrupted and ignore the box.
Comment on attachment 374125 [details] [review] qtdemux: Ignore corrupted CTTS box Why >2x? Even >1x the stream duration would seem wrong, at most it could in theory be the stream duration or not?
(In reply to Sebastian Dröge (slomo) from comment #27) > Comment on attachment 374125 [details] [review] [review] > qtdemux: Ignore corrupted CTTS box > > Why >2x? Even >1x the stream duration would seem wrong, at most it could in > theory be the stream duration or not? Actually, I could not find any reference (from spec.) that defines the allowed sample_offset bounds exactly in this case. So, 2x is a just "magic number" to avoid some possible edge cases, such as one video sample only case, etc. Note that ffmpeg is avoiding this issue as follows. "If the sample_offset is sufficiently large and this entry is not the last... then ignore this!". I don't know where "-(1<<28)" came from... if (FFNABS(duration) < -(1<<28) && i+2<entries) { av_log(c->fc, AV_LOG_WARNING, "CTTS invalid\n"); av_freep(&sc->ctts_data); sc->ctts_count = 0; return 0; }
Attachment 374125 [details] pushed as 5d54203 - qtdemux: Ignore corrupted CTTS box