GNOME Bugzilla – Bug 606307
mastroska muxer bad in case of 1fps
Last modified: 2010-01-11 20:57:32 UTC
we try to put a h264 stream into a matroska stream with gstreamer. the strange thing everything seems to working if it's at least 2fps, but if it's only 1fps then the resulting file is corrupted. we try many ways: - generate the raw h264 stream file directly from capture card with (1 or 2 fps): gst-launch lannersrc channel=0 ! capsfilter caps="video/x-h264,framerate=2/1" ! filesink location=2fps_raw.h264 these are working (but of course seek is not possible which is normal). - put the h264 stream into a matroska file (1 or 2 fps): gst-launch filesrc location=1fps_raw.h264 ! h264parse ! matroskamux ! filesink location=1fps_file2matroska.mkv it can be played but not able to seek. why? - generate matroska file directly from capture card (1 or 2 fps): gst-launch lannersrc channel=0 ! capsfilter caps="video/x-h264,framerate=1/1" ! matroskamux ! filesink location=1fps_lanner2matroska.mkv in case of 1 fps it's not playable with gstreamer (mplayer play it but with errors) in case of 2 fps it's playable but not seekable. - but if we use mkvtoolnix anything is perfect, so it seems to be that mkvmerge can do it better the gstreamer's matroska. in this case both 1 and 2 fps works, can be played with gstreamer and be able to seek. so is it a bug in gstreamer's matroska or just we don't know something? mkvmerge -o 1fps.mkv --forced-track 0:no --default-duration 0:1fps -d 0 -A -S -T --no-global-tags --no-chapters /tmp/1fps.h264 --track-order 0:0 the usual error messages from gstreamer: - number of reference frames exceeds max (probably corrupt input), discarding one - non-existing PPS referenced - sps_id out of range - Missing reference picture - decode_slice_header error - mmco: unref short failure
I tested some stuff, here are the results: 1) videotestsrc ! x264enc ! matroskamux ! filesink This works 2) videotestsrc ! "video/x-raw-yuv, framerate=(fraction)*whatever*" ! x264enc ! matroskamux ! filesink This works and seeks, even with 1 and 2fps 3) videotestsrc ! "video/x-raw-yuv, framerate=(fraction)*whatever*" ! x264enc ! filesink and then filesrc ! h264parse ! matroskamux ! filesink Didn't work for any framerate. The matroskademux threw an EBML error nothing happens Can you reproduce the problem replacing lannersrc with videotestsrc somehow?
Could you test if it works if you use a capsfilter right after the filesrc with the exact same caps that the h264 had when it was writen to the file?
I had a look at the files that were mentioned in the [gst-devel] mail. The problematic 1fps_* files have either no or very few frames, so there is not much to play (or seek). I have no idea why, since using the pipelines mentioned above on the raw h264 file produces mkv files that have a lot more stuff in it. At the core of the matter, you might possibly not know something here. That is, h264 data can be passed around in (at least) 2 ways, e.g. the "bytestream" format (as defined in the H264 spec) or the "AVC sample format" (as defined in some MP4 file spec). The former is typically used for 'raw h264' data (e.g. on its way to be rtp payloaded etc), the latter must be used for storage in a mp4/mov container (and by extension say also in other containers). Bytestream format stashed into a container can lead to all sorts of surprises for player/decoder (and corresponding messages). The raw file (and possibly also lannersrc output) provide bytestream formatted data. Until recently, h264parse only provided bytestream formatted output, which should as such explain that it could not work that way. And no (matroska)muxer is not buggy here. It works with mkvmerge as it may be aware of all this and perform the necessary transformations. Using a (properly configured) recent h264parse, such as in following pipeline, the resulting file runs/seek fine in mplayer/gstreamer: gst-launch filesrc location=1fps_raw.h264 ! h264parse access-unit=true output-format=0 ! matroskamux ! filesink location=1fps_file2matroska.mkv BTW: The problem in 3) given above is due to x264enc producing avc sample format output by default (for convenient muxing). However, when dumped into a file, there is no real decent way for a parser to reconstruct frames, so no decent mkv files results from it. In summary, this is not really a bug (least of all in matroskamux), at best a distant relative/duplicate of bug #604925 (as in: some more caps fields might help with automagic negotiation).
Hi, i'm working with Levente on this project. We don't have already the solution of our problem, it can be either in the videosrc h264 stream or in one of the element in the pipeline. > Could you test if it works if you use a capsfilter right after the filesrc with > the exact same caps that the h264 had when it was writen to the file? I used this pipeline, but the ffmpeg errors still coming while i'm trying to seek: gst-launch videotestsrc ! capsfilter caps="video/x-raw-yuv,framerate=1/1" ! x264enc ! matroskamux ! filesink location=vsrc_1fps.mkv Seeking with totem: ERROR ffmpeg :0:: number of reference frames exceeds max (probably corrupt input), discarding one The same happend when I set the x264enc element's byte-stream property to true or false, so it looks like for me the byte-stream is not our problem (btw very thanks Mark for the additional infos). The pipeline was: gst-launch videotestsrc ! capsfilter caps="video/x-raw-yuv,framerate=1/1" ! x264enc byte-stream=false ! matroskamux ! filesink location=vsrc_1fps.mkv Another test case: disabling the B frames at convert, there isn't any ERROR while seeking the video: ... ! x264enc key-int-max=1 ! ... Any ideas? I'm using gstreamer-plugins-bad-0.10.17-2.fc12.i686 but the h264parse element only has two properties: name and split-packetized. From where can I get a recent version?
(In reply to comment #4) > Hi, i'm working with Levente on this project. We don't have already the > solution of our problem, it can be either in the videosrc h264 stream or in one > of the element in the pipeline. > > > Could you test if it works if you use a capsfilter right after the filesrc with > > the exact same caps that the h264 had when it was writen to the file? > > I used this pipeline, but the ffmpeg errors still coming while i'm trying to > seek: > gst-launch videotestsrc ! capsfilter caps="video/x-raw-yuv,framerate=1/1" ! > x264enc ! matroskamux ! filesink location=vsrc_1fps.mkv > > Seeking with totem: > > ERROR ffmpeg :0:: number of reference frames exceeds max (probably corrupt > input), discarding one I don't see how the above pipeline can lead to a valid file. If you terminate it with Ctrl-C, the muxer will be unable to finalize the file (e.g. write an index). In particular, the resulting file should not be seekable at all. If some player were to make some (random) attempt to do so, it would probably not properly seek to a keyframe, and (decoding) problems could arise that way [but I do not believe mplayer or totem would make such attempts]. On the other hand, if I use the following pipeline, the result plays and seeks fine: gst-launch videotestsrc num-buffers=60 ! video/x-raw-yuv,framerate=1/1 ! x264enc ! matroskamux ! filesink location=vsrc_1fps.mkv [alternatively, you might use the -e option to gst-launch] > > The same happend when I set the x264enc element's byte-stream property to true > or false, so it looks like for me the byte-stream is not our problem (btw very > thanks Mark for the additional infos). The pipeline was: > > gst-launch videotestsrc ! capsfilter caps="video/x-raw-yuv,framerate=1/1" ! > x264enc byte-stream=false ! matroskamux ! filesink location=vsrc_1fps.mkv > > Another test case: disabling the B frames at convert, there isn't any ERROR > while seeking the video: > > ... ! x264enc key-int-max=1 ! ... Default x264enc settings do not lead to B-frames anyway. key-int-max=1 should actually force every frame to be an IDR-frame (= sort of super I-frame). > > Any ideas? There could be problems in a particular version of x264 encoder library, or similarly in the ffmpeg decoder, or a corrupt index due to reasons explained above. Basically, I have not been able to reproduce anything so far (provided files are constructed properly). > > I'm using gstreamer-plugins-bad-0.10.17-2.fc12.i686 but the h264parse element > only has two properties: name and split-packetized. From where can I get a > recent version? The most recent one is available in git (source), see e.g. http://gstreamer.freedesktop.org/dev/ for info on obtaining it.
> I don't see how the above pipeline can lead to a valid file. If you terminate > it with Ctrl-C, the muxer will be unable to finalize the file (e.g. write an > index). In particular, the resulting file should not be seekable at all. Maybe muxers should post a warning message when they're shut down without having finalised the file properly? (Might even want to add a new error/warning code for that)
I terminate gst-launch with ctrl-c but after than i use this pipeline to close the file: gst-launch filesrc location=file.mkv ! matroskademux ! matroskamux ! filesink location=file_closed.mkv As far as i know this is a good way to closing the output file... Maybe not? (Your example works for me without any errors, too) I made some test with the num-buffers property to avoid it, but the result generates "non-existing PPS referenced" errors, and with 1 fps the results either can't be playable with gstreamer. (Btw what means exactly this message? i can't find a proper description about that.) Pipeline: gst-launch lannersrc num-buffers=100 ! capsfilter caps="video/x-h264,framerate=10/1" ! matroskamux ! filesink location=10fps_lanner2matroska.mkv Then i tried reencoding the stream of our capture card, and everything was fine and seekable: gst-launch lannersrc num-buffers=20 ! capsfilter caps="video/x-h264,framerate=1/1" ! ffdec_h264 ! x264enc byte-stream=false ! matroskamux ! filesink location=10fps_reencoded.mkv Maybe there's a problem with the h264 stream of the capture card? Tim it would be great a warning message like that.
(In reply to comment #6) > > I don't see how the above pipeline can lead to a valid file. If you terminate > > it with Ctrl-C, the muxer will be unable to finalize the file (e.g. write an > > index). In particular, the resulting file should not be seekable at all. > > Maybe muxers should post a warning message when they're shut down without > having finalised the file properly? (Might even want to add a new error/warning > code for that) Good idea.
does anybody know any usable tool for linux which are able to examine the frame level info in a video file? it'd be very useful for this kind of bug hunting...
(In reply to comment #7) > I terminate gst-launch with ctrl-c but after than i use this pipeline to close > the file: > > gst-launch filesrc location=file.mkv ! matroskademux ! matroskamux ! filesink > location=file_closed.mkv > > As far as i know this is a good way to closing the output file... Maybe not? > (Your example works for me without any errors, too) Unfortunately, it is not. Matroska version 1 files (which matroskamux produces by default, see version property) actually do not record whether or not a frame is a keyframe. So, the remuxing pipeline above has no real keyframe input info. As such, the output file may have sort of an index, but hardly one that is really certified for "complaint-free" seeking etc [it might work for version 2 files, but such abort-and-patch later approach would not work for many formats, not for avi, qt, mkv version 1, etc]. > > I made some test with the num-buffers property to avoid it, but the result > generates "non-existing PPS referenced" errors, and with 1 fps the results > either can't be playable with gstreamer. (Btw what means exactly this message? > i can't find a proper description about that.) > > Pipeline: > gst-launch lannersrc num-buffers=100 ! capsfilter > caps="video/x-h264,framerate=10/1" ! matroskamux ! filesink > location=10fps_lanner2matroska.mkv PPS (picture parameter set) contain some 'global' data needed/useful for decoding frames/slices. Such message would suggest a referenced (by number) PPS has not been seen before in the stream. Either it really was not there (corrupt stream) or the id has been misparsed (possibly due to bytestream vs avc sample problems), or the decoder is in a wrong state due to seeking to wrong place (non-keyframe). > > Then i tried reencoding the stream of our capture card, and everything was fine > and seekable: > > gst-launch lannersrc num-buffers=20 ! capsfilter > caps="video/x-h264,framerate=1/1" ! ffdec_h264 ! x264enc byte-stream=false ! > matroskamux ! filesink location=10fps_reencoded.mkv > > Maybe there's a problem with the h264 stream of the capture card? Could be, or with the plugin. Maybe its output caps contain codec-data while the stream itself is in byte-format, which could confuse matters. Or much more likely, it simply never sets the DELTA_UNIT buffer flag. This misleads the muxer into believing every frame is a keyframe and every frame can be seeked to, which is likely wrong and messes up seeking. Btw, mkvinfo (which comes alongside mkvmerge) should allow for some dumping of a mkv (though apparently unfortunately not frame content itself). In particular, using mkvinfo on a 2fps lannersrc sample file shows that every frame has a cue (index) entry, which typically means muxer was told every frame is a keyframe. It is then an exercise to confirm that really not every frame is an I(DR)-frame ;)
(In reply to comment #10) > [it might work for version 2 files, but such abort-and-patch later approach > would not work for many formats, not for avi, qt, mkv version 1, etc]. what's the difference between version 1 and 2? i can't find any info about it.
(In reply to comment #11) > (In reply to comment #10) > > [it might work for version 2 files, but such abort-and-patch later approach > > would not work for many formats, not for avi, qt, mkv version 1, etc]. > > what's the difference between version 1 and 2? i can't find any info about it. The most prominent one (if any other) I can remember is that SimpleBlock only exists in v2. This used to be mentioned much more clearly on e.g. http://www.matroska.org/technical/specs/index.html, but now it seems only indicated in the pdf that is mentioned there.