GNOME Bugzilla – Bug 587799
h264parse not chopping the correct timestamp
Last modified: 2011-08-19 23:38:49 UTC
Please describe the problem: When the upstream element is a filesrc, h264parse is not putting the correct timestamp in the buffer it pushes out. Steps to reproduce: When trying to mux an h264 video stream with an AAC audio stream within a MPEG2 TS container, I use the following pipeline: gst-launch --gst-debug=h264parse:4 \ filesrc location=test.264 ! h264parse ! mux. filesrc location=test.aac ! aacparse ! mux. mpegtsmux name=mux ! filesink location=output.ts > test.ts.log 2>&1 And inspect the log file test.ts.log Actual results: By inspecting the debug output file test.ts.log, I saw that h264parse is outputting timestamp 99:99.99.999 on every buffer it pushes out. An example line in the log file 0:00:03.384807000 38630 0x662740 DEBUG h264parse gsth264parse.c:543:gst_h264_parse_chain_forward:<h264parse0> pushing buffer 0x5f8388, size 3878, ts 99:99:99.999999999 Expected results: I think h264parse should be able to read into the NAL's a bit to put the correct timestamp. Does this happen every time? yes Other information: I also looked into the source code of h264pares. It looks that it is not doing anything to the timestamp at all, as it's merely passing the original timestamp on the incoming buffer directly to the new buffer. Not sure if this is the desired behavior... but I think the parser should be able to detect if the timestamp is something like GST_CLOCK_TIME_NONE and upon this case put something more meaningful to the timestamp field.
Yes, h264parse should put sensible timestamps on the buffers if upstream doesn't provide any.
AFAIK, h264 specs do not require the h264 stream to contain timing information (there may only optionally be some such info). Current h264parse already tries to do something useful if so available, but if none is available, I do not see what it could do more. In such cases, only meaningful option seems to force some framerate into the caps (if that fits the semantics of the stream) and have some generic element generate timestamps based on that (akin to the datarate property on identity but then a bufferrate or so instead). IIRC, gentrans' stamp element will in fact do so (when properly configured): http://gentrans.sourceforge.net/docs/head/gst-entrans-plugins/html/gst-entrans-plugins-stamp.html
I agree with Mark. According to what I know, timing info is often absent from raw h264 stream and the only thing we rely on is the frame rate and the picture order count (POC). (There may even be samples where frame rate is not available at all; but even in such cases some commercial parser, like Elecard stream eye, can still get meaningful time stamp estimates. I don't know how they do it.) As what I know, the current h264parse is not parsing timing info even if it exists, given a raw h264 stream. I think there is still something the parser can do, like when the frame rate is available and the POCs are provided. My concern here is that the POC is not necessarily mono increasing; I don't know how to reflect that in the current framework. That why I propose to calculate estimated time stamp and write on the buffers. With this we can solve the problem that some muxers like mpegtsmux relies critically on the time stamp of the buffers to interleave A/V data correctly. But maybe forcing frame rate suffices for this purpose in most cases. Need some experiments to see if this works. Does the mentioned element that generates time stamp basing on frame rate exist now? Or we need to write a new one?
Following pipeline fragment should generate h264 access-units with timestamps based on framerate (using a recent h264parse and 'stamp' element [see aforementioned link]): filesrc location=test.h264 ! h264parse access-unit=true ! video/x-h264,framerate=25/1 ! stamp ! fakesink Note that recent h264parse has a few more tricks (e.g. converting from Annex B bytestream format to AVC sample format), and that the above pipeline may obviously need extending (e.g. for the audio part, where aacparse probably generates decent timestamps with ok input).
See bug #625673, which is the same as this one just for mpeg4 video
What should we do about this? IMHO it can just be closed, h264 streams without any timing information are completely useless unless they are inside a container that has timing information. In that case h264parse will passthrough the timing information and everything's fine.