GNOME Bugzilla – Bug 530513
[h264parse] no width and height in output caps
Last modified: 2009-12-10 12:05:26 UTC
Please describe the problem: h264parse doesnt provide enough capabilities at its src caps , such as width and height. This can be used nicely by the downstream element for negotiation. Steps to reproduce: 1. gst-inspect h264parse Actual results: its src caps dont have width and height Expected results: Does this happen every time? Other information:
Created attachment 110095 [details] [review] gsth264parse_patch Patch to add width and height caps. This is done by parsing sps.
Patch looks sane, but haven't applied/tested it.
One small thing: please don't use C++ style comments, some C compilers refuse to compile them.
Sorry for that, I will make sure that its not repeated.
what is this used for?
(In reply to comment #5) > what is this used for? > (In reply to comment #5) > what is this used for? > Its used for negotiation. I was just trying to make it like other parsers. In the current h264parse the caps is just the mime type and if I am not wrong that can also be served from the typefind.
What kind of negotiation? Maybe you mean transcoding h264 into a new container format?
It would be useful for muxing h264 into matroska for example
At the risk of veering off-topic, perhaps (?) a similar situation exists e.g. with theoraenc or vorbisenc, where matroskamux has to do some bitpicking on vorbis or theora caps streamheaders to obtain samplerate, channels, width, height, etc (in addition to the manipulation to put these headers into private data). Also, though a less basic property, as mp3 caps (output by lame) do not include bitrate, avimux has to compute the bitrate based on #bytes and time (and it is actually this bitrate that typically determines video/audio sync quality later on).
IMHO vorbisenc/theoraenc should put these things into the caps then (or one has to plug vorbis/theoraparse before matroskamux which puts it in the caps). For the bitrate in lame... it's nothing that can be known in advance unless you're encoding with constant bitrate so there's nothing better than guessing by avimux.
Ok, better have this functionality in a parser instead of the muxer.
As for the bitrate: why can't muxers pick up the bitrate from the tags?
If I understand vorbisparse/theoraparse code/documentation/test result correctly, it will "parse the header packets of the stream and put them as the streamheader in the caps", which vorbisenc/theoraenc already do. ...parse do not put a width or height entry in the caps structure, apparently/perhaps (?) because it is defined (? somewhere) that theora/vorbis caps should not have these properties (?). As such, matroskamux can take the header packets from the caps (not the stream) but still has to start digging for the proper bytes providing width, height, etc (without depending on theora/vorbis libs to do so). Regarding (constant) bitrate, muxers might pick these up from (event) tags if: a) e.g. lame would send such (pretty sure it does not do so now), b) and, there would be a way to distinguish the ones sent by e.g. mad from those by lame. Or otherwise the undesirable ones should be eaten before causing ambiguity somewhere (however, right now, lame explicitly passes on any incoming event tags ...), c) and, having muxers use e.g. GstCollectPads2 (in bug 415754) would give them a clean chance to deal with incoming events (rather than a fighting/hacked one) In any case, a great deal can be done here with caps, events, etc, but it seems to be primarily a matter of having specs/agreement (?) that sort these things out so that elements can rely on it in their (loose) interaction with one another. After all, the original topic here also seems to revolve around such a matter of taste/opinion/usefulness (so as not to appear getting off-topic :) )
Ok, I've tested and looked at the patch now... needs some more work: - The width/height that is read from the SPS is wrong. I get different values for every SPS and they're all wrong. test file was created by gst-launch-0.10 -v videotestsrc num-buffers=100 ! ffmpegcolorspace ! x264enc ! filesink location=test.h264 - It should only set width/height if they're != 0 - The caps should always be set, even if upstream already set caps, if the fields are different or they contain less information (i.e. if there are already caps, simply set whatever we read from the SPS on them, check if they're still the same and if not set the new caps).
Hey Sebastian, you've set this bug to NEEDINFO with comment #14 on 2008-05-13 18:07:23 UTC. I can't see any open question, so us there any? This bug is NEEDINFO for quite some time now and I'd like to see it closed :) Also, if this is issue is accepted as a valid bug, would you mind to set this bug to NEW?
I'm also quite keen on making parsers in general put this info into their output caps, this is very useful for tagreadbin.
This should now be parsed and so provided in caps in recent h264parse, notably around following commits: commit f3a153eecbf665fbf93e1c5f4857abeb10d2a1f0 Author: Mark Nauwelaerts <mark.nauwelaerts@collabora.co.uk> Date: Fri Nov 20 18:49:56 2009 +0100 h264parse: provide w/h from stream in src caps if not in sink caps commit 291f45b383d18878ca41f324fa4090e36ab43e24 Author: Mark Nauwelaerts <mark.nauwelaerts@collabora.co.uk> Date: Fri Nov 6 12:58:33 2009 +0100 h264parse: parse SPS some more (e.g. for width/height)