GNOME Bugzilla – Bug 502814
Adds profile (hex string), sps and pps (base64 encoded) in h264 payloader src caps
Last modified: 2007-12-12 16:05:39 UTC
I would like to add the following properties to the src caps of the h264 payloader to facilitate SDP generation: - profile as a hexadecimal string - sps and pps as a base64 encoded string Here's an example output of the gstrtph264pay's src caps after successful implementation: /pipeline0/rtph264pay0.src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, profile=(string)42c00d, sps=(string)\"J0LADZWQFB+wEAg\\=\", pps=(string)\"KM4G8g\\=\\=\", ssrc=(guint)750654104, payload=(int)96, clock-base=(guint)169274195, seqnum-base=(guint)31568
Created attachment 100679 [details] [review] Proposed patch to provide requested feature This is a patch proposal to gstrtph264pay to provide the requested feature.
*** Bug 502813 has been marked as a duplicate of this bug. ***
Interesting indeed. Currently the RFC requires you to produce a sprop-parameter-sets string, which we then copy into the SDP. Is this the concatenation of the profile/sps/pps this patch creates?
I think I found the answer to my question, sprop-parameter-sets is the concatenation of SPS/PPS. I'm going to apply this patch and expose the sprop-parameter-sets property instead then.
You got it. The "profile" item is used to provide the profile-level-id= attribute, as described in the RFC. This completes the minimum set of SDP attributes for a H264 stream.
Another thing, sometimes we expose the configuration string in the GstCaps in the codec_data. This is in the AVC Decoder Configuration Record ver. 1 format and could also be used to generate the sprop-parameter-sets. It seems x264enc only produces this when not operating in bit-stream mode.
Patch by: Wai-Ming Ho <webregbox at yahoo dot co dot uk> * gst/rtp/gstrtph264pay.c: (gst_rtp_h264_pay_init), (gst_rtp_h264_pay_finalize), (gst_rtp_h264_pay_setcaps), (next_start_code), (is_nal_equal), (gst_rtp_h264_pay_decode_nal), (encode_base64), (gst_rtp_h264_pay_parse_sps_pps), (gst_rtp_h264_pay_handle_buffer): * gst/rtp/gstrtph264pay.h: Use higher performance start-code searching. Parse NALs and store SPS, PPS and profile in the caps so that they can be used in the SDP. Fixes #502814.
In 'byte-stream' mode, the SPS & PPS go directly in the data stream, usually repeated regularly, like the Sequence header in an MPEG-2 stream. When it's being sent as a stream of NALs, it's more common to expect the PPS & SPS to be carried in the container and provided to the decoder separately, like this. Most H.264 streams have exactly 1 SPS and 1 PPS, as this patch assumes, but it's possible to have more, indexed by SPS/PPS id. I have no idea if the RTP RFC covers that case?
I have not been able to find anything in the RFC that explicitly talk about multiple SPS/PPS per stream. If I understand it correctly (RFC 3984, 8.1 Mime Registration; optional parameters "sprop-parameter-sets"), it may be possible to specify sprop-parameter-sets as a series of comma separated pairs of SPS/PPS, each with a unique SPS/PPS id.
Comment #9, yes that is my understanding as well. It's a bit strange now because we put the prop sets both in the stream (twice when decoding) and on the caps. The caps are needed for SDP negotiation.
Some RTP clients relies only on the SDP caps, some relies on the in-band SPS/PPS. I suppose it doesn't hurt to have both?