GNOME Bugzilla – Bug 599585
gstrtph263pay does not fragment frame with limited GOBs
Last modified: 2010-04-30 14:41:45 UTC
I use as streaming queue something like the following: ... ! ffenc_h263 ! rtph263enc ! ... The following code inside gstrtph263pay.c is wrong: ##############################<begin code>################################## (line 1711) for (i = 0; i < format_props[context->piclayer->ptype_srcformat][0]; i++) { GST_DEBUG ("Searching for gob %d", i); bound = gst_rtp_h263_pay_gobfinder (rtph263pay, bound); if (!bound) { GST_WARNING ("No GOB's were found in data stream! Please enable RTP mode in encoder. Forcing mode A for now."); ret = gst_rtp_h263_send_entire_frame (rtph263pay, context); goto end; } ##############################<end code>################################### It checks the picture for the GOBs contained. Even if it already found a bunch of GOBs (e.g. "i" is 10 or even higher) and it does not find another GOB, its says that "NO" GOB is found and sends the entire frame in "Mode A". When I set export GST_DEBUG=rtph263pay:5 This wrong behaviour can seen very good. Due to that programming error it never sends a "Mode B" package. which is very bad, because the "Mode A" packages get fragmented, because they have (at my computer) a size of 16 Kilobyte or more, but the MTU is only about 1.4 kilobyte. I tried to fix the above if clause to the following: if (i==0 && !bound) { then indeed real "Mode B" packages get send, but the program then aborts at the following place: ##############################<begin code>################################## (line 1030) //Step 3 decode CBPY I cbpy_type_index = gst_rtp_h263_pay_decode_cbpy (context->window, cbpy_I); GST_DEBUG ("CBPY index: %d", cbpy_type_index); if (cbpy_type_index == -1) { GST_ERROR ("CBPY index shouldn't be -1 in window: %08x", context->window); return NULL; } ##############################<end code>################################### It tries to decode a "0x0180" but in the cbpy_I table is no such code. While looking for a reason of that I found something other strange: ##############################<begin code>################################# (line 1206) //check if the block has TCOEF if (i > 3) { ind = mcbpc_P[mb_type_index][i - 1]; } else { if (mcbpc_P[mb_type_index][5] > 2) { ind = cbpy_I[cbpy_type_index][i + 3]; } else { ind = cbpy_P[cbpy_type_index][i + 3]; } } #############################<end code>#################################### It uses the table cbpy_I (the suffix _I means that it is for I-Frames), but it is a in the else part relevant for only P-Frames.
I forgot to say that I set "modea-only=false"
I took a quick look at the specification for H263 at http://www.itu.int/rec/T-REC-H.263-200501-I/en and also ffmpeg-0.5/libavcodec/h263.c but was not able to find the problem.
In the change history of that file you can see that "Mode B" is a feature that other people would like to have also: http://cgit.freedesktop.org/gstreamer/gst-plugins-good/commit/?id=645b9449a5610f488f90953115399eca5a5a4b5c even if that setting did not really change anything due to the wrong "if" statement from my Bugreport.
To reproduce the problem the following command can be used: gst-launch videotestsrc ! video/x-raw-yuv, width=704, height=576 ! ffenc_h263 ! rtph263pay modea-only=false ! rtph263depay ! ffdec_h263 ! xvimagesink
Couldn't the following callback be used to hand over the packets directly from FFMPEG (see file ffmpeg-0.5/libavcodec/avcodec.h) void (*rtp_callback)(struct AVCodecContext *avctx, void *data, int size, int mb_nb); So that VERY UGLY AND NASTY parsing for the Mode B packets inside gstrtph263pay.c would be obsolete. I don't know how this could be accomplished by the standard gstreamer pipeline interface, but maybe somebody has a good idea?
We look at the option of hooking up the thing in gst-ffmpeg, but gst currently lacks meaningful per-buffer custom metadata, so we have no place to store the information... Hopefully, in gst 0.11 we can fix that and remove the parsing.
Is there a possibility that a element of a pipeline get access to the previous element in the pipeline and just querys if a certain property (like a pointer) on that element? So rtph263depay would check its previous element (in our case it is ffenc_h263) for a property named "rtp_callback_ptr". It would set that property with a pointer to its own rtp_callback service function, and voila everything is fine. Just the thing I don't know is how a element of a pipeline get a reference to the previous element in the pipeline? Does anybody know that?
Created attachment 158879 [details] [review] Apply Mode A fragmenting to GOBs The description, code fragment and explanation in the original report actually comes down to rtph264pay foregoing to do any fragmenting (any Mode X whatsoever) if the number of GOB startcodes is not the maximum expected (other GOBs then presumably being present without startcodes). Attached patch tweaks matters to at least perform Mode A payloading w.r.t. to the GOB (startcodes) that have been located. This should then cater for a typical case ffenc_h263 rtp-payload-size=yyy ! rtph263pay (the parameter being needed to have at least some GOB startcodes).
commit a08f76a92e117a85ef63d136b83ae279f54f6dad Author: Mark Nauwelaerts <mark.nauwelaerts@collabora.co.uk> Date: Fri Apr 16 11:53:17 2010 +0200 rtph263pay: use found GOBs to apply Mode A payloading ... rather than falling back to sending the whole frame in one packet if number of GOB startcodes < maximum. One might take this further and still perform Mode B/C payloading, but at least this should cater for decent fragments in typical cases. Fixes #599585.