GNOME Bugzilla – Bug 700773
gstrtpjpegdepay.c -- index numbering of image components in SOF header off by one
Last modified: 2013-06-07 08:07:28 UTC
Applies to both gstreamer 0.10 and 1.0 Error description ----------------- GStreamer fails to play MJPEG videos over RTP, i.e. RTP+JPEG according to RFC2435. Sample display pipelines ------------------------ -- RTSP, RTP unicast $ gst-launch rtspsrc debug=true location=rtsp://bla ! rtpjpegdepay ! jpegdec ! xvimagesink $ gst-launch rtspsrc debug=true location=rtsp://bla ! rtpjpegdepay ! ffdec_mjpeg ! xvimagesink -- RTP unicast $ gst-launch rtpsrc uri=rtp://bla ! rtpjpegdepay ! jpegdec ! xvimagesink $ gst-launch rtpsrc uri=rtp://bla ! rtpjpegdepay ! ffdec_mjpeg ! xvimagesink Remedy ------ in gst-plugins-good/gst/rtp/gstrtpjpegdepay.c:MakeHeaders() - *p++ = 0; /* comp 0 */ + *p++ = 1; /* comp 0 */ if ((type & 0x3f) == 0) *p++ = 0x21; /* hsamp = 2, vsamp = 1 */ else *p++ = 0x22; /* hsamp = 2, vsamp = 2 */ *p++ = 0; /* quant table 0 */ - *p++ = 1; /* comp 1 */ + *p++ = 2; /* comp 1 */ *p++ = 0x11; /* hsamp = 1, vsamp = 1 */ *p++ = 1; /* quant table 1 */ - *p++ = 2; /* comp 2 */ + *p++ = 3; /* comp 2 */ *p++ = 0x11; /* hsamp = 1, vsamp = 1 */ *p++ = 1; /* quant table 1 */ Component table indexing starts at One, not Zero, at least mjpegdec.c of libav expects it this way. The standard (ISO/IEC 10918-1) depicts start of indexing at One, albeit only graphically. Frame header parameters as stated in Table B.2 allow component index values in the range from 0-255 nonetheless. Remaining Bugs -------------- Still poor image quality in comparison to vlc visualization. Chroma seems to be off (displaced by 1-4 pix) Double checked AC luma/chroma coefficients, Quantization tables and the likes to no avail.
-- RTP unicast (another one) $ gst-launch uridecodebin uri=rtp://bla ! xvimagesink Poor Image Quality ------------------ Apparently luma/chroma quantization coefficients other than defined by ISO/IEC 10918-1 are more suitable. Did some copy+paste from tables found within the live lib that is employed by vlc. These work much better, don't know about their rationale, though. see: http://www.live555.com/liveMedia, JPEGVideoRTPSource.cpp Remaining Bugs -------------- none.
Created attachment 245069 [details] header
Created attachment 245071 [details] source
In gst_rtp_jpeg_depay_process(): Throw out everything between .. /* start scan for leading JPEG header garbage .. and .. /* end scan for leading JPEG header garbage */ .. and out-comment superfluous debugging tidbits: guint8 header[1024]; /* guchar strheader[1024 * 3 + 1]; memset (strheader, 0, sizeof (strheader)); GST_INFO_OBJECT (rtpjpegdepay, "JPEG header param type %d, precision %d, dri %d", type, precision, dri); */ size = MakeHeaders (header, type, width, height, qtable, precision, dri); g_byte_array_append (rtpjpegdepay->hold_buf, header, size < 1024 ? size : 1024); /* GST_INFO_OBJECT (rtpjpegdepay, "JPEG header size %d, start of image? %s", size, header[0] == 0xff && header[1] == 0xd8 ? "Yup" : "Nope"); */ rtpjpegdepay->len_header = size; /* for (int k = 0; k < size; k++) sprintf (strheader + k * 3, "%02x ", header[k]); GST_INFO_OBJECT (rtpjpegdepay, "%s", strheader); */
Created attachment 245141 [details] polished source
> Sample display pipelines > ------------------------ > > -- RTP unicast > $ gst-launch rtpsrc uri=rtp://bla ! rtpjpegdepay ! jpegdec ! xvimagesink > $ gst-launch rtpsrc uri=rtp://bla ! rtpjpegdepay ! ffdec_mjpeg ! xvimagesink These pipelines don't make sense, rtspsrc does not have a uri property and the rtp protocol is not defined in GStreamer. > > component table indexing starts at One, not Zero, at least mjpegdec.c of libav > expects it this way. The standard (ISO/IEC 10918-1) depicts start of indexing > at One, albeit only graphically. > Frame header parameters as stated in Table B.2 allow component index values in > the range from 0-255 nonetheless. They are just IDs, any number is fine, the RFC example code starts from 0, I see many encoders start from 1. Either way it should work. > Poor Image Quality > ------------------ > Apparently luma/chroma quantization coefficients other than defined > by ISO/IEC 10918-1 are more suitable. > > Did some copy+paste from tables found within the live lib that is employed > by vlc. These work much better, don't know about their rationale, though. > see: http://www.live555.com/liveMedia, JPEGVideoRTPSource.cpp Those tables are already in zigzag order. We store the ISO tables and apply the zigzag order later. So, this code is equivalent. Let's start from the start. Could you make available the output (out.gdp) of the following pipeline: gst-launch rtspsrc debug=true location=rtsp://bla ! gdppay ! filesink location=out.gdp
Created attachment 246146 [details] JPEG with standard luma and chroma coefficients
Created attachment 246147 [details] JPEG with luma/chroma coefficients taken from live555
> > -- RTP unicast > > $ gst-launch rtpsrc uri=rtp://bla ! rtpjpegdepay ! jpegdec ! xvimagesink > > $ gst-launch rtpsrc uri=rtp://bla ! rtpjpegdepay ! ffdec_mjpeg ! xvimagesink > > These pipelines don't make sense, rtspsrc does not have a uri property and the > rtp protocol is not defined in GStreamer. Yes, the bug should be adjusted to use udpsrc; the effect will remain the same. rtpsrc (and rtpsink) are bins we wrote around udpsrc in order to be able to reuse the sockets on port+1 for RTCP. Those bins register the rtp:// protocol.
Created attachment 246166 [details] rtspsrc ! gdppay ! filesink
Created attachment 246168 [details] fixed: rtspsrc ! gdppay ! filesink
Both streams work fine for me, what's the problem? I tried: gst-launch-1.0 filesrc location=/home/wim/Downloads/fixed-out.gdp ! gdpdepay ! decodebin ! xvimagesink gst-launch-1.0 filesrc location=/home/wim/Downloads/fixed-out.gdp ! gdpdepay ! rtpjpegdepay ! jpegdec ! xvimagesink gst-launch-1.0 filesrc location=/home/wim/Downloads/fixed-out.gdp ! gdpdepay ! rtpjpegdepay ! avdec_mjpeg ! xvimagesink same with the faulty-out.gdp. What's the difference between those two files?
Seems that the issue only applies to GStreamer 0.10. In fact, quantization tables in 1.0 have changed to those I have found in live555, presumably defined in 10918-5. Currently I cannot reproduce my notion that 1.0 is affected to. Way to much repositories I'm pulling from, I guess. Component indexing starting from One is a non-issue in 1.0 as well. Iirc a possibly outdated jpeg lib decremented the component numbering by one to denote the maximum index. You may close that bug as resolved / bogus at this point.