GNOME Bugzilla – Bug 556648
[typefind] detect lossless jpeg
Last modified: 2011-10-28 11:13:55 UTC
Hi, Using playbin or decodebin on an avi file (for example) containing encoded video with ljpeg, the playbin prefers jpegdec than ffdec_mjpeg and it fails. I do not know if jpegdec can or cannot decode ljpeg in abosulte way but: gst-launch-0.10 videotestsrc ! ffenc_ljpeg ! jpegdec ! fakesink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... ERROR: from element /pipeline0/jpegdec0: Failed to decode JPEG image Additional debug info: E:\devel-cvs\src\gst-plugins-good\ext\jpeg\gstjpegdec.c(1043): ??? (): /pipeline0/jpegdec0: Error #60: Unsupported JPEG process: SOF type 0x%02x ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... FREEING pipeline ... whereas this one works: gst-launch-0.10 videotestsrc ! ffenc_ljpeg ! ffdec_mjpeg ! fakesink Anyway, if jpegdec never decodes ljpeg it should have something in the caps to make the link impossible because the media type is "image/jpeg" for both of them. For now I do not know if it's but or something else so some clarifications about that from experts would be apreciated. (more infos, the 2 followings pipelines work: gst-launch-0.10 videotestsrc ! ffenc_mjpeg ! jpegdec ! fakesink and gst-launch-0.10 videotestsrc ! jpegenc ! ffdec_mjpeg ! fakesink ) Sincerely Julien
ljpeg = lossless jpeg. This is a simillar issue like with profile and levels on e.g. mpeg decoders. We don't have these details in the caps :/
I'll mark this as a typefind problem for now, since that needs fixing first.
Created attachment 161451 [details] [review] Make jpeg typefinder extract "profile" etc. (work in progress) First stab at making the jpeg typefinder extract the JPEG flavour used (not sure what the right terminology is for JPEG - "dct type"?). The field names probably need some work. Currently it's either profile="baseline" or profile="foobar" and entropy-coding="huffman"|"arithmetic". Could merge those together of course and make it part of the profile/flavour string.
Had a look at the patch, looks promising. However, when muxed in AVI as in my case, the typefinder isn't used, the caps are set based on the RIFF type. Would one then have to change avidemux to use typefindhelper or similar?
Why do we do that in a typefinder. Shouldn't this be done in jpegparse?
The most interesting question at the moment is what things to put in the caps and how to put this into the caps. When that's done we can make other elements expose that info as well where applicable. jpegparse isn't auto-plugged at the moment, and since it's simple to do there's no reason not to do it in the typefinder as well regardless.
(In reply to comment #6) > The most interesting question at the moment is what things to put in the caps > and how to put this into the caps. True, > > When that's done we can make other elements expose that info as well where > applicable. > > jpegparse isn't auto-plugged at the moment, and since it's simple to do there's > no reason not to do it in the typefinder as well regardless. But thats the plan (autoplug it). The typefinder does not fix it for e.g. mjpeg in containers.
Created attachment 162819 [details] [review] typefinding: extract profile and coding in jpeg typefinder So we can tell lossless jpeg apart from baseline jpeg, for example, and make sure decoders only announce what they can handle in their template caps. Fixes #556648. Updated patch. Primary questions are: a) have 'profile' and 'coding' as separate fields, or just merge them into one field (profile="lossless-huffman")? b) is there a better name than 'profile' ?
This is from memory... The only "profile", recommended, but not required by JFIF. The two hardware decoders that I have dealt with only decoded baseline: + /* SOF0 = 0xc0: baseline Supported by older versions of IJG libjpeg (6.2): + * SOF1 = 0xc1: extended sequential, huffman + * SOF2 = 0xc2: progressive, huffman + * SOF5 = 0xc5: differential sequential, huffman + * SOF6 = 0xc6: differential progressive, huffman + * SOF7 = 0xc7: differential lossless, huffman Supported by IJG libjpeg 8: + * SOF9 = 0xc9: extended sequential, arithmetic + * SOF10 = 0xca: progressive, arithmetic + * SOF13 = 0xcd: differential sequential, arithmetic + * SOF14 = 0xce: differential progressive, arithmetic Not supported by IJG-jpeg: + * SOF11 = 0xcb: lossless, arithmetic + * SOF15 = 0xcf: differential lossless, arithmetic + * SOF11 = 0xcb: lossless, arithmetic + * SOF11 = 0xcb: lossless, arithmetic As a rough measure, we have "profiles" libjpeg8 > libjpeg62 > baseline, and a separate format "lossless".
I'm not sure there's much utility in splitting these into several fields, if so one could also split the differential from non-differential. I think 'profile' is just fine.
Created attachment 186686 [details] [review] typefind: extract profile (SOF) in jpeg typefinder (string) My take on a single property, using rather verbose strings.
Created attachment 186687 [details] [review] typefind: extract profile (SOF) in jpeg typefinder (integer) The profiles are given in the spec numerically as SOF0-SOF15 (sans 4,8,12). Simply using this makes things much easier, we avoid the need for any int<->string mappings at all, as the strings won't be very pretty anyway. Are there objections to this approach? Exposing the correct values on jpegdec and ffdec_mjpeg should be easy enough, but what is the correct way to do this in demuxers? Is it to update all demuxers to plug a typefinder for image/jpeg, or is it to make sure that jpegparse is autoplugged and then replaced? What I want is for my AVI files made with ffenc_ljpeg to work with decodebin2, so any pointers on what code is likely to be accepted in code review would be helpful.
I would prefer the strings. You could use a table of strings to make the int->string mapping less ugly. For demuxers you would put the profile in the caps if known inside the demuxer and leave them out if the demuxer does not know. Downstream should plug jpegparse if it really needs to know the profile and decodebin2 for example would do that if the rank of jpegparse was high enough.
Tim, David? Which of the two patches should it be? :)
I would prefer calling it what it is: "sof" or "sof-marker" rather than "profile", as "profile" usually means something specific that JPEG doesn't have. No opinion on the names or numbers, but I think it's better than what I suggested, i.e., two separate fields.
Ok, went with the KISS approach, which is an int field named "sof-marker". commit 88491ff2e7822c5ad09b648dd4bae0844ac22946 Author: Philip Jägenstedt <philipj@opera.com> Date: Tue Apr 26 22:20:29 2011 +0200 typefind: extract SOF marker in jpeg typefinder The SOF types are defined by http://www.w3.org/Graphics/JPEG/itu-t81.pdf This is needed to make sure that we plug a jpeg decoder that can handle the type of JPEG we have (e.g. lossless JPEG) https://bugzilla.gnome.org/show_bug.cgi?id=556648
commit ff40deb139b49262c253f8ca284fdae82d256442 Author: Tim-Philipp Müller <tim.muller@collabora.co.uk> Date: Fri Oct 28 09:57:36 2011 +0100 jpegdec: add sof-marker to template caps, so we don't get plugged for lossless jpeg jpegdec (using libjpeg 6.2/8) can't decode some lossless types of JPEG. https://bugzilla.gnome.org/show_bug.cgi?id=556648