GNOME Bugzilla – Bug 739196
video/x-xvid changed to video/mpeg,mpegversion=4
Last modified: 2018-05-05 16:12:59 UTC
Back to GStreamer 0.10 days xvid was reveived by dvbmediasink[1] as video/x-xvid. But this is not hapening any more because of the following commits: video/x-xvid -> video/mpeg,mpegversion=4 http://cgit.freedesktop.org/gstreamer/gst-plugins-base/commit/?id=ba7d738d9e53b33907c607b8003014f7b5b0cb7f http://cgit.freedesktop.org/gstreamer/gst-plugins-good/commit/?id=9bf90f47cf789876c91d807bc70866f23875d3a5 Now we are receiving video/mpeg and dvbmediasink uses wrong ioctl against hardware drivers, causing xvid videos not playing smoothly any more. (video/mpeg http://sourceforge.net/p/openpli/gst-plugin-dvbmediasink/ci/gst-1.0/tree/gstdvbvideosink.c#l1297 informs drivers with ioctl 4 and video/x-xvid http://sourceforge.net/p/openpli/gst-plugin-dvbmediasink/ci/gst-1.0/tree/gstdvbvideosink.c#l1455 informs drivers with ioctl 10) We tried to get initial fourcc information using the following ways inside gst_dvbvideosink_set_caps: const char* fourccc = gst_structure_get_string(structure, "format"); fourcc = GST_STR_FOURCC(value); GstVideoInfo vinfo; gst_video_info_from_caps (&vinfo, caps) fourcc = gst_video_format_to_fourcc (GST_VIDEO_INFO_FORMAT(&vinfo)); But unfortunatelly none of the above returned something in fourcc. Is there any other way to find that video/mpeg, mpegversion=4 is actually from video/x-xvid? Above "issue" initially asked on GStreamer devel mailing list[2] but without any activity. [1] http://sourceforge.net/p/openpli/gst-plugin-dvbmediasink/ci/gst-1.0/tree/ [2] http://lists.freedesktop.org/archives/gstreamer-devel/2014-October/050084.html
video/x-xvid in 0.10 should be equivalent to video/mpeg,mpegversion=4. xvid is creating standards compatible MPEG-4 Part 2 video. Are you maybe confusing this with video/x-divx?
Here is the part that informs hardware for video/x-xvid: else if (!strcmp (mimetype, "video/x-xvid")) { streamtype = 10; #ifdef PACK_UNPACKED_XVID_DIVX5_BITSTREAM self->must_pack_bitstream = TRUE; #endif GST_INFO_OBJECT (self, "MIMETYPE video/x-xvid -> VIDEO_SET_STREAMTYPE, 10"); } And this one differs from the "video/mpeg" witch uses streamtype = 4 Unfortunately with streamtype = 4 video playback is not smooth. Only if we send streamtype = 10 to hardware drivers video playback is smooth.
Can you provide a sample stream that is not compatible with MPEG-4 Part 2 and requires this change? Also ideally with the encoder that was used and the settings?
Here is a sample shared on OpenPLi forum: https://dl.dropboxusercontent.com/u/38760017/xvid.avi If there is nothing special/strange with the above xvid I guess we have to get additiotional information for this from Enigma2 STB manufactures, because drivers are closed, so we don't know why they require something different on xvid. Also inside the the dvbmediasink the is a special code that handles (un)packed bitstream. (PACK_UNPACKED_XVID_DIVX5_BITSTREAM), if that helps or might be related.
Yeah that specifically says DIVX5 though, which is different from normal XVID :)
Here is another one: FaF_SaMple.avi 5.7 MB https://mega.co.nz/#!XF0VCaDY!kOK16NqBakVukJlCZcnFZBlbIz7_S-Re5ezpXHnMWWU
Created attachment 289607 [details] [review] add mpeg4codec flavor field to mpeg4 caps Add a flavor field to the caps for xvid and 3ivx
Review of attachment 289607 [details] [review]: This would also need to be added to mpeg4videoparse then. But what exactly is the difference between real MPEG-4 Part 2, and XVID now? :)
We can use the flavor field to instruct the STB drivers about the codec. Why Enigma2 manufactures need a differect ioctl for MPEG-4 Part 2 and XVID, I really don't know. I will bring that to their attencion and come back with any new info.
The drivers on enigma2 boxes rely on SDK features provided to manufacturers by the chipset vendor. The SDK has it's requirements and because most of the work is done by custom hardware accessed via binary blob APIs, there's not much scope to change these requirements by the hardware manufacturers. The hardware manufacturer prefers to supply and support their own alternative solutions to gstreamer, so the SDK features are not strongly driven by gstreamer compatibility goals. However, let's not concentrate on the politics and let's find a technical solution... When it comes to parsing the incoming bitstream, the drivers on enigma2 boxes need to know (or correctly guess) the original container type to deal with things like frame reordering, packed bitstreams and PTS vs DTS buffer timestamps. Some of the things that a software decoder can do without too much trouble (like propagate information up) are hard to do with the hardware based information where the information flows only in one direction and is handled by hardware pipelines that are configured once and then left to their own devices. Ideally, dvbmediasink would have access to as much metadata about the stream as possible. This would include things such as container format and information stored there, for example, SAR and PAR. Even though some of that information may be present in the bitstream, it is simpler to set up the decode pipeline if you have the information upfront. The specific case of real MPEG-4 Part 2 vs XVID comes down to issues with "packed" format - where the frames are re-ordered and also the use of DTS instead of PTS when the data comes from the AVI container. In 0.10, the required information could be deduced based on the difference between video/mpeg mpegversion=4, video/x-xvid, video/x-divx and video/x-msmpeg. If 1.x makes everything look like video/mpeg mpegversion=4, then there's not much that dvbmediasink can do.
There should be no "packing" differences between MPEG-4 Part 2 and XVID. DTS and PTS should be added in all cases to the buffer. And reordering of frames (B frames) is independent of XVID vs MPEG-4 Part 2. It depends on the profile. B frames are not allowed with the SimpleProfile, but are allowed with the AdvancedSimpleProfile and upwards. You can get this information from the video/mpeg,mpegversion=4 caps. There is a profile field that can contain the values "simple", "simple-scalable", "core", "main", "n-bit", "scalable", "simple-face", "simple-fba", "basic-animated-texture", "hybrid", "advanced-real-time-simple", "core-scalable", "advanced-coding-efficiency", "advanced-core", "advanced-scalable-texture", "simple-studio", "core-studio", "advanced-simple", "fine-granularity-scalable". mpeg4videoparse should add the profile field to the caps. Yes there are really that many profiles of MPEG-4 Part 2 unfortunately. MPEG at its best. You have to check with your hardware which setting it wants for which profile.
I should mention that I never saw anything other than simple and advanced-simple, and I would guess that your hardware codec only supports those two.
Created attachment 290138 [details] [review] riff-media: added fourcc to all mpeg4 video caps With this patch we know the original fourcc and we can use it in special cases where hardware requires different approach (eg for mpeg4 xvid).
Comment on attachment 290138 [details] [review] riff-media: added fourcc to all mpeg4 video caps As said before this is not the right approach considering all the discussion we had so far. Your hardware probably needs different settings depending on the MPEG-4 Part 2 *profile* but not based on this magic fourccs. And mpeg4videoparse should already add the profile to the caps.
When the video is xvid and the container is avi you don't have any *profile* /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstPad:src_0: caps = "video/mpeg, mpegversion=(int)4, systemstream=(boolean)false, framerate=(fraction)24000/1001, width=(int)640, height=(int)272" /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0 Same video xvid remuxed in mp4 (ffmpeg -c copy) has profile set. /GstMultiQueue:multiqueue0.GstPad:src_0: caps = "video/mpeg, mpegversion=(int)4, systemstream=(boolean)false, profile=(string)advanced-simple, level=(string)5, codec_data=(buffer)000001b0f5000001b5090000010000000120088685770307d31404221463000001b244697658353033623133393370000001b25876694430303634, width=(int)640, height=(int)272, framerate=(fraction)24000/1001, pixel-aspect-ratio=(fraction)1/1"
That would be a bug in mpeg4videoparse then. It should add the profile in any case, and parse it from the stream if necessary.
Can you open another bug against mpeg4videoparse for this? And then let's keep this bug open in case something else has to be done on top of that.
Ok, bug https://bugzilla.gnome.org/show_bug.cgi?id=739773 is opened to track mpeg4videoparse issue.
It seems that we are getting unknown behaviour only when we are using packed bitstream in avi xvid files. When xvid uses unpacked bitstream everything plays normaly using mpeg4 ioctl for hardware drivers. When xvid uses packed bitstream we need to use xvid ioctl for hardware drivers. So most probably a flag indicating if bitstream is packed or unpacked would be helpful (and most probably will solve the problem).
Packed format is really just a workaround for deficiencies in Microsoft software. It would be nice if gstreamer undid the packing in an upstream node, such that dvbmediasink only ever receives unpacked MPEG4_Part2 streams. Perhaps there is already a method to request that?
Would it be possible to point out to the specification for these packed/unpacked format. These thing should go in caps and mpeg4videoparse should detect and adapt the stream. Obviously, we need documentation in order to consider anything.
The packing should already be undone by mpeg4videoparse. Downstream of it you should get all MPGE4 Part 2 packets up to and including one video frame per buffer. So you won't get the B frames packed together with the I/P frame around it (IIRC that was the packing, right?).
The following is what we receive to render function of our sink: unpacked xvid: 0x000000: 00 00 01 b0 f5 00 00 01 ........ 0x000008: b5 09 00 00 01 00 00 00 ........ 0x000010: 01 20 08 86 87 ff ff 0a . ...... 0x000018: ad 88 82 19 0a 31 00 00 .....1.. 0x000020: 01 b2 58 76 69 44 30 30 ..XviD00 0x000028: 34 33 00 00 01 b6 10 00 43...... 0x000030: 0c 06 30 47 b6 fe 36 db ..0G..6. 0x000038: f8 db 6f e3 6d bf 8d b6 ..o.m... 0x000040: fe 36 db f8 db 6f e3 6d .6...o.m 0x000048: bf 8d b6 fe 36 db f8 db ....6... 0x000050: 6f e3 6d bf 8d b6 fe 36 o.m....6 0x000058: db f8 db 6f e3 6d bf 8d ...o.m.. 0x000060: b6 fe 36 db f8 db 6f e3 ..6...o. 0x000068: 6d bf 8d b6 fe 36 db f8 m....6.. packed xvid: 0x000000: 00 00 01 b0 f5 00 00 01 ........ 0x000008: b5 09 00 00 01 00 00 00 ........ 0x000010: 01 20 08 86 87 ff ff 0a . ...... 0x000018: ad 88 82 19 0a 31 00 00 .....1.. 0x000020: 01 b2 44 69 76 58 35 30 ..DivX50 0x000028: 33 62 31 33 39 33 70 00 3b1393p. 0x000030: 00 01 b2 58 76 69 44 30 ...XviD0 0x000038: 30 34 33 00 00 01 b6 10 043..... 0x000040: 00 0c 06 30 47 b6 fe 36 ...0G..6 0x000048: db f8 db 6f e3 6d bf 8d ...o.m.. 0x000050: b6 fe 36 db f8 db 6f e3 ..6...o. 0x000058: 6d bf 8d b6 fe 36 db f8 m....6.. 0x000060: db 6f e3 6d bf 8d b6 fe .o.m.... 0x000068: 36 db f8 db 6f e3 6d bf 6...o.m. Probably I have to check with a small sample that render receives the "same" data and ofc that GST_BUFFER_PTS(buffer) is same. Maybe the drivers look at DivX503b1393p (packed bitstream) and decide do handle stream in another way.
There is another difference between packed and unpacked xvid. unpacked xvid: XviD Packet Dump 0x000000: 00 00 01 b0 f5 00 00 01 ........ 0x000008: b5 09 00 00 01 00 00 00 ........ -- XviD Packet Dump 0x000000: 00 00 01 b6 52 22 78 10 ....R"x. 0x000008: b1 93 b1 95 90 be 97 04 ........ -- XviD Packet Dump 0x000000: 00 00 01 b6 91 28 38 29 .....(8) 0x000008: 3f f4 fc a9 b9 a6 3d 06 ?.....=. -- XviD Packet Dump 0x000000: 00 00 01 b6 91 a5 58 31 ......X1 0x000008: 34 ff fe 8e be 48 e1 ef 4....H.. -- XviD Packet Dump 0x000000: 00 00 01 b6 53 99 dc 14 ....S... 0x000008: be fb eb 62 e5 f0 70 60 ...b..p` -- XviD Packet Dump 0x000000: 00 00 01 b6 92 9f 98 29 .......) 0x000008: 3f a7 fc 89 b4 b3 83 83 ?....... -- XviD Packet Dump 0x000000: 00 00 01 b6 93 1c b8 31 .......1 0x000008: 3f ff 4f ff ff 4b ff 09 ?.O..K.. -- XviD Packet Dump 0x000000: 00 00 01 b6 55 11 38 10 ....U.8. 0x000008: b7 8b 56 c1 62 af d8 70 ..V.b..p -- XviD Packet Dump 0x000000: 00 00 01 b6 94 16 f8 39 .......9 0x000008: 3a 3f fe 08 8e 74 18 9c :?...t.. -- XviD Packet Dump 0x000000: 00 00 01 b6 94 94 18 31 .......1 0x000008: 3d 3f ff 4e 9f d1 bf fa =?.N.... -- XviD Packet Dump 0x000000: 00 00 01 b6 56 88 9c 10 ....V... 0x000008: b7 85 88 78 2f 01 29 3c ...x/.)< packed xvid: XviD Packet Dump 0x000000: 00 00 01 b0 f5 00 00 01 ........ 0x000008: b5 09 00 00 01 00 00 00 ........ -- XviD Packet Dump 0x000000: 00 00 01 b6 52 22 78 10 ....R"x. 0x000008: b1 93 b1 95 90 be 97 04 ........ -- XviD Packet Dump 0x000000: 00 00 01 b6 91 28 38 29 .....(8) 0x000008: 3f f4 fc a9 b9 a6 3d 06 ?.....=. -- XviD Packet Dump 0x000000: 00 00 01 b6 91 a5 58 31 ......X1 0x000008: 34 ff fe 8e be 48 e1 ef 4....H.. -- XviD Packet Dump 0x000000: 00 00 01 b6 52 22 73 ....R"s -- XviD Packet Dump 0x000000: 00 00 01 b6 53 99 dc 14 ....S... 0x000008: be fb eb 62 e5 f0 70 60 ...b..p` -- XviD Packet Dump 0x000000: 00 00 01 b6 92 9f 98 29 .......) 0x000008: 3f a7 fc 89 b4 b3 83 83 ?....... -- XviD Packet Dump 0x000000: 00 00 01 b6 93 1c b8 31 .......1 0x000008: 3f ff 4f ff ff 4b ff 09 ?.O..K.. -- XviD Packet Dump 0x000000: 00 00 01 b6 53 99 d3 ....S -- XviD Packet Dump 0x000000: 00 00 01 b6 55 11 38 10 ....U.8. 0x000008: b7 8b 56 c1 62 af d8 70 ..V.b..p -- XviD Packet Dump 0x000000: 00 00 01 b6 94 16 f8 39 .......9 0x000008: 3a 3f fe 08 8e 74 18 9c :?...t.. -- XviD Packet Dump 0x000000: 00 00 01 b6 94 94 18 31 .......1 0x000008: 3d 3f ff 4e 9f d1 bf fa =?.N.... -- XviD Packet Dump 0x000000: 00 00 01 b6 55 11 33 ....U.3 -- XviD Packet Dump 0x000000: 00 00 01 b6 56 88 9c 10 ....V... 0x000008: b7 85 88 78 2f 01 29 3c ...x/.)< -- Packed has extra packets received... 0x000000: 00 00 01 b6 52 22 73 ....R"s -- 0x000000: 00 00 01 b6 53 99 d3 ....S -- 0x000000: 00 00 01 b6 55 11 33 ....U.3 So if those packets removed and DivX503b1393p (packed bitstream) removed, then most probably the stream will be exacly the same.
Some introduction to that found here: http://itsjustonesandzeros.blogspot.ca/2007/01/what-is-packed-bitstream.html So I think the way forward is to add a new (optional) field to the parser, so packed bit stream can be negotiated (and transformed on demand). Would bee nice if some public doc could be sited to state this. Note that older Xvid encoder would use a different tag to identify packaged/unpacked, and switch to DivX.. for compatibility reason, upon DivX request. Worst case we should have a look at ffmpeg parser. This most likely worked in 0.10 when we where using ffmpeg parsers.
Raspberry Pi also used to have problem with packed B frames, they workaround it in the player and later a fix was provided in the PI firmware. [rbp/omxplayer] fixed playback of mpeg4 with packed B frames. thx popcornmix for the workaround https://github.com/xbmc/xbmc/commit/e08fee30d1c9ca755ddb5c9833a19b8d1cbabff5 [rbp/omxplayer] reverted e08fee3 the PI firmware handles this now. https://github.com/xbmc/xbmc/commit/f868fc4808e19bee60d7a83e8aa5f09b3bc861c2 // avi files with packed B frames have "not-coded" dummy frames that decoder doesn't like if (m_codingType == OMX_VIDEO_CodingMPEG4 && iSize <= 7) return true; I will ckeck if above workaround "temporary" fixes the issue in the sink. Although most probably mpeg4videoparser should do this.
So either the packed'ness should be a stream-format, or mpeg4videoparse should always unpack. Right?
I think mpeg4videoparse unpacks the packed bitstream already. The only diffence is the packed bitstream indication in user data (DivX503b1393p) and the dummy frames (N-VOP?). Adding the following check in dvbmediasink: /* remove dummy packed B-Frame */ if (self->codec_type == CT_MPEG4_PART2 && data_len <= 7) goto ok; Seems that fixes the playback of xvid packed files passing the mpeg4 ioctl in drivers (although currenlty only two users tested above fix). So removing packed leftovers should make packed bitstreams fully unpacked. PS. The xvid ioctl used in GStreamer 0.10 seem to work only for packed xvid and thats why there was a special code in dvbmediasink to convert unpacked xvid back to packed!
So is anything left to be done here?
Yes, we need to fix mpeg4videoparser not to include dummy packed B-Frames and most probably remove DivX503b1393p (packed bitstream) from userdata. Create new bug?
Yes, please create a new bug and put all the information to the relevant specs of this packed format in there.
Assuming this bug is fixed then. Closing.