GNOME Bugzilla – Bug 755723
REAL Media File make system Hang <gst_rmdemux_chain:Bogus looking header, unprintable FOURCC>
Last modified: 2018-11-03 15:35:16 UTC
Hello, I were checking the GST-Real media Parser code and found that one clip (Test file) makes my system Hangs (Windows x86, Linux, Win CE etc). this problem for the video file is present from GST 0.1 to Even in GST 1.6. please help to guide me which part of code can be debugged further for the same. >1: Test clip run command gst-launch-1.0 --gst-debug-level=5 playbin uri=file:///D:/11_OnePiece_RV4.rmvb >2: Test clip source file Will attach somewhere and share the location in comment later on this bug. >3: Logs with Debug level 5 Attached. >4: My Analysis: a. Test real media clip headers are correct. I checked each and every information and links (including size and other details for all Types of FOURCC. b. While decoding, it detects [.RMF, PROP , MDPR , MDPR, MDPR, CONT] correctly next available FOURCC is "DATA", which were never detected. c. This file i am able to play on lots of other frameworks and plateform (VLC (Win/ Linux), Android devices etc.). Logs analysis is as below (Line# 97156) ---- 0:00:05.198350818 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2350:gst_base_src_update_length:<source> reading offset 858, length 31, size 178754120, segment.stop -1, maxsize 178754120 0:00:05.198403458 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2451:gst_base_src_get_range:<source> calling create offset 858 length 31, time 0 0:00:05.198453449 6904 0F7AEA08 DEBUG GST_MEMORY gstmemory.c:138:gst_memory_init: new memory 005683F0, maxsize:34 offset:0 size:31 0:00:05.198512048 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2305:gst_base_src_do_sync:<source> we have no clock 0:00:05.198562370 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2515:gst_base_src_get_range:<source> buffer ok 0:00:05.198611699 6904 0F7AEA08 DEBUG default rmutils.c:83:gst_rm_utils_read_tags: File Content : (CONT) len = 31 0:00:05.198662022 6904 0F7AEA08 DEBUG default rmutils.c:111:gst_rm_utils_read_tags: title = 0:00:05.198718965 6904 0F7AEA08 DEBUG default rmutils.c:104:gst_rm_utils_read_tags: converting tag from CP1252 to UTF-8 0:00:05.198790807 6904 0F7AEA08 DEBUG default rmutils.c:111:gst_rm_utils_read_tags: artist = û¤·¤ÎZHU 0:00:05.198845433 6904 0F7AEA08 DEBUG default rmutils.c:111:gst_rm_utils_read_tags: copyright = bbs.jumpcn.com 0:00:05.198896418 6904 0F7AEA08 DEBUG default rmutils.c:111:gst_rm_utils_read_tags: comment = 0:00:05.198949058 6904 0F7AEA08 DEBUG GST_MEMORY gstmemory.c:87:_gst_memory_free: free memory 005683F0 0:00:05.199001698 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2350:gst_base_src_update_length:<source> reading offset 178576568, length 10, size 178754120, segment.stop -1, maxsize 178754120 0:00:05.199054999 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2451:gst_base_src_get_range:<source> calling create offset 178576568 length 10, time 0 0:00:05.199105653 6904 0F7AEA08 DEBUG GST_MEMORY gstmemory.c:138:gst_memory_init: new memory 0F7EC500, maxsize:13 offset:0 size:10 0:00:05.199173191 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2305:gst_base_src_do_sync:<source> we have no clock 0:00:05.199224506 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2515:gst_base_src_get_range:<source> buffer ok 0:00:05.199275822 6904 0F7AEA08 WARN rmdemux rmdemux.c:1039:gst_rmdemux_chain:<rmdemux0> Bogus looking header, unprintable FOURCC 0:00:05.199327800 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2350:gst_base_src_update_length:<source> reading offset 178576568, length 10, size 178754120, segment.stop -1, maxsize 178754120 0:00:05.199381102 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2451:gst_base_src_get_range:<source> calling create offset 178576568 length 10, time 0 0:00:05.199475787 6904 0F7AEA08 DEBUG GST_MEMORY gstmemory.c:138:gst_memory_init: new memory 0F7EC4A0, maxsize:13 offset:0 size:10 0:00:05.199536372 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2305:gst_base_src_do_sync:<source> we have no clock 0:00:05.199586695 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2515:gst_base_src_get_range:<source> buffer ok 0:00:05.199635362 6904 0F7AEA08 DEBUG GST_PERFORMANCE gstadapter.c:502:gst_adapter_map: copy remaining 10 bytes from adapter 0:00:05.199684691 6904 0F7AEA08 DEBUG adapter gstadapter.c:296:copy_into_unchecked: bsize 10, skip 4, csize 6 0:00:05.199736338 6904 0F7AEA08 WARN rmdemux rmdemux.c:1039:gst_rmdemux_chain:<rmdemux0> Bogus looking header, unprintable FOURCC 0:00:05.199784673 6904 0F7AEA08 DEBUG GST_PERFORMANCE gstadapter.c:502:gst_adapter_map: copy remaining 10 bytes from adapter 0:00:05.199833340 6904 0F7AEA08 DEBUG adapter gstadapter.c:296:copy_into_unchecked: bsize 10, skip 8, csize 2 0:00:05.199883994 6904 0F7AEA08 WARN rmdemux rmdemux.c:1039:gst_rmdemux_chain:<rmdemux0> Bogus looking header, unprintable FOURCC 0:00:05.199931999 6904 0F7AEA08 DEBUG GST_MEMORY gstmemory.c:87:_gst_memory_free: free memory 0F7EC500 0:00:05.199983645 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2350:gst_base_src_update_length:<source> reading offset 178576568, length 10, size 178754120, segment.stop -1, maxsize 178754120 0:00:05.200037278 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2451:gst_base_src_get_range:<source> calling create offset 178576568 length 10, time 0 0:00:05.200087932 6904 0F7AEA08 DEBUG GST_MEMORY gstmemory.c:138:gst_memory_init: new memory 0F7EC440, maxsize:13 offset:0 size:10 0:00:05.200147855 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2305:gst_base_src_do_sync:<source> we have no clock 0:00:05.200198177 6904 0F7AEA08 DEBUG basesrc gstbasesrc.c:2515:gst_base_src_get_range:<source> buffer ok 0:00:05.200246844 6904 0F7AEA08 DEBUG GST_PERFORMANCE gstadapter.c:502:gst_adapter_map: copy remaining 10 bytes from adapter 0:00:05.200295511 6904 0F7AEA08 DEBUG adapter gstadapter.c:296:copy_into_unchecked: bsize 10, skip 2, csize 8 0:00:05.200346827 6904 0F7AEA08 WARN rmdemux rmdemux.c:1039:gst_rmdemux_chain:<rmdemux0> Bogus looking header, unprintable FOURCC ----
Thanks for the bug report. Any chance you could make the file available to us so we can investigate ourselves?
Thanks for your quick response. Can you please share any medium or link accessible to you where i can put this file. As i am not able to upload on bugzilla Size 157 MB). -- Ashu
(In reply to Tim-Philipp Müller from comment #1) > Thanks for the bug report. Any chance you could make the file available to > us so we can investigate ourselves? Can you please search and download any of the files from this keyword search. Most of the links are Torrent file: [one piece][Big5][688][848x480].rmvb This is Chinese translated One piece Animation series Episode#688 Real media Content. Complete File Information is like: [猪猪字幕组][one piece][Big5][688][848x480].rmvb(170.5MB) Like: http://www.kisssub.org/show-64bc143163ba6a55ac55ee07073b4b221cd2dda9.html Attaching a reference page layout.
Created attachment 312276 [details] Test Real Media file Download Reference
Hello, I just checked some more details about the Test video File, in RMVB test file, INDX FOURCC word is not present in the last bytes of the data. Also the Last bytes are marked with ZERO's (0x00). So, i think the issue related logs "Bogus looking header, unprintable FOURCC" pointing to me about these 0x00 only. and one more experiment i did with removing these zero's. and then Real parser is able to parse the file and play it. i am checking more, but if you can guide me some portions of Real media parser and importance of INDX part of it, it will be really helpful. Also the file present on the link shared above is not exactly the same. Hence please neglect my last comments. You please share the file sharing media you can use, will upload my test contents on the same. -- Ashu
Created attachment 312425 [details] Test Content_which Hangs the System Dear Developers, Please find the Test contents which hang the system while I use playbin to play the file. This content I created/ Edited to replicate my test case. for your reference I will add the original content too in the next attachment. Thanks & Regards Ashu
Created attachment 312426 [details] Original Working content Hello Developers, Please find the original test content for your reference. To generate the other non-working content First : I removed the INDX part --> the content played well at this stage Secondly : I added four HEX digits 0x00 to enter invalid FourCC. -- Thanks and regards Orange Ashu
(In reply to Tim-Philipp Müller from comment #1) > Thanks for the bug report. Any chance you could make the file available to > us so we can investigate ourselves? Hello Tim-Phillipp Muller Hope you are dong good. Sorry to keep you waiting, finally I were able to get the buggy content with in 1600 KB size. Uploaded here, please check the attachments. Also Please find the original test content for your reference. To generate the other non-working content First : I removed the INDX part --> the content played well at this stage Secondly : I added four HEX digits 0x00 to enter invalid FourCC. -- Thanks and regards Ashu
Thanks. Someone will hopefully be able to have a look soon.
Hello all The Test Content (With "INDX" part is wrong file) is not working with Gstreamer (It make it hang), but it is playing fine with FFMPEG player. Can you please check and update me your views. or please share any of hints which part I need to look further. -- Ashu
fwiw, it works fine in push mode (pushfile:///location/to/that/file.rmvb) In pull mode it doesn't properly exit the index loop detection
Created attachment 312674 [details] [review] rmdemux: Handle files that have bogus index location Some files have a non-zero index_offset entry ... but doesn't contain a valid index at that location. While it will work fine in push-mode, it will enter a never-ending loop constantly looking for the index. When we detect there is no valid index at the specified location, just resumes to normal playback.
Hello, Thanks Edward for your quick review and inputs. Yes, PUSH mode the file is working. : checked. But the normal file mode the file is still not playing after adding the patch. Note: I am having buildable Gstreamer code base (is earlier version 1.0.10 and merged for the WIN CE systems), merged the patch their and checked, it's is not working there yet. Now I am setting up newer code base for Linux, will check their too. Let me confirm the Idea behind this patch: "After this patch test file work for normal file mode too?" gst-launch-1.0 --gst-debug-level=5 playbin uri=file:///D:/11_OnePiece_RV4.rmvb -- Ashu
Hello Edward Hervey, I complied and checked the patch with the GST 1.6.0 linux system. The test file is still not playing (though the playback exited, No hang this time). Correct Start Offset of "DATA" is 0x3c2, so once bogus INDX hits, now it is coming back to DATA OFFSET. but here too i think it is ended with BOGUS FOUR CC. Can you please re-look over the issue. Summary of logs: rmdemux rmdemux.c:917:gst_rmdemux_loop:<rmdemux0>[00m Switching from HEADER to INDEX (offset:0x3c2, data_offset:0x3c2) rmdemux rmdemux.c:857:gst_rmdemux_loop:<rmdemux0>[00m loop with state=1 and offset=0x128898 rmdemux rmdemux.c:1048:gst_rmdemux_chain:<rmdemux0>[00m Bogus looking header, unprintable FOURCC rmdemux rmdemux.c:1028:gst_rmdemux_chain:<rmdemux0>[00m looping in chain, avail 6 rmdemux rmdemux.c:857:gst_rmdemux_loop:<rmdemux0>[00m loop with state=2 and offset=0x3c2 rmdemux rmdemux.c:1023:gst_rmdemux_chain:<rmdemux0>[00m Chaining buffer of size 10 rmdemux rmdemux.c:1028:gst_rmdemux_chain:<rmdemux0>[00m looping in chain, avail 16 rmdemux rmdemux.c:1048:gst_rmdemux_chain:<rmdemux0>[00m Bogus looking header, unprintable FOURCC rmdemux rmdemux.c:1028:gst_rmdemux_chain:<rmdemux0>[00m looping in chain, avail 12 rmdemux rmdemux.c:1048:gst_rmdemux_chain:<rmdemux0>[00m Bogus looking header, unprintable FOURCC rmdemux rmdemux.c:1028:gst_rmdemux_chain:<rmdemux0>[00m looping in chain, avail 8 rmdemux rmdemux.c:857:gst_rmdemux_loop:<rmdemux0>[00m loop with state=2 and offset=0x3cc Adding logs in the attachment for your kinf refrence. Another Querry: Indeed Pushfile method worked for the file, can we set it bydefault true for the RM parser only? -- Orange Ashu
Created attachment 312714 [details] GST_1.6.0_RM_Parser_PATCH_Logs_Debug_LEVEL_6 Please check this with reference to my latest comment.
Hello Edward Hervey, Sorry for the confusion, Yes you were right the attached test file working with your patch. I have 3 types of BOGUS INDX contents (possible combinations) 0. Name : Custom_katamari.rmvb [Earlier attached] Details : Didn't contain any INDX BYTES Earlier : Playing Correctly After Patch : Playing Correctly 1. Name : Custom_katamari_04_0x00.rmvb Details : In place of INDX it contains FOUR 0x00 Bytes Earlier : HANG After Patch : Playing Correctly 2. Name : Custom_katamari_14_0x00.rmvb [Will attach from now] Details : In place of INDX it contains 14 0x00 Bytes [More then 4 Case] Earlier : HANG After Patch : No Hang, launch exits now, NOT PLAYED 3. Name : Custom_katamari_AK47_14_0x00.rmvb [Will attach from now] Details : In place of INDX contains bogus printable but bogus Bytes Earlier : HANG After Patch : HANG Case 3: Can be left but Case(2) is very prone to the uncomplete downloaded files of RMVB type, hence i think it will be really helpful if you can support or guide me further. The logs shared earlier were for the case b type test contents. Sorry for the complicated test contents types, in case you have any sort of doubt please post here, will revert you ASAP. -- Thanks & Regards Orange Ashu
Created attachment 312721 [details] Case(2) Custom_katamari_14_0x00 rmvb Test Content Name : Custom_katamari_14_0x00.rmvb [Will attach from now] Details : In place of INDX it contains 14 0x00 Bytes [More then 4 Case] Earlier : HANG After Patch : No Hang, launch exits now, NOT PLAYED
Created attachment 312722 [details] Case (3) Custom_katamari_AK47_14_0x00 RMVB Test content 3. Name : Custom_katamari_AK47_14_0x00.rmvb [Will attach from now] Details : In place of INDX contains bogus printable but bogus Bytes Earlier : HANG After Patch : HANG
Created attachment 312833 [details] [review] Gstreamer 1.6.0 patch for BOGUS FOUR CC INDX patch Hello All, I made a tentative patch over the last patch which Edward provided. to play all kind of ODD scenario's mentioned with BOGUS FOUR CC's. I will check more on Case (3) mentioned above [Printable but INVALID FOUR CC], similar to this patch will work for that too, will share with in a day or two for further review. Can you please review my patch attached here. Thanks and regards Orange Ashu
Created attachment 312908 [details] [review] Gstreamer 1.6.0 patch for BOGUS FOUR CC INDX patch [Seek also restricted] Hello All, Please allow me to update my last day patch, as with that patch files started playing, but due to corrupted INDX part, we must need to stop the seek. Hence set the Can you please review my patch attached here, In case it seems ok, please help to mainline the same. Thanks and regards Orange Ashu
Created attachment 313108 [details] [review] Gstreamer 1.6.0 patch for BOGUS FOUR CC INDX patch [Seek restricted + Clean Exit in the last] Gstreamer 1.6.0 patch for BOGUS FOUR CC INDX patch [Seek restricted + Clean Exit in the last] Hello All, Please allow me to update my last patch (Again) for BOGUS indexed RMVB files a. RM Parser Started playing b. Seek also restricted c. Smooth or graceful exiting of player Can you please review my patch attached here, In case it seems ok, please help to mainline the same. Thanks and regards Orange Ashu
Comment on attachment 312674 [details] [review] rmdemux: Handle files that have bogus index location What's with this one? Already as "accept-commit-now" since a long time but was never merged.
Thanks for the patches, but they are a bit confusing. Please attach patches in "git format-patch" format, see e.g. https://gstreamer.freedesktop.org/documentation/contribute/index.html#how-to-prepare-a-patch-for-submission Ideally prepare one patch for each problem and describe the problem and solution in the commit message. Do not include Edward's changes in your patch, create your fixes on top of Edward's patch.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-ugly/issues/13.