GNOME Bugzilla – Bug 785077
Can't unpack archive with broken encoding in some cases
Last modified: 2017-08-09 12:51:06 UTC
When unpacking an archive with broken filenames on a remote directory, nautilus doesn't try to convert the filenames as file-roller would. This throws "Invalid argument" errors. The same problem would likely happen when unpacking on any filesystem with similar restrictions. The soundtrack for the "Monaco - What's Yours Is Mine" game $ bsdtar tvf '/run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643.zip' -rw-rw-r-- 0 0 0 4096 Sep 4 2014 Monaco - What's Yours Is Mine Original Soundtrack/._Monaco Cover - Main.jpg -rw-rw-r-- 0 0 0 4096 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/._Thumbs.db -rw-rw-r-- 0 0 0 1314892 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/01 What's Yours Is Mine.mp3 -rw-rw-r-- 0 0 0 6226201 Sep 11 2014 Monaco - What's Yours Is Mine Original Soundtrack/02 Prison Life.mp3 -rw-rw-r-- 0 0 0 3349584 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/03 Hijack at the Hairpin.mp3 -rw-rw-r-- 0 0 0 7058445 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/04 Foreign Affairs.mp3 -rw-rw-r-- 0 0 0 5951895 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/05 Liquidity.mp3 -rw-rw-r-- 0 0 0 5434971 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/06 Manoir Moucharder.mp3 -rw-rw-r-- 0 0 0 2866207 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/07 Turf War.mp3 -rw-rw-r-- 0 0 0 4157964 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/08 Le Port de la Condamine.mp3 -rw-rw-r-- 0 0 0 2873195 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/09 Centre Hospitalier.mp3 -rw-rw-r-- 0 0 0 8999811 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/10 False Teeth.mp3 -rw-rw-r-- 0 0 0 5533366 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/11 The Devil's Trick.mp3 -rw-rw-r-- 0 0 0 5630765 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/12 Discoth\202que Rouge.mp3 -rw-rw-r-- 0 0 0 8007787 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/13 Quartier Diamant.mp3 -rw-rw-r-- 0 0 0 4445620 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/14 Pearls Before Swine.mp3 -rw-rw-r-- 0 0 0 4360448 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/15 Casino De Monte Carlo.mp3 -rw-rw-r-- 0 0 0 4160296 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/16 H\223tel De Monaco.mp3 -rw-rw-r-- 0 0 0 3314926 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/17 Can't Resist (ft. Laura Vall).mp3 -rw-rw-r-- 0 0 0 29320 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/cover.jpg -rw-rw-r-- 0 0 0 35328 Sep 5 2014 Monaco - What's Yours Is Mine Original Soundtrack/Thumbs.db You can see the broken filenames for file 12 and 16. I can make this file available for the purpose of testing. See: https://git.gnome.org//browse/file-roller/tree/src/fr-process.c#n245 https://git.gnome.org//browse/file-roller/tree/src/fr-process.c#n858 and: https://git.gnome.org//browse/totem-pl-parser/tree/plparse/totem-pl-parser-lines.c#n417
(In reply to Bastien Nocera from comment #0) > When unpacking an archive with broken filenames on a remote directory, That functionality was added in bug 784941, ftr.
$ ./test-extract '/run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643.zip' '/run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/' Scanning OK, 21 files to be extracted. Destination Path: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643 Destination URI: smb://diskstation.local/music/Video%20Games/FIXME/monaco_hib12_ost_mp3_1410479643 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/._Monaco Cover - Main.jpg File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/._Thumbs.db File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/01 What's Yours Is Mine.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/02 Prison Life.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/03 Hijack at the Hairpin.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/04 Foreign Affairs.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/05 Liquidity.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/06 Manoir Moucharder.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/07 Turf War.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/08 Le Port de la Condamine.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/09 Centre Hospitalier.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/10 False Teeth.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/11 The Devil's Trick.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/12 Discoth�que Rouge.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/13 Quartier Diamant.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/14 Pearls Before Swine.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/15 Casino De Monte Carlo.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/16 H�tel De Monaco.mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/17 Can't Resist (ft. Laura Vall).mp3 File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/cover.jpg File: /run/user/1000/gvfs/smb-share:server=diskstation.local,share=music/Video Games/FIXME/monaco_hib12_ost_mp3_1410479643/Monaco - What's Yours Is Mine Original Soundtrack/Thumbs.db Progress: Archive Size 64.20 %, Files 61.90 % Error 13: Invalid argument Reproduced with gnome-autoar's test case.
Created attachment 355865 [details] [review] private: Add helper to convert filenames to UTF-8 Archives in general and zip files in particular contain filenames in legacy encodings. If a pathname is not in UTF-8 encoding already, try to convert it using commonly used encoding in various archive types. See also https://git.gnome.org//browse/file-roller/tree/src/fr-process.c#n245
Created attachment 355866 [details] [review] AutoarExtractor: Fix decompressing archive with legacy encoding filenames The archive in question contains filenames in IBM 437 encoding, likely generated with an older version of Windows. Convert the filenames to UTF-8 before writing the files out to disk to avoid "Invalid argument" errors for filesystems which don't like non-UTF-8 encodings.
Review of attachment 355865 [details] [review]: ::: gnome-autoar/autoar-private.c @@ +300,3 @@ + + if (g_utf8_validate (pathname, -1, NULL)) + * could not be converted or is already in UTF-8. Free the string with is the g_debug on pourpose?
That looks like a typo, oops.
Review of attachment 355866 [details] [review]: ::: gnome-autoar/autoar-extractor.c @@ +857,3 @@ + utf8_pathname ? + utf8_pathname : + pathname_bytes); maybe worth a new bool var to avoid this inside the call itself
Created attachment 357257 [details] [review] private: Add helper to convert filenames to UTF-8 Archives in general and zip files in particular contain filenames in legacy encodings. If a pathname is not in UTF-8 encoding already, try to convert it using commonly used encoding in various archive types. See also https://git.gnome.org//browse/file-roller/tree/src/fr-process.c#n245
Review of attachment 355866 [details] [review]: ::: gnome-autoar/autoar-extractor.c @@ +857,3 @@ + utf8_pathname ? + utf8_pathname : + pathname_bytes); you mean: if (utf8_pathname != NULL) extracted_filename = g_file_get_child (self->destination_dir, utf8_pathname); else extracted_filename = g_file_get_child (self->destination_dir, pathname_bytes); ? I can certainly do that. What about the other g_file_get_child() further down the patch?
(In reply to Bastien Nocera from comment #9) > Review of attachment 355866 [details] [review] [review]: > > ::: gnome-autoar/autoar-extractor.c > @@ +857,3 @@ > + utf8_pathname ? > + utf8_pathname : > + pathname_bytes); > > you mean: > if (utf8_pathname != NULL) > extracted_filename = g_file_get_child (self->destination_dir, > utf8_pathname); > else > extracted_filename = g_file_get_child (self->destination_dir, > pathname_bytes); > ? > > I can certainly do that. What about the other g_file_get_child() further > down the patch? Yeah. If you feel like it same for the other one further down, but that one is more like a personal preference of mine.
Created attachment 357261 [details] [review] AutoarExtractor: Fix decompressing archive with legacy encoding filenames The archive in question contains filenames in IBM 437 encoding, likely generated with an older version of Windows. Convert the filenames to UTF-8 before writing the files out to disk to avoid "Invalid argument" errors for filesystems which don't like non-UTF-8 encodings.
Attachment 357257 [details] pushed as 99e5ae6 - private: Add helper to convert filenames to UTF-8 Attachment 357261 [details] pushed as b08c420 - AutoarExtractor: Fix decompressing archive with legacy encoding filenames