After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 613808 - Tarballs containing filenames longer than 99 characters cannot be unpacked without applying attached patch to tar
Tarballs containing filenames longer than 99 characters cannot be unpacked wi...
Status: RESOLVED FIXED
Product: Release Engineering
Classification: Infrastructure
Component: General
unspecified
Other Linux
: Normal normal
: ---
Assigned To: Release Engineering People
Release Engineering People
Depends on:
Blocks:
 
 
Reported: 2010-03-24 14:29 UTC by Pacho Ramos
Modified: 2011-06-23 16:22 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
tar-revert-pipe.patch (3.61 KB, application/octet-stream)
2010-03-24 14:29 UTC, Pacho Ramos
Details

Description Pacho Ramos 2010-03-24 14:29:44 UTC
Created attachment 156984 [details]
tar-revert-pipe.patch

This is a problem that seems to affect to some gnome tarballs usually related with glibmm (like glibmm-2.18.1.tar.bz2) and, also, evolution-data-server (like evolution-data-server-2.28.3.1.tar.bz2)

Please read first http://bugs.gentoo.org/show_bug.cgi?id=309001 for seeing a full explanation of the problem. Also take a look on http://lists.gnu.org/archive/html/bug-tar/2008-12/msg00028.html

In summary, when portage tries to unpack them, it fails because of this:
bzip2: I/O or other error, bailing out.  Possible reason follows.
bzip2: Broken pipe
        Input file =
/var/tmp/portage/gnome-extra/evolution-data-server-2.28.3.1/distdir/evolution-data-server-2.28.3.1.tar.bz2,
output file = (stdout)
(See http://bugs.gentoo.org/show_bug.cgi?id=309001#c7 and http://bugs.gentoo.org/show_bug.cgi?id=309001#c8 )

Thanks a lot for your help
Comment 1 Vincent Untz 2010-03-24 15:56:49 UTC
Any idea what can be done? Is it really an issue with our tarballs? I don't think we do anything special, so I'd blame tar or another tool instead...
Comment 2 Pacho Ramos 2010-03-24 16:16:41 UTC
I will ask downstream (I forgot CC proper people sorry)
Comment 3 Pacho Ramos 2010-03-24 17:33:33 UTC
As commented downstream:
"If you look at the bunzip2ed glibmm-2.18.1.tar you'll see that it's two
concatenated tarballs of lengths 28303360 and 28395520 bytes, respectively. The
first part contains 3880 files, the second part 3892 files (a superset of the
first part!). So something is definitely wrong with this tarball.

tar-1.22 on the whole glibmm-2.18.1.tar unpacks only the first part (3880
files) and ignores the rest."
http://bugs.gentoo.org/show_bug.cgi?id=309001#c34
Comment 4 Vincent Untz 2010-03-24 19:56:31 UTC
But I'm nearly sure that this was generated with the standard autotools. So it wouldn't be a bug on our side.

cc'ing Daniel Elstner who should know for glibmm.
Comment 5 Pacho Ramos 2010-03-24 20:24:40 UTC
But, what tool do you use to create the tarball?
Comment 6 Vincent Untz 2010-03-24 20:27:18 UTC
The autotools use tar, I guess.
Comment 7 Pacho Ramos 2010-03-24 20:42:23 UTC
And, what tar version? Maybe that could cause the problem :-/
Comment 8 Vincent Untz 2010-03-24 20:43:27 UTC
I'm not the one who's releasing those tarballs, so I can't know. Let's wait for Daniel.
Comment 9 Daniel Elstner 2010-03-24 23:59:17 UTC
We are using Autotools of course, meaning the tarball is produced with either "make dist" or "make distcheck".  The glibmm-2.18.1 tarball was released by Jonathon Jongsma, but I'm pretty sure he didn't do anything other than that.

Investigating the matter, I had a closer look at the configure.{ac,in} files of glibmm-2.18.1 and evolution-data-server-2.28.3.1.  At the time the releases were made, both modules used the default tar format of Automake.  However, both modules later switched to tar-ustar or tar-pax.  For glibmm, we had to do this because some files of the reference documentation had names that were too long for the default tar format to handle.

On inspection of the two concatenated tar archives of glibmm-2.18.1, it turns out that the files which are contained in the second archive but are missing from the first all have names longer than 99 characters.

In the Automake manual [1], I found this paragraph in the description of the v7 archive format:

"This format has traditionally been used by Automake when producing Makefiles. This practice will change in the future, in the meantime, however this means that projects containing file names more than 99 characters long will not be able to use GNU tar 1.23 and Automake prior to 1.9."

I'm almost certain now that this is the problem we are seeing here.  In addition to the limitation described above, there is apparently a bug in at least some versions of either Automake or GNU tar that causes the concatenated archives.  My guess is on an automatic fallback mechanism gone haywire.

Independently of this apparent Automake/tar bug, I think that any module which requires extra-long filenames in the archive should explicitly use either the "tar-ustar" or the "tar-pax" option with AM_INIT_AUTOMAKE().  Unfortunately it can easily go unnoticed, since apparently tar does not abort if a filename is too long but only shows a warning.

[1] http://www.gnu.org/software/tar/manual/html_node/Formats.html
Comment 10 Pacho Ramos 2010-03-26 08:34:49 UTC
Thanks a lot Daniel for your investigation
Comment 11 Murray Cumming 2011-02-03 08:04:09 UTC
As of glibmm-2.20, we used the tar-pax option in configure.ac, which should fix this problem:
http://git.gnome.org/browse/glibmm/tree/configure.ac#n28
So, this is not a glibmm bug now.

evolution-data-servers uses tar-ustar, which should fix it too:
http://git.gnome.org/browse/evolution-data-server/tree/configure.ac#n23
so it's not an evolution-data-server bug now either.

If any modules still have this problem, then I suggest that you file individual bugs against them, pointing them at this comment for a possible solution.
Comment 12 Pacho Ramos 2011-02-03 09:43:43 UTC
Thanks a lot for your help :-)
Comment 13 Behdad Esfahbod 2011-06-20 16:02:02 UTC
Has anyone pinged upstream about changing the default to ustar?  That's what autotools is all about afterall.
Comment 14 Javier Jardón (IRC: jjardon) 2011-06-23 16:22:54 UTC
(In reply to comment #13)
> Has anyone pinged upstream about changing the default to ustar?  That's what
> autotools is all about afterall.

http://lists.gnu.org/archive/html/automake/2011-06/msg00016.html