GNOME Bugzilla – Bug 105611
uudecoding corruption on large files
Last modified: 2006-06-18 05:22:37 UTC
The version of Pan with the problem is 0.13.3.91. It appears that any file that is larger than 2 parts is corrupted when downloading. This definately affects uuencoded files, I am not sure about yencoded files. I downloaded MANY corrupted files with 0.13.3.91. I downgraded to 0.13.3 and the files downloaded perfectly. This should be fairly easy to verify.
I think I know what was causing this, could you please upgrade to 0.13.3.92 and see if this fixes the problem? Also, what version of gnet are you running? Did you install it from tarball or RPM? If from RPM, where did you get the RPM from? The reason I ask is that Pan is now using GIOChannels via GNet. GIOChannel is defined in glib, and in glib1 they were fairly simple raw socket wrappers. In glib2 they are buffered by default and have a charset encoding. When compiling, GNet has an #ifdef to check for glib1 or glib2, and if glib2, turns off the charset encoding. ... if the GNet RPM was built against glib1, then installed on a glib2 machine, AFAICT the effect would be that the GIOChannel's charset encoding is still turned on. Pan 0.13.3.92 adds code to turn off the charset encoding, regardless of what GNet did.
*** Bug 105136 has been marked as a duplicate of this bug. ***
Unfortunately, no, upgrading to 0.13.3.92 did not solve my problem. I have the RPMs of both glib1 and glib2 installed: libglib1.2-1.2.10-6mdk libglib1.2-devel-1.2.10-6mdk libglib2.0_0-2.0.6-2mdk glib-gettextize-2.0.6-2mdk libglib2.0_0-devel-2.0.6-2mdk And I installed gnet 1.1.8 from tarball. It seems, however, that gnet is linked against glib1: [dogshu@delta-9 dogshu]$ ldd /usr/local/lib/libgnet.so libgthread-1.2.so.0 => /usr/lib/libgthread-1.2.so.0 (0x4001c000) libglib-1.2.so.0 => /usr/lib/libglib-1.2.so.0 (0x4001f000) libpthread.so.0 => /lib/i686/libpthread.so.0 (0x40047000) libresolv.so.2 => /lib/libresolv.so.2 (0x4005b000) libnsl.so.1 => /lib/libnsl.so.1 (0x4006c000) libc.so.6 => /lib/i686/libc.so.6 (0x40080000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) So, I just tried recompiling and reinstalling gnet with the --enable-glib2 flag, then recompiling and reinstalling pan 0.13.3.92. Gnet seems to have linked correctly: [dogshu@delta-9 gnet-1.1.8]$ ldd /usr/local/lib/libgnet.so libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0 (0x4001c000) libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x40021000) libpthread.so.0 => /lib/i686/libpthread.so.0 (0x40093000) libresolv.so.2 => /lib/libresolv.so.2 (0x400a7000) libnsl.so.1 => /lib/libnsl.so.1 (0x400b8000) libc.so.6 => /lib/i686/libc.so.6 (0x400cc000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) But, unfortunately, Pan is still downloading corrupt files, even after flushing the cache. I am willing to help find the source of this problem, but I'd prefer it if you would contact me directly via e-mail.
Charles: I've been away since Thursday afternoon, returning only this evening I can confirm that 0.13.3.92 does not solve the problem.
Hmm, I'm wondering if it's the new task manager that's causing corruption, or something to do with the new socket routines. What happens if you go into tools|servers and lower your maximum connections to 1, so that the download is run in serial? Pick, say, a large multipart picture (the wallpaper group will usualy have a few of >= 3 parts) to test this out quickly. jfaulkne: if we moev this to mail, then Adam (and anyone else) will not be able to contribute.
I just changed the maximum connections setting as you suggested, flushed the cache, and re-downloaded a file. The file is corrupt. I am using a PAR file for this test, because it is easy to tell that it is corrupt. I know this file is corrupt when downloading with 0.13.3.91+, and I know it is not corrupt when downloading with 0.13.3. If you contact me via e-mail, I am willing you give you an account on my machine with TightVNC access so you can do all the testing yourself.
Okay, then it probably is the sockets, rather than the task manager rewrite. Perhaps a faster next step than my setting up TightVNC, is to mail me the .msg files for a multipart that you're seeing as corrupt: _ clear your cache _ find a small multipart posted in the last 6 hours or so, which exhibits the behavior _ set the cache size very large _ save the mutlipart _ mail me a tarball of your ~/.pan/messages/cache/ directory.
*** Bug 105788 has been marked as a duplicate of this bug. ***
I submitted bug 105788. I didn't see this one...Sorry. One thing I might add to this. I was able to go to my ~/.pan/messages/cache/ directory, combine the .msg files by hand and successfully uudecode.
Ken: That's a great clue. That narrows down the possible problems quite a bit. Adam, Jim, could one of you confirm that you can cleanly uudecode from the raw .msg files that Pan said were corrupt?
Yes, you can cleanly uudecode the raw .msg files (after removing the article header), even if Pan decodes them incorrectly.
Sigh, maybe I should take you up on that VNC offer. :) Since the decoding step seems to be the culprit, let's get all the other suspects out of the way. What happens if you run Pan with the par file still in the cache, go offline, and hit `save'? Does the decode still fail? When offline, the queue and sockets are removed from the picture altogether, so we will know it's down to the actual decode process. I wonder why some people see this problem consistently, while I haven't even seen it once.
I'm sure this is grasping at straws, but ... in pan/base/acache.c, try changing - fopen (filename, "wb+"); + fopen (filename, "w+"); and - fopen (filename, "rb"); + fopen (filename, "r");
The decode still fails when Pan is offline. The code changes didn't help either. Just e-mail me and I'll set you up with an account.
Okay. Incidentally, you've been very helpful in running out and testing all these questions I've had so far. Thanks...
Update on this: Jim mailed his cache dir to me, and after stripping out the headers (to take care of the differences between the two servers' paths) the md5sums were identical to the messages in my cache dir.
I thought this was because I am SMP but is that not the case? UP ppl are seeing the bug too? I am RH 7.3 dual athlon XP.
got the same problem , using a single pentium processor ,2.4 kernel
Created attachment 14275 [details] [review] test patch to fix this bug
This patch fixes it for me. Could someone confirm this for me before I it into CVS?
Yep, it fixes it for me too!
seems it's fixed for me
Fixed in CVS: http://cvs.gnome.org/bonsai/cvsview2.cgi?diff_mode=context&whitespace_mode=show&subdir=pan/pan/base&command=DIFF_FRAMESET&file=article-thread.c&rev1=1.41&rev2=1.42&root=/cvs/gnome http://cvs.gnome.org/bonsai/cvsview2.cgi?diff_mode=context&whitespace_mode=show&subdir=pan&command=DIFF_FRAMESET&file=ChangeLog&rev1=1.1687&rev2=1.1688&root=/cvs/gnome http://cvs.gnome.org/bonsai/cvsview2.cgi?diff_mode=context&whitespace_mode=show&subdir=pan&command=DIFF_FRAMESET&file=ANNOUNCE.html&rev1=1.59&rev2=1.60&root=/cvs/gnome