GNOME Bugzilla – Bug 400242
libxslt ignores ENOMEM and crashes on SIGSEGV
Last modified: 2010-11-25 03:14:32 UTC
When I run xsltproc and system runs out of memory, a segmentation fault is thrown somewhere in libxslt. (Reporter says that it can even hang the machine, however I was not able to reproduce hang). Small test case to show this behavior is included (attached file). Run ./testcase after unpacking. If you can not get segfault, try changing ULIMIT_M and ULIMIT_V variables at the start of the script. GDB backtrace and strace are also present in the archive (files debug.backtrace and debug.strace).
Created attachment 81080 [details] testcase
Note that the library does not "ignore" ENOMEM, but rather provides the caller with a NULL pointer. The crashes are caused when code doesn't check for this return value. The distinction is important, because the underlying philosophy is that the library attempts to continue after various errors are detected, in order to assist the user in locating additional errors. Still, anytime the library produces a SIGSEGV, that should certainly be fixed. I'm not certain whether your "testcase" was intentionally prepared with some missing files, which cause a large number of warnings and errors. I fixed two spots within libxslt and one within libxml2 which were certainly causing crashes with reduced memory. Please try the current SVN versions of both libraries and see whether your original problem is now OK. I'll leave this bug report open for now in case any further problems are found. Also, please note that libxslt-1.1.20 is now current (but still contains the errors which I just fixed).
(In reply to comment #2) > I'm not certain whether your "testcase" was intentionally prepared with some > missing files, which cause a large number of warnings and errors. No, I took directories "customization" and "docbook" from /opt/kde3/share/apps/ksgmltools2 and cleaned them to reduce size. File gdm.xml is from /opt/gnome/share/gnome/help/gdm/C. It was working without any warnings or errors on my box. > I fixed two spots within libxslt and one within libxml2 which were certainly > causing crashes with reduced memory. Please try the current SVN versions of > both libraries and see whether your original problem is now OK. Thanks, SVN versions are not producing SIGSEGVs anymore.