GNOME Bugzilla – Bug 378766
xsltproc with libxslt 1.1.18 sometimes segfaults, sometimes pegs CPU, when processing XCB protocol descriptions
Last modified: 2007-04-26 07:44:36 UTC
Bug #398327 originally sent to Debian; see <http://bugs.debian.org> for details. As reported by Jamey Sharp, with updates for more recent information: When building libxcb, xsltproc 1.1.18 with libxslt 1.1.18 segfaults on some of our XML sources and hangs chewing 100% of the CPU on other sources. 1.1.17 worked fine. The stylesheet we're using is here: http://gitweb.freedesktop.org/?p=xcb/libxcb.git;a=blob_plain;f=src/c-client.xsl And the XML documents we're transforming can be found here: http://xcb.freedesktop.org/dist/xcb-proto-1.0.tar.bz2 All of our XML triggers either a hang or a segfault. You can run the transformation with arguments like these, if you're in the src directory of the above tarball: xsltproc --stringparam mode header --stringparam base-path ./ c-client.xsl xproto.xml Processing 'bigreq.xml' illustrates a case where it hangs consuming 100% of the CPU. From a little fiddling in gdb, I think it's infinite-looping in xsltReleaseLocalRVTs from transform.c, but I haven't looked at the source to see if that makes sense. The cases that segfault have the following stack trace. I can regenerate it with full debugging information if needed. I'm hoping that Josh Triplett can help if more than this is needed to debug the problem, since this XSLT is his code. :-) (gdb) bt #0 0x40271909 in free () from /lib/tls/libc.so.6 #1 0x4009f1bc in xsltFreeDocumentKeys (idoc=0x81625a8) at keys.c:154 #2 0x4009d6c1 in xsltReleaseRVT (ctxt=0x80e1d20, RVT=0x81607d8) at variables.c:365 #3 0x400ae781 in xsltApplyXSLTTemplate (ctxt=0x80e1d20, contextNode=0x80889b8, list=0x80704c8, templ=0x8061920, withParams=0x0) at transform.c:3031 #4 0x400aef1b in xsltProcessOneNode (ctxt=0x80e1d20, contextNode=0x80889b8, withParams=0x0) at transform.c:2024 #5 0x400afb78 in xsltApplyTemplates (ctxt=0x80e1d20, node=0x8089f30, inst=0x805c128, castedComp=0x8067270) at transform.c:4977 #6 0x400ad264 in xsltApplySequenceConstructor (ctxt=0x80e1d20, contextNode=0x8089f30, list=0x805b9b0, templ=0x809fb00) at transform.c:2560 #7 0x400ae74e in xsltApplyXSLTTemplate (ctxt=0x80e1d20, contextNode=0x8089f30, list=0x805b9b0, templ=0x809fb00, withParams=0x0) at transform.c:3008 #8 0x400aef1b in xsltProcessOneNode (ctxt=0x80e1d20, contextNode=0x8089f30, withParams=0x0) at transform.c:2024 #9 0x400af3bd in xsltProcessOneNode (ctxt=0x80e1d20, contextNode=0x8083c58, withParams=0x0) at transform.c:1854 #10 0x400afb78 in xsltApplyTemplates (ctxt=0x80e1d20, node=0x8083c58, inst=0x805b4a0, castedComp=0x8066740) at transform.c:4977 #11 0x400ad264 in xsltApplySequenceConstructor (ctxt=0x80e1d20, contextNode=0x8083c58, list=0x805b4a0, templ=0x0) at transform.c:2560 #12 0x4009e13a in xsltEvalGlobalVariable (elem=0x819fa48, ctxt=0x80e1d20) at variables.c:1178 #13 0x4009e311 in xsltGlobalVariableLookup (ctxt=0x80e1d20, name=0x80635e7 "pass1-rtf", ns_uri=<value optimized out>) at variables.c:1852 #14 0x4009e781 in xsltXPathVariableLookup (ctxt=0x80e1d20, name=0x80635e7 "pass1-rtf", ns_uri=0x0) at variables.c:2278 #15 0x401436fc in xmlXPathVariableLookup () from /usr/lib/libxml2.so.2 #16 0x40144903 in xmlXPathFunctionLookup () from /usr/lib/libxml2.so.2 #17 0x40143b0f in xmlXPathFunctionLookup () from /usr/lib/libxml2.so.2 #18 0x4014432a in xmlXPathFunctionLookup () from /usr/lib/libxml2.so.2 #19 0x40144392 in xmlXPathFunctionLookup () from /usr/lib/libxml2.so.2 #20 0x40143b0f in xmlXPathFunctionLookup () from /usr/lib/libxml2.so.2 #21 0x40145846 in xmlXPathFunctionLookup () from /usr/lib/libxml2.so.2 #22 0x40149ed6 in xmlXPathEval () from /usr/lib/libxml2.so.2 #23 0x4014a0d2 in xmlXPathCompiledEval () from /usr/lib/libxml2.so.2 #24 0x4009df09 in xsltEvalGlobalVariable (elem=0x819f6f8, ctxt=0x80e1d20) at variables.c:1115 #25 0x40112a2f in xmlBuildQName () from /usr/lib/libxml2.so.2 #26 0x40112acb in xmlHashScanFull () from /usr/lib/libxml2.so.2 #27 0x40112b1a in xmlHashScan () from /usr/lib/libxml2.so.2 #28 0x4009d39b in xsltEvalGlobalVariables (ctxt=0x80e1d20) at variables.c:1286 #29 0x400b2f80 in xsltApplyStylesheetInternal (style=0x8058800, doc=0x8083c58, params=0x804cb20, output=0x804d2e8 "xproto.c", profile=0x0, userCtxt=0x80e1d20) at transform.c:5968 #30 0x400b375b in xsltRunStylesheetUser (style=0x8058800, doc=0x8083c58, params=0x804cb20, output=0x804d2e8 "xproto.c", SAX=0x0, IObuf=0x0, profile=0x0, userCtxt=0x80e1d20) at transform.c:6275 #31 0x080498cc in xsltProcess (doc=0x8083c58, cur=0x8058800, filename=0xbfe406f6 "xproto.xml") at xsltproc.c:468 #32 0x0804a37c in main (argc=14, argv=0xbfe3f304) at xsltproc.c:854 #33 0x40220ea8 in __libc_start_main () from /lib/tls/libc.so.6 #34 0x080492d1 in _start () at ../sysdeps/i386/elf/start.S:119 Further information from me: I imported libxslt CVS into GIT with git cvsimport and ran git bisect between 1.1.17 and 1.1.18. I managed to track both the segfault and the hang (which manifests differently on different platforms; on x86-64 it causes "*** glibc detected *** corrupted double-linked list: 0x0000000000633f70 ***", and an abort) down to this commit: 7fc9680614c7dbd5407fc718ac5dcb601a378da7 is first bad commit commit 7fc9680614c7dbd5407fc718ac5dcb601a378da7 Author: kbuchcik <kbuchcik> Date: Fri Jul 14 16:10:17 2006 +0000 * libxslt/attributes.c libxslt/documents.c libxslt/functions.c libxslt/keys.c libxslt/namespaces.c libxslt/pattern.c libxslt/preproc.c libxslt/templates.c libxslt/templates.h libxslt/transform.c libxslt/variables.c libxslt/xslt.c libxslt/xsltInternals.h libxslt/xsltutils.c libxslt/xsltutils.h libexslt/common.c libexslt/dynamic.c libexslt/functions.c libexslt/strings.c: Refactored xsltValueOf(). Changed to use xmlXPathCastToString() directly, rather than creating an intermediate object with xmlXPathConvertString(). This now does not add a text-node to the result if the string is empty (this has impact on serialization, since an empty text-node is serialized as <foo></foo>, and now it will be serialized as <foo/>). Refactored other functions in transform.c: Mostly code cleanup/restructuring. Minimized number of function variables for instruction which eat up function stack memory when recursing templates (xsltIf(), xsltChoose(), xsltApplyTemplates(), xsltCallTemplate()). Changed XSLT tests to use xmlXPathCompiledEvalToBoolean(). Implemented redefinition checks at compilation-time and eliminating them at transformation time in the refactored code paths. Introduced the field @currentTemplateRule on xsltTransformContext to reflect the "Current Template Rule" as defined by the spec. NOTE that ctxt->currentTemplateRule and ctxt->templ is not the same; the former is the "Current Template Rule" as defined by the XSLT spec, the latter is simply the template struct being currently processed by Libxslt. Added XML_COMMENT_NODE and XML_CDATA_SECTION_NODE to the macro IS_XSLT_REAL_NODE. Misc code cleanup/restructuring and everything else I already forgot. Refactored lifetime of temporary result tree fragments. Substituted all calls to the now deprecated xsltRegisterTmpRVT() for the new xsltRegisterLocalRVT(). Fragments of xsl:variable and xsl:param are freed when the variable/pram is freed. Fragments created when evaluating a "select" of xsl:varible and xsl:param are also bound to the lifetime of the var/param. EXSLT's func:function now uses the following functions to let take care the transformation's garbage collector of returned tree fragments: xsltExtensionInstructionResultRegister(), xsltExtensionInstructionResultFinalize() Fixes: #339222 - xsl:param at invalid position inside an xsl:template is not catched #346015 - Non-declared caller-parameters are accepted #160400 - Compiles invalid XSLT; unbound variable accepted #308441 - namespaced parameters become unregistered #307103 - problem with proximity position in predicates of match patterns #328218 - problem with exsl:node-set() when converting strings to node sets #318088 - infinite recursion detection #321505 - Multiple contiguous CDATA in output #334493 - "--param" option does not have root context #114377 - weird func:result/xsl:variable/exsl:node-set interaction #150309 - Regression caused by fix for 142768 From Ed Casas: I'm seeing this bug as well. If I enable '--debug' the last trace message before the segfault is: freeing dictionary from stylesheet and then gdb reports: Program received signal SIGSEGV, Segmentation fault. 0xa7cbd909 in free () from /lib/tls/libc.so.6 (gdb) where #0 0xa7cbd909 in free () from /lib/tls/libc.so.6 #1 0xa7ef01bc in xsltFreeDocumentKeys () from /usr/lib/libxslt.so.1 #2 0xa7eee6c1 in xsltReleaseRVT () from /usr/lib/libxslt.so.1 #3 0xa7efdf1e in xsltCopyTextString () from /usr/lib/libxslt.so.1 #4 0xa7efe277 in xsltCopyTextString () from /usr/lib/libxslt.so.1 #5 0xa7eff74e in xsltChoose () from /usr/lib/libxslt.so.1 #6 0xa7efff1b in xsltProcessOneNode () from /usr/lib/libxslt.so.1 #7 0xa7f00b78 in xsltApplyTemplates () from /usr/lib/libxslt.so.1 #8 0xa7efe264 in xsltCopyTextString () from /usr/lib/libxslt.so.1 #9 0xa7eff74e in xsltChoose () from /usr/lib/libxslt.so.1 #10 0xa7efff1b in xsltProcessOneNode () from /usr/lib/libxslt.so.1 #11 0xa7f003bd in xsltProcessOneNode () from /usr/lib/libxslt.so.1 #12 0xa7f0403c in xsltNewTransformContext () from /usr/lib/libxslt.so.1 #13 0x08049a49 in ?? () #14 0x00000000 in ?? () (gdb) The handling of exslt function 'param's seems to have been broken as well and I have seen xsltproc hang so there seem to have been some major errors introduced by the above changes.
Both the segfault and the hang were caused by some new code which handles the caching of internal RVT's. This probably explains your problem with exslt functions as well, but with no specific example I can't prove that. The fixed code (libxslt/variables.c) is in CVS.
*** Bug 379208 has been marked as a duplicate of this bug. ***
Reopening bug. While the exact symptom has changed, this bug still appears present in latest CVS; now, I get either: xsltproc --stringparam mode source \ --stringparam base-path /tmp/xcb-inst/share/xcb/ \ --stringparam extension-path /tmp/xcb-inst/share/xcb/ \ -o xfixes.c ./c-client.xsl xfixes.xml *** glibc detected *** realloc(): invalid pointer: 0x08274df0 *** Aborted with this backtrace:
+ Trace 90695
Well .... your original bug report concerned either a dead loop or a segfault. I traced through the library code, using the stylesheet and xml file you specified, and found (and fixed) the problem that caused that bug. Now you have encountered another problem (granted that, from your point of view, it's still the same "job" which is using xsltproc as a tool), but this is totally unrelated. Please open a separate bugzilla entry. And, from what you have provided above, my immediate response to this new bug would be to mark it as NEEDINFO - you have specified --stringparam 's which specify /tmp/xcb-share/xcb as a base directory, and it is not immediately obvious to me where that directory resides (and I must be able to reproduce the problem in order to try to fix it).
On second thought, wait for a bit - I made a couple of guesses as to what you intended with the stringparams (your original testfile xml/xcb-proto-1.0/src/), and can now reproduce the problem as a segfault. I still don't think it's related, but let me look further.
OK, my apologies - it's related (at least as far as the fact that it concerns xsl RVT's, and the internal housekeeping associated with the new code). In my defense, at least the problem was in a different module (transform.c). Fixed code is in CVS. There may be yet more incidents - some very substantial modifications were made to the library to improve it's efficiency. Many of these changes involved a complete re-write of the handling of "result value transforms" (RVT's). Your stylesheets apparently make (relatively) heavy usage of exslt functions involving node-sets (RVT's), and the libxslt regression tests have very few test cases in this area. Please re-open the bug (once again) if you find any more :-).
I can confirm that current libxslt CVS handles all the transforms in the libxcb build process correctly. Thanks! If you need further test cases, feel free to use any or all of the xcb-proto XML and libxcb XSLT as part of the regression test suite. We'd feel honored, and somewhat relieved. :)
(In reply to comment #6) > OK, my apologies - it's related (at least as far as the fact that it concerns > xsl RVT's, and the internal housekeeping associated with the new code). In my > defense, at least the problem was in a different module (transform.c). Fixed > code is in CVS. > > There may be yet more incidents - some very substantial modifications were made > to the library to improve it's efficiency. Many of these changes involved a > complete re-write of the handling of "result value transforms" (RVT's). Your > stylesheets apparently make (relatively) heavy usage of exslt functions > involving node-sets (RVT's), and the libxslt regression tests have very few > test cases in this area. > > Please re-open the bug (once again) if you find any more :-). Ok, I guess I have found two more examples, afaiu. For all information, please see these mails on the libxsl mailing list (no URLs in the archive yet ;() titled: 1. "Segfault shutting down exslt/functions (proposed patch)" 2. "Endless loop in xsltExtensionInstructionResultFinalize" Pure URLs you need for 1) http://tmp.stephan-suerken.de/libxslt-segfault/libxslt.patch http://tmp.stephan-suerken.de/libxslt-segfault/xsl.xsl http://tmp.stephan-suerken.de/libxslt-segfault/xml.xml 2) http://tmp.stephan-suerken.de/libxslt-loop/xsl.xsl http://tmp.stephan-suerken.de/libxslt-loop/xml.xml Thanks! Stephan
(In reply to comment #8) Seems I may not re-open with score=0 (?).
I'm not sure what's required to re-open, but guess I should be able to re-open it for you :-). Concerning your "please see these mails", I certainly did not receive either of them, and I can find nothing on either the archives or on xmlsoft.org. I'll check the results of the two sets of URL's you provided, as well as your proposed patch, and report further here.
Your proposed patch is not right - the code you want to change is performing correctly. I have fixed the problem (libxslt/transform.c), and the fixed code is in SVN. I regret there have been so many problems in this area of the code. This particular one concerned the handling of a new feature involving the caching and re-use of memory blocks, which gives a very good improvement in overall performance. Thanks for the report, and for your interest.
(In reply to comment #11) > Your proposed patch is not right - the code you want to change is performing > correctly. I have fixed the problem (libxslt/transform.c), and the fixed code > is in SVN. > > I regret there have been so many problems in this area of the code. This > particular one concerned the handling of a new feature involving the caching > and re-use of memory blocks, which gives a very good improvement in overall > performance. Ok. I have verified that the patch fixes both problems. Thanks for the prompt reaction ;). I can't find my own mails to the libxslt mailing list archives (although I received them via the mailing list). Who cares, it's fixed now ;). Info for the debian bug: The following upstream patch should fix this additional problem (for 1.1.19/etch and 1.1.20/lenny): svn diff -r1414:1425 libxslt/transform.c Thx, Stephan