GNOME Bugzilla – Bug 59281
Transformation could fail, if document and stylesheet is in shared memory, and result is not.
Last modified: 2009-08-15 18:40:50 UTC
I use libxml2+libxslt in multi-process environment. I cache compiled documents and stylesheets in shared memory (one pool for stylesheets and one for docs). I use libmm to create shared memory pools. When document and stylesheet are compiled, I call xsltApplyStlyesheet() many times in different processes. On this stage memory allocation routines allocate memory via standard libc malloc() routines (I set global flag to swith my allocation routines to this mode). I don't want results to be shared! And sometimes ALL processes core dumps in xpath routines (always in same line!). If allocation routines uses one of shared piils (it doesn't make diffirence -- which one, doc or style) everything works ok for hours... But I could not process documents in shared memory -- race coditions could destroy whole pool, and write-locking of pool eat any advatages of using cached docs and styles. Here is stack dump after crash (it is always the same).
+ Trace 8694
P.S. If I don't use any complex xpath expressions (like '*[@lang and @lang! =$language]') everything works Ok too.
I think you are hitting the fact that the XPath optimizer makes the trees not read-only. edit libxml/xpath.c search xmlXPathCompSwap() empty it. recompile, relink and retry. Give me some feedback on this. There is another place in libxslt/pattern.c in xsltTestCompMatch() where a part of the stylesheet is modified, but it should be okay thread-wise (or far less likely to give troubles) Daniel
I comment out whole body of xmlXPathCompSwap(), and it doesn't help at all. The same core dump on same line. I could send very small program (108 lines) and XML+XSL (8 and 10 lines), which give me this result in 50% of cases (one of two or three runs gives core dump ont my computer). BTW, if it is problem with non-readonly tree of stylesheet, then I could not understand -- why storing result and temporary data in any shared memory eliminates this bug. It is strange. I don't lock any shared memory for writing (and don't want to do it, because exclusive working eats all benefits of shared memory). If you need any dups, values of variables in functions (from core file), etc. I could send it to you -- my libxml2 and libxslt was compiled with debug information and, again, core dump always accures in the SAME plase.
Okay in this case, provide the code and XML/XSL source. The problem is not related to the memeory being read only it is related to the absence of synchronization between mutiple processes modifying the same data structures. If you play with threads or multiples processes making change to shared data you should really understand that doing this without interprocess communication facilities or at least an atomic test and set cannot be done reliably. Provide the code and I will see what I can do, Daniel
Created attachment 922 [details] Code, which demonstrate this bug, xml and style files.
I know about locking, etc. I'm lock shared memory (in real project) when parse new document or stylesheet, and I use libc heap (inter- process, not shared!) when apply stylesheet to avoid any race conditions and memory corruption. I've thought, applying stylesheet create new document, but is read- only process for stylesheet and original document :(. And, again, I don't understand, why processing document in shared memory helps to avoid this core dumps. It likes paradox -- using additional shared memory (without any locking) eliminates bug!
I think your t_mm_free code is wrong. It base the decision on whether to call free() instead mm_free() on the g_mm value instead of checking if p pertains to doc_mm or g_mm . Moreover your assumption that libxslt won't modify its imput document is wrong, it will at least update the _private field of some nodes. This is documented in the libxslt internal doc. Daniel
Confirmed, what I see is an error in the alloaction system. At some point the wrong freeing function got called. I don't know why, I honnestly can't spend time debugging this, let's say that switching the allocation function multiple time is not a supported use of xmlMemSetup(). You will have to debug memory error problems related to this yourself: (gdb) r Starting program: /u/veillard/tmp/59281/test DOC Memory avail: -1141854591 STYLE Memory avail: 327193780 Program received signal SIGSEGV, Segmentation fault. 0x400d900a in chunk_free (ar_ptr=0x40181f00, p=0x39ec0) at malloc.c:3125 3125 malloc.c: No such file or directory. in malloc.c (gdb) where
+ Trace 8712
Closing the bug, there isn't much I can do, Daniel