GNOME Bugzilla – Bug 122483
exslt-function does not work in every context
Last modified: 2009-08-15 18:40:50 UTC
It's possible to define a function by using the EXSLT func:function extension, however it does not work everywhere. <func:function name="func:initial"> <xsl:param name="s" /> <func:result select="substring($s,1,1)" /> </func:function> defines a function and <xsl:value-of select="func:initial(.)"/> works as expected. But if the function is used in the use-attribute of a key <xsl:key name="Fails" match="test" use="func:initial(.)" /> xsltproc reports an error: aylee /home/thimo> xsltproc bug.xsl bug.xml xmlXPathCompOpEval: function initial not found XPath error Unregistered function in func:initial(.) Exitcode 10 Commenting out the xsl:key-statement shows that the error isn't triggered by the value-of. This is Debian bug #204583, reported by Thimo Neubauer.
Can you provide a complete stylesheet showing the problem ? thanks, Daniel
Created attachment 21669 [details] example stylesheet
I changed initialisation of external functions to take place at an earlier time. I have tested the results with your example, and it now seems satisfactory. Updated source (transform.c) is in CVS. Thanks for your report.
The additional problem occuring when the function and the key were defined in an imported module have been fixed too, Daniel
This should be closed by release of libxslt-1.1.5, thanks, Daniel
Unfortunately this issue is not solved completely. The DocBook XSL stylesheets implemented support for internationalized indexing that uses user defined function inside xsl:key. However the function calls named template that is using other global parameter and this is probably too much for xsltproc. If you try the following stylesheet: <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:import href="http://docbook.sourceforge. net/release/xsl/current/html/docbook.xsl"/> <xsl:include href="http://docbook.sourceforge. net/release/xsl/current/html/autoidx-ng.xsl"/> </xsl:stylesheet> You will get following error message: runtime error: file ../docbook-xsl-1.65.1/common/l10n.xsl line 26 element choose unregistered variable l10n.gentext.language xmlXPathCompiledEval: evaluation failed I'm now finishing article for xml.com about DocBook and indexes and it would be very cool if I can write that internationalized indexing works also in xsltproc. Jirka
paphio:~/tmp -> wget http://docbook.sourceforge.net/release/xsl/current/html/autoidx-ng.xsl --11:31:12-- http://docbook.sourceforge.net/release/xsl/current/html/autoidx-ng.xsl => `autoidx-ng.xsl' Resolving docbook.sourceforge.net... done. Connecting to docbook.sourceforge.net[66.35.250.209]:80... connected. HTTP request sent, awaiting response... 404 Not Found 11:31:13 ERROR 404: Not Found. paphio:~/tmp -> please provide all the informations needed to test, because there are missing resources, a tarball attached to bugzilla would help Daniel
Using docbook-xsl-1.65.1 the current library shows no problem:- bill@billsuper bug122483 $ xsltproc bug3.xsl test.xml No template for "/test" (or any of its leaves) exists in the context named "title" in the "en" localization. No template matches test. <html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO- 8859-1"><title></title><meta name="generator" content="DocBook XSL Stylesheets V1.65.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><font color="red"><test>Test succeeded</test></font></body></html> bill@billsuper bug122483 $ cat bug3.xsl <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:import href="../bug140861/docbook-xsl-1.65.1/html/docbook.xsl"/> <xsl:include href="../bug140861/docbook-xsl-1.65.1/html/autoidx-ng.xsl"/> </xsl:stylesheet>
You must place at least one indexterm element into DocBook source in order to be xsl:key evaluated and error raised. E.g.: <indexterm> <primary>foo</primary> </indexterm>
jirka - have you tried with libxml2-2.6.10 and libxslt-1.1.7 ? - what exact published version of the docbook stylesheet are needed to reproduce the bug. - provide a complete example for the problem. It is not proper to give an XSLT example which simply doesn't work (broken url) and and incomplete XML input file. To fix any problem we need to be able to reproduce it first, and currently we are not able to reproduce it, so provide the full informations needed after checking on your side that the bug shows up with the latest version. This is all part of the normal process for reporting bugs, as explained in: http://xmlsoft.org/XSLT/bugs.html thanks, Daniel
Created attachment 29020 [details] Test XML document
Created attachment 29021 [details] Sample stylesheet
Hi William and Daniel, I isolated bug out of the DocBook stylesheets, see attached files test.xml and test.xsl. Transformation yields in: E:\work\xsltproc>xsltproc test.xsl test.xml runtime error: file test.xsl line 24 element if unregistered variable globalparam xmlXPathCompiledEval: evaluation failed I'm using: E:\work\xsltproc>xsltproc -V Using libxml 20610, libxslt 10107 and libexslt 805 xsltproc was compiled against libxml 20610, libxslt 10107 and libexslt 805 libxslt 10107 was compiled against libxml 20610 libexslt 805 was compiled against libxml 20610 It seems that problem is if xsl:key uses user-defined function which calls template that references global parameter. Probably xsl:keys and xsl:params are evaluated in a wrong order.
Okay, william managed to reproduce it now, thanks, Daniel
Okay after analysis we think the stylesheet is in error. This is related to the use of variable in keys. You cannot use variables to define keys: http://www.w3.org/TR/xslt#key "It is an error for the value of either the use attribute or the match attribute to contain a VariableReference." Before the addition of extension function, VariableReference in the attributes were the only way a key could reference a variable, I think the principle should remains through extension function, because it generate a chicken and egg problem on the bootstrapping process, if you allow keys to reference variables, you need to evaluate variables first, but to evaluate variables you may need the document and keys. The cycle had to be breaken in one way and the spec authors decided to forbid variables in keys. So I think the stylesheet you made is in error w.r.t. the XSLT-1 specification, and is not portable as a result. I think this would be an error to base default DocBook XSLT stylesheets on code which is clearly breaking part of the XSLT spec, Daniel
Hi Daniel, the internationalized indexing in the DocBook stylesheets is not active by default. User must include autoidx-ng.xsl stylesheet in order to override some key indexing templates. So the default stylesheets still work with xsltproc. Problem is with optional internationalized indexing feature. The autoidx-ng.xsl stylesheet depends not only on XSLT 1.0, but also on EXSLT. Without relying on EXSLT it would be impossible to implement internationalized indexing mechanism in the DocBook stylesheets. I know that behaviour of XSLT 1.0 + EXSLT is sometimes underspecified but IMHO your interpretation of spec in this particular issue is too strict. I dont't see technical reason why there could not be reference to variable in a key definition (indirect via user defined function) as long as there is no circular dependency between variables. Detection of such circular dependencies should be pretty straightforward in general. At the same time I know that current architecture of libxslt might not allow easy implementation of such detection. Anyway I would like kindly ask you to reappraise your decision that this is not a bug. Without such feature it is impossible to implement internationalized indexing feature in the DocBook stylesheets. That means that non-English users of xsltproc will not be able to produce high-quality indexes and they will be forced to switch to another XSLT implementation. I think that this will not be good situation as many users appreciate xsltproc speed and stability which is result of your tough development effort. Please, listen to French, German, Spanish, Denmark, Czech, Turkish, ... users of your software. Jirka
"Detection of such circular dependencies should be pretty straightforward in general." Easy to say, can be very tricky to implement, hence the wording from the spec, as you recognize it makes things very hard. It also break some possible optimizations on keys. Similary, the exslt:node-set "should be straigtforward" too, guess what, no it isn't. One of the reason libxslt is fast (and stable ?) is because It was implemented with a good overall knowledge of XSLT-1.0 principles. If you go and violate those principles then the code will loose those properties or just not work. I do listen to people needing I18N support, but I do not see any reason whatsoever why this should justify to change the working of XSLT, or any given implementation. i You're doing code based on XSLT, this code breaks because it does things XSLT specifically called against, change your code to work within the constraints of the language you decided to use, or change your language. Code not following the standards have very restricted usefulness and shorter lifetime, don't do that, it's not just a libxslt issue, this will break with other implementations I'm sure. Daniel
Daniel, I can fully understand your position and decision. OTOH xsltproc claims support for XSLT 1.0 and some of EXSLT extensions. And I think that my code is perfectly legal within the scope of XSLT + EXSLT. (Of course and unfortunately this scope is sometimes quite fuzzy.) Jirka
So you don't want to change your code to make it portable and prefer to rely on unportable constructs ? Daniel
Hi Daniel, it is not matter of willing but technical problem. I tried to implement internationalized index term grouping without keys. It will be slower than with keys, but at the end it could work in xsltproc. I removed all keys dependant on variables from my code, but I still must use node-set() function. I think that resulting code should work, but instead I'm getting following strange messages from xsltproc: output error : string is not in UTF-8 output error : invalid character value output error : invalid character value xmlEscapeEntities : char out of range xmlEscapeEntities : char out of range output error : string is not in UTF-8 xmlEscapeEntities : char out of range After waiting three hours for finishing transformation I interrupted xsltproc manually. Do you have any idea why and when such error messages can occur? I want to get new indexing code work under xsltproc, but I can't debug xsltproc code. Any ideas?
Basically it's in libxml2, the function(s) taking an internal (DOM) tree and doing the serialization complains because string in the tree content are not UTF-8. It's is one of the key implementation choices made in libxml2, once parsed, all text in trees must be UTF-8 (so to have a single representation internally and a single coherent set of APIs using only UTF-8). This error means libxml2 found a violation of this constraint at saving time. This can be caused by using DOM API to modify trees with non-UTF-8 string or raw internal memory corruption. The problem is that if it takes 3 hours, nobody is gonna use it anyway, no ? I appreciate your willingness to work around the initial problem ! I hope we will get to a working stage. Daniel
So, in that case I probably hit another issue in xsltproc -- I'm not using DOM API directly, I'm just using XSLT transformation and both input XML document and XSLT stylesheet are well-formed so UTF-8 error must be caused by xsltproc. I think that speed problem is related to above UTF-8 or other issue. Same code successfully finishes within few seconds in Saxon. I will try to isolate the problem, but it's quite hard to extract something simple and short from DocBook stylesheets :-( I'm going to vacations for next 10 days, so don't expect news from me in a very near future. Jirka
Okay, have nice vacations ! Daniel
what's the current status ? We just released 1.1.9 Daniel
Thanks Daniel, my node-set() problem is away in 1.1.9. I run some test on stylesheet that performs internationalized indexing without using keys. Transformation now finishes but it is *very* slow -- it tooks almost 4 minutes to finish transformation of 400K DocBook document (the same stylesheet processed with Saxon finishes in 28 seconds). This is too slow to be used in any real word application. For that reason I would appreciate if you could reappraise your decission to not support references to global variables/parameters inside user defined functions that are used inside xsl:key definition. xsl:keys can significantly boost speed of grouping. The same document processed with stylesheet that uses xsl:keys to get internationalized index tooks 8 seconds in Saxon and 10 in Xalan. Moreover I think that users appreciate if different implementations of same language give same results. In this particular case libxslt behaves differently from both Xalan and Saxon. I think that this situation when three most used XSLT processors are not compatible is very frustrating for XSLT developers.
Get on the mailing-list and try to sell your case ! And please join the list before posting there. I still think it's an abuse of the spec and a request for a non-conformance deviation, i.e. I have near zero inclination to work on it myself. Someone else might find this worthwhile. I still think this deviation should not be shipped as a requirement for DocBook stylesheets. And I'm not seeing kindly pressure like "product A and B implement deviations from the spec, so you should do so". If you really believe it's an error you should work publicly to get the XSLT Working Group to release an errata on XSLT-1.0 instead, try to be logical, Daniel