After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 122483 - exslt-function does not work in every context
exslt-function does not work in every context
Status: VERIFIED NOTABUG
Product: libxslt
Classification: Platform
Component: general
1.0.30
Other All
: Normal enhancement
: ---
Assigned To: Daniel Veillard
Daniel Veillard
Depends on:
Blocks:
 
 
Reported: 2003-09-16 21:26 UTC by Graham Wilson
Modified: 2009-08-15 18:40 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
example stylesheet (725 bytes, text/xml)
2003-11-20 21:21 UTC, Graham Wilson
Details
Test XML document (73 bytes, text/xml)
2004-06-26 07:39 UTC, jirka
Details
Sample stylesheet (1.22 KB, text/xml)
2004-06-26 07:40 UTC, jirka
Details

Description Graham Wilson 2003-09-16 21:26:27 UTC
It's possible to define a function by using the EXSLT func:function
extension, however it does not work everywhere.
                                                                          
     
  <func:function name="func:initial">
    <xsl:param name="s" />
    <func:result select="substring($s,1,1)" />
  </func:function>
                                                                          
     
defines a function and
                                                                          
     
  <xsl:value-of select="func:initial(.)"/>
                                                                          
     
works as expected. But if the function is used in the use-attribute of
a key
                                                                          
     
  <xsl:key name="Fails" match="test" use="func:initial(.)" />
                                                                          
     
xsltproc reports an error:
                                                                          
     
aylee /home/thimo> xsltproc bug.xsl bug.xml
xmlXPathCompOpEval: function initial not found
XPath error Unregistered function in func:initial(.)
Exitcode 10
                                                                          
     
Commenting out the xsl:key-statement shows that the error isn't
triggered by the value-of.                                                
                               

This is Debian bug #204583, reported by Thimo Neubauer.
Comment 1 Daniel Veillard 2003-11-20 18:16:14 UTC
Can you provide a complete stylesheet showing the problem ?

 thanks,

Daniel
Comment 2 Graham Wilson 2003-11-20 21:21:49 UTC
Created attachment 21669 [details]
example stylesheet
Comment 3 William M. Brack 2003-11-30 10:52:23 UTC
I changed initialisation of external functions to take place at an 
earlier time.  I have tested the results with your example, and it 
now seems satisfactory.  Updated source (transform.c) is in CVS.  
Thanks for your report.
Comment 4 Daniel Veillard 2003-12-21 11:55:00 UTC
The additional problem occuring when the function and the
key were defined in an imported module have been fixed too,

Daniel
Comment 5 Daniel Veillard 2004-03-25 11:51:32 UTC
This should be closed by release of libxslt-1.1.5,
                                                                                
  thanks,
                                                                                
Daniel
Comment 6 jirka 2004-06-23 09:57:52 UTC
Unfortunately this issue is not solved completely. The DocBook XSL stylesheets 
implemented support for internationalized indexing that uses user defined 
function inside xsl:key. However the function calls named template that is using 
other global parameter and this is probably too much for xsltproc. If you try 
the following stylesheet:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                version="1.0">

<xsl:import href="http://docbook.sourceforge.
net/release/xsl/current/html/docbook.xsl"/>

<xsl:include href="http://docbook.sourceforge.
net/release/xsl/current/html/autoidx-ng.xsl"/>

</xsl:stylesheet>

You will get following error message:

runtime error: file ../docbook-xsl-1.65.1/common/l10n.xsl line 26 element
choose
unregistered variable l10n.gentext.language
xmlXPathCompiledEval: evaluation failed

I'm now finishing article for xml.com about DocBook and indexes and it would be 
very cool if I can write that internationalized indexing works also in xsltproc.

Jirka
Comment 7 Daniel Veillard 2004-06-25 09:32:35 UTC
paphio:~/tmp -> wget
http://docbook.sourceforge.net/release/xsl/current/html/autoidx-ng.xsl
--11:31:12--  http://docbook.sourceforge.net/release/xsl/current/html/autoidx-ng.xsl
           => `autoidx-ng.xsl'
Resolving docbook.sourceforge.net... done.
Connecting to docbook.sourceforge.net[66.35.250.209]:80... connected.
HTTP request sent, awaiting response... 404 Not Found
11:31:13 ERROR 404: Not Found.
                                                                                
paphio:~/tmp ->


  please provide all the informations needed to test, because there are missing
resources, a tarball attached to bugzilla would help

Daniel
Comment 8 William M. Brack 2004-06-25 09:38:16 UTC
Using docbook-xsl-1.65.1 the current library shows no problem:-
bill@billsuper bug122483 $ xsltproc bug3.xsl test.xml
No template for "/test" (or any of its leaves) exists
in the context named "title" in the "en" localization.
No template matches test.
<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-
8859-1"><title></title><meta name="generator" content="DocBook XSL Stylesheets 
V1.65.1"></head><body bgcolor="white" text="black" link="#0000FF" 
vlink="#840084" alink="#0000FF"><font color="red">&lt;test&gt;Test 
succeeded&lt;/test&gt;</font></body></html>
bill@billsuper bug122483 $ cat bug3.xsl
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                version="1.0">

<xsl:import href="../bug140861/docbook-xsl-1.65.1/html/docbook.xsl"/>

<xsl:include href="../bug140861/docbook-xsl-1.65.1/html/autoidx-ng.xsl"/>

</xsl:stylesheet>
Comment 9 jirka 2004-06-25 09:50:59 UTC
You must place at least one indexterm element into DocBook source in order to be 
xsl:key evaluated and error raised. E.g.:

<indexterm>
<primary>foo</primary>
</indexterm>
Comment 10 Daniel Veillard 2004-06-25 12:37:13 UTC
jirka

 - have you tried with libxml2-2.6.10 and libxslt-1.1.7 ?
 - what exact published version of the docbook stylesheet are needed to 
reproduce the bug.
 - provide a complete example for the problem. It is not proper to give
   an XSLT example which simply doesn't work (broken url) and and incomplete
   XML input file.

To fix any problem we need to be able to reproduce it first, and currently 
we are not able to reproduce it, so provide the full informations needed
after checking on your side that the bug shows up with the latest version.
This is all part of the normal process for reporting bugs, as explained in:

     http://xmlsoft.org/XSLT/bugs.html

  thanks,

Daniel
Comment 11 jirka 2004-06-26 07:39:42 UTC
Created attachment 29020 [details]
Test XML document
Comment 12 jirka 2004-06-26 07:40:17 UTC
Created attachment 29021 [details]
Sample stylesheet
Comment 13 jirka 2004-06-26 07:43:55 UTC
Hi William and Daniel,

I isolated bug out of the DocBook stylesheets, see attached files test.xml and 
test.xsl. Transformation yields in:

E:\work\xsltproc>xsltproc test.xsl test.xml
runtime error: file test.xsl line 24 element if
unregistered variable globalparam
xmlXPathCompiledEval: evaluation failed

I'm using:

E:\work\xsltproc>xsltproc -V
Using libxml 20610, libxslt 10107 and libexslt 805
xsltproc was compiled against libxml 20610, libxslt 10107 and libexslt 805
libxslt 10107 was compiled against libxml 20610
libexslt 805 was compiled against libxml 20610


It seems that problem is if xsl:key uses user-defined function which calls 
template that references global parameter. Probably xsl:keys and xsl:params are 
evaluated in a wrong order.
Comment 14 Daniel Veillard 2004-06-26 10:57:30 UTC
Okay, william managed to reproduce it now,

 thanks,

Daniel
Comment 15 Daniel Veillard 2004-06-26 12:34:13 UTC
Okay after analysis we think the stylesheet is in error. This is
related to the use of variable in keys. You cannot use variables
to define keys:
  http://www.w3.org/TR/xslt#key
 "It is an error for the value of either the use attribute or the match
  attribute to contain a VariableReference."

Before the addition of extension function, VariableReference in the attributes
were the only way a key could reference a variable, I think the principle
should remains through extension function, because it generate a chicken and
egg problem on the bootstrapping process, if you allow keys to reference
variables, you need to evaluate variables first, but to evaluate variables
you may need the document and keys. The cycle had to be breaken in one way
and the spec authors decided to forbid variables in keys.

  So I think the stylesheet you made is in error w.r.t. the XSLT-1
specification, and is not portable as a result. I think this would be
an error to base default DocBook XSLT stylesheets on code which is clearly
breaking part of the XSLT spec,

Daniel
Comment 16 jirka 2004-06-26 14:24:13 UTC
Hi Daniel,

the internationalized indexing in the DocBook stylesheets is not active by 
default. User must include autoidx-ng.xsl stylesheet in order to override some 
key indexing templates. So the default stylesheets still work with xsltproc. 
Problem is with optional internationalized indexing feature.

The autoidx-ng.xsl stylesheet depends not only on XSLT 1.0, but also on EXSLT. 
Without relying on EXSLT it would be impossible to implement internationalized 
indexing mechanism in the DocBook stylesheets.

I know that behaviour of XSLT 1.0 + EXSLT is sometimes underspecified but IMHO 
your interpretation of spec in this particular issue is too strict. I dont't see 
technical reason why there could not be reference to variable in a key 
definition (indirect via user defined function) as long as there is no circular 
dependency between variables. Detection of such circular dependencies should be 
pretty straightforward in general. At the same time I know that current 
architecture of libxslt might not allow easy implementation of such detection.

Anyway I would like kindly ask you to reappraise your decision that this is not 
a bug. Without such feature it is impossible to implement internationalized 
indexing feature in the DocBook stylesheets. That means that non-English users 
of xsltproc will not be able to produce high-quality indexes and they will be 
forced to switch to another XSLT implementation. I think that this will not be 
good situation as many users appreciate xsltproc speed and stability which is 
result of your tough development effort. Please, listen to French, German, 
Spanish, Denmark, Czech, Turkish, ... users of your software.


Jirka
Comment 17 Daniel Veillard 2004-06-26 19:05:27 UTC
"Detection of such circular dependencies should be pretty straightforward
 in general."
Easy to say, can be very tricky to implement, hence the wording from
the spec, as you recognize it makes things very hard. It also break 
some possible optimizations on keys. 
Similary, the exslt:node-set "should be straigtforward" too, guess what,
no it isn't. One of the reason libxslt is fast (and stable ?) is because
It was implemented with a good overall knowledge of XSLT-1.0 principles.
If you go and violate those principles then the code will loose those 
properties or just not work. I do listen to people needing I18N support,
but I do not see any reason whatsoever why this should justify to change
the working of XSLT, or any given implementation.
i  You're doing code based on XSLT, this code breaks because it does things
XSLT specifically called against, change your code to work within the
constraints of the language you decided to use, or change your language.
Code not following the standards have very restricted usefulness and shorter
lifetime, don't do that, it's not just a libxslt issue, this will break 
with other implementations I'm sure.

Daniel
Comment 18 jirka 2004-06-26 19:59:28 UTC
Daniel, I can fully understand your position and decision. OTOH xsltproc claims 
support for XSLT 1.0 and some of EXSLT extensions. And I think that my code is 
perfectly legal within the scope of XSLT + EXSLT. (Of course and unfortunately 
this scope is sometimes quite fuzzy.)

Jirka
Comment 19 Daniel Veillard 2004-06-27 16:27:04 UTC
So you don't want to change your code to make it portable and prefer
to rely on unportable constructs ?

Daniel
Comment 20 jirka 2004-07-02 12:27:06 UTC
Hi Daniel,

it is not matter of willing but technical problem. I tried to implement 
internationalized index term grouping without keys. It will be slower than with 
keys, but at the end it could work in xsltproc. I removed all keys dependant on 
variables from my code, but I still must use node-set() function. I think that 
resulting code should work, but instead I'm getting following strange messages 
from xsltproc:

output error : string is not in UTF-8
output error : invalid character value
output error : invalid character value
xmlEscapeEntities : char out of range
xmlEscapeEntities : char out of range
output error : string is not in UTF-8
xmlEscapeEntities : char out of range

After waiting three hours for finishing transformation I interrupted xsltproc 
manually. Do you have any idea why and when such error messages can occur?
I want to get new indexing code work under xsltproc, but I can't debug xsltproc 
code. Any ideas?
Comment 21 Daniel Veillard 2004-07-02 12:53:44 UTC
Basically it's in libxml2, the function(s) taking an internal (DOM) tree
and doing the serialization complains because string in the tree content
are not UTF-8. It's is one of the key implementation choices made in
libxml2, once parsed, all text in trees must be UTF-8 (so to have a single
representation internally and a single coherent set of APIs using only
UTF-8). This error means libxml2 found a violation of this constraint
at saving time. This can be caused by using DOM API to modify trees with
non-UTF-8 string or raw internal memory corruption.
The problem is that if it takes 3 hours, nobody is gonna use it anyway, no ?
I appreciate your willingness to work around the initial problem ! I hope
we will get to a working stage.

Daniel
Comment 22 jirka 2004-07-02 13:06:31 UTC
So, in that case I probably hit another issue in xsltproc -- I'm not using DOM 
API directly, I'm just using XSLT transformation and both input XML document and 
XSLT stylesheet are well-formed so UTF-8 error must be caused by xsltproc.

I think that speed problem is related to above UTF-8 or other issue. Same code 
successfully finishes within few seconds in Saxon.

I will try to isolate the problem, but it's quite hard to extract something 
simple and short from DocBook stylesheets :-(
I'm going to vacations for next 10 days, so don't expect news from me in a very 
near future.

Jirka
Comment 23 Daniel Veillard 2004-07-02 13:13:05 UTC
Okay, have nice vacations !

Daniel
Comment 24 Daniel Veillard 2004-08-22 20:55:37 UTC
what's the current status ? We just released 1.1.9 

Daniel
Comment 25 jirka 2004-09-06 21:17:37 UTC
Thanks Daniel, my node-set() problem is away in 1.1.9.

I run some test on stylesheet that performs internationalized indexing without 
using keys. Transformation now finishes but it is *very* slow -- it tooks almost 
4 minutes to finish transformation of 400K DocBook document (the same stylesheet 
processed with Saxon finishes in 28 seconds). This is too slow to be used in any 
real word application.

For that reason I would appreciate if you could reappraise your decission to not 
support references to global variables/parameters inside user defined functions 
that are used inside xsl:key definition. xsl:keys can significantly boost speed 
of grouping. The same document processed with stylesheet that uses xsl:keys to 
get internationalized index tooks 8 seconds in Saxon and 10 in Xalan.

Moreover I think that users appreciate if different implementations of same 
language give same results. In this particular case libxslt behaves differently 
from both Xalan and Saxon. I think that this situation when three most used XSLT 
processors are not compatible is very frustrating for XSLT developers.
Comment 26 Daniel Veillard 2004-09-06 23:17:54 UTC
Get on the mailing-list and try to sell your case ! And please join the list
before posting there.
I still think it's an abuse of the spec and a request for a non-conformance
deviation, i.e. I have near zero inclination to work on it myself. Someone
else might find this worthwhile. I still think this deviation should not
be shipped as a requirement for DocBook stylesheets. And I'm not seeing
kindly pressure like "product A and B implement deviations from the spec,
so you should do so". If you really believe it's an error you should work
publicly to get the XSLT Working Group to release an errata on XSLT-1.0
instead, try to be logical,

Daniel