After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 789558 - [ODS] Our ods files with objects fail to validate
[ODS] Our ods files with objects fail to validate
Status: RESOLVED OBSOLETE
Product: Gnumeric
Classification: Applications
Component: import/export OOo / OASIS
git master
Other All
: Normal normal
: ---
Assigned To: Andreas J. Guelzow
Jody Goldberg
Depends on: 789703
Blocks:
 
 
Reported: 2017-10-27 13:17 UTC by Morten Welinder
Modified: 2018-05-22 14:31 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Morten Welinder 2017-10-27 13:17:19 UTC
../test/t6150-ods-syntax.pl --corpus /obj/
-------------------------------------------------------------------------------
t6150-ods-syntax.pl: Check that the ods exporter produces valid files.
Checking ../samples/excel/objs.xls (strict conformance)
While checking content.xml from objs.ods:
| -:141: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:141: element forms: Relax-NG validity error : Did not expect element forms there
| -:141: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:141: element forms: Relax-NG validity error : Element table failed to validate content
| -:364: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:364: element forms: Relax-NG validity error : Did not expect element forms there
| -:364: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:364: element forms: Relax-NG validity error : Element table failed to validate content
| - fails to validate
Checking ../samples/excel/objs.xls (extended conformance)
While checking content.xml from objs.ods:
| -:141: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:141: element forms: Relax-NG validity error : Did not expect element forms there
| -:141: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:141: element forms: Relax-NG validity error : Element table failed to validate content
| -:367: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:367: element forms: Relax-NG validity error : Did not expect element forms there
| -:367: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:367: element forms: Relax-NG validity error : Element table failed to validate content
| - fails to validate
Checking ../samples/object-tests.gnumeric (strict conformance)
While checking content.xml from object-tests.ods:
| -:584: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:584: element forms: Relax-NG validity error : Did not expect element forms there
| -:584: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:584: element forms: Relax-NG validity error : Element table failed to validate content
| -:1272: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:1272: element forms: Relax-NG validity error : Did not expect element forms there
| -:1272: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:1272: element forms: Relax-NG validity error : Element table failed to validate content
| -:1446: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:1446: element forms: Relax-NG validity error : Did not expect element forms there
| -:1446: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:1446: element forms: Relax-NG validity error : Element table failed to validate content
| - fails to validate
Checking ../samples/object-tests.gnumeric (extended conformance)
While checking content.xml from object-tests.ods:
| -:584: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:584: element forms: Relax-NG validity error : Did not expect element forms there
| -:584: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:584: element forms: Relax-NG validity error : Element table failed to validate content
| -:1287: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:1287: element forms: Relax-NG validity error : Did not expect element forms there
| -:1287: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:1287: element forms: Relax-NG validity error : Element table failed to validate content
| -:1467: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:1467: element forms: Relax-NG validity error : Did not expect element forms there
| -:1467: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:1467: element forms: Relax-NG validity error : Element table failed to validate content
| - fails to validate
Fail
Comment 1 Andreas J. Guelzow 2017-10-28 00:27:22 UTC
I do not see this:

aguelzow@kirkman ~/git/gnumeric/src $ jhbuild run ../test/t6150-ods-syntax.pl download
-------------------------------------------------------------------------------
t6150-ods-syntax.pl: Check that the ods exporter produces valid files.
Downloading OpenDocument-v1.2-os-schema.rng...
# /usr/bin/curl -s -S -o ../test/ods-schema/OpenDocument-v1.2-os-schema.rng.tmp http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-schema.rng
Download ok.
Downloading OpenDocument-v1.2-os-manifest-schema.rng...
# /usr/bin/curl -s -S -o ../test/ods-schema/OpenDocument-v1.2-os-manifest-schema.rng.tmp http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-manifest-schema.rng
Download ok.
# cp ../test/ods-schema/OpenDocument-v1.2-os-schema.rng ../test/ods-schema/OpenDocument-v1.2-os-ext-schema.rng
# (cd ../test && patch -i ods-ext-schema.patch ods-schema/OpenDocument-v1.2-os-ext-schema.rng)
patching file ods-schema/OpenDocument-v1.2-os-ext-schema.rng
aguelzow@kirkman ~/git/gnumeric/src $ jhbuild run ../test/t6150-ods-syntax.pl --corpus /obj/
-------------------------------------------------------------------------------
t6150-ods-syntax.pl: Check that the ods exporter produces valid files.
Checking ../samples/excel/objs.xls (strict conformance)
Checking ../samples/excel/objs.xls (extended conformance)
Checking ../samples/object-tests.gnumeric (strict conformance)
Checking ../samples/object-tests.gnumeric (extended conformance)
Pass
aguelzow@kirkman ~/git/gnumeric/src $
Comment 2 Andreas J. Guelzow 2017-10-28 00:41:32 UTC
Looking a the content of content.xml after saving in extended format I see on line 140 to 143:

      <table:table table:name="Sheet1" table:style-name="ta-0" table:print="true">
        <office:forms form:automatic-focus="false" form:apply-design-mode="false">
          <form:form>
            <form:value-range xml:id="CTRL0000" form:id="CTRL0000" form:control-implementation="gnm:spinbutton" form:orientation="vertical" form:value="1" form:

In the schema we have:
	<define name="table-table">
		<element name="table:table">
			<ref name="table-table-attlist"/>
			<optional>
				<ref name="table-title"/>
			</optional>
			<optional>
				<ref name="table-desc"/>
			</optional>
			<optional>
				<ref name="table-table-source"/>
			</optional>
			<optional>
				<ref name="office-dde-source"/>
			</optional>
			<optional>
				<ref name="table-scenario"/>
			</optional>
			<optional>
				<ref name="office-forms"/>
			</optional>
			<optional>
				<ref name="table-shapes"/>
			</optional>
			<ref name="table-columns-and-groups"/>
			<ref name="table-rows-and-groups"/>
			<optional>
				<ref name="table-named-expressions"/>
			</optional>
			<optional>
				<ref name="gnm-selections"/>
			</optional>
		</element>
	</define>

So clearly

| -:141: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
| -:141: element forms: Relax-NG validity error : Did not expect element forms there
| -:141: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
| -:141: element forms: Relax-NG validity error : Element table failed to validate content

is not a valid error.
Comment 3 Morten Welinder 2017-10-28 01:01:03 UTC
Hmm...  doesn't happen at home for me.  I might have different files at work.
Here's what I have at home:

welinder@lenovo ~/gnome/gnumeric/test $ sha1sum ods-schema/OpenDocument-v1.2-os-*
e1d10cc874b694240cf49342026157b540601a2c  ods-schema/OpenDocument-v1.2-os-ext-schema.rng
661ab5bc695f9a8266e89cdf2747d8d76eacfedf  ods-schema/OpenDocument-v1.2-os-manifest-schema.rng
adc746cbb415ac3a17199442a15b38a5858fc7ef  ods-schema/OpenDocument-v1.2-os-schema.rng
Comment 4 Andreas J. Guelzow 2017-10-28 01:54:13 UTC
This appears to match what I have (and what I downloaded earlier today):

aguelzow@kirkman ~/git/gnumeric/test $ sha1sum ods-schema/OpenDocument-v1.2-os-*
e1d10cc874b694240cf49342026157b540601a2c  ods-schema/OpenDocument-v1.2-os-ext-schema.rng
661ab5bc695f9a8266e89cdf2747d8d76eacfedf  ods-schema/OpenDocument-v1.2-os-manifest-schema.rng
adc746cbb415ac3a17199442a15b38a5858fc7ef  ods-schema/OpenDocument-v1.2-os-schema.rng
Comment 5 Morten Welinder 2017-10-30 11:39:59 UTC
Same files at work; still fails.  Let's see if we generate the same files.



../src/ssconvert -T Gnumeric_OpenCalc:odf ../samples/excel/objs.xls /tmp/ooo.ods
welinder@dicentra:~/gnome-src/gnumeric/test> unzip -p /tmp/ooo.ods content.xml | xmllint --noout --relaxng ods-schema/OpenDocument-v1.2-os-ext-schema.rng  -
-:141: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
-:141: element forms: Relax-NG validity error : Did not expect element forms there
-:141: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
-:141: element forms: Relax-NG validity error : Element table failed to validate content
-:367: element forms: Relax-NG validity error : Expecting element table-column-group, got forms
-:367: element forms: Relax-NG validity error : Did not expect element forms there
-:367: element forms: Relax-NG validity error : Expecting element table-header-columns, got forms
-:367: element forms: Relax-NG validity error : Element table failed to validate content
- fails to validate
welinder@dicentra:~/gnome-src/gnumeric/test> unzip -p /tmp/ooo.ods content.xml | sha1sum 
614881f193d13ac05880e3d602d2f0a9e5888f8c  -
Comment 6 Andreas J. Guelzow 2017-10-30 15:30:55 UTC
aguelzow@bourbaki ~/git/gnumeric/test $ ../src/ssconvert -T Gnumeric_OpenCalc:odf ../samples/excel/objs.xls /tmp/ooo.ods
aguelzow@bourbaki ~/git/gnumeric/test $ unzip -p /tmp/ooo.ods content.xml | xmllint --noout --relaxng ods-schema/OpenDocument-v1.2-os-ext-schema.rng  -
- validates
aguelzow@bourbaki ~/git/gnumeric/test $ unzip -p /tmp/ooo.ods content.xml | sha1sum
614881f193d13ac05880e3d602d2f0a9e5888f8c  -
aguelzow@bourbaki ~/git/gnumeric/test $ 


aguelzow@bourbaki ~/git/gnumeric/test $ xmllint --version
xmllint: using libxml version 20901
   compiled with: Threads Tree Output Push Reader Patterns Writer SAXv1 FTP HTTP DTDValid HTML Legacy C14N Catalog XPath XPointer XInclude Iconv ISO8859X Unicode Regexps Automata Expr Schemas Schematron Modules Debug Zlib Lzma
Comment 7 Morten Welinder 2017-10-30 15:45:55 UTC
So: same grammar files, same xml file.

welinder@dicentra:~> xmllint --version
xmllint: using libxml version 20904
   compiled with: Threads Tree Output Push Reader Patterns Writer SAXv1 FTP HTTP DTDValid HTML Legacy C14N Catalog XPath XPointer XInclude Iconv ISO8859X Unicode Regexps Automata Expr Schemas Schematron Modules Debug Zlib Lzma 

Slightly different xmllint, but I am having a hard time believing that to
be the cause.  My version at home is likely to be even newer.

I am going to have to narrow down what it is complaining about.
Comment 8 Morten Welinder 2017-10-30 23:00:25 UTC
Home:

welinder@lenovo ~/gnome/gnumeric $ xmllint --version
xmllint: using libxml version 20903
   compiled with: Threads Tree Output Push Reader Patterns Writer SAXv1 FTP HTTP DTDValid HTML Legacy C14N Catalog XPath XPointer XInclude Iconv ISO8859X Unicode Regexps Automata Expr Schemas Schematron Modules Debug Zlib Lzma 


So the newest is actually the one that complains.
Comment 9 Morten Welinder 2017-10-31 00:35:18 UTC
Fails with libxml2 git HEAD.
Comment 10 Morten Welinder 2017-10-31 01:07:58 UTC
Bisections identifies cad102b861f74d56e3f6e710c466cf1a38a5db56 in libxml.
The relevant part is...

commit cad102b861f74d56e3f6e710c466cf1a38a5db56
Author: Audric Schiltknecht <audric.schiltknecht@viagenie.ca>
Date:   Fri Apr 15 22:41:24 2016 +0800

    Do normalize string-based datatype value in RelaxNG facet checking
    
    Original patch is from Jan Pokorný <jpokorny redhat com>
    https://mail.gnome.org/archives/xml/2013-November/msg00028.html
    
    Improve it according to reviews and add test files.

[...]

diff --git a/xmlschemastypes.c b/xmlschemastypes.c
index ff64f50..3c9514c 100644
--- a/xmlschemastypes.c
+++ b/xmlschemastypes.c
@@ -5315,7 +5315,15 @@ xmlSchemaValidateFacetInternal(xmlSchemaFacetPtr facet,
            */
            if (value == NULL)
                return(-1);
-           ret = (facet->regexp, value);
+           /*
+           * If string-derived type, regexp must be tested on the value space of
+           * the datatype.
+           * See https://www.w3.org/TR/xmlschema-2/#rf-pattern
+           */
+           const int stringType = val && ((val->type >= XML_SCHEMAS_STRING && val->type <= XML_SCHEMAS_NORMSTRING)
+                                           || (val->type >= XML_SCHEMAS_TOKEN && val->type <= XML_SCHEMAS_NCNAME));
+           ret = xmlRegexpExec(facet->regexp,
+                               (stringType && val->value.str) ? val->value.str : value);
            if (ret == 1)
                return(0);
            if (ret == 0)

It't not really obvious why we get hit by that.




However, with a little bit of debugging it turns out that
val->value.str) is "spinbutton" while value is "gnm:spinbutton".  The former
obviously does not match [^:]+:[^:]+
Comment 11 Morten Welinder 2017-10-31 01:13:11 UTC
Incidentally, "git bisect" is a stunningly effective tool.
Comment 12 Andreas J. Guelzow 2017-10-31 04:43:53 UTC
If I understand this correctly we should be able to work around this by patching the schema to change:
	<define name="namespacedToken">
		<data type="QName">
			<param name="pattern">[^:]+:[^:]+</param>
		</data>
	</define>
to
	<define name="namespacedToken">
		<data type="QName">
			<param name="pattern">.+</param>
		</data>
	</define>
In that case we just fail to verify that we have properly namespaced values in these cases.
Comment 13 Morten Welinder 2017-10-31 11:42:41 UTC
Correct -- that does work around the problem and it's an easy thing to do
for the extended format.  We use the raw schema for the non-extended case,
though, so it's a bit more work.

I'll try filing a bug with libxml2 and see if they think they are at fault
or if the schema is wrong.
Comment 14 Morten Welinder 2017-10-31 11:58:57 UTC
Filed bug 789703.  Let's see.
Comment 15 GNOME Infrastructure Team 2018-05-22 14:31:27 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to GNOME's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/gnumeric/issues/327.