GNOME Bugzilla – Bug 122619
Some regression tests failed under IRIX64 for libxml2-2.5.9
Last modified: 2009-08-15 18:40:50 UTC
I am unsing the MIPSPro compiling system 7.3.1.3m and compiled the libxml2 in -n32 mode which is the default under IRIX. Running the regression tests I observed the following failures: 1st) ## XML regression tests Testing utf16bom.xml Binary files ./result/utf16bom.xml and result.utf16bom.xml differ result.utf16bom.xml:1: error: Document is empty ^ result.utf16bom.xml:1: error: Start tag expected, '<' not found ^ Binary files result.utf16bom.xml and result2.utf16bom.xml differ 2nd) ## XML push regression tests . . Testing utf16bom.xml Binary files ./result/utf16bom.xml and result.utf16bom.xml differ result.utf16bom.xml:1: error: Document is empty ^ Binary files result.utf16bom.xml and result2.utf16bom.xml differ 3rd) ## XML regression tests on memory Testing utf16bom.xml Binary files ./result/utf16bom.xml and result.utf16bom.xml differ Entity: line 1: error: Document is empty ^ Entity: line 1: error: Start tag expected, '<' not found ^ Binary files result.utf16bom.xml and result2.utf16bom.xml differ 4th) ## XML entity subst regression tests Testing utf16bom.xml Binary files ./result/noent/utf16bom.xml and result.utf16bom.xml differ result.utf16bom.xml:1: error: Document is empty ^ result.utf16bom.xml:1: error: Start tag expected, '<' not found ^ Binary files result.utf16bom.xml and result2.utf16bom.xml differ 5th) ## Schemas regression tests Testing length3_0_0 1c1 < compilation error --- > compilation error: file ./test/schemas/length3_0.xsd line 5 element xs:element 6th) Testing po1_0_0 1d0 < ./test/schemas/po1_0.xml fails to validate 0a1 > compilation error: file ./test/schemas/po1_0.xsd line 20 element xsd:element 1a3 > compilation error: file ./test/schemas/po1_0.xsd line 48 element xsd:element 2a5 > compilation error: file ./test/schemas/po1_0.xsd line 19 element xsd:element 3a7 > compilation error: file ./test/schemas/po1_0.xsd line 51 element xsd:attribute5d8 < Element purchaseOrder content check failed 7th) Testing docbook_0 0a1,6 > ./test/relaxng/docbook_0.xml:16: warning: failed to load external entity "test/dtd/4.1.2/docbookx.dtd" > ]> > ^ > ./test/relaxng/docbook_0.xml:1864: error: Entity 'copy' not defined > <sgmltag>&copy;</sgmltag> — copyright sign (©) > ^ 8th) ## Relax-NG streaming regression tests Testing docbook_0 1c1,3 < ./test/relaxng/docbook_0.xml validates --- > Expand failed ! > ./test/relaxng/docbook_0.xml fails to validate > ./test/relaxng/docbook_0.xml : failed to parse 0a1,69 > ./test/relaxng/docbook_0.xml:16: warning: failed to load external entity "test/dtd/4.1.2/docbookx.dtd" > ]> > ^ > ./test/relaxng/docbook_0.xml:1760: error: Entity 'copy' not defined > <sgmltag>&copy;</sgmltag> — copyright sign (©) > ^ > RNG validity error: file ./test/relaxng/docbook_0.xml line 1763 element text > Invalid attribute > for element itemizedlist > RNG validity error: element > Element ^P1\236Xtag has extra content: keys > RNG validity error: file ./test/relaxng/docbook_0.xml line 1752 element text > Invalid attribute > for element sgmltag > RNG validity error: file ./test/relaxng/docbook_0.xml line 1754 element text > Invalid attribute > for element sgmltag > RNG validity error: file ./test/relaxng/docbook_0.xml line 1738 element sect3 > Invalid attribute id for element sect3 > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect element emphasis there > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect element sgmltag there > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect element para there > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect element itemizedlist there > RNG validity error > Did not expect element listitem there > RNG validity error > Did not expect element para there > RNG validity error > Did not expect element sgmltag there > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect element listitem there > RNG validity error > Did not expect element para there > RNG validity error > Did not expect element sgmltag there > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect element listitem there > RNG validity error > Did not expect element para there > RNG validity error > Did not expect element sgmltag there > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect element listitem there > RNG validity error > Did not expect element para there > RNG validity error > Did not expect element sgmltag there 9th) Testing tutor10_1_4 1,2c1,2 < RNG validity error: file ./test/relaxng/tutor10_1_4.xml line 1 element foo < Expecting a namespace for element foo --- > RNG validity error > Did not expect element foo there Testing tutor10_1_5 1,2c1,2 < RNG validity error: file ./test/relaxng/tutor10_1_5.xml line 1 element foo < Element foo has wrong namespace: expecting http://www.example.com --- > RNG validity error > Did not expect element foo there Testing tutor10_1_6 1,2c1,2 < RNG validity error: file ./test/relaxng/tutor10_1_6.xml line 1 element foo < Element foo has wrong namespace: expecting http://www.example.com --- > RNG validity error > Did not expect element foo there 10th) Testing tutor10_2_3 1,2c1,2 < RNG validity error: file ./test/relaxng/tutor10_2_3.xml line 1 element foo < Expecting no namespace for element foo --- > RNG validity error > Did not expect element foo there Testing tutor10_2_4 1,2c1,2 < RNG validity error: file ./test/relaxng/tutor10_2_4.xml line 1 element foo < Expecting no namespace for element foo --- > RNG validity error > Did not expect element foo there 11th) Testing tutor3_2_1 1c1 < RNG validity error: file ./test/relaxng/tutor3_2_1.xml line 1 element email --- > RNG validity error 2a3,8 > RNG validity error > Did not expect text in element TODO content > RNG validity error > Did not expect element name there > RNG validity error > Did not expect text in element TODO content
Please update to the latest version first. Use 2.5.11 or test/2.6.0beta1 . The UTF16 bom problems are not normal, though that may have been fixed. The W3C Schemas errors are normal... well sort of The non-streaming Relax-NG validation error are not normal Some divergence in the error messages between streaming and standard Relax-NG validation are normal. I can't make use of your report as is... Daniel
What do you need to make the report usable for you? Let me know how to assist you. Reiner
Well first it should be upgraded to the latest version, and then I would need some analysis of why the BOM tests fails, if they still occurs. Or even a patch if you manage to get to a complete analysis. A bug I can't reproduce, I usually can't debug and fix, Daniel
Okay, did a upgrade to the latest libxml2 2.5.11. The BOM tests still fail. I did the following things manually: ../xmllint utf16bom.xml > result.utf16bom.xml diff ../result/utf16bom.xml result.utf16bom.xml Binary files ../result/utf16bom.xml and result.utf16bom.xml differ ../xmllint result.utf16bom.xml > result2.utf16bom.xml result.utf16bom.xml:1: error: Document is empty ^ result.utf16bom.xml:1: error: Start tag expected, '<' not found ^ In deed the very first entry or line in utf16bom.xml was missing in result.utf16bom.xml Question: Was the binary information in utf16bom.xml generated on a little endian machine? A MIPS-Irix machine is big endian. Or is the endianess unimportant for that test?
I forgot to give you the output of od utf16bom.xml under Irix and on a Linux Intel Pentium III box: Irix: reiner@dcm23 148> od utf16bom.xml 0000000 177776 036000 037400 074000 066400 066000 020000 073000 0000020 062400 071000 071400 064400 067400 067000 036400 021000 0000040 030400 027000 030000 021000 020000 062400 067000 061400 0000060 067400 062000 064400 067000 063400 036400 021000 052400 0000100 052000 043000 026400 030400 033000 021000 020000 071400 0000120 072000 060400 067000 062000 060400 066000 067400 067000 0000140 062400 036400 021000 074400 062400 071400 021000 037400 0000160 037000 005000 036000 071000 062400 070000 067400 071400 0000200 064400 072000 067400 071000 074400 020000 071000 062400 0000220 070000 067400 071400 064400 072000 071000 067400 074400 0000240 057400 064400 062000 036400 021000 072000 062400 071400 0000260 072000 021000 037000 036000 067000 060400 066400 062400 0000300 071400 070000 060400 061400 062400 020000 067000 060400 0000320 066400 062400 036400 021000 072000 062400 071400 072000 0000340 021000 027400 037000 036000 027400 071000 062400 070000 0000360 067400 071400 064400 072000 067400 071000 074400 037000 0000400 Linux: reiner@zam160:~/tmp> od utf16bom.xml 0000000 177377 000074 000077 000170 000155 000154 000040 000166 0000020 000145 000162 000163 000151 000157 000156 000075 000042 0000040 000061 000056 000060 000042 000040 000145 000156 000143 0000060 000157 000144 000151 000156 000147 000075 000042 000125 0000100 000124 000106 000055 000061 000066 000042 000040 000163 0000120 000164 000141 000156 000144 000141 000154 000157 000156 0000140 000145 000075 000042 000171 000145 000163 000042 000077 0000160 000076 000012 000074 000162 000145 000160 000157 000163 0000200 000151 000164 000157 000162 000171 000040 000162 000145 0000220 000160 000157 000163 000151 000164 000162 000157 000171 0000240 000137 000151 000144 000075 000042 000164 000145 000163 0000260 000164 000042 000076 000074 000156 000141 000155 000145 0000300 000163 000160 000141 000143 000145 000040 000156 000141 0000320 000155 000145 000075 000042 000164 000145 000163 000164 0000340 000042 000057 000076 000074 000057 000162 000145 000160 0000360 000157 000163 000151 000164 000157 000162 000171 000076 0000400 So you may recognize that the outputs differ
utf16bom.xml result should be machine independant by definition. I'm sorry but the right way to attack this is with a debugger and I can't do that ! Daniel
Just for completeness I am adding the comments by Dimitri Papadopoulos-Orfanos <papadopo@shfj.cea.fr> Yes, I can reproduce the problem on these platforms: IRIX 6.5.18 MIPS MIPSpro 7.3.1.3m Solaris 8 6.5.18 SPARC Sun ONE Studio 7 but not on this one: Red Hat Linux 9 i686 Red Hat Linux gcc 3.2.2-5 Therefore this is indeed related to endianess. The test file starts with an endianess marker FFFE: $ ad test/utf16bom.xml 00000000: FFFE 3C00 3F00 7800 6D00 6C00 2000 7600 ..<.?.x.m.l. .v. On MIPS and SPARC platforms the output file lacks an endianess marker: $ ./xmllint test/utf16bom.xml > foobar.xml $ ad foobar.xml 00000000: 003C 003F 0078 006D 006C 0020 0076 0065 .<.?.x.m.l. .v.e On the Intel platform the output file does have an endianess marker: $ ./xmllint test/utf16bom.xml > foobar.xml $ ad foobar.xml 00000000: FFFE 3C00 3F00 7800 6D00 6C00 2000 7600 ..<.?.x.m.l. .v. This is probably a regression in the libxml2 library. The library believes it doesn't need to insert an endianess marker in the output file because the output's file endianess is the same as the platform's endianess. This proves it's better to always insert an endianess marker. That said this isn't necessarily a bug, but I really suggest an endianess marker is always insrted in UTF16 files.
I find a couple of minor errors in the bigendian output within encoding.c, but nothing to explain the reported problem. In particular, there seems to be no way a BOM is not output by the routine. Are you possibly using iconv routines? I would be happy to pursue the problem, but (like Daniel) I don't have any access to bigendian architecture. If you can provide a login account with access to libxml2 source plus C-compiler, debugger and library, let me know by separate email.
Dear Bill, I guess that I am using iconv since configure found successfully iconv. I let you where you can run some tests. Nany thanks for your assistance Reiner
Testing on the reporter's system showed that the test failure was because the iconv in use produced a UTF-16BE text stream without a BOM. When re-configured --without-iconv the tests passed. It appears that, if an external iconv library produces UTF-16BE *with* a correct BOM, the test would also fail. This is because the test method uses a simple "diff" between the test program output and the "expected" result (which is UTF-16LE, with a correct BOM). I will try to implement something to take care of this scenario, but since it has (as yet) never happened it has a somewhat lower priority.
The libxml2 library has now been enhanced to, by default, produce "correct" UTF-16 output (BOM with UTF-16LE encoding) whether or not iconv is configured. See the ChangeLog, or xml mailing list, for further details. I am, therefore, closing this bug report.
So let's close this old bug, Daniel