I can't reproduce that one with valgrind.
If you build it from git head, can you stop in gdb and tell me at this point
what is the value of len, and give the stack trace again because the lines
in HTMLparser.c don't match git head,

  thanks,

Daniel

len is 1 directly before the oob happens.

valgrind is not capable of catching stack oob reads, only heap as far as I know.

Stack trace line numbers are correct for git head as far as I can see. To confirm, in HTMLparser.c this is 5730:
					ctxt->sax->characters(
						ctxt->userData, &cur, 1);
And this is 6097:
    htmlParseTryOrFinish(ctxt, terminate);

 Sorry git head master say line 5730 of HTMLparser.c is in the comment of

---------------------
               if ((xmlStrEqual(ctxt->name, BAD_CAST"script")) ||
                    (xmlStrEqual(ctxt->name, BAD_CAST"style"))) {
                    /*
                     * Handle SCRIPT/STYLE separately
                     */
--------------------

 so no your report is not w.r.t. git head
Can you try to patch I attaching ?

  thanks,

Daniel

Created attachment 307206 [details] [review]
potential patch for the issue

I'm really confused now, I re-checked. Are we looking into different git-repos?
I checked out this:
git clone --depth=1 https://git.gnome.org/browse/libxml2

I re-tried with the git URl in case https is doing something strange, same result:
git clone --depth=1 git://git.gnome.org/libxml2

Am I using the wrong repo?

Regarding your patch: Produces hunks (as we're probably looking at different repos for whatever reason), but it fixes the issue. Thanks!

Haha, I know I hadn't update my repo on that machine and I pushed updates to
HTMLparser.c last week from another machine :-)

If it fixes the issue I will push after a bit more checking on my side.

Daniel

Additional info, I was a bit too fast: While your patch fixes the issue originally reported another one pops up later in the test suite.

Line numbers are current git + your patch :-)

[...]
## Parsing non-recursive test cases
## Parsing non-recursive huge case
.........
Total 9 tests, no errors
Testing HTMLparser : 32 of 38 functions ...
=================================================================
==25757==ERROR: AddressSanitizer: global-buffer-overflow on address 0x00000047b300 at pc 0x7f22de2fad23 bp 0x7ffd04a1c080 sp 0x7ffd04a1c070
READ of size 122 at 0x00000047b300 thread T0
    #0 0x7f22de2fad22 in xmlBufAdd /mnt/ram/libxml2/buf.c:908
    #1 0x7f22de1db32e in xmlParserInputBufferCreateMem__internal_alias /mnt/ram/libxml2/xmlIO.c:3038
    #2 0x7f22de2333d3 in htmlCreateMemoryParserCtxt__internal_alias /mnt/ram/libxml2/HTMLparser.c:4971
    #3 0x41d361 in test_htmlCreateMemoryParserCtxt /mnt/ram/libxml2/testapi.c:1484
    #4 0x41fd81 in test_HTMLparser /mnt/ram/libxml2/testapi.c:2784
    #5 0x41cc43 in testlibxml2 /mnt/ram/libxml2/testapi.c:1249
    #6 0x41b343 in main /mnt/ram/libxml2/testapi.c:154
    #7 0x7f22dd3c4f9f in __libc_start_main (/lib64/libc.so.6+0x1ff9f)
    #8 0x41a948 (/mnt/ram/libxml2/.libs/testapi+0x41a948)

0x00000047b304 is located 0 bytes to the right of global variable '*.LC12' from 'testapi.c' (0x47b300) of size 4
  '*.LC12' is ascii string 'foo'
SUMMARY: AddressSanitizer: global-buffer-overflow /mnt/ram/libxml2/buf.c:908 xmlBufAdd
Shadow bytes around the buggy address:
  0x000080087610: 00 02 f9 f9 f9 f9 f9 f9 00 00 00 00 00 00 00 00
  0x000080087620: 00 00 00 01 f9 f9 f9 f9 00 00 00 00 06 f9 f9 f9
  0x000080087630: f9 f9 f9 f9 04 f9 f9 f9 f9 f9 f9 f9 00 04 f9 f9
  0x000080087640: f9 f9 f9 f9 00 03 f9 f9 f9 f9 f9 f9 03 f9 f9 f9
  0x000080087650: f9 f9 f9 f9 00 00 00 00 06 f9 f9 f9 f9 f9 f9 f9
=>0x000080087660:[04]f9 f9 f9 f9 f9 f9 f9 07 f9 f9 f9 f9 f9 f9 f9
  0x000080087670: 00 02 f9 f9 f9 f9 f9 f9 03 f9 f9 f9 f9 f9 f9 f9
  0x000080087680: 00 01 f9 f9 f9 f9 f9 f9 05 f9 f9 f9 f9 f9 f9 f9
  0x000080087690: 06 f9 f9 f9 f9 f9 f9 f9 00 04 f9 f9 f9 f9 f9 f9
  0x0000800876a0: 00 00 00 00 f9 f9 f9 f9 00 00 00 00 02 f9 f9 f9
  0x0000800876b0: f9 f9 f9 f9 00 00 02 f9 f9 f9 f9 f9 00 00 01 f9
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Contiguous container OOB:fc
  ASan internal:           fe
==25757==ABORTING

I doubt it's related, you didn't even called htmlParseTryOrFinish()
with that context and that's the only function modified by the patch.

Daniel

I don't think it's related, just wanted to mention it. Without the patch the test suite just didn't go that far. Should I open a new bug for it?

nahh, that's fine :-) just wanted to make sure
compiling libxml2 with address sanitizer: just give the CFLAGS and compile say on
Fedora 22 works ? If yes I might just add a target next to 'make valgrind'

Daniel

Yeah, that should work. Just need to add it to ldflags as well, because asan is using a library provided by gcc.
On older gcc's (before 4.9) the output is not as clear (there's a script to decode it), but it should still work catching the bugs.
(you can also add ubsan, you may want to have a look at my tutorial here: https://fuzzing-project.org/tutorial-cflags.html )

Just noted that you made a new release, unfortunately the last issue is still unfixed (global out of bounds read in xmlBufAdd).

https://bugzilla.gnome.org/show_bug.cgi?id=758572

The test suite should run fine with ASan now.