After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 611807 - unclean regression tests
unclean regression tests
Status: RESOLVED FIXED
Product: libxml2
Classification: Platform
Component: general
2.7.6
Other Linux
: Normal normal
: ---
Assigned To: Daniel Veillard
libxml QA maintainers
Depends on:
Blocks:
 
 
Reported: 2010-03-04 15:16 UTC by Mark Overmeer
Modified: 2016-05-22 02:02 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
configuration and test output (10.34 KB, text/plain)
2010-03-04 15:16 UTC, Mark Overmeer
  Details
Implement "runtest -u" mode (1.30 KB, patch)
2016-04-05 00:15 UTC, David Kilzer
none Details | Review
Implement "runtest -u" mode v2 (3.55 KB, patch)
2016-04-05 02:42 UTC, David Kilzer
none Details | Review

Description Mark Overmeer 2010-03-04 15:16:39 UTC
Created attachment 155232 [details]
configuration and test output

Various errors where reported, with components disabled. The weirdest is the complaint the "xpath" is disabled where that is not true. For some output, it is unclear whether it means: "successful" or "broken".

See attachment.
Comment 1 Daniel Veillard 2010-03-10 14:19:45 UTC
Use "make check" not "make tests"

the only problem raised is that   --without-valid
makes the following fail

## Error cases regression tests
Error for ./test/errors/attr3.xml failed
File ./test/errors/attr3.xml generated an error
## Error cases stream regression tests
Error for ./test/errors/attr3.xml failed
File ./test/errors/attr3.xml generated an error

since it tests a validity warning

Daniel
Comment 2 David Kilzer 2016-04-04 22:34:15 UTC
(In reply to Daniel Veillard from comment #1)
> Use "make check" not "make tests"

What's the correct way to generate new test results for "make check" when adding a test case?

I really want a "./runtests -u" mode that simply writes out new output.
Comment 3 David Kilzer 2016-04-05 00:15:06 UTC
Created attachment 325392 [details] [review]
Implement "runtest -u" mode

This patch implements a really dumb (always overwrites existing results) "runtest -u" mode that makes it much easier to create baselines for new tests.

This is much easier than using "make tests", which doesn't match "make check" output anyway.
Comment 4 David Kilzer 2016-04-05 02:42:48 UTC
Created attachment 325397 [details] [review]
Implement "runtest -u" mode v2

Changes since implement-runtest-update.diff:
- Writes out new file when calling compareFiles().
- Creates output files if they don't exist.

Does not:
- Attempt to delete unneeded output files.
- Skip writing output files if the content already matches.
Comment 5 Daniel Veillard 2016-05-22 02:02:33 UTC
This makes a lot of sense, very good idea !
Pushed as 

https://git.gnome.org/browse/libxml2/commit/?id=5c37382f0fe298d946f4dad0b4c3626fbd8591e2

  thanks a lot !

Daniel