GNOME Bugzilla – Bug 525592
Provide 'diff' like output for regression test failures
Last modified: 2008-04-19 17:01:28 UTC
See http://bugzilla.gnome.org/show_bug.cgi?id=523451#c7. See also http://docs.python.org/lib/differ-examples.html where difflib.Differ might be useful in getting us there.
Created attachment 108795 [details] [review] Revision #1. Here's a first pass at this. It's currently written to try to import difflib. If that succeeds, then it'll generate diffs of the EXPECTED results and the ACTUAL results. If the import fails, it falls back to generating what is currently there (EXPECTED results followed by ACTUAL results). Here's what the results now look like for role_label.py: $ ./runone.sh ../keystrokes/gtk-demo/role_label.py gtk-demo 0 starting test application gtk-demo ... Test 1 of 9 SUCCEEDED: ../keystrokes/gtk-demo/role_label.py:This message box label Test 2 of 9 SUCCEEDED: ../keystrokes/gtk-demo/role_label.py:This message box label Where Am I Test 3 of 9 SUCCEEDED: ../keystrokes/gtk-demo/role_label.py:This message box label Extended Where Am I Test 4 of 9 SUCCEEDED: ../keystrokes/gtk-demo/role_label.py:This message box label caret movement to 'h' Test 5 of 9 FAILED: ../keystrokes/gtk-demo/role_label.py:This message box label caret select 'his' of 'This' DIFFERENCES FOUND: - BUG? - no selection is announced? + BRAILLE LINE: 'gtk-demo Application Information Alert This message box has been popped up the following $l' + VISIBLE: 'This message box has been popped', cursor=2 [FAILURE WAS UNEXPECTED] Test 6 of 9 SUCCEEDED: ../keystrokes/gtk-demo/role_label.py:This message box label caret selection Where Am I Test 7 of 9 SUCCEEDED: ../keystrokes/gtk-demo/role_label.py:This message box label caret selection Extended Where Am I Test 8 of 9 SUCCEEDED: ../keystrokes/gtk-demo/role_label.py:This message box label caret select 'T' in 'This' Test 9 of 9 SUCCEEDED: ../keystrokes/gtk-demo/role_label.py:This message box label caret select rest of 'This' SUMMARY: 8 SUCCEEDED and 1 FAILED (1 UNEXPECTED) of 9 for ../keystrokes/gtk-demo/role_label.py /usr/bin/orca: line 92: 11393 Killed /usr/bin/python -c "import orca.orca; orca.orca.main()" "$ARGS" ./runone.sh: line 187: 11400 Killed $APP_NAME $ARGS $PARAMS $
(In reply to comment #1) > Created an attachment (id=108795) [edit] > Revision #1. I think this looks great. Feel free to check it in. :-)
Changed committed to SVN trunk. Moving to '[pending]'. Thanks.
I'm not sure I'm the best one to verify this one.
(In reply to comment #4) > I'm not sure I'm the best one to verify this one. > I'll do the honors then as I've been using it quite a bit. :-) It's definitely an improvement. Thanks Rich!