After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 657456 - Folks test output not terribly useful
Folks test output not terribly useful
Status: RESOLVED FIXED
Product: folks
Classification: Platform
Component: general
git master
Other Linux
: Normal normal
: Unset
Assigned To: Jeremy Whiting
folks-maint
Depends on:
Blocks:
 
 
Reported: 2011-08-26 17:11 UTC by Travis Reitter
Modified: 2012-07-12 20:30 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
capture unit test output like tp-glib does (1.49 KB, patch)
2012-07-10 20:26 UTC, Jeremy Whiting
reviewed Details | Review

Description Travis Reitter 2011-08-26 17:11:31 UTC
See if you can spot which test failed in this list by visually scanning:

/IndividualRetrieval/singleton individuals: OK
PASS: individual-retrieval
/NameDetailsInterfaceTests/test name details interface: OK
PASS: name-details-interface
/AvatarDetailsInterfaceTests/test avatar details interface: OK
PASS: avatar-details-interface
/ImDetailsInterfaceTests/test im details interface: OK
PASS: im-details-interface
/PhoneDetailsInterfaceTests/test phone details interface: OK
PASS: phone-details-interface
/EmailDetailsInterfaceTests/test email details interface: OK
PASS: email-details-interface
/UrlDetailsInterfaceTests/test url details interface: OK
PASS: url-details-interface
/FavouriteDetailsInterfaceTests/test favourite details interface: OK
PASS: favourite-details-interface
/FullnameUpdates/fullname updates: OK
PASS: fullname-updates
/AddContactTests/test adding contacts : OK
PASS: add-contact
/DefaultContactTests/test default contact: OK
PASS: default-contact
/RemoveContactTests/test removing contacts : OK
PASS: remove-contact
/NicknameUpdates/nickname updates: OK
PASS: nickname-updates
/FamilyNameUpdates/family name updates: OK
PASS: family-name-updates
/GivenNameUpdates/given name updates: OK
PASS: given-name-updates
/AdditionalNamesUpdates/additional names updates: OK
PASS: additional-names-updates
/PrefixNameUpdates/prefix name updates: OK
PASS: prefix-name-updates
/SuffixNameUpdates/suffix name updates: OK
PASS: suffix-name-updates
/WebsiteUpdates/websites updates: OK
PASS: website-updates
/FavouriteUpdates/favourite update: OK
PASS: favourite-updates
/EmailsUpdates/emails updates: Aborted
FAIL: emails-updates
/PhonesUpdates/phones updates: OK
PASS: phones-updates
/IMAddressesUpdates/im addresses updates: OK
PASS: imaddresses-updates
/RoleDetailsInterfaceTests/test role details interface: OK
PASS: role-details-interface
/BirthdayDetailsInterfaceTests/test birthday details interface: OK
PASS: birthday-details-interface
/BirthdayUpdates/birthday updates: OK
PASS: birthday-updates
/NoteDetailsInterfaceTests/test note details interface: OK
PASS: note-details-interface
/GenderDetailsInterfaceTests/test gender details interface: OK
PASS: gender-details-interface
/PostalAddressDetailsInterfaceTests/test postal address details interface: OK
PASS: postal-address-details-interface
/AvatarUpdates/avatar updates: OK
PASS: avatar-updates
/AddPersonaTests/test adding personas to Tracker : OK
PASS: add-persona
/RemovePersonaTests/test adding personas to Tracker : OK
PASS: remove-persona
/SetNicknameTests/test setting nickname : OK
PASS: set-nickname
/SetFavouriteTests/test setting favourite : OK
PASS: set-favourite
/SetPhonesTests/test setting phones : OK
PASS: set-phones
/SetEmailsTests/test setting emails : OK
PASS: set-emails
/SetAvatarTests/test setting avatar : OK
PASS: set-avatar
/SetStructuredNameTests/test setting structured name : OK
PASS: set-structured-name
/SetFullNameTests/test setting structured name : OK
PASS: set-full-name
/SetURLsTests/test setting urls : OK
PASS: set-urls
/SetIMAddressesTests/test setting im_addresses : OK
PASS: set-im-addresses
/SetPostalAddressesTests/test setting postal addresses : OK
PASS: set-postal-addresses
/SetRolesTests/test setting roles : OK
PASS: set-roles
/SetNotesTests/test setting notes : OK
PASS: set-notes
/SetBirthdayTests/test setting bithday : OK
PASS: set-birthday
/SetGenderTests/test setting gender : OK
PASS: set-gender
/LinkPersonasTests/test linking personas: OK
PASS: link-personas
/DuplicatedEmailsTests/test adding 2 personas with the same email address : OK
PASS: duplicated-emails
/DuplicatedPhonesTests/test adding 2 personas with the same phone number : OK
PASS: duplicated-phones
/LinkPersonasViaLocalIDsTests/test linking personas: OK
PASS: link-personas-via-local-ids
/MatchIMAddressesTests/test potential match by IM addresses : OK
PASS: match-im-addresses
/MatchEmailAddressesTests/test potential match with same email addresses : OK
PASS: match-email-addresses
/MatchKnownEmailsTests/test potential match with same email addresses : OK
PASS: match-known-emails
/MatchPhoneNumberTests/test potential match with phone numbers : OK
PASS: match-phone-number
/MatchNameTests/test potential match by name #1 : OK
/MatchNameTests/test potential match by name #2 : OK
/MatchNameTests/test potential match by name #3 : OK
PASS: match-name
/MatchAllTests/test potential match all : OK
PASS: match-all
/SetDuplicateEmailTests/test re-setting an existing e-mail address: OK
PASS: set-duplicate-email
/SetNullAvatarTests/test setting null avatar : OK
PASS: set-null-avatar
====================================================================
1 of 58 tests failed
Please report to https://bugzilla.gnome.org/browse.cgi?product=folks
====================================================================

This output is mostly useless:
  * it's hard to spot which tests failed
  * who cares about which tests passed? (the "PASS" and preceding lines are just noise)

We should just summarize the test results and expand upon which tests fail. Use of bold to highlight important details would be very nice, as would color (in an a11y-friendly, aesthetically-pleasing way).
Comment 1 Raul Gutierrez Segales 2011-09-08 15:55:30 UTC
We could enable colors by:

diff --git a/configure.ac b/configure.ac
index d46b61f..a902ba1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -43,7 +43,7 @@ AC_CONFIG_SRCDIR([Makefile.am])
 AC_CONFIG_HEADERS(config.h)
 AC_CONFIG_SRCDIR([configure.ac])
 AM_INIT_AUTOMAKE([1.11 dist-bzip2 no-define
-                  no-dist-gzip tar-ustar -Wno-portability])
+                  no-dist-gzip tar-ustar -Wno-portability color-tests])
 AM_MAINTAINER_MODE
 
 AC_PROG_CC

I've tried cutting out output as tp-glib does it, but I ended up with non-function make some-test.gdb runs. Gotta to look into it.
Comment 2 Jeremy Whiting 2012-07-10 18:23:40 UTC
Adding color-tests as Raul mentioned does make it much more readable.  I think the PASS is still important to see but it's very useful to quickly find the red FAIL messages also.
Comment 3 Jeremy Whiting 2012-07-10 20:26:14 UTC
Created attachment 218469 [details] [review]
capture unit test output like tp-glib does

This patch makes PASS green, and FAIL red.  It also captures test output and cats the captured output on test failure or if tests are skipped just like tp-glib unit tests.
Comment 4 Philip Withnall 2012-07-11 19:34:27 UTC
Review of attachment 218469 [details] [review]:

Looks OK to me, but we might want to investigate http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/Parallel-Test-Harness.html#Parallel-Test-Harness instead of doing our own log redirection and detection of skipped tests.
Comment 5 Philip Withnall 2012-07-11 19:36:00 UTC
(In reply to comment #4)
> Looks OK to me, but we might want to investigate
> http://www.gnu.org/savannah-checkouts/gnu/automake/manual/html_node/Parallel-Test-Harness.html#Parallel-Test-Harness
> instead of doing our own log redirection and detection of skipped tests.

That’s not to say that this patch can’t be merged in the meantime, though…