GNOME Bugzilla – Bug 747830
gst-validate-launcher --sync performs tests when annex failed
Last modified: 2015-06-24 14:44:10 UTC
In systems without git annex installed, 'gst-validate-launcher --sync' fails to sync test medias but still performs tests which mostly fail, shows long messages that each tests failed, and takes time. It would be better if gst-validate-launcher can abort right after annex failed and doesn't show test fail messages.
Created attachment 301519 [details] [review] testsuites: Error out if the assets could not be synced
commit b0aa782694c053237c625af73765f42b8b5fa331 Author: Thibault Saunier <tsaunier@gnome.org> Date: Tue Apr 14 11:41:57 2015 +0200 testsuites: Error out if the assets could not be synced https://bugzilla.gnome.org/show_bug.cgi?id=747830 commit b0aa782694c053237c625af73765f42b8b5fa331 Author: Thibault Saunier <tsaunier@gnome.org> Date: Tue Apr 14 11:41:57 2015 +0200 testsuites: Error out if the assets could not be synced https://bugzilla.gnome.org/show_bug.cgi?id=747830
With this patch applied, testsuite becomes empty and running another 'gst-validate-launcher --sync' has no test to perform.
In main.py: tests_launcher.set_settings(options, []) ... tests_launcher.list_tests() If there's a way to check if testsuite is not loaded, then exit before list_tests()?
(In reply to Hyung Song from comment #3) > With this patch applied, testsuite becomes empty and running another > 'gst-validate-launcher --sync' has no test to perform. Isn't that the expected behaviour? If we can't sync we should just exit. (In reply to Hyung Song from comment #4) > In main.py: > > tests_launcher.set_settings(options, []) > ... > tests_launcher.list_tests() > > If there's a way to check if testsuite is not loaded, then exit before > list_tests()? What would that bring us? (from what I understand it would be a very minor optimization in the failure case)
(In reply to Thibault Saunier from comment #5) > (In reply to Hyung Song from comment #3) > > With this patch applied, testsuite becomes empty and running another > > 'gst-validate-launcher --sync' has no test to perform. > > Isn't that the expected behaviour? If we can't sync we should just exit. > Before exiting, message is printed for removing each test case and testsuite becomes empty. To rollback testsuite I'm removing gst-validate directory before 'gst-validate-launcher --sync'. > (In reply to Hyung Song from comment #4) > > In main.py: > > > > tests_launcher.set_settings(options, []) > > ... > > tests_launcher.list_tests() > > > > If there's a way to check if testsuite is not loaded, then exit before > > list_tests()? > > What would that bring us? (from what I understand it would be a very minor > optimization in the failure case) Removing test cases in the testsuite seems to happen in list_tests.
(In reply to Hyung Song from comment #6) > (In reply to Thibault Saunier from comment #5) > > (In reply to Hyung Song from comment #3) > > > With this patch applied, testsuite becomes empty and running another > > > 'gst-validate-launcher --sync' has no test to perform. > > > > Isn't that the expected behaviour? If we can't sync we should just exit. > > > > Before exiting, message is printed for removing each test case and testsuite > becomes empty. To rollback testsuite I'm removing gst-validate directory > before 'gst-validate-launcher --sync'. Ah, ok I understand what you are talking about now. It does not remove any test, the list of test not being run is just informative to the user to let him now something changed/something went wrong. If you install git annex and --sync again, you will get the media files and it will afterward print the list of now 'added' tests. > > (In reply to Hyung Song from comment #4) > > > In main.py: > > > > > > tests_launcher.set_settings(options, []) > > > ... > > > tests_launcher.list_tests() > > > > > > If there's a way to check if testsuite is not loaded, then exit before > > > list_tests()? > > > > What would that bring us? (from what I understand it would be a very minor > > optimization in the failure case) > > Removing test cases in the testsuite seems to happen in list_tests. As I explained previously no tests are removed, it is just a notification to the user.
(In reply to Thibault Saunier from comment #7) > (In reply to Hyung Song from comment #6) > > (In reply to Thibault Saunier from comment #5) > > > (In reply to Hyung Song from comment #3) > > > > With this patch applied, testsuite becomes empty and running another > > > > 'gst-validate-launcher --sync' has no test to perform. > > > > > > Isn't that the expected behaviour? If we can't sync we should just exit. > > > > > > > Before exiting, message is printed for removing each test case and testsuite > > becomes empty. To rollback testsuite I'm removing gst-validate directory > > before 'gst-validate-launcher --sync'. > > Ah, ok I understand what you are talking about now. It does not remove any > test, the list of test not being run is just informative to the user to let > him now something changed/something went wrong. If you install git annex and > --sync again, you will get the media files and it will afterward print the > list of now 'added' tests. > > > > (In reply to Hyung Song from comment #4) > > > > In main.py: > > > > > > > > tests_launcher.set_settings(options, []) > > > > ... > > > > tests_launcher.list_tests() > > > > > > > > If there's a way to check if testsuite is not loaded, then exit before > > > > list_tests()? > > > > > > What would that bring us? (from what I understand it would be a very minor > > > optimization in the failure case) > > > > Removing test cases in the testsuite seems to happen in list_tests. > > As I explained previously no tests are removed, it is just a notification to > the user. I understand. Thank you very much!