GNOME Bugzilla – Bug 725156
Add UI tests
Last modified: 2018-01-23 09:21:15 UTC
Add Dogtail-based UI tests that are run both at make check time and as installed tests. See gnome-software as an example.
Some thoughts for someone who might pick this up: - The most important thing that I want to test is the transitions between the different modes. eg., from "photos" to "search", then into a collection and finally "preview", etc.. There are lots of such transitions which are very easy to unknowingly break. - Need to figure out a way to run the tests in Continuous. The problem is that Continuous does not allow network access, nor does it have any content on its own. Unless we populate it with some data, we will only get an "empty" application.
Created attachment 272010 [details] [review] WIP: added infrastructure for tests.
Created attachment 272051 [details] [review] added infrastructure for tests. applying this patch adds an empty dogtail test. further patches are required over this infrastructure to extend the tests. I am working on writing more patches.
Review of attachment 272051 [details] [review]: Good job. A few comments while reading through it. ::: tests/Makefile.am @@ +4,3 @@ + basic.py + +TESTS_ENVIRONMENT += LC_ALL=C Maybe we should also add "GSETTINGS_BACKEND=memory" to avoid tampering with the dconf database on the machine where the test is run? eg., the location and size of the window. ::: tests/testutil.py @@ +39,3 @@ +def start(): + builddir = os.environ.get('G_TEST_BUILDDIR', None) + if not builddir and not 'TESTUTIL_DONT_START' in os.environ: I find that it is "if builddir and ..." in gnome-weather. Typo? @@ +54,3 @@ + global settings + + settings = Gio.Settings(APPLICATION_ID) Do we really need the settings for our tests? There isn't much in there.
Created attachment 272278 [details] [review] view-container: Make it accessible by setting AtkObject:name
Created attachment 272363 [details] [review] added infrastructure for tests.
Review of attachment 272363 [details] [review]: Looks good, apart from some minor comments. ::: tests/basic.py @@ +19,3 @@ + recent_button = app.child('Recent') + recent_button.click() + Favorites_button = app.child('Favorites') Typo alert! ::: tests/testutil.py @@ +49,3 @@ +def start(): + builddir = os.environ.get('G_TEST_BUILDDIR', None) + if builddir and not 'TESTUTIL_DONT_START' in os.environ: The TESTUTIL_DONT_START should not be a part of this patch. It is part of the lcov integration, so lets add that when we do lcov. @@ +55,3 @@ + _do_bus_call("Activate", GLib.Variant('(a{sv})', ([],))) + else: + _do_bus_call("Activate", GLib.Variant('(a{sv})', ([],))) We can get rid of the extra else and have the Activate be called unconditionally.
(In reply to comment #7) > Review of attachment 272363 [details] [review]: > ::: tests/testutil.py > @@ +49,3 @@ > +def start(): > + builddir = os.environ.get('G_TEST_BUILDDIR', None) > + if builddir and not 'TESTUTIL_DONT_START' in os.environ: > > The TESTUTIL_DONT_START should not be a part of this patch. It is part of the > lcov integration, so lets add that when we do lcov. Actually, no. Forget what I said. It might be useful to have it even without lcov.
Created attachment 272480 [details] [review] added infrastructure for tests.
Review of attachment 272480 [details] [review]: Perfect.
Now that we have the basic infrastructure in place, we need to add the bits that would reset $HOME and all the XDG variables in ~/.config/user-dirs.dirs to our own test data directory. This way we can run the application against a fixed set of test images that is part of the test suite. Then we need to work up a list of test cases.
Created attachment 272902 [details] [review] change in test infrastructure To make tests compatible with application after https://bugzilla.gnome.org/726791
Review of attachment 272902 [details] [review]: Looks good. I adjusted the commit message to mention this bug as well.
Created attachment 273611 [details] [review] add coverage reports for tests
Review of attachment 273611 [details] [review]: I think you should add a hunk like this to src/Makefile.am so that the *.gcda and *.gcno files generated during the build do not show up in 'git status' and are cleaned up during 'make clean'. diff --git a/src/Makefile.am b/src/Makefile.am index 069d510..b027754 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -326,6 +326,13 @@ CLEANFILES = \ xgen-tmh \ $(NULL) +if ENABLE_COVERAGE +CLEANFILES += \ + *.gcda \ + *.gcno \ + $(NULL) +endif + gegl-gtk-enums.c: $(srcdir)/gegl-gtk-enums.h $(AM_V_GEN) ( \ glib-mkenums \ ::: configure.ac @@ +144,3 @@ + + CFLAGS="$CFLAGS -g -O0 --coverage" + LDFLAGS="$LDFLAGS -lgcov" It should be LIBS instead of LDFLAGS. LIBS is for passing additional libraries using -l, and LDFLAGS is for passing linker flags like -L for library paths.
Created attachment 275424 [details] [review] add coverage reports for tests
Created attachment 275425 [details] [review] add coverage reports for tests minor tweak
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/gnome-photos/issues/15.