GNOME Bugzilla – Bug 713387
Unit test infrastructure
Last modified: 2016-12-23 02:13:16 UTC
---- Reported by geary-maint@gnome.bugs 2012-04-01 05:21:00 -0700 ---- Original Redmine bug id: 4989 Original URL: http://redmine.yorba.org/issues/4989 Searchable id: yorba-bug-4989 Original author: Christian Dywan Original description: When looking at issue #4313 and adding a function to Geary.RFC822.MailboxAddress I thought, wouldn't it be trivial to add a cuple of test cases, if only we had a test infrastructure. Let's make it so easy to add test cases to any class that it's tempting to do it, not tedious. "./waf check" should just work. Adding a test case should not require work on the build system. Related issues: related to geary - 5325: continuous automated testing (Open) related to geary - Feature #6934: Autopilot UI testing (Open) ---- Additional Comments From geary-maint@gnome.bugs 2013-07-11 15:13:00 -0700 ---- ### History #### #1 Updated by Adam Dingle over 1 year ago Yes - this would be great. We need to decide: do we use an existing unit test framework for Vala? Or do we just roll our own inside the Geary code? A year ago, we evaluated all the Vala-based unit test frameworks we could find, and decided that the most promising was Valadate, originally at http://gitorious.org/valadate . We then did a bunch of work to make it work with the then-current version of Vala. The result is at http://redmine.yorba.org/projects/valadate . We meant to start using this in Shotwell, but never got too far with that. #### #2 Updated by Eric Gregory over 1 year ago Christian, you might want to wait on this one since we're looking into switching away from Waf. #### #3 Updated by Christian Dywan over 1 year ago So I did take a look at Valadate in any event. It provides a handful of utility functions, such as "copy file" and "get file contents into a string". It scans the build (read: vapi files) for files with "test" in the name. It's an extra, optional dependency by design. I feel the main flaw is being an optional dependency. I would really strongly prefer something mandatory. It's hard to say if the utility functions are useful over GLib. From my experience it's hard to abstract much. For example, if you want to compare URIs or email addresses, you'll have a format for a list of expected results, and that's going to be mostly tailored to the Geary engine. #### #4 Updated by Adam Dingle over 1 year ago Christian, we just met to discuss this at Yorba. We think it would be fine to skip Valadate and just code in the tests directly, at least for now. (If we eventually have lots of tests we can always introduce a testing meta-framework at some later time.) If the tests run very quickly (say, in less than 0.1 second) I think we can always run them at startup. If they take any longer, they should be run via "waf check" or some similar mechanism. Anyway, feel free to work on a simple built-in testing infrastructure if that makes sense to you. #### #5 Updated by Adam Dingle over 1 year ago * **Target version** set to _0.1_ #### #6 Updated by Adam Dingle over 1 year ago * **Tracker** changed from _Bug_ to _Task_ #### #7 Updated by Adam Dingle over 1 year ago * **Tracker** changed from _Task_ to _Feature_ #### #8 Updated by Adam Dingle over 1 year ago * **Tracker** changed from _Feature_ to _Task_ #### #9 Updated by Adam Dingle over 1 year ago * **Target version** deleted (<strike>_0.1_</strike>) #### #10 Updated by Adam Dingle over 1 year ago * **Target version** set to _0.2_ #### #11 Updated by Adam Dingle about 1 year ago * **Assignee** deleted (<strike>_Christian Dywan_</strike>) * **Target version** deleted (<strike>_0.2_</strike>) Unassigning since there's been no talk about this in a while. Christian, if you want to work on this in the near future then feel free to reassign to yourself. #### #12 Updated by Jim Nelson about 1 year ago * **Target version** set to _0.3.0_ #### #13 Updated by Jim Nelson 8 months ago * **Target version** changed from _0.3.0_ to _0.4.0_ #### #14 Updated by Jim Nelson 4 months ago * **Target version** deleted (<strike>_0.4.0_</strike>) --- Bug imported by chaz@yorba.org 2013-11-21 20:20 UTC --- This bug was previously known as _bug_ 4989 at http://redmine.yorba.org/show_bug.cgi?id=4989 Unknown Component Using default product and component set in Parameters Unknown version " in product geary. Setting version to "!unspecified". Unknown milestone "unknown in product geary. Setting to default milestone for this product, "---". Setting qa contact to the default for this product. This bug either had no qa contact or an invalid one. Resolution set on an open status. Dropping resolution
Informative post on unit testing vala apps using GLib's GTest/gtester and libgee's TestCase: https://esite.ch/2012/06/writing-tests-for-vala/ Since Builder 3.22 now has support for GTest in vala projects as well, this may well be a good way forward? As a side note, for #geary-wk2 I've had to port some DOM code written in vala to JavaScript. Since JS is so brittle, it'd be awesome to be able to have unit tests for it. I wonder if that could be done with gjs or something?
This suddenly became of interest to me since today I sent a few emails with some sketchy formatting courtesy the wk2 port. Proposed implementation pushed as branch wip/713387-unit-tests, will attach patches momentarily. Niels & Gautier, can I get a review and/or your opinion of this approach?
Created attachment 341813 [details] [review] Basic unit test build configuration
Created attachment 341814 [details] [review] Add an initial/example unit test
One note: As-is it's only really useful for testing the engine (and I should be able to use it to test JS in the WK2 port), but not really for testing the client codebase. To allow that, the client build would need to be split up, building basically all of the client code (except main.vala) as a static lib (a'la the engine) before building the executable itself, so that it can be linked with both that and the test executable(s). I'm not sure how important/easy it will be to test the client codebase anyway at the moment, so I haven't bothered doing this, but it wouldn't be too hard to add on when we actually have something in the client codebase to test. :)
Okay, per feedback on IRC just updated the second patch to use a subdir for the engine test, and use the same namespace as the engine. These have been pushed to master as commit 6ce262a and commit 7683044. I tried (but not very hard) to find something in the client that was easy to test and worth the effort to actually build a separate client static lib, but didn't. So lets leave that until we have a use case for it, I might need to for the WK2 port anyway.