GNOME Bugzilla – Bug 738580
Old FTS data not removed when file changes - false positives on old content query
Last modified: 2016-05-06 11:48:44 UTC
Identified by functional-tests/310-fts-indexing.py, test cases test_02_empty_the_file, test_03_update_the_file (and more) on Nemomobile. STEPS TO REPRODUCE: ~$ echo "big red automobile on road" > ~/foo.txt ~$ tracker-control -F # wait until it gets IDLE ^C ~$ tracker-query -q "SELECT ?url WHERE { ?u a nfo:TextDocument ; nie:url ?url ; fts:match 'automobile'. }" Results: file:///home/user/foo.txt <-------- OK ~$ echo "big red kangaroo on road" > ~/foo.txt ~$ tracker-control -F # wait until it gets IDLE ^C ~$ tracker-query -q "SELECT ?url WHERE { ?u a nfo:TextDocument ; nie:url ?url ; fts:match 'automobile'. }" Results: file:///home/user/foo.txt <------- UNEXPECTED ~$ tracker-query -q "SELECT ?url WHERE { ?u a nfo:TextDocument ; nie:url ?url ; fts:match 'kangaroo'. }" Results: file:///home/user/foo.txt <------- OK ~$ tracker-query -q "SELECT ?url WHERE { ?u a nfo:TextDocument ; nie:url ?url ; fts:match 'turtle'. }" Results: None <-------- OK EXPECTED RESULT: Searching for "automobile" gives no match after changing file contents replacing "automobile" with "kangaroo" ACTUAL RESULT: Searches for both previous and current content give positive results My best guess is this has been broken since 6460779dc792878807cb7921cfc9d946868429e0 (Use fts4 external content tables on Tracker FTS). SQLite documentation state it is necessary to remove the affected data from the FTS table *prior to* updating the external content table: """If the content table row cannot be found, or if it contains values inconsistent with the contents of the FTS index, the results can be difficult to predict. [...]""" http://www.sqlite.org/fts3.html#section_6_2_2 Tracking down the use of tracker_db_interface_sqlite_fts_update_text() and tracker_db_interface_sqlite_fts_delete_text(), it seems Tracker implementation was not updated to obey this when the use of FTS4 external content tables was introduced. SQLite documentation (later after the quoted paragraph) gives an example of creating triggers on the external content table to update the FTS virtual table at the right time - unfortunately Tracker implementation uses an SQL view as the external content table and SQLite does not allow creating triggers on views so it cannot be fixed this easy way.
This was coincidentally fixed with the switch to FTS5. Fixed in >=1.8.