After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 738580 - Old FTS data not removed when file changes - false positives on old content query
Old FTS data not removed when file changes - false positives on old content q...
Status: RESOLVED FIXED
Product: tracker
Classification: Core
Component: FTS
1.1.x
Other Linux
: Normal normal
: ---
Assigned To: tracker-general
tracker-general
Depends on:
Blocks: 739235
 
 
Reported: 2014-10-15 14:27 UTC by Martin Kampas
Modified: 2016-05-06 11:48 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Martin Kampas 2014-10-15 14:27:34 UTC
Identified by functional-tests/310-fts-indexing.py, test cases test_02_empty_the_file, test_03_update_the_file (and more) on Nemomobile.

STEPS TO REPRODUCE:
~$ echo "big red automobile on road" > ~/foo.txt
~$ tracker-control -F # wait until it gets IDLE
^C
~$ tracker-query -q "SELECT ?url WHERE { ?u a nfo:TextDocument ; nie:url ?url ; fts:match 'automobile'. }"
Results:
  file:///home/user/foo.txt     <-------- OK

~$ echo "big red kangaroo on road" > ~/foo.txt
~$ tracker-control -F # wait until it gets IDLE
^C
~$ tracker-query -q "SELECT ?url WHERE { ?u a nfo:TextDocument ; nie:url ?url ; fts:match 'automobile'. }"
Results:
  file:///home/user/foo.txt     <------- UNEXPECTED

~$ tracker-query -q "SELECT ?url WHERE { ?u a nfo:TextDocument ; nie:url ?url ; fts:match 'kangaroo'. }"
Results:
  file:///home/user/foo.txt     <------- OK

~$ tracker-query -q "SELECT ?url WHERE { ?u a nfo:TextDocument ; nie:url ?url ; fts:match 'turtle'. }"
Results:
  None                                 <-------- OK

EXPECTED RESULT:
Searching for "automobile" gives no match after changing file contents replacing "automobile" with "kangaroo"

ACTUAL RESULT:
Searches for both previous and current content give positive results

My best guess is this has been broken since 6460779dc792878807cb7921cfc9d946868429e0 (Use fts4 external content tables on Tracker FTS).

SQLite documentation state it is necessary to remove the affected data from the FTS table *prior to* updating the external content table:

"""If the content table row cannot be found, or if it contains values inconsistent with the contents of the FTS index, the results can be difficult to predict. [...]""" http://www.sqlite.org/fts3.html#section_6_2_2

Tracking down the use of tracker_db_interface_sqlite_fts_update_text() and tracker_db_interface_sqlite_fts_delete_text(), it seems Tracker implementation was not updated to obey this when the use of FTS4 external content tables was introduced.

SQLite documentation (later after the quoted paragraph) gives an example of creating triggers on the external content table to update the FTS virtual table at the right time - unfortunately Tracker implementation uses an SQL view as the external content table and SQLite does not allow creating triggers on views so it cannot be fixed this easy way.
Comment 1 Carlos Garnacho 2016-05-06 11:48:44 UTC
This was coincidentally fixed with the switch to FTS5. Fixed in >=1.8.