GNOME Bugzilla – Bug 785883
tracker search doesn't work with sqlite 3.20.0 due to changes in fts5 interfaces
Last modified: 2017-08-22 12:04:19 UTC
After upgrading from sqlite 3.19.3 to 3.20.0 tracker can no longer find files, which makes searching in the Activities overview of Gnome Shell not show results from Files. Running `tracker search -f` shows files are indexed fine but running `tracker search -f something` returns: "Could not get search results, no such tokenizer: TrackerTokenizer". Doing `tracker reset --soft` didn't help. Downgrading to sqlite 3.19.3 fixed the issue. Tested with both tracker version 1.12.1 and 1.99.1. Also tested with rebuilding them from source after installing sqlite 3.20.0. Same issue with both. Operating system: Arch Linux Architecture: x86_64 Desktop environment: GNOME Shell 3.24.3
Thanks for the heads up, sqlite indeed made an incompatible change on their fts5 interfaces. I'm attaching (and pushing after I get to test with sqlite 3.20) a patch that should cater for the old and new way of doing things.
Created attachment 357061 [details] [review] libtracker-fts: Adapt to incompatible change in sqlite 3.20 From that version on, we must use the brand new sqlite3_bind_pointer() interface to access the fts5 interface pointer. Adapt to it without giving up older versions.
Could you also provide a backport to tracker 1.12.1? You provided another patch for what looks to be a possibly related bug https://bugzilla.gnome.org/show_bug.cgi?id=785876 earlier today. I tried the patch you provided there on suggestion of downstream but that patch doesn't solve my issue.
I am getting a build failure with the patch. https://paste.gnome.org/psnsukuxe (it expects 5 arguments).
(In reply to Jake Dane from comment #3) > Could you also provide a backport to tracker 1.12.1? Yeah, had that in mind, the patch should apply mostly as is. > > You provided another patch for what looks to be a possibly related bug > https://bugzilla.gnome.org/show_bug.cgi?id=785876 earlier today. I tried the > patch you provided there on suggestion of downstream but that patch doesn't > solve my issue. Right, I didn't know of the sqlite 3.20.0 incompatibility back then, if debian also updated to sqlite 3.20.0 it makes sense for him to have this error. The pushed fix still makes sense though, because that database interface plugged by the patch there had no FTS set up, so this error was certainly and legitimately possible. (In reply to Hussam Al-Tayeb from comment #4) > I am getting a build failure with the patch. > > https://paste.gnome.org/psnsukuxe (it expects 5 arguments). Meh, was deceived by the recipe at http://sqlite.org/fts5.html#extending_fts5 , seeing the api docs it indeed takes 5 arguments. I'm updating the patch.
Created attachment 357074 [details] [review] libtracker-fts: Adapt to incompatible change in sqlite 3.20 From that version on, we must use the brand new sqlite3_bind_pointer() interface to access the fts5 interface pointer. Adapt to it without giving up older versions.
That patch worked for me on tracker 1.12.1. First upgrading to sqlite 3.20.0 and afterwards installing tracker recompiled with this patch made tracker do file searches again. Thank you ever so much for the quick response and patch!
Thanks for testing! After ensuring all is correct I'm pushing to master/tracker-1.12, I'm sorting out new releases for both versions today. Attachment 357074 [details] pushed as 4aadfc9 - libtracker-fts: Adapt to incompatible change in sqlite 3.20
*** Bug 786130 has been marked as a duplicate of this bug. ***
Reopening. There are still problems because the fts5 module from sqlite 3.20.0 (as shipped in Tracker) is incompatible with sqlite >= 3.20.0, and fts5 modules generated from sqlite >= 3.20 are backwards incompatible with sqlite < 3.20. As I refuse to ship 2 copies of fts5 for pre/post 3.20.0, I plan to mandate that sqlite >= 3.20.0 must have fts5 enabled itself, and preserve the internal copy to keep the requirement optional for sqlite < 3.20.0. Distros were eager to push API/ABI-incompatible sqlite 3.20.0 quickly to users, they'll hopefully be as eager to do --enable-fts5 on it.
*** Bug 785876 has been marked as a duplicate of this bug. ***
(In reply to Carlos Garnacho from comment #10) > Reopening. There are still problems because the fts5 module from sqlite > 3.20.0 (as shipped in Tracker) is incompatible with sqlite >= 3.20.0, and ^ That should read < 3.20.0 I should also mention, the change shall affect 1.12 and 2.x
Created attachment 358126 [details] [review] build: Mandate that sqlite3 >= 3.20.0 has fts5 builtin That sqlite3 version brought incompatibilities in the fts5 interface, fts5 modules generated from < 3.20.0 trees won't work with sqlite3 >= 3.20.0, and viceversa. Since it's not feasible nor desirable to include an additional fts5 module copy nor an embedded sqlite3 amalgamation copy, mandate that SQLite was compiled with --enable-fts5 if >= 3.20.0 is found. The current copy of the fts5 module is kept around for sqlite3 < 3.20.0, so we don't make this a mandatory requirement for earlier versions.
Pushed to git, I am doing new 1.12 and 1.99 releases. Attachment 358126 [details] pushed as 84c8cc2 - build: Mandate that sqlite3 >= 3.20.0 has fts5 builtin