After adding an account application does not launch any more, outputting an error in a small GTK window. However, after deleting the database in ~/.local/share/geary it can be launched again, but it does not solve the account adding crash problem.

Hi, this seems to be a problem with more recent versions of SQLite. What version you have installed on your system?

This may require an update to src/sqlite3-unicodesn for newer versions of SQLite?

I am also seeing this error on Gentoo Linux using 0.10.0 and a freshly updated system.

(In reply to Christopher Dale from comment #4)
> I am also seeing this error on Gentoo Linux using 0.10.0 and a freshly
> updated system.

Can you say what version of SQLite you are using? Thanks!

(In reply to Michael Gratton from comment #5)
> (In reply to Christopher Dale from comment #4)
> > I am also seeing this error on Gentoo Linux using 0.10.0 and a freshly
> > updated system.
> 
> Can you say what version of SQLite you are using? Thanks!

Sure thing, I've got:

$ sqlite3 --version
3.12.0 2016-03-29 10:14:15 e9bb4cf40f4971974a74468ef922bdee481c988b

Hi,

I tried geary on the freshly installed and updated fedora 24 alpha and got the same error.

$ sqlite3 --version
3.11.0 2016-02-15 17:29:24 3d862f207e3adc00f78066799ac5a8c282430a5f

$ geary --version
geary 0.10.0


--snip--
** (geary:22741): ERROR **: imap-db-database.vala:247: Error creating search table: (Connection.exec /home/beni/.local/share/geary/test@test.ch/geary.db) [err=1] - unknown tokenizer: unicodesn (
                CREATE VIRTUAL TABLE MessageSearchTable USING fts4(
                    body,
                    attachment,
                    subject,
                    from_field,
                    receivers,
                    cc,
                    bcc,
                    
                    tokenize=unicodesn "stemmer=german",
                    prefix="2,4,6,8,10",
                );
            )

Thread 13 "Geary.Db.Versio" received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 0x7fff76770700 (LWP 22757)]
0x00007ffff0ae126b in _g_log_abort (breakpoint=1) at gmessages.c:325
325	    G_BREAKPOINT ();

Ok, probably got it.

https://www.sqlite.org/fts3.html#section_8_1 says:
Because of security concerns, SQLite version 3.11.0 only enabled the second form of the fts3_tokenizer() function when the library is compiled with the -DSQLITE_ENABLE_FTS3_TOKENIZER option. In earlier versions it was always available. Beginning with SQLite version 3.12.0, the second form of fts3_tokenizer() can also be activated at run-time by calling sqlite3_db_config(db,SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER,1,0).

recompiled sqlite3 it with the mentioned flag "-DSQLITE_ENABLE_FTS3_TOKENIZER"
and it worked. geary started and is ready.

--snip--
[deb] 17:49:51 0.077779 db-versioned-database.vala:126: Upgrading database to version 11 with /usr/share/geary/sql/version-011.sql
 [deb] 17:49:51 0.044120 imap-db-database.vala:228: Creating search table using german stemmer
 [deb] 17:49:51 0.054547 db-versioned-database.vala:126: Upgrading database to version 12 with /usr/share/geary/sql/version-012.sql
--snip--

reported it for fedora:
https://bugzilla.redhat.com/show_bug.cgi?id=1326884

(In reply to Benjamin from comment #8)
> Ok, probably got it.
> 
> https://www.sqlite.org/fts3.html#section_8_1 says:
> Because of security concerns, SQLite version 3.11.0 only enabled the second
> form of the fts3_tokenizer() function when the library is compiled with the
> -DSQLITE_ENABLE_FTS3_TOKENIZER option. In earlier versions it was always
> available. Beginning with SQLite version 3.12.0, the second form of
> fts3_tokenizer() can also be activated at run-time by calling
> sqlite3_db_config(db,SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER,1,0).

Hi Benjamin,

Thanks for looking into this one. I'm going to try re-compiling sqlite3 with the flag you mentioned tonight and see if that resolves the issue :D

Thanks man!

Huh, just re-compiled the latest sqlite3 and ensured SQLITE_ENABLE_FTS3_TOKENIZER was defined; however, when I attempted to setup a new account I received the same error messages. I also `rm -rf ~/.local/share/geary` to make sure the old DB's were removed. Is there something I'm missing?

Thanks for looking into this, guys. It looks like the Debian/Ubuntu packages for 3.11 have been built with SQLITE_ENABLE_FTS3_TOKENIZER, which explains why I haven't seen the issue. Presumably there's no way of telling at compile time for Geary whether sqlite has been compiled with that flag?

In any case I guess we want to start issuing that sqlite3_db_config call, to take care of it down the track.

(In reply to Christopher Dale from comment #11)
> Huh, just re-compiled the latest sqlite3 and ensured
> SQLITE_ENABLE_FTS3_TOKENIZER was defined; however, when I attempted to setup
> a new account I received the same error messages. I also `rm -rf
> ~/.local/share/geary` to make sure the old DB's were removed. Is there
> something I'm missing?

Did you rebuild the RPM and reinstall that or build it from upstream source? If the latter, you might need to also rebuild geary and ensure it's linked with the correct version.

(In reply to Michael Gratton from comment #12)
> Did you rebuild the RPM and reinstall that or build it from upstream source?
> If the latter, you might need to also rebuild geary and ensure it's linked
> with the correct version.

Ah, gotcha, that makes sense. I'll give that a go. Thanks Michael!

By the way, I'm working on gentoo :D

Hey guys,

So I just re-compiled geary after re-installing sqlite3 with the enabled flag. After re-creating my profile, the main window disappeared again with the same error message as before:

Gtk-Message: GtkDialog mapped without a transient parent. This is discouraged.

** (geary:11156): ERROR **: imap-db-database.vala:247: Error creating search table: (Connection.exec /home/monkeyman/.local/share/geary/chris@dalejail.net/geary.db) [err=1] - unknown tokenizer: unicodesn (
                CREATE VIRTUAL TABLE MessageSearchTable USING fts4(
                    body,
                    attachment,
                    subject,
                    from_field,
                    receivers,
                    cc,
                    bcc,
                    
                    tokenize=unicodesn "stemmer=english",
                    prefix="2,4,6,8,10",
                );
            )
Trace/breakpoint trap (core dumped)

And when I try to open geary again, I get the same error as before:

 [msg] 20:34:17 0.047648 geary-application.vala:193: Geary 0.10.0 prefix=/usr exec_dir=/usr/bin is_installed=true
 [deb] 20:34:17 0.062279 null-indicator.vala:13: No messaging menu support in this build
 [deb] 20:34:17 0.053353 imap-client-session-manager.vala:525: Checking if IMAP host host.com/default:143 reachable...
 [deb] 20:34:17 0.073811 db-versioned-database.vala:77: VersionedDatabase.upgrade: current database schema for /home/username/.local/share/geary/fake@host.com/geary.db: 15
 [deb] 20:34:17 0.000103 db-versioned-database.vala:126: Upgrading database to version 16 with /usr/share/geary/sql/version-016.sql
 [deb] 20:34:17 0.000084 db-connection.vala:397: Connection.exec_transaction: transaction threw error: (Connection.exec /home/username/.local/share/geary/fake@host.com/geary.db) [err=1] - no such table: MessageSearchTable (--
-- For a while there, we weren't properly indexing attachment filenames in the
-- search table.  To be proper (and since this is right before a major release)
-- we want to make sure anyone who's been running the dailies has a good
-- database, which unfortunately means ditching the search table and letting
-- Geary recreate it properly.
--

DELETE FROM MessageSearchTable
)
*[wrn] 20:34:17 0.000018 db-versioned-database.vala:134: Error upgrading database to version 16: (Connection.exec /home/username/.local/share/geary/fake@host.com/geary.db) [err=1] - no such table: MessageSearchTable (--
-- For a while there, we weren't properly indexing attachment filenames in the
-- search table.  To be proper (and since this is right before a major release)
-- we want to make sure anyone who's been running the dailies has a good
-- database, which unfortunately means ditching the search table and letting
-- Geary recreate it properly.
--

DELETE FROM MessageSearchTable
)
*[wrn] 20:34:17 0.001208 imap-db-account.vala:87: Unable to open database: (Connection.exec /home/username/.local/share/geary/fake@host.com/geary.db) [err=1] - no such table: MessageSearchTable (--
-- For a while there, we weren't properly indexing attachment filenames in the
-- search table.  To be proper (and since this is right before a major release)
-- we want to make sure anyone who's been running the dailies has a good
-- database, which unfortunately means ditching the search table and letting
-- Geary recreate it properly.
--

DELETE FROM MessageSearchTable
)
 [deb] 20:34:17 0.000328 geary-controller.vala:1123: Unable to open account Other:fake@host.com: (Connection.exec /home/username/.local/share/geary/fake@host.com/geary.db) [err=1] - no such table: MessageSearchTable (--
-- For a while there, we weren't properly indexing attachment filenames in the
-- search table.  To be proper (and since this is right before a major release)
-- we want to make sure anyone who's been running the dailies has a good
-- database, which unfortunately means ditching the search table and letting
-- Geary recreate it properly.
--

DELETE FROM MessageSearchTable
)
 [msg] 20:34:17 0.010601 imap-client-session-manager.vala:536: IMAP host host.com/default:143 considered reachable
 [deb] 20:34:17 0.009445 conversation-web-view.vala:207: Loading new message viewer style from /home/username/.config/geary/user-message.css...
 [deb] 20:34:20 2.756937 geary-controller.vala:355: Closing Engine...
 [deb] 20:34:20 0.000071 geary-controller.vala:357: Closed Engine

I'm going to do some reading and see if there is a way I can verify my build of sqlite3 does in fact have support for the FTS3 tokenizer. If you guys know of a way to check, could you let me know in the comments?

Thanks everyone

(In reply to Christopher Dale from comment #14)
> 
> I'm going to do some reading and see if there is a way I can verify my build
> of sqlite3 does in fact have support for the FTS3 tokenizer. If you guys
> know of a way to check, could you let me know in the comments?

You can try manually compiling the tokeniser from Geary's source tree and following the use instructions in the README there: https://git.gnome.org/browse/geary/tree/src/sqlite3-unicodesn/README

(In reply to Michael Gratton from comment #15)
> (In reply to Christopher Dale from comment #14)
> > 
> > I'm going to do some reading and see if there is a way I can verify my build
> > of sqlite3 does in fact have support for the FTS3 tokenizer. If you guys
> > know of a way to check, could you let me know in the comments?
> 
> You can try manually compiling the tokeniser from Geary's source tree and
> following the use instructions in the README there:
> https://git.gnome.org/browse/geary/tree/src/sqlite3-unicodesn/README

Nice, thanks for the tip Michael. So far, I've tried a couple things and still no luck with geary. I was able to get the examples listed under "Querying Tokenizers" to work though after re-compiling sqlite3 using the command listed here:

http://stackoverflow.com/a/6578661

I'll try manually compiling the tokenizer from the source tree and see where that gets me.

(In reply to Christopher Dale from comment #16)
> (In reply to Michael Gratton from comment #15)
> > (In reply to Christopher Dale from comment #14)
> > > 
> > > I'm going to do some reading and see if there is a way I can verify my build
> > > of sqlite3 does in fact have support for the FTS3 tokenizer. If you guys
> > > know of a way to check, could you let me know in the comments?
> > 
> > You can try manually compiling the tokeniser from Geary's source tree and
> > following the use instructions in the README there:
> > https://git.gnome.org/browse/geary/tree/src/sqlite3-unicodesn/README
> 
> Nice, thanks for the tip Michael. So far, I've tried a couple things and
> still no luck with geary. I was able to get the examples listed under
> "Querying Tokenizers" to work though after re-compiling sqlite3 using the
> command listed here:
> 
> http://stackoverflow.com/a/6578661
> 
> I'll try manually compiling the tokenizer from the source tree and see where
> that gets me.

Unfortunately, following the instructions I ran into a snag after issuing the following command:

CREATE VIRTUAL TABLE fts USING fts3(text, tokenize=unicodesn "stemmer=russian");

The output was as follows:

sqlite> CREATE VIRTUAL TABLE fts USING fts3(text, tokenize=unicodesn "stemmer=russian");
Error: unknown tokenizer: unicodesn
sqlite>

(In reply to Christopher Dale from comment #17)
> 
> Unfortunately, following the instructions I ran into a snag after issuing
> the following command:
> 
> CREATE VIRTUAL TABLE fts USING fts3(text, tokenize=unicodesn
> "stemmer=russian");
> 
> The output was as follows:
> 
> sqlite> CREATE VIRTUAL TABLE fts USING fts3(text, tokenize=unicodesn
> "stemmer=russian");
> Error: unknown tokenizer: unicodesn

Did the preceding .load command successfully complete? According to this you may need to do a make install first: https://wiki.gnome.org/Apps/Geary/FAQ#Why_can.2BIBk-t_I_examine_the_Geary_MessageSearchTable_with_an_SQLite_explorer_tool_.28or_the_sqlite3_command-line_app.29.3F

to clarify:

I set "-DSQLITE_ENABLE_FTS3_TOKENIZER=1", not only -DSQLITE_ENABLE_FTS3_TOKENIZER

all options, that fedora uses:
export CFLAGS="$RPM_OPT_FLAGS -DSQLITE_ENABLE_COLUMN_METADATA=1 \
               -DSQLITE_DISABLE_DIRSYNC=1 -DSQLITE_ENABLE_FTS3=3 \
               -DSQLITE_ENABLE_RTREE=1 -DSQLITE_SECURE_DELETE=1 \
               -DSQLITE_ENABLE_UNLOCK_NOTIFY=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1 \
               -DSQLITE_ENABLE_FTS3_PARENTHESIS=1 -DSQLITE_ENABLE_JSON1=1 \
               -Wall -fno-strict-aliasing"

(In reply to Benjamin from comment #19)
> to clarify:
> 
> I set "-DSQLITE_ENABLE_FTS3_TOKENIZER=1", not only
> -DSQLITE_ENABLE_FTS3_TOKENIZER
> 
> all options, that fedora uses:
> export CFLAGS="$RPM_OPT_FLAGS -DSQLITE_ENABLE_COLUMN_METADATA=1 \
>                -DSQLITE_DISABLE_DIRSYNC=1 -DSQLITE_ENABLE_FTS3=3 \
>                -DSQLITE_ENABLE_RTREE=1 -DSQLITE_SECURE_DELETE=1 \
>                -DSQLITE_ENABLE_UNLOCK_NOTIFY=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1
> \
>                -DSQLITE_ENABLE_FTS3_PARENTHESIS=1 -DSQLITE_ENABLE_JSON1=1 \
>                -Wall -fno-strict-aliasing"

Hey Benjamin,

Ahh, I see. I'll give that a shot and see where it gets me. Thanks for the info!

(In reply to Michael Gratton from comment #18)
> (In reply to Christopher Dale from comment #17)
> > 
> > Unfortunately, following the instructions I ran into a snag after issuing
> > the following command:
> > 
> > CREATE VIRTUAL TABLE fts USING fts3(text, tokenize=unicodesn
> > "stemmer=russian");
> > 
> > The output was as follows:
> > 
> > sqlite> CREATE VIRTUAL TABLE fts USING fts3(text, tokenize=unicodesn
> > "stemmer=russian");
> > Error: unknown tokenizer: unicodesn
> 
> Did the preceding .load command successfully complete? According to this you
> may need to do a make install first:
> https://wiki.gnome.org/Apps/Geary/FAQ#Why_can.2BIBk-
> t_I_examine_the_Geary_MessageSearchTable_with_an_SQLite_explorer_tool_.
> 28or_the_sqlite3_command-line_app.29.3F

Hi Michael,

It did actually. Sorry, I didn't post that part, just the bit that failed for me. I did follow all the steps in the instructions though. Currently, I'm trying things with the CFLAGS that Benjamin posted and then I'll post the results.

Thanks Michael!

Huh, I'm afraid I'm still not able to get this to work. The steps I took are outlined below:

root@computer $ cd sqlite-autoconf-3120100
root@computer $ make distclean
 rm -f sqlite3
test -z "libsqlite3.la" || rm -f libsqlite3.la
rm -f ./so_locations
rm -rf .libs _libs
rm -f *.o
rm -f *.lo
rm -f *.tab.c
test -z "sqlite3.pc" || rm -f sqlite3.pc
test . = "." || test -z "" || rm -f 
rm -f libtool config.lt
rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags
rm -f cscope.out cscope.in.out cscope.po.out cscope.files
rm -f config.status config.cache config.log configure.lineno config.status.lineno
rm -rf ./.deps
rm -f Makefile

root@computer $ ./configure --prefix=/usr
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/x86_64-pc-linux-gnu/bin/ld
checking if the linker (/usr/x86_64-pc-linux-gnu/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/x86_64-pc-linux-gnu/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/x86_64-pc-linux-gnu/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for fdatasync... yes
checking for usleep... yes
checking for fullfsync... no
checking for localtime_r... yes
checking for gmtime_r... yes
checking whether strerror_r is declared... yes
checking for strerror_r... yes
checking whether strerror_r returns char *... no
checking for library containing readline... no
checking for library containing pthread_create... -lpthread
checking for library containing pthread_mutexattr_init... none required
checking for library containing dlopen... -ldl
checking for whether to support dynamic extensions... yes
checking for posix_fallocate... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating sqlite3.pc
config.status: executing depfiles commands
config.status: executing libtool commands

root@computer $ export CFLAGS="$RPM_OPT_FLAGS -DSQLITE_ENABLE_COLUMN_METADATA=1 \
>                -DSQLITE_DISABLE_DIRSYNC=1 -DSQLITE_ENABLE_FTS3=3 \
>                -DSQLITE_ENABLE_RTREE=1 -DSQLITE_SECURE_DELETE=1 \
>                -DSQLITE_ENABLE_UNLOCK_NOTIFY=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1 \
>                -DSQLITE_ENABLE_FTS3_PARENTHESIS=1 -DSQLITE_ENABLE_JSON1=1 \
>                -Wall -fno-strict-aliasing"

root@computer $ make
/bin/sh ./libtool  --tag=CC   --mode=compile gcc -DPACKAGE_NAME=\"sqlite\" -DPACKAGE_TARNAME=\"sqlite\" -DPACKAGE_VERSION=\"3.12.1\" -DPACKAGE_STRING=\"sqlite\ 3.12.1\" -DPACKAGE_BUGREPORT=\"http://www.sqlite.org\" -DPACKAGE_URL=\"\" -DPACKAGE=\"sqlite\" -DVERSION=\"3.12.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_FDATASYNC=1 -DHAVE_USLEEP=1 -DHAVE_LOCALTIME_R=1 -DHAVE_GMTIME_R=1 -DHAVE_DECL_STRERROR_R=1 -DHAVE_STRERROR_R=1 -DHAVE_POSIX_FALLOCATE=1 -I.    -D_REENTRANT=1 -DSQLITE_THREADSAFE=1    -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -g -O2 -MT sqlite3.lo -MD -MP -MF .deps/sqlite3.Tpo -c -o sqlite3.lo sqlite3.c
libtool: compile:  gcc -DPACKAGE_NAME=\"sqlite\" -DPACKAGE_TARNAME=\"sqlite\" -DPACKAGE_VERSION=\"3.12.1\" "-DPACKAGE_STRING=\"sqlite 3.12.1\"" -DPACKAGE_BUGREPORT=\"http://www.sqlite.org\" -DPACKAGE_URL=\"\" -DPACKAGE=\"sqlite\" -DVERSION=\"3.12.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_FDATASYNC=1 -DHAVE_USLEEP=1 -DHAVE_LOCALTIME_R=1 -DHAVE_GMTIME_R=1 -DHAVE_DECL_STRERROR_R=1 -DHAVE_STRERROR_R=1 -DHAVE_POSIX_FALLOCATE=1 -I. -D_REENTRANT=1 -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -g -O2 -MT sqlite3.lo -MD -MP -MF .deps/sqlite3.Tpo -c sqlite3.c  -fPIC -DPIC -o .libs/sqlite3.o
libtool: compile:  gcc -DPACKAGE_NAME=\"sqlite\" -DPACKAGE_TARNAME=\"sqlite\" -DPACKAGE_VERSION=\"3.12.1\" "-DPACKAGE_STRING=\"sqlite 3.12.1\"" -DPACKAGE_BUGREPORT=\"http://www.sqlite.org\" -DPACKAGE_URL=\"\" -DPACKAGE=\"sqlite\" -DVERSION=\"3.12.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_FDATASYNC=1 -DHAVE_USLEEP=1 -DHAVE_LOCALTIME_R=1 -DHAVE_GMTIME_R=1 -DHAVE_DECL_STRERROR_R=1 -DHAVE_STRERROR_R=1 -DHAVE_POSIX_FALLOCATE=1 -I. -D_REENTRANT=1 -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -g -O2 -MT sqlite3.lo -MD -MP -MF .deps/sqlite3.Tpo -c sqlite3.c -o sqlite3.o >/dev/null 2>&1
mv -f .deps/sqlite3.Tpo .deps/sqlite3.Plo
/bin/sh ./libtool  --tag=CC   --mode=link gcc -D_REENTRANT=1 -DSQLITE_THREADSAFE=1    -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -g -O2 -no-undefined -version-info 8:6:8  -o libsqlite3.la -rpath /usr/lib sqlite3.lo  -ldl -lpthread 
libtool: link: gcc -shared  -fPIC -DPIC  .libs/sqlite3.o   -ldl -lpthread  -g -O2   -Wl,-soname -Wl,libsqlite3.so.0 -o .libs/libsqlite3.so.0.8.6
libtool: link: (cd ".libs" && rm -f "libsqlite3.so.0" && ln -s "libsqlite3.so.0.8.6" "libsqlite3.so.0")
libtool: link: (cd ".libs" && rm -f "libsqlite3.so" && ln -s "libsqlite3.so.0.8.6" "libsqlite3.so")
libtool: link: ar cru .libs/libsqlite3.a  sqlite3.o
libtool: link: ranlib .libs/libsqlite3.a
libtool: link: ( cd ".libs" && rm -f "libsqlite3.la" && ln -s "../libsqlite3.la" "libsqlite3.la" )
gcc -DPACKAGE_NAME=\"sqlite\" -DPACKAGE_TARNAME=\"sqlite\" -DPACKAGE_VERSION=\"3.12.1\" -DPACKAGE_STRING=\"sqlite\ 3.12.1\" -DPACKAGE_BUGREPORT=\"http://www.sqlite.org\" -DPACKAGE_URL=\"\" -DPACKAGE=\"sqlite\" -DVERSION=\"3.12.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_FDATASYNC=1 -DHAVE_USLEEP=1 -DHAVE_LOCALTIME_R=1 -DHAVE_GMTIME_R=1 -DHAVE_D  L_STRERROR_R=1 -DHAVE_STRERROR_R=1 -DHAVE_POSIX_FALLOCATE=1 -I.    -D_REENTRANT=1 -DSQLITE_THREADSAFE=1    -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_EXPLAIN_COMMENTS -g -O2 -MT sqlite3-shell.o -MD -MP -MF .deps/sqlite3-shell.Tpo -c -o sqlite3-shell.o `test -f 'shell.c' || echo './'`shell.c
mv -f .deps/sqlite3-shell.Tpo .deps/sqlite3-shell.Po
gcc -DPACKAGE_NAME=\"sqlite\" -DPACKAGE_TARNAME=\"sqlite\" -DPACKAGE_VERSION=\"3.12.1\" -DPACKAGE_STRING=\"sqlite\ 3.12.1\" -DPACKAGE_BUGREPORT=\"http://www.sqlite.org\" -DPACKAGE_URL=\"\" -DPACKAGE=\"sqlite\" -DVERSION=\"3.12.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_FDATASYNC=1 -DHAVE_USLEEP=1 -DHAVE_LOCALTIME_R=1 -DHAVE_GMTIME_R=1 -DHAVE_DECL_STRERROR_R=1 -DHAVE_STRERROR_R=1 -DHAVE_POSIX_FALLOCATE=1 -I.    -D_REENTRANT=1 -DSQLITE_THREADSAFE=1    -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_EXPLAIN_COMMENTS -g -O2 -MT sqlite3-sqlite3.o -MD -MP -MF .deps/sqlite3-sqlite3.Tpo -c -o sqlite3-sqlite3.o `test -f 'sqlite3.c' || echo './'`sqlite3.c
mv -f .deps/sqlite3-sqlite3.Tpo .deps/sqlite3-sqlite3.Po
/bin/sh ./libtool  --tag=CC   --mode=link gcc -D_REENTRANT=1 -DSQLITE_THREADSAFE=1    -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_EXPLAIN_COMMENTS -g -O2   -o sqlite3 sqlite3-shell.o sqlite3-sqlite3.o  -ldl -lpthread 
libtool: link: gcc -D_REENTRANT=1 -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_EXPLAIN_COMMENTS -g -O2 -o sqlite3 sqlite3-shell.o sqlite3-sqlite3.o  -ldl -lpthread

$ # Navigate to the sqlite3-unicodesn directory
$ make clean
rm -f *.o unicodesn.sqlext
$ make
cc -Os -DNDEBUG -s -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS4_UNICODE61  -DWITH_STEMMER_danish  -DWITH_STEMMER_dutch  -DWITH_STEMMER_english  -DWITH_STEMMER_finnish  -DWITH_STEMMER_french  -DWITH_STEMMER_german  -DWITH_STEMMER_hungarian  -DWITH_STEMMER_italian  -DWITH_STEMMER_norwegian  -DWITH_STEMMER_porter  -DWITH_STEMMER_portuguese  -DWITH_STEMMER_romanian  -DWITH_STEMMER_russian  -DWITH_STEMMER_spanish  -DWITH_STEMMER_swedish  -DWITH_STEMMER_turkish -Ilibstemmer_c/runtime -Ilibstemmer_c/src_c -fPIC -shared -fvisibility=hidden -o unicodesn.sqlext \
   fts3_unicode2.c fts3_unicodesn.c extension.c libstemmer_c/runtime/api_sq3.c libstemmer_c/runtime/utilities_sq3.c  libstemmer_c/src_c/stem_UTF_8_danish.c  libstemmer_c/src_c/stem_UTF_8_dutch.c  libstemmer_c/src_c/stem_UTF_8_english.c  libstemmer_c/src_c/stem_UTF_8_finnish.c  libstemmer_c/src_c/stem_UTF_8_french.c  libstemmer_c/src_c/stem_UTF_8_german.c  libstemmer_c/src_c/stem_UTF_8_hungarian.c  libstemmer_c/src_c/stem_UTF_8_italian.c  libstemmer_c/src_c/stem_UTF_8_norwegian.c  libstemmer_c/src_c/stem_UTF_8_porter.c  libstemmer_c/src_c/stem_UTF_8_portuguese.c  libstemmer_c/src_c/stem_UTF_8_romanian.c  libstemmer_c/src_c/stem_UTF_8_russian.c  libstemmer_c/src_c/stem_UTF_8_spanish.c  libstemmer_c/src_c/stem_UTF_8_swedish.c  libstemmer_c/src_c/stem_UTF_8_turkish.c -lsqlite3
$ su
Password: 
root@computer $ make install
mkdir -p /usr/lib 2> /dev/null
install -D -o root -g root -m 644 unicodesn.sqlext /usr/lib
# exit

$ sqlite3 
SQLite version 3.12.1 2016-04-08 15:09:49
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .load unicodesn.sqlext
sqlite>  CREATE VIRTUAL TABLE fts USING fts3(text, tokenize=unicodesn "stemmer=russian");
Error: unknown tokenizer: unicodesn
sqlite> .quit

*** Bug 763086 has been marked as a duplicate of this bug. ***

Adding link to Fedora bug report.

Christopher: You may need to export the CFLAGS before running ./configure in the sqlite source tree. If that doesn't help, look me up on irc.gimp.org #geary (I'm mjog) or send an email to the list <http://mail.gnome.org/mailman/listinfo/geary-list> - bugzilla isn't the best place for tech support.

Hi Michael,

Ah, thanks for the tip man. I'll give that a go and report the results. LOL, unfortunately in the interim here I've switched to XFCE4. I tend to switch desktop environs quite often so I might have to install some other deps before hand.

Also, if I run into problems I'll try to jump into the IRC channel you recommended and lookup mjog.

Thanks again,

Christopher

Created attachment 326987 [details] [review]
Enable use of custom FTS3 tokeniser in SQLite 3.12 and later

This enables the two-arg form of fts3_tokenizer in SQLite for the Geary databases, used set the custom unicodesn tokeniser for full-text message search. The two-arg form was disabled by default in 3.11 for secuity reasons, but can be programmatically enabled in 3.12.
    
Note that this does not fix SQLite 3.11, to do that it must be recompiled with -DSQLITE_ENABLE_FTS3_TOKENIZER.

Christopher: No problem!

Adam: This patch is good to commit, thanks!

Pushed to master.  Mike, you can close this bug if appropriate.

I think this is good to close. Thanks Adam.