GNOME Bugzilla – Bug 783466
sqlite parser stack overflow when running gnome-music
Last modified: 2017-08-07 22:40:24 UTC
Created attachment 353251 [details] Output log of gnome-music -d Hi, I have about 22000 songs in my Music/ folder. Every time I try to start gnome-music 3.24.2 on Fedora 26, I get the errors: ------------------------------------------ (gnome-music:15980): Grilo-WARNING **: [tracker-source-request] grl-tracker-source-api.c:504: Could not execute sparql query id=2: parser stack overflow (gnome-music:15980): Grilo-WARNING **: [tracker-source-request] grl-tracker-source-api.c:504: Could not execute sparql query id=3: parser stack overflow ------------------------------------------ The covers' and artists' views are then empty. The songs' view, however, shows the full list. It is my understanding that this is due to the stack size limit of sqlite3, which is compiled in. sqlite> .limit length 1000000000 sql_length 1000000000 column 2000 expr_depth 1000 compound_select 500 vdbe_op 250000000 function_arg 127 attached 10 like_pattern_length 50000 variable_number 999 trigger_depth 1000 worker_threads 0 However, it appears that hitting the limit shows that there is a problem in the complexity of the underlying query. I wonder if this bug is on Grilo's side rather than GNOME Music. Attached is a sample log of a run obtained with -d.
Same error here; version 3.24.2 on a fresh Antergos install, less than 500 files in my Music folder. I tried to reset tracker (tracker reset --hard) with no success. Thank you, lorenzo
It would be interesting to know what tracker queries trigger this for you, 'TRACKER_VERBOSITY=3 gnome-music' should hopefully show this. Could you attach a log?
Created attachment 355783 [details] Logs at verbosity level 3
Logs attached. Please let me know if I can do something else. Thank you, lorenzo
(In reply to Matteo Settenvini from comment #0) > Created attachment 353251 [details] > Output log of gnome-music -d > > Hi, > > I have about 22000 songs in my Music/ folder. Every time I try to start > gnome-music 3.24.2 on Fedora 26, I get the errors: > > ------------------------------------------ > > (gnome-music:15980): Grilo-WARNING **: [tracker-source-request] > grl-tracker-source-api.c:504: Could not execute sparql query id=2: parser > stack overflow > > (gnome-music:15980): Grilo-WARNING **: [tracker-source-request] > grl-tracker-source-api.c:504: Could not execute sparql query id=3: parser > stack overflow > > ------------------------------------------ > > The covers' and artists' views are then empty. The songs' view, however, > shows the full list. > > It is my understanding that this is due to the stack size limit of sqlite3, > which is compiled in. > > sqlite> .limit > length 1000000000 > sql_length 1000000000 > column 2000 > expr_depth 1000 > compound_select 500 > vdbe_op 250000000 > function_arg 127 > attached 10 > like_pattern_length 50000 > variable_number 999 > trigger_depth 1000 > worker_threads 0 > > However, it appears that hitting the limit shows that there is a problem in > the complexity of the underlying query. Nice analysis. However the limit we are hitting here is a bit more sneaky, can be triggered with just: sqlite3 ":memory:" "select (((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((1)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))));" This limit is not advertised in https://sqlite.org/limits.html, maybe it's related to point 5 (Maximum depth on an expression tree, defaulting to 1000 in docs and your provided info), but however I see I just need 93 parens on each side to trigger a parser overflow error, which is oddly specific. > > I wonder if this bug is on Grilo's side rather than GNOME Music. > > Attached is a sample log of a run obtained with -d. Thanks, that helped. I see that the Italian translation adds a few extra articles to cut down from the beginning of titles, each of which increments the depth of the query. This can be simplified, at least in a way that the current locale doesn't influence the complexity of the query.
Created attachment 355942 [details] [review] query: Use tracker:title-order() SPARQL function for title sorting This function can be used to remove common articles when sorting, so avoid open-coding that in the SPARQL queries. This also brings the benefit that queries turn a lot less complex, so there's less chances to hit parser limits.
Note that this depends on https://git.gnome.org/browse/tracker/commit/?id=80606e70475409f1e3dea47aee15a4c3b038aef9 in tracker master.
(In reply to Carlos Garnacho from comment #7) > Note that this depends on > https://git.gnome.org/browse/tracker/commit/ > ?id=80606e70475409f1e3dea47aee15a4c3b038aef9 in tracker master. Thanks, I was trying to make sense of the patch without seeing this commit. Now I get it. I noticed that you always assume that an article is followed by a space, though. This is not true at least in Italian and French. You have "l'enfant prodige" or "all'amore cieco" Which contain an elision of a vowel with the next word (the apostrophe). No space is present there. Should I open a separate bug?
(In reply to Matteo Settenvini from comment #8) > (In reply to Carlos Garnacho from comment #7) > > Note that this depends on > > https://git.gnome.org/browse/tracker/commit/ > > ?id=80606e70475409f1e3dea47aee15a4c3b038aef9 in tracker master. > > Thanks, I was trying to make sense of the patch without seeing this commit. > Now I get it. > > I noticed that you always assume that an article is followed by a space, > though. This is not true at least in Italian and French. You have "l'enfant > prodige" or "all'amore cieco" Which contain an elision of a vowel with the > next word (the apostrophe). No space is present there. > > Should I open a separate bug? Oh, good point. Yes, please open one on tracker product.
(In reply to Carlos Garnacho from comment #9) > (In reply to Matteo Settenvini from comment #8) > > > > Thanks, I was trying to make sense of the patch without seeing this commit. > > Now I get it. > > > > I noticed that you always assume that an article is followed by a space, > > though. This is not true at least in Italian and French. You have "l'enfant > > prodige" or "all'amore cieco" Which contain an elision of a vowel with the > > next word (the apostrophe). No space is present there. > > > > Should I open a separate bug? > > Oh, good point. Yes, please open one on tracker product. I filed https://bugzilla.gnome.org/show_bug.cgi?id=785146
Attachment 355942 [details] pushed as 6d63667 - query: Use tracker:title-order() SPARQL function for title sorting