After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 783466 - sqlite parser stack overflow when running gnome-music
sqlite parser stack overflow when running gnome-music
Status: RESOLVED FIXED
Product: gnome-music
Classification: Applications
Component: general
unspecified
Other Linux
: Normal major
: ---
Assigned To: gnome-music-maint
gnome-music-maint
Depends on:
Blocks:
 
 
Reported: 2017-06-06 10:19 UTC by Matteo Settenvini
Modified: 2017-08-07 22:40 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Output log of gnome-music -d (75.62 KB, text/x-log)
2017-06-06 10:19 UTC, Matteo Settenvini
  Details
Logs at verbosity level 3 (101.33 KB, text/plain)
2017-07-17 21:19 UTC, lorenzo benvenuti
  Details
query: Use tracker:title-order() SPARQL function for title sorting (3.96 KB, patch)
2017-07-19 12:25 UTC, Carlos Garnacho
committed Details | Review

Description Matteo Settenvini 2017-06-06 10:19:51 UTC
Created attachment 353251 [details]
Output log of gnome-music -d

Hi,

I have about 22000 songs in my Music/ folder. Every time I try to start gnome-music 3.24.2 on Fedora 26, I get the errors:

------------------------------------------

(gnome-music:15980): Grilo-WARNING **: [tracker-source-request] grl-tracker-source-api.c:504: Could not execute sparql query id=2: parser stack overflow

(gnome-music:15980): Grilo-WARNING **: [tracker-source-request] grl-tracker-source-api.c:504: Could not execute sparql query id=3: parser stack overflow

------------------------------------------

The covers' and artists' views are then empty. The songs' view, however, shows the full list.

It is my understanding that this is due to the stack size limit of sqlite3, which is compiled in.

sqlite> .limit
              length 1000000000
          sql_length 1000000000
              column 2000
          expr_depth 1000
     compound_select 500
             vdbe_op 250000000
        function_arg 127
            attached 10
 like_pattern_length 50000
     variable_number 999
       trigger_depth 1000
      worker_threads 0

However, it appears that hitting the limit shows that there is a problem in the complexity of the underlying query.

I wonder if this bug is on Grilo's side rather than GNOME Music.

Attached is a sample log of a run obtained with -d.
Comment 1 lorenzo benvenuti 2017-06-20 20:26:10 UTC
Same error here; version 3.24.2 on a fresh Antergos install, less than 500 files in my Music folder. I tried to reset tracker (tracker reset --hard) with no success.

Thank you,

lorenzo
Comment 2 Marinus Schraal 2017-07-17 10:15:15 UTC
It would be interesting to know what tracker queries trigger this for you, 'TRACKER_VERBOSITY=3 gnome-music' should hopefully show this. Could you attach a log?
Comment 3 lorenzo benvenuti 2017-07-17 21:19:06 UTC
Created attachment 355783 [details]
Logs at verbosity level 3
Comment 4 lorenzo benvenuti 2017-07-17 21:19:39 UTC
Logs attached. Please let me know if I can do something else.

Thank you,

lorenzo
Comment 5 Carlos Garnacho 2017-07-18 17:09:00 UTC
(In reply to Matteo Settenvini from comment #0)
> Created attachment 353251 [details]
> Output log of gnome-music -d
> 
> Hi,
> 
> I have about 22000 songs in my Music/ folder. Every time I try to start
> gnome-music 3.24.2 on Fedora 26, I get the errors:
> 
> ------------------------------------------
> 
> (gnome-music:15980): Grilo-WARNING **: [tracker-source-request]
> grl-tracker-source-api.c:504: Could not execute sparql query id=2: parser
> stack overflow
> 
> (gnome-music:15980): Grilo-WARNING **: [tracker-source-request]
> grl-tracker-source-api.c:504: Could not execute sparql query id=3: parser
> stack overflow
> 
> ------------------------------------------
> 
> The covers' and artists' views are then empty. The songs' view, however,
> shows the full list.
> 
> It is my understanding that this is due to the stack size limit of sqlite3,
> which is compiled in.
> 
> sqlite> .limit
>               length 1000000000
>           sql_length 1000000000
>               column 2000
>           expr_depth 1000
>      compound_select 500
>              vdbe_op 250000000
>         function_arg 127
>             attached 10
>  like_pattern_length 50000
>      variable_number 999
>        trigger_depth 1000
>       worker_threads 0
> 
> However, it appears that hitting the limit shows that there is a problem in
> the complexity of the underlying query.

Nice analysis. However the limit we are hitting here is a bit more sneaky, can be triggered with just:

sqlite3 ":memory:" "select (((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((1)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))));"

This limit is not advertised in https://sqlite.org/limits.html, maybe it's related to point 5 (Maximum depth on an expression tree, defaulting to 1000 in docs and your provided info), but however I see I just need 93 parens on each side to trigger a parser overflow error, which is oddly specific.

> 
> I wonder if this bug is on Grilo's side rather than GNOME Music.
> 
> Attached is a sample log of a run obtained with -d.

Thanks, that helped. I see that the Italian translation adds a few extra articles to cut down from the beginning of titles, each of which increments the depth of the query. This can be simplified, at least in a way that the current locale doesn't influence the complexity of the query.
Comment 6 Carlos Garnacho 2017-07-19 12:25:16 UTC
Created attachment 355942 [details] [review]
query: Use tracker:title-order() SPARQL function for title sorting

This function can be used to remove common articles when sorting, so
avoid open-coding that in the SPARQL queries. This also brings the
benefit that queries turn a lot less complex, so there's less chances
to hit parser limits.
Comment 7 Carlos Garnacho 2017-07-19 12:26:25 UTC
Note that this depends on https://git.gnome.org/browse/tracker/commit/?id=80606e70475409f1e3dea47aee15a4c3b038aef9 in tracker master.
Comment 8 Matteo Settenvini 2017-07-19 16:55:50 UTC
(In reply to Carlos Garnacho from comment #7)
> Note that this depends on
> https://git.gnome.org/browse/tracker/commit/
> ?id=80606e70475409f1e3dea47aee15a4c3b038aef9 in tracker master.

Thanks, I was trying to make sense of the patch without seeing this commit. Now I get it.

I noticed that you always assume that an article is followed by a space, though. This is not true at least in Italian and French. You have "l'enfant prodige" or "all'amore cieco" Which contain an elision of a vowel with the next word (the apostrophe). No space is present there.

Should I open a separate bug?
Comment 9 Carlos Garnacho 2017-07-19 17:05:22 UTC
(In reply to Matteo Settenvini from comment #8)
> (In reply to Carlos Garnacho from comment #7)
> > Note that this depends on
> > https://git.gnome.org/browse/tracker/commit/
> > ?id=80606e70475409f1e3dea47aee15a4c3b038aef9 in tracker master.
> 
> Thanks, I was trying to make sense of the patch without seeing this commit.
> Now I get it.
> 
> I noticed that you always assume that an article is followed by a space,
> though. This is not true at least in Italian and French. You have "l'enfant
> prodige" or "all'amore cieco" Which contain an elision of a vowel with the
> next word (the apostrophe). No space is present there.
> 
> Should I open a separate bug?

Oh, good point. Yes, please open one on tracker product.
Comment 10 Matteo Settenvini 2017-07-19 19:19:28 UTC
(In reply to Carlos Garnacho from comment #9)
> (In reply to Matteo Settenvini from comment #8)
> >
> > Thanks, I was trying to make sense of the patch without seeing this commit.
> > Now I get it.
> > 
> > I noticed that you always assume that an article is followed by a space,
> > though. This is not true at least in Italian and French. You have "l'enfant
> > prodige" or "all'amore cieco" Which contain an elision of a vowel with the
> > next word (the apostrophe). No space is present there.
> > 
> > Should I open a separate bug?
> 
> Oh, good point. Yes, please open one on tracker product.

I filed https://bugzilla.gnome.org/show_bug.cgi?id=785146
Comment 11 Marinus Schraal 2017-08-07 22:40:20 UTC
Attachment 355942 [details] pushed as 6d63667 - query: Use tracker:title-order() SPARQL function for title sorting