(In reply to Adrien Plazas from comment #0)
> Games takes quite a long time to look for games and to build the games list.
> 
> I suspect one of the factors to be that we are running one Tracker query per
> system.

I built gnome-games this weekend, and it turns out tracker is nowhere close to being the hot spot here :). Although that said, I still have suggestions about how to make the info available more readily.

What I saw is that gnome-games spends by far most of the time in https://git.gnome.org/browse/gnome-games/tree/src/ui/collection-icon-view.vala#n128 , commenting out the flowbox sort function at https://git.gnome.org/browse/gnome-games/tree/src/ui/collection-icon-view.vala#n52 brings much improved startup times, although still generally high.

Now the bad news, I don't think flowbox can put up with potential thousands of elements here, I think you'd be better off if provided some search/pagination, so the number of elements in the flowbox is always under control.

> 
> I suggest that when a game file is found it should be tagged as such so that
> prior to looking for games per-system, we could just make one query
> demanding all the known games.
> 
> We could have such a workflow:
> 
> - init the game sources
> - query the known games
> - forward the game file to the game source which initially tagged it
> - does the file still exist and is still a game file?
>   - yes: build the game object
>   - no: remove the tag stating it's a game
> - for each game source, query to look for new games
> - is the found game already tagged?
>   - yes: skip it
>   - no: tag the file as a game file
> 
> Not all game sources would have to follow this workflow and they could just
> build games when their own query is ran as it's the case now.
> 
> Pros:
> - it should hopefully make the games list to be loaded faster,
> - if at some point we support games which are hard to detect and we require
> to user to manually locate (ScummVM or DOS games for example) we could
> easily remember which files are games.
> 
> Cons:
> - this workflow could be problematic with games spread across multiple files
> such as multi-disc games.
> 
> Any feedback is welcome as I don't know if it's a good idea. :)

Elaborating further on what we talked on IRC here, I propose a similar workflow  to what tracker-extract does, with the only difference that gnome-games is also the consumer of that data.

1) We add ontology and extract rules to Tracker so that files with well known game mimetypes are tagged as having an eg. gnome:Game tag (class name TBD, I would like at least an intermediate/additional class for software requiring an emulator to work).

That'd make looking up for those games Tracker can distinguish pretty straightforward:

  SELECT ?urn nie:url(?urn)
  WHERE { ?urn a gnome:Game }

2) Now, adding additional metadata, and only doing it once. The query above would give you all games that tracker found, in no special order. You want to add metadata (eg. title) to Tracker and tag the file as "already sniffed" so gnome-games can obtain the info directly and avoid any extra processing for the next times, eg. doing an update like:

  INSERT DATA { <urn:uuid:xxxx> nie:title "Double Dragon" ;
                                nie:dataSource <gnome:games:urn:xxxx> }

Where "urn:uuid:xxxx" is the ?urn you'd obtain in a SELECT query like the above [1] , and "gnome:games:urn:xxxx" is a gnome-games defined identifier. That'd allow querying for previously indexed and unindexed elements separately:

  SELECT ?urn nie:url(?urn) nie:title(?urn)
  WHERE { ?urn a gnome:Game ;
               nie:dataSource <gnome:games:urn:xxxx> }
  ORDER BY (nie:title(?urn))

  SELECT ?urn nie:url(?urn)
  WHERE { ?urn a gnome:Game .
          FILTER (! EXISTS (?urn nie:dataSource <gnome:games:urn:xxxx>)) }

Note the ORDER BY in the first query would allow you to avoid entirely the flowbox sorting function :).

3) There's still some mimetypes which Tracker can't/won't tag as games right away (eg. application/zip for mame roms), there's not many ways to improve the lookup for those (querying by mimetype as you do), however there's a couple of ways to improve this. First, after finding out a game, gnome-games should be promoting those as having the "game" class, eg:

  INSERT DATA { <...> a gnome:Game ;
                      nie:title "..." ;
                      nie:dataSource <...> }

It is actually safe to add again a rdf:type if the resource has it, so you could be doing this kind of query for all games you find. This would have those files already tagged as games for the next times you start gnome-games, so there would be no need to sniff those again.

Second, you certainly don't want to sniff over and over again the files with dubious mimetype that turned out not to be a game. We could maybe use another special nie:dataSource (or the same) to tag those files so that gnome-games can avoid sniffing those after it did the first time. Although tagging resources for the purpose of blacklisting could be seen as polluting the database... I would suggest that you query for the tracker:modified timestamp of those files, keep the most recent and store it somewhere (eg. in ~/.cache) so you can query for the files that changed after your last check:

  SELECT ?urn nie:url(?urn) tracker:modified(?urn)
  WHERE { ?urn a nfo:FileDataObject ;
               nie:mimeType "foo/bar" .
          FILTER (tracker:modified (?urn) > $last_timestamp) }

[1] inserts/selects are a bit simplified, IMO gnome-games should insert data into its own graph, instead of the default one. We'll elaborate further on this when really needed.

Created attachment 351306 [details] [review]
ui: Remove a sort invalidation in CollectionIconView

Don't invalidate the sorting after adding new games as it seems that
FlowBox already add them sorted and hence is always sorted.

This roughly doubles the loading time on my fairly large collection.

Comment on attachment 351306 [details] [review]
ui: Remove a sort invalidation in CollectionIconView

Attachment 351306 [details] pushed as 25ec101 - ui: Remove a sort invalidation in CollectionIconView

I leave the bug open as loading the games is still quite slow.

I can confirm this bug on Fedora 26 with gnome-games 3.25 installed via Flatpak. It takes more than 10 seconds to load the main UI. Scrolling through the collection causes gnome-games to stutter and freeze several times until the whole collection is loaded or until gnome-games crashes. Here are some of the displayed errors:

Tracker-Message: Could not get mtime for 'file:///home/User/.var/app/org.gnome.Games/cache/tracker/meta.db': Error when getting information for file “/home/User/.var/app/org.gnome.Games/cache/tracker/meta.db”: No such file or directory

(gnome-games:2): Tracker-WARNING **: Falling back to bus backend, the direct backend failed to initialize: Could not open sqlite3 database:'/home/User/.var/app/org.gnome.Games/cache/tracker/meta.db': unable to open database file

(gnome-games:2): Gtk-WARNING **: Could not load a pixbuf from icon theme.
This may indicate that pixbuf loaders or the mime database could not be found.



(gnome-games:2): Grilo-WARNING **: [lua-library] grl-lua-library.c:508: Can't fetch element 1 (URL: http://thegamesdb.net/api/GetGamesList.php?name=Pokemon%20Trozei&platform=Nintendo%20DS): 'Service Unavailable'



** (gnome-games:2): WARNING **: game-thumbnail.vala:203: Couldn’t load the icon: Unrecognized image file format


(gnome-games:2): GamesDesktop-WARNING **: nintendo-ds-icon.vala:25: Operation not supported


This bug is rather severe and it makes using gnome-games often difficult.

This should be much better now than it was before, although there are still problems e.g. with MAME plugin that has to load the entire supported-games db each time.

This is basically same as https://gitlab.gnome.org/GNOME/gnome-games/issues/11