That error comes from:

https://git.gnome.org/browse/tracker/tree/src/libtracker-data/tracker-db-interface-sqlite.c#n380

That's the implementation of tracker:uri-is-descendant, which AFAICS it's used as:

  tracker:uri-is-descendant ($location_uri, nie:url(?urn))

The warning is about the last argument nie:url(?urn), which tbh I find strange that it's NULL, given that we're talking about nfo:FileDataObjects... Could you tell me if the following query returns values?:

tracker sparql -q "select ?urn { ?urn a nfo:FileDataObject . FILTER (! BOUND (nie:url (?urn))) }"

If some value is returned, I'd appreciate you'd paste here the output of "tracker info" with some of those URNs.

However, I guess it is not completely impossible to hit that, I'm attaching a couple of patches nonetheless. The first should get rid of this if I'm right here, and the second actually gets rid of tracker:uri-is-descendant/parent, as the performance of those functions doesn't shine.

Created attachment 315985 [details] [review]
searchenginetracker: ensure nie:url is bound

This could produce strange warnings as it is currently passed to
tracker:uri-is-* sparql functions, as these expect no NULLs.

Created attachment 315986 [details] [review]
searchenginetracker: Optimize direct/recursive folder lookups

tracker:uri-is-descendant/parent has the unfortunate side effect of
rendering the collation mechanisms in the database useless, so those
require full table scans to be validated.

Performing these as pure string comparisons will perform much better,
as those allow the underlying sqlite to rely on its own collation
to perform the search, which can be significantly faster with many
elements in the database.

I have no clue how any of this works, but I can copy commands and press enter!



$ tracker sparql -q "select ?urn { ?urn a nfo:FileDataObject . FILTER (! BOUND (nie:url (?urn))) }"
Results:
  urn:theme-icon:libreoffice-startcenter
  urn:theme-icon:/usr/share/ibus-libpinyin/icons/ibus-pinyin.svg
  urn:theme-icon:system-software-install
  file:///usr/share/applications/gpk-install-catalog.desktop
  file:///usr/share/applications/gnome-software.desktop
  file:///usr/share/desktop-directories/X-GNOME-WebApplications.directory
  urn:uuid:22da2de3-abf3-1d4e-8766-a686aaa57ebb
  urn:uuid:a2a0494f-338c-e340-7a30-b85a90b7ec82
  urn:uuid:7f97e21a-fe84-944b-19d6-46d3beefdea6
  urn:uuid:4fb4a0a8-6a30-43b5-a028-ff388e7ec5c0
  urn:uuid:cd3ce175-83d9-57d4-30f4-8dedf72b6076
  urn:uuid:6583a051-963b-522d-5794-8d4c9979339f
  urn:uuid:c07e6540-e9bf-79cc-89b4-911a8e7c0d3d
  urn:uuid:0b72d53f-fb35-c3bb-a37c-dff002ab8a77
  urn:uuid:e8b22457-8737-3680-2d97-6df59c346bf5
  urn:uuid:2caa5e32-08cd-eced-6ea0-3cbc41c23597
  urn:uuid:64aec8ac-262b-0399-3387-a71a7ffff128
  urn:uuid:97afc242-0d79-ecc1-9737-a5e9ec002556
  urn:uuid:e7f68332-e3ef-98a0-da48-1714a4a9b67b
  (null)
  urn:theme-icon:gnome-nibbles
  urn:theme-icon:four-in-a-row
  (null)
  urn:uuid:56ae4888-af6e-82e8-254f-e1f16032ead7
  (null)
  urn:uuid:41200457-f173-a280-4eb9-8521cf2e740b
  (null)
  urn:uuid:2aa585bc-287f-3aa4-34bc-ad7763c131d8
  urn:uuid:07abceef-6033-cb0c-e04b-314fc31628e4
  (null)
  (null)
  urn:theme-icon:application-vnd.iccprofile
  (null)
  urn:uuid:8b41db63-392f-cbd9-4f5b-d52842a5cd65
  urn:uuid:713decc2-c6b0-741e-94b1-7bdc632e57b5
  https://docs.google.com/feeds/id/document%3A1I76cRnRv69t4FXEMx4PaLQ5L8TZ0G_2vIaoyst_bu_g
  urn:uuid:1ba3f0be-7582-cfae-6cec-c8e44e82e184
  urn:uuid:83f81c2b-dcb9-18f6-e896-e35b43363ca8
  (null)
  (null)
  (null)
  urn:uuid:84cf745c-3344-d0f3-1ba0-af79034f0843
  (null)
  (null)
  (null)
  urn:theme-icon:gpk-log
  (null)
  (null)
  (null)
  (null)
  (null)
  (null)
  (null)
  (null)
  (null)
  (null)
  (null)
  urn:theme-icon:/usr/share/ibus-libpinyin/icons/ibus-bopomofo.svg
  (null)
  (null)
  urn:uuid:e9d9e06c-4f10-19cd-69a4-231b3d95a1fe
  (null)
  urn:theme-icon:applications-other
  urn:theme-icon:preferences-desktop-font
  urn:theme-icon:gnome-disks
  urn:theme-icon:drive-removable-media
  urn:theme-icon:preferences-color
  urn:theme-icon:preferences-desktop-wallpaper
  urn:theme-icon:gnome-abrt
  (null)
  (null)
  urn:theme-icon:security-medium
  urn:theme-icon:gnome-color-manager
  file:///usr/share/applications/gcalctool.desktop
  (null)
  urn:theme-icon:firefox
  file:///usr/share/applications/fedora-release-notes.desktop
  urn:theme-icon:evolution
  (null)
  (null)
  urn:theme-icon:deja-dup



$ tracker info urn:theme-icon:libreoffice-startcenter
Querying information for entity:'urn:theme-icon:libreoffice-startcenter'
Results:
  'tracker:added' = '2015-08-21T18:06:23Z'
  'rdf:type' = 'http://www.w3.org/2000/01/rdf-schema#Resource'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/01/19/nie#DataObject'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/01/19/nie#InformationElement'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Media'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Visual'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Image'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Executable'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Orientation'



$ tracker info file:///usr/share/applications/gpk-install-catalog.desktop
Querying information for entity:'file:///usr/share/applications/gpk-install-catalog.desktop'
Results:
  'tracker:added' = '2015-07-25T15:03:14Z'
  'rdf:type' = 'http://www.w3.org/2000/01/rdf-schema#Resource'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/01/19/nie#DataObject'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/01/19/nie#InformationElement'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Executable'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Orientation'



$ tracker info urn:uuid:22da2de3-abf3-1d4e-8766-a686aaa57ebb
Querying information for entity:'urn:uuid:22da2de3-abf3-1d4e-8766-a686aaa57ebb'
Results:
  'tracker:added' = '2015-04-27T18:51:35Z'
  'rdf:type' = 'http://www.w3.org/2000/01/rdf-schema#Resource'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/01/19/nie#DataObject'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/01/19/nie#InformationElement'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Media'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Visual'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Image'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Executable'
  'rdf:type' = 'http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Orientation'
  'nie:contentCreated' = '2011-12-09T19:06:03Z'

Hmm, right, the applications miner could produce these. Although that doesn't explain all I see there, those nulls returned look somewhat suspicious, and it's not completely clear to me who created that last item you provided the tracker info from...

Anyway, it looks like my patches should fix the gtk+ side, I would be especially interested in tracking how we got those nulls there, but that should probably be taken to a tracker bug.

Is there a way to find out what the last item actually is?

Not quite... actually the rdf:types of any those elements don't make much sense together (eg. nfo:Orientation is completely out of place here, so is mixing media/visual types with executable). It looks like your database got botched at some point in the not too distant past, and those items were left stale.

I eg. see that the second example you pasted (gpk-install-catalog.desktop), has tracker:modified 2015-07-25T15:03:14Z , I'm not sure if that can be relied on at this point, but I certainly remember no recent commits that could have caused anything similar.

I could maybe do some forensics with your ~/.local/tracker/meta.db , if you're willing to hand it :).

(In reply to Carlos Garnacho from comment #7)
> I could maybe do some forensics with your ~/.local/tracker/meta.db , if
> you're willing to hand it :).

Ehm, ~/.cache/...

(In reply to Carlos Garnacho from comment #7)
> Not quite... actually the rdf:types of any those elements don't make much
> sense together (eg. nfo:Orientation is completely out of place here, so is
> mixing media/visual types with executable). It looks like your database got
> botched at some point in the not too distant past, and those items were left
> stale.

Could that explain why 90% of my music has mysteriously disappeared from GNOME Music?

> I eg. see that the second example you pasted (gpk-install-catalog.desktop),
> has tracker:modified 2015-07-25T15:03:14Z , I'm not sure if that can be
> relied on at this point, but I certainly remember no recent commits that
> could have caused anything similar.
>
> I could maybe do some forensics with your ~/.local/tracker/meta.db , if
> you're willing to hand it :).

I have ~/.local/share/tracker/data/.meta.isrunning. Do I have to somehow tell tracker to quit in order to get a meta.db file?

(In reply to Michael Catanzaro from comment #9)
> (In reply to Carlos Garnacho from comment #7)
> > Not quite... actually the rdf:types of any those elements don't make much
> > sense together (eg. nfo:Orientation is completely out of place here, so is
> > mixing media/visual types with executable). It looks like your database got
> > botched at some point in the not too distant past, and those items were left
> > stale.
> 
> Could that explain why 90% of my music has mysteriously disappeared from
> GNOME Music?
> 
> > I eg. see that the second example you pasted (gpk-install-catalog.desktop),
> > has tracker:modified 2015-07-25T15:03:14Z , I'm not sure if that can be
> > relied on at this point, but I certainly remember no recent commits that
> > could have caused anything similar.
> >
> > I could maybe do some forensics with your ~/.local/tracker/meta.db , if
> > you're willing to hand it :).
> 
> I have ~/.local/share/tracker/data/.meta.isrunning. Do I have to somehow
> tell tracker to quit in order to get a meta.db file?

Sorry, I meant ~/.cache/tracker/meta.db. Although bear in mind, that's the actual place where tracker data is stored, so there's the usual privacy implications... I always handle this with due diligence, although I most usually receive the odd file that failed to be extracted, not the full DB.

Anyway, the more I think about this, the more of a legit database corruption it looks to me. This is the first time I see this bug, plus it sounds a bit weird to me due to the way we handle the DB schema and query construction.

So let me change the suggestion, please make a copy of your ~/.cache/tracker directory, reset/restart everything by doing:

  tracker control -r
  tracker control -s

And I'll beg for that data if something like this is ever seen again.

The patches should still be applied probably, it's a fine sanity check, since we can guarantee it's only tracker-miner-fs which inserts nfo:FileDataObjects, nor as nicely.

(In reply to Carlos Garnacho from comment #10)
> The patches should still be applied probably, it's a fine sanity check,
> since we can guarantee it's only tracker-miner-fs which inserts
> nfo:FileDataObjects, nor as nicely.

I don't think anybody is going to complain if you push those. ;)

(In reply to Carlos Garnacho from comment #10)
> So let me change the suggestion, please make a copy of your ~/.cache/tracker
> directory, reset/restart everything by doing:
> 
>   tracker control -r
>   tracker control -s
> 
> And I'll beg for that data if something like this is ever seen again.

Now all my music returned. That's good.

Attachment 315985 [details] pushed as 61d6c1a - searchenginetracker: ensure nie:url is bound
Attachment 315986 [details] pushed as f6dd043 - searchenginetracker: Optimize direct/recursive folder lookups