After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 792337 - Can't index non-native directories
Can't index non-native directories
Status: RESOLVED OBSOLETE
Product: tracker
Classification: Core
Component: Miners
2.0.x
Other All
: Normal normal
: ---
Assigned To: tracker-general
tracker-general
Depends on:
Blocks: 751212
 
 
Reported: 2018-01-08 18:20 UTC by Debarshi Ray
Modified: 2021-05-26 22:24 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
libtracker-miner: Support indexing non-native directories (1.27 KB, patch)
2018-01-08 18:31 UTC, Debarshi Ray
none Details | Review
tests: Add a test for indexing non-native directories (2.31 KB, patch)
2018-01-08 18:32 UTC, Debarshi Ray
none Details | Review
libtracker-miner: Support indexing non-native directories (1.41 KB, patch)
2018-01-09 13:11 UTC, Debarshi Ray
none Details | Review
tests: Add a test for indexing non-native directories (2.31 KB, patch)
2018-01-09 13:11 UTC, Debarshi Ray
none Details | Review
libtracker-miner: Support caching non-native files (2.31 KB, patch)
2018-01-09 13:12 UTC, Debarshi Ray
none Details | Review
tests: Add a test for caching non-native files (1.75 KB, patch)
2018-01-09 13:12 UTC, Debarshi Ray
none Details | Review
libtracker-miner: Rename a variable for consistency (1.63 KB, patch)
2018-01-09 13:13 UTC, Debarshi Ray
committed Details | Review
libtracker-miner: Support enumerating non-native files (4.80 KB, patch)
2018-01-09 13:13 UTC, Debarshi Ray
none Details | Review
libtracker-miner: Don't mix sync & async GFileEnumerator APIs (4.91 KB, patch)
2018-01-29 22:02 UTC, Debarshi Ray
committed Details | Review

Description Debarshi Ray 2018-01-08 18:20:50 UTC
It is not possible to index non-native directories (eg., gphoto2://.../) using IndexFile and IndexFileForProcess because tracker_indexing_tree_add expects everything to get resolved as a file:/// child. Directly indexing a non-native file works because it doesn't go through TrackerIndexingTree.

One option is to special-case such locations as file:/// children. Even though g_file_has_prefix doesn't work across different URI schemas, conceptually, it still makes sense to think of a remote WebDAV mount or an attached camera as a child of file:///.
Comment 1 Debarshi Ray 2018-01-08 18:31:35 UTC
Created attachment 366504 [details] [review]
libtracker-miner: Support indexing non-native directories
Comment 2 Debarshi Ray 2018-01-08 18:32:03 UTC
Created attachment 366505 [details] [review]
tests: Add a test for indexing non-native directories
Comment 3 Debarshi Ray 2018-01-09 12:55:41 UTC
There is a similar issue in TrackerFileSystem, and GVfs doesn't like how TrackerFileDataProvider creates a GFileEnumerator through the synchronous API but uses it asynchronously later.
Comment 4 Debarshi Ray 2018-01-09 13:11:13 UTC
Created attachment 366547 [details] [review]
libtracker-miner: Support indexing non-native directories
Comment 5 Debarshi Ray 2018-01-09 13:11:39 UTC
Created attachment 366548 [details] [review]
tests: Add a test for indexing non-native directories
Comment 6 Debarshi Ray 2018-01-09 13:12:11 UTC
Created attachment 366549 [details] [review]
libtracker-miner: Support caching non-native files
Comment 7 Debarshi Ray 2018-01-09 13:12:49 UTC
Created attachment 366550 [details] [review]
tests: Add a test for caching non-native files
Comment 8 Debarshi Ray 2018-01-09 13:13:16 UTC
Created attachment 366551 [details] [review]
libtracker-miner: Rename a variable for consistency
Comment 9 Debarshi Ray 2018-01-09 13:13:39 UTC
Created attachment 366552 [details] [review]
libtracker-miner: Support enumerating non-native files
Comment 10 Carlos Garnacho 2018-01-29 14:51:37 UTC
Review of attachment 366552 [details] [review]:

Looks like it's even a cleanup :). Minor nit with the commit log, I think it would be clearer to make "avoid mixing sync & async enumerator API" be the subject (as it's the gist of the commit) and move the "fixes non-native files" into the blurb.

::: src/libtracker-miner/tracker-file-data-provider.c
@@ +178,3 @@
+	                                 cancellable,
+	                                 enumerate_children_cb,
+	                                 g_object_ref (task));

Looks like this ref is being leaked in enumerate_children_cb(), I think there should be a g_object_unref after g_task_return* there.
Comment 11 Debarshi Ray 2018-01-29 22:02:29 UTC
Created attachment 367613 [details] [review]
libtracker-miner: Don't mix sync & async GFileEnumerator APIs
Comment 12 Carlos Garnacho 2018-01-30 20:40:08 UTC
Comment on attachment 367613 [details] [review]
libtracker-miner: Don't mix sync & async GFileEnumerator APIs

Looks good!
Comment 13 Debarshi Ray 2018-01-31 12:00:20 UTC
Comment on attachment 366551 [details] [review]
libtracker-miner: Rename a variable for consistency

From #tracker on GIMPNet:

10:57 <rishi> garnacho: Hey! I just realized that the async/sync                
      FileEnumerator patch is on top of                                         
      https://bug792337.bugzilla-attachments.gnome.org/attachment.cgi?id=366551
10:58 <rishi> It renames a variable.
11:04 <garnacho> rishi: hey :), makes sense
11:05 <garnacho> well, I guess it'll make more sense when remote locations are  
      actually supported :p
11:05 <garnacho> but seems fine to have
11:07 <rishi> garnacho: The s/dir/url/ change is to just bring it in line with  
      the sync implementation, and the public tracker_data_provider_begin*      
      methods.
11:10 <garnacho> oh, indeed
Comment 14 Sam Thursfield 2021-05-26 22:24:48 UTC
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org.
As part of that, we are mass-closing older open tickets in bugzilla.gnome.org
which have not seen updates for a longer time (resources are unfortunately
quite limited so not every ticket can get handled).

If you can still reproduce the situation described in this ticket in a recent
and supported software version, then please follow
  https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines
and create a new enhancement request ticket at
  https://gitlab.gnome.org/GNOME/tracker/-/issues/

Thank you for your understanding and your help.