After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 706380 - checkout: too many open files
checkout: too many open files
Status: RESOLVED FIXED
Product: ostree
Classification: Infrastructure
Component: general
unspecified
Other Linux
: Normal normal
: ---
Assigned To: OSTree maintainer(s)
OSTree maintainer(s)
Depends on:
Blocks:
 
 
Reported: 2013-08-20 11:11 UTC by Colin Walters
Modified: 2013-08-21 13:30 UTC
See Also:
GNOME target: ---
GNOME version: ---

Attachments
libostree: Check out directories depth-first in serial (rather than parallel) (3.97 KB, patch)
2013-08-20 23:37 UTC, Colin Walters 		none Details | Review
libostree: Check out directories depth-first in serial, switch to sync API (29.75 KB, patch)
2013-08-21 13:09 UTC, Colin Walters 		accepted-commit_now Details | Review

Description Colin Walters 2013-08-20 11:11:22 UTC 
See:

https://bugzilla.gnome.org/show_bug.cgi?id=687223#c1

This is really a fundamental design issue with GLib; however, we can work around it in a few ways:

1) When we're root, bump up our own fd limit
2) Drop the parallelism =(
Comment 1 Colin Walters 2013-08-20 23:37:25 UTC 
Created attachment 252495 [details] [review]
libostree: Check out directories depth-first in serial (rather than parallel)

The way we recurse into subdirectories in parallel makes it far too
easy to hit up against the arbitrary Linux fd limit of 1024.

This is a speed hit, but it's the best we can do for now.
Comment 2 Colin Walters 2013-08-21 13:09:59 UTC 
Created attachment 252551 [details] [review]
libostree: Check out directories depth-first in serial, switch to sync API

The way we recurse into subdirectories in parallel makes it far too
easy to hit up against the arbitrary Linux fd limit of 1024.

Since the fix here is about dropping parallelism, let's just go all
the way for now and make a plain old synchronous API =(

This does simplify both internal callers which wanted a sync API
anyways.
Comment 3 Jasper St. Pierre (not reading bugmail) 2013-08-21 13:21:22 UTC 
Review of attachment 252551 [details] [review]:

Looks good.
Comment 4 Colin Walters 2013-08-21 13:30:14 UTC 
Attachment 252551 [details] pushed as caf6be3 - libostree: Check out directories depth-first in serial, switch to sync API