After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 696713 - gvfsd-http crashed with SIGSEGV in create_reply()
gvfsd-http crashed with SIGSEGV in create_reply()
Status: RESOLVED FIXED
Product: gvfs
Classification: Core
Component: http backend
1.16.x
Other Linux
: Normal normal
: ---
Assigned To: gvfs-maint
gvfs-maint
Depends on:
Blocks:
 
 
Reported: 2013-03-27 16:11 UTC by Sebastien Bacher
Modified: 2014-07-03 19:19 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
bug fix suggested by Stephen M. Webb (1.42 KB, patch)
2013-03-27 20:19 UTC, Sebastien Bacher
committed Details | Review

Description Sebastien Bacher 2013-03-27 16:11:23 UTC
The bug has been reported on https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/1065082

The issue seems to happen often when using remote searches in the unity dash (that's doing quite some http requests to fetch thumbnails and content for the results)

"#0  0x0000000000413d8d in create_reply (job=0x7f2c8c0221f0, object=0x2f2f370, invocation=0x2f2e4c0) at gvfsjobopenforread.c:183
        open_job = 0x7f2c8c0221f0
        channel = 0x2995c40
        error = 0x0
        remote_fd = -1
        fd_id = -1
        fd_list = 0x2691ad0
        __PRETTY_FUNCTION__ = "create_reply"
  • #1 send_reply
    at gvfsjobdbus.c line 160
  • #2 _g_closure_invoke_va
    at /build/buildd/glib2.0-2.34.0/./gobject/gclosure.c line 840
  • #3 g_signal_emit_valist
    at /build/buildd/glib2.0-2.34.0/./gobject/gsignal.c line 3211
  • #4 g_signal_emit
    at /build/buildd/glib2.0-2.34.0/./gobject/gsignal.c line 3356
  • #5 g_vfs_job_send_reply
    at gvfsjob.c line 236
  • #6 g_vfs_job_succeeded
    at gvfsjob.c line 302
  • #7 open_for_read_ready
    at gvfsbackendhttp.c line 367
  • #8 wrapper_callback
    at soup-input-stream.c line 529
  • #9 g_simple_async_result_complete
    at /build/buildd/glib2.0-2.34.0/./gio/gsimpleasyncresult.c line 775
  • #10 send_async_finished
    at soup-input-stream.c line 591
  • #11 g_closure_invoke
    at /build/buildd/glib2.0-2.34.0/./gobject/gclosure.c line 777
  • #12 signal_emit_unlocked_R
    at /build/buildd/glib2.0-2.34.0/./gobject/gsignal.c line 3551
  • #13 g_signal_emit_valist
    at /build/buildd/glib2.0-2.34.0/./gobject/gsignal.c line 3300
  • #14 g_signal_emit
    at /build/buildd/glib2.0-2.34.0/./gobject/gsignal.c line 3356
  • #15 soup_message_got_headers
    at soup-message.c line 1062
  • #16 io_read
    at soup-message-io.c line 568
  • #17 io_run_until
    at soup-message-io.c line 828
  • #18 io_run
    at soup-message-io.c line 891
  • #19 g_main_dispatch
    at /build/buildd/glib2.0-2.34.0/./glib/gmain.c line 2715
  • #20 g_main_context_dispatch
    at /build/buildd/glib2.0-2.34.0/./glib/gmain.c line 3219
  • #21 g_main_context_iterate
    at /build/buildd/glib2.0-2.34.0/./glib/gmain.c line 3290
  • #22 g_main_loop_run
    at /build/buildd/glib2.0-2.34.0/./glib/gmain.c line 3484
  • #23 daemon_main
    at daemon-main.c line 396

Comment 1 Sebastien Bacher 2013-03-27 20:19:53 UTC
Created attachment 239986 [details] [review]
bug fix suggested by Stephen M. Webb

This patch is a suggest fix from Stephen M. Webb, the segfaults happens in the case where socketpair() is hitting an error
Comment 2 Tomas Bzatek 2013-04-02 14:39:06 UTC
(In reply to comment #0)
>         fd_list = 0x2691ad0

We're passing non-NULL pointer to g_unix_fd_list_append() so I was wondering why it returns -1 and doesn't set an error. Perhaps remote_fd is <= 0 because g_vfs_channel_steal_remote_fd() failed?

Any chance to reproduce this and getting a value of remote_fd?

(the patch looks generally good but we may want to fail in nicer way - expecting the code would fail/crash somewhere else with even uglier error)
Comment 3 Sebastien Bacher 2013-04-02 14:43:37 UTC
the bottom line of the issue is that the system is running out of available fds, it should be easy enough to write a small buggy program that open fds in loop until hitting that situation ... I can try to have a go to that later in the week, what info would you need exactly from then?
Comment 4 Tomas Bzatek 2013-04-03 13:30:54 UTC
(In reply to comment #3)
> the bottom line of the issue is that the system is running out of available
> fds, it should be easy enough to write a small buggy program that open fds in
> loop until hitting that situation ... I can try to have a go to that later in
> the week, what info would you need exactly from then?

Eh, scratch my question please, all the information I need is in the first post. I've eventually ended up reproducing the issue, it's preceded by the following error:

> ** (process:22711): WARNING **: Error creating socket pair: Too many open files

I've committed the segfault fix based on the above patch:

commit 94f8380381c46b1512712be1192894dd9a9e12f0
Author: Tomas Bzatek <tbzatek@redhat.com>
Date:   Wed Apr 3 15:08:20 2013 +0200

    gvfschannel: Return proper error if we're out of free fds
    
    In case of too many open files within the process the
    g_vfs_channel_init() call fails on socketpair and subsequent
    g_vfs_channel_steal_remote_fd() call returns -1 for the fd.
    Then g_unix_fd_list_append() hits the assert and doesn't set an
    error we're dereferencing afterwards.
    
    This patch doesn't solve the lack of free fds situation and since glib
    heavily depends on it it would fail somewhere else. We're just fixing
    the segfault and returning nicer error.
    
    Based on a fix suggested by Stephen M. Webb
    
    https://bugzilla.gnome.org/show_bug.cgi?id=696713


This however doesn't fix the issue fully - we don't segfault anymore but fail elsewhere, randomly depending on things that require fds. In http case, I get 

> (process:22711): GLib-ERROR **: Creating pipes for GWakeup: Too many open files
> 
> Trace/breakpoint trap

resulting in process abortion. We're already handling SIGPIPE signals gracefully, this is SIGTRAP. Ignoring the signal makes the backend loop forever on random places (seen libsoup for example).

This is something that would be nice to have fixed all over gvfs, only not sure if it's worth the effort given how much we are dependent on free fds.
Comment 5 Ross Lagerwall 2014-07-03 19:19:19 UTC
The fd leak bug should be fixed in bug 732090.

Regarding free fds, we are dependent on them and auditing the entire codebase to ensure that we are well-behaved when running out of fds is probably more effort than gain.  We should just fix any reported problems that are reported.