After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 571722 - shout2send: use nonblocking network I/O
shout2send: use nonblocking network I/O
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal enhancement
: 1.13.1
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2009-02-14 13:39 UTC by marco
Modified: 2017-05-21 13:37 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
add nonblocking io option to shout2send (7.49 KB, patch)
2009-02-14 13:42 UTC, marco
none Details | Review
add nonblocking i/o for shout2send (7.49 KB, patch)
2009-02-14 14:47 UTC, marco
none Details | Review
nonblocking i/o patch against current git tree (7.40 KB, patch)
2009-05-10 06:44 UTC, marco
none Details | Review
shout2send: use non-blocking I/O and a configurable network queue limit (6.07 KB, patch)
2017-04-24 14:25 UTC, George Kiagiadakis
none Details | Review
shout2send: use non-blocking I/O and a configurable network operations timeout (8.77 KB, patch)
2017-05-21 13:00 UTC, George Kiagiadakis
committed Details | Review

Description marco 2009-02-14 13:39:34 UTC
Currently shout2send use libshout in blocking mode. This leads to the complete stall of a pipeline when network problems are detected (even more, the "connection-problem" signal is emitted with big delay cause shout2send will block on a shout_send() call whenever there are network problems.
Comment 1 marco 2009-02-14 13:42:29 UTC
Created attachment 128716 [details] [review]
add nonblocking io option to shout2send

this works only for the mp3 format. It allows to recovere from network problems reconnecting to the icecast server.
Comment 2 Tim-Philipp Müller 2009-02-14 13:47:29 UTC
Cool, thanks for the patch. What's the problem with the vorbis headers/reconnect exactly? It would be nice if we could make this work and then just use non-blocking I/O by default instead of using a property and/or changing any other defaults.

Comment 3 marco 2009-02-14 14:05:24 UTC
i added as a property as the code is not yet tested well (it's a quick workaround at least, sorry for this).
I did not investigate the vorbis problem but:
- on reconnect the mountpoint is correctly listed by the icecast server
  (but song metadatata are not listed anymore, for example)
- trying to connect to the mountpoint with a client (tried vlc, totem, mplayer) does not even reach the buffering state

i collected some data with tcpdumo -i interface -s0 -X and i saw a single difference:
on the very first connect, the stream includes an header (specified by the vorbis format) (OggS....vorbis.....) which is not sent on reconnecting (probably all the pipleline must be stopped and then restarted on each reconnection tentative, and in this way the encoder will send that header again). take all this as "chatting", as i am not a developer and have little knowledge of gstreamer. 

the proposed patch also change SHOUT_FORMAT_VORBIS (seems deprecated) in SHOUT_FORMAT_OGG.
Comment 4 marco 2009-02-14 14:47:45 UTC
Created attachment 128720 [details] [review]
add nonblocking i/o for shout2send
Comment 5 marco 2009-05-10 06:44:03 UTC
Created attachment 134346 [details] [review]
nonblocking i/o patch against current git tree

This is an update of the nonblocking i/o patch against the current git tree.
Comment 6 Sebastian Dröge (slomo) 2009-08-09 10:38:56 UTC
Any news on this? The patch still needs some work because of the vorbis-reconnect problems...
Comment 7 Timo Häkli 2010-05-10 08:18:52 UTC
Please, complete this bug. I'm waiting for vorbis fixes. Thank you!
Comment 8 George Kiagiadakis 2017-04-24 14:25:06 UTC
Created attachment 350303 [details] [review]
shout2send: use non-blocking I/O and a configurable network queue limit

I happened to come across this bug recently and took the liberty of creating a new patch. At this moment in time there seems to be no problem with vorbis (not sure what was the problem back then either...). Everything seems to work like in the blocking version.
Comment 9 Jérôme Lebleu 2017-04-29 17:25:08 UTC
I'm experiencing this issue too while trying to send a vorbis encoded audio stream to an Icecast server.

The audio input stream is both recorded to a file and sent to Icecast using the tee plugin. Even if both tasks are done in a queue - correct me if I'm speaking nonsense, I'm just learning GStreamer... -, the whole pipeline is blocked when there is a network connexion issue. As a result, it stops to record the input stream to the file too and I have to kill the program. It's quite embarrassing for a live record if there is no fallback, even if a file record. It may also be my mistake too, I can paste my code as needed.

I'm running Debian Stretch with GStreamer 1.10.4 so.

Thanks in advance!
Comment 10 George Kiagiadakis 2017-05-20 16:53:04 UTC
(In reply to Jérôme Lebleu from comment #9)
> The audio input stream is both recorded to a file and sent to Icecast using
> the tee plugin. Even if both tasks are done in a queue - correct me if I'm
> speaking nonsense, I'm just learning GStreamer... -, the whole pipeline is
> blocked when there is a network connexion issue. As a result, it stops to
> record the input stream to the file too and I have to kill the program. It's
> quite embarrassing for a live record if there is no fallback, even if a file
> record. It may also be my mistake too, I can paste my code as needed.

You should use a leaky queue before shout2send to avoid these issues if you have a live pipeline. The patch that I submitted doesn't solve this issue, it will still block upstream when the network is not responsive and still expects you to handle this matter upstream. The *difference* though, is that you can now change state while it's blocking and it will unblock. Previously, you had to kill the process.

Btw, anyone to review this?
Comment 11 Tim-Philipp Müller 2017-05-20 17:09:36 UTC
The 30s connect timeout should probably be a separate property.

What I don't think is right is to treat the internal queue building up as a fatal error, it could happen when there's a transient network problem, in which case we don't want to error out, or if bandwidth is too low. This makes things more brittle than robust IMHO, at least if I understand the patch correctly.

I think there should be a time-based timeout that is reset whenever we successfully send more data, or was there a reason that can't be done?
Comment 12 George Kiagiadakis 2017-05-21 13:00:32 UTC
Created attachment 352271 [details] [review]
shout2send: use non-blocking I/O and a configurable network operations timeout

Here is a better approach, as we discussed. The idea now is that I monitor the rate at which libshout's internal queue empties out and trigger an error when this rate is 0 for a certain, configurable, amount of time.
Comment 13 Tim-Philipp Müller 2017-05-21 13:19:05 UTC
Cool, but let's make the timeout property future-proof by making it millisecs, as discussed.
Comment 14 George Kiagiadakis 2017-05-21 13:36:10 UTC
Committed, with a small change to take the timeout in milliseconds, with a 500ms accuracy. Reviewed by Tim.

commit b26d44501c2e7e720cd4108aec7b835fe8555dab
Author: George Kiagiadakis <george.kiagiadakis@collabora.com>
Date:   Mon Apr 24 16:55:22 2017 +0300

    shout2send: use non-blocking I/O and a configurable network operations timeout
    
    This allows timing out on network errors much earlier
    (currently it takes ~15min to timeout) and we can still
    unlock and change state in the meantime.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=571722