After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 500447 - EOS and EOF across a named pipe is .... difficult
EOS and EOF across a named pipe is .... difficult
Status: RESOLVED INCOMPLETE
Product: GStreamer
Classification: Platform
Component: gstreamer (core)
git master
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2007-11-29 16:48 UTC by Peter Maersk-Moller
Modified: 2011-08-11 12:51 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Peter Maersk-Moller 2007-11-29 16:48:32 UTC
Hi.

Im trying to make gstreamer to work with an external application through named pipes. Named pipes are like files in many aspects, but opening a named pipe reading is blocked until it is also opened somhow for writing. And likewise when openeing for writing.

This is how I set it up using CVS from yesterday (in this example 'cat' is the external application) 

$ ls -l >myfile1
$ mkfifo raw_pipe encoded_pipe
$ cat <raw_pipe >encoded_pipe &
$ gst-launch -v filesrc location=myfile1 \
    ! queue \
    ! filesink location=raw_pipe \
    filesrc location=encoded_pipe \
    ! queue \
    ! filesink  location=myfile2
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

What you can observer is that a) not all data is copied from myfile1 to myfile2 and b) gst-launch never exits.

This is what I conclude :

When the first filesrc detects EOF, it sends an EOS downstream but thats all. The first filsink receives the EOS event and probably flushes its buffers, but keeps its fd to raw_pipe open so far.

The 'cat' process reading from raw_pipe never gets an EOF because Gstreamer keeps its fd on raw_pipe open. Subsequently 'cat' never closes its fd to encoded_pipe and since it is opened in a BLOCKING rather than a NONBLOCKING mode, 'cat' probably even keeps some data back in its internal buffers.

Now since cat never closes encoded_pipe for writing, Gstreamer never receives an EOF on it second filesrc reading encoded_pipe. Subsequently Gstreamer can not close its open fd's and it can not terminate. 

You can't exactly say it is a bug in filesink, but it limits the usability of Gstreamer together with named pipes.

If the event handling of EOS in gstfilesink.c line xxx is changed from
----------------------
  case GST_EVENT_EOS:
        if (fflush (filesink->file))
          goto flush_failed;
        break;
----------------------
to
----------------------
  case GST_EVENT_EOS:
        if (fflush (filesink->file))
          goto flush_failed;
        if (!filesink->seekable) gst_file_sink_close_file (filesink);
      break;
----------------------
then non seekable files (like named pipes) are closed upon EOS and subsequently signalling EOF to external sources reading the named pipes.

Closing a fd is the only way to signal EOF across a named pipe.

Kind regards

Peter Maersk-Moller
Comment 1 David Schleef 2007-12-07 23:36:15 UTC
I can't say that I care that filesrc/sink don't work with named pipes.  Few things do.

I recommend using sockets instead of named pipes.  Their behavior is much better overall.  You'll still need to use something other than filesrc and filesink.
Comment 2 Mark Nauwelaerts 2008-03-02 21:56:15 UTC
I could be overlooking something here (at this time of the day :) ),
but wouldn't the problem also be solved by running the 2 pipelines in separate instances of gst-launch (rather than in 1 instance as seems to be the case now).
After all, with 2 processes, if the first one reaches EOS, gst-launch should exit, which will take down the first filesink, and close the fd.
So, cat gets an EOS, can finish its business, and the second filesrc should get EOS, and shut down.  In summary, the following works for me:
mkfifo rpipe epipe
cat <rpipe >epipe &
gst-0.10 gst-launch filesrc location=myfile1 ! queue ! filesink location=rpipe &
gst-0.10 gst-launch filesrc location=epipe ! queue ! filesink location=myfile2

Alternatively, the following element (or one like it) might be useful (part of a transcoding oriented set):
http://gentrans.sourceforge.net/docs/head/gst-entrans-plugins/html/gst-entrans-plugins-procpipe.html
It is meant to make piping data through external processes easier to integrate into a pipeline, and has been tested sucessfully (e.g. using mp2enc from mjpegtools as a layer 1 encoder, if one would really want to ...).
Comment 3 Sebastian Dröge (slomo) 2011-05-19 06:12:06 UTC
Is here still a problem? Mark's suggestion sounds like it would fix your problem
Comment 4 Fabio Durán Verdugo 2011-06-29 04:04:59 UTC
Mark ping
Comment 5 Stefan Sauer (gstreamer, gtkdoc dev) 2011-06-30 05:18:33 UTC
(In reply to comment #4)
> Mark ping

The balls is back on Peters side to try this.
Comment 6 Akhil Laddha 2011-08-11 12:51:20 UTC
Closing this bug report as no further information has been provided. Please feel free to reopen this bug if you can provide the information asked for.
Thanks!