Created attachment 316341 [details]
receiver pipeline log

(In reply to Arjen Veenhuizen from comment #0)

> Please also note that the examples in [2,3] are broken since rtprtx*
> elements don't have the rtx-payload-type property anymore but rather the
> payload-type-map property which takes a different value type.

What do you mean with this? The two examples (server-rtpaux.c and client-rtpaux.c) don't use the rtx-payload-type property.


So to reproduce your problem, using those two examples is enough or do you also see other problems that only your manual pipeline exposes?

Ah, I now see that my explanation wasn't that clear :).
What I meant is that the docs on the rtprtxreceive and rtprtxsend elements [1,2] do no longer work due to the changed properties. I was not referencing to your examples in C.

Anyway, with the pipelines given in the first post I do not observe any working RTX. server-rtx.c and client-rtx.c are, however, working just fine.

[1] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good-plugins/html/gst-plugins-good-plugins-rtprtxsend.html
[2] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good-plugins/html/gst-plugins-good-plugins-rtprtxreceive.html

Thanks, that makes sense. I'll try to find some time to take a look at that in the next days if nobody else is faster.

That it's working with the C examples is also what I can confirm here, that's why I was a bit confused :)

Btw, in the server-rtx.c example, I do see the following warning every couple of seconds when running with GST_DEBUG=3:
rtpsource rtpsource.c:1499:rtp_source_get_new_sr: no clock-rate, cannot interpolate rtp time

That also doesn't seem right. So two things to look into: your manual pipelines not working, and server-rtpaux.c complaining about not being able to get clock-rate (which it should after some changes that happened during 1.5).

I assume you're using GIT master?

Yes, I am using GIT master (actually compiled it again yesterday and problems still exist)

I have been digging deeper and I have created a test Sender and Receiver application in Python. This app sends MP2TS in RTP over UDP from one host to another. For now I am just streaming on localhost with sender and receiver running on the same host. Pipelines are attached to this bug report.

At a low bitrate < 30Mbps:
* Virtually no visual distortion is noticed at the receiver with do-retransmission=TRUE
* Virtually no visual distortion is noticed at the receiver with do-retransmission=FALSE
* I see a hand-full of retransmission requests (querying num-rtx-requests and num-rtx-packets) when do-retransmission=TRUE
* I see no retransmission requests when do-retransmission=FALSE (which makes sense)

At a high bitrate > 50Mbps:
* Significant visual distortion is noticed at the receiver with do-retransmission=TRUE
* Virtually no visual distortion is noticed at the receiver with do-retransmission=FALSE
* I see hundreds of retransmission requests (querying num-rtx-requests and num-rtx-packets) when do-retransmission=TRUE
* I see no retransmission requests when do-retransmission=FALSE (which makes sense)

In case of even higher bitrates (250Mbps and up), the problem gets worse and worse. And to make matters even more worse, VLC is playing the raw RTP stream (so without any RTCP at all) just fine. This leads me to assume that this problem is caused by the receiver pipeline. Any thoughts?

Created attachment 316602 [details]
Python based test utility receiver pipeline.

This pipeline does not relate to the already attached receiver log text file.

Created attachment 316603 [details]
Python based test utility sender pipeline image

This pipeline does not relate to the already attached sender log text file.

Sebastian, were you able to reproduce these problems? If you need additional information, please let me know.

Sorry not yet, I was busy with other things

This seems to be related to bug #771383

I have made some tests and it seems that using C examples NACKs also do not work properly.


For testing, I have disabled this piece of code in rtpsession.c, which tries to limit rate of early RTCP packets:

    /*  RFC 4585 section 3.5.2 step 4a and
     *  RFC 4585 section 3.5.2 step 6 */
    allow_early = TRUE;
    if (sess->last_rtcp_check_time == sess->last_rtcp_send_time) {
      /* Last time we sent a full RTCP packet, we can now immediately
       * send an early one as allow_early was reset to TRUE */
      allow_early = TRUE;
    } else if (sess->last_rtcp_check_time + T_rr <= current_time + max_delay) {
      /* Last packet we sent was an early RTCP packet and more than
       * T_rr has passed since then, meaning we would have suppressed
       * a regular RTCP packet already and reset allow_early to TRUE */
      allow_early = TRUE;
    //
      /* We have to offset a bit as T_rr has not passed yet, but will before
       * max_delay */
      if (sess->last_rtcp_check_time + T_rr > current_time)
        offset = (sess->last_rtcp_check_time + T_rr) - current_time;
    } else {
      GST_DEBUG_OBJECT (sess,
          "can't allow early RTCP yet: last regular %" GST_TIME_FORMAT ", %"
          GST_TIME_FORMAT " + %" GST_TIME_FORMAT " > %" GST_TIME_FORMAT " + %"
          GST_TIME_FORMAT, GST_TIME_ARGS (sess->last_rtcp_send_time),
          GST_TIME_ARGS (sess->last_rtcp_check_time), GST_TIME_ARGS (T_rr),
          GST_TIME_ARGS (current_time), GST_TIME_ARGS (max_delay));
    }
    //
    if (!allow_early) {
      /* Ignore the request a scheduled packet will be in time anyway */
      if (current_time + max_delay > sess->next_rtcp_check_time) {
        GST_LOG_OBJECT (sess,
            "next scheduled time is soon %" GST_TIME_FORMAT " + %" GST_TIME_FORMAT
            " > %" GST_TIME_FORMAT, GST_TIME_ARGS (current_time),
            GST_TIME_ARGS (max_delay),
            GST_TIME_ARGS (sess->next_rtcp_check_time));
        ret = TRUE;
      } else {
        GST_LOG_OBJECT (sess,
            "can't allow early feedback and next scheduled time is too late %"
            GST_TIME_FORMAT " + %" GST_TIME_FORMAT " < %" GST_TIME_FORMAT,
            GST_TIME_ARGS (current_time), GST_TIME_ARGS (max_delay),
            GST_TIME_ARGS (sess->next_rtcp_check_time));
        ret = FALSE;
      }
      goto end;
    }

And NACKs are transmitted properly at least with the C code, lost packets are retransmitted but probably RFC is violated.

I just pushed a bunch of RTX related patches, please let us know if they help. If they don't, can you attach your python script?

I will test your patches asap and keep you up to date. Thx!

I have fixed the examples of rtprtxreceive (see bug 771383) to mention correct property usage and to make them work.

Unfortunately I can confirm some NACKs being dropped occasionally. By studying the algorithm, it looks like it is the intention of RFC 4585 to occasionally drop some retransmission requests in order to regulate the bandwidth of the RTCP stream. This can be circumvented by setting a higher bandwidth limit for RTCP, which makes it send receiver reports more often and therefore less NACKs are being lost. I am not sure if this is a bug.

Hi all, I run the example client-rtpaux/server-rtpaux in gst-plugins-good/tests/examples/rtp in order to evaluate the retransmission feature. But I discovered that NACK of packet loss is reported by receiver but the sender doesn't respond it. As a results, retranmission doesn't happen. The number of rtx requests increases but the number of rtx packets are always zero.

Does someone here provide valuable hints on how to make it work? I'm using gstreamer master branch. Are any other branches working?

Today I modified the code of rtpsession.c in rtpmanager according to Marcin's work (Comment 14). It is working. It seems that the commented code does early feedback check. I'm trying to reading about RFC4585 to understand it. Is anyone here can provide some hints? Is this kind of modification valid for gstreamer?

I tried another way to change the RTCP bandwidth configuration in rtp session element without the modification of code. But the retransmission doesn't work either.

When "profile" is set to AVPF, the early feedback should work (the examples set this). From running the server-rtpaux / client-rtpaux examples and checking the data in wireshark, it seems whenever the client requests a packet to be resent via NACK, the server properly replies with a resent packet with payload type 99.
This is with git master and 1.12.

How do you detect that it does not work for you exactly?

Also a log for the things that Marcin mentioned in https://bugzilla.gnome.org/show_bug.cgi?id=758719#c14 would be useful to see why things are not handled for you. But that part of the code is for the RTCP sending on the client, not for anything on the server.

I used wireshark to check whether there is pt-99 retransmission stream from sender to receiver. I only discovered NACK before. However, when I modified the rtpsession code and re-compiled and re-run it, I was able to see the pt-99 stream.

I just run the server-rtpaux/client-rtpaux to demonstrate the retransmission.

I debugged it, and noticed that several rtx options, like rtx-retry, rtx-timeout, rtx-deadline, etc, are involved. I just use the default value which is automatically determined by gstreamer.

I understand it is really strange. I will continue working on it.

So the client is requesting retransmissions, but they never happen. Are the NACKs arriving in the sender rtpbin, are they going to rtprtxsend, and what does that do with them? Do the NACKs arrive after the time that is queued in rtprtxsend?

Sorry for late reply. I reused the same RTCP port at receiver side. But it seems not working with gstreamer and also caused the retransmission unable to work properly. Finally I successfully make everything work.

What exactly did you change? You said that also with the example server-rtpaux/client-rtpaux it did not work

Sorry for misunderstanding. The example does work. The problem is that I reused the RTCP port on the receiver side for other purpose. I changed my application to remove this logic and rtx works quite well.