After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 792664 - souphttpsrc: Should send "Range" when offset is 0
souphttpsrc: Should send "Range" when offset is 0
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal major
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2018-01-18 23:57 UTC by Paul Kim
Modified: 2018-11-03 15:25 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Should send Range when offset is 0 (1.89 KB, patch)
2018-01-18 23:59 UTC, Paul Kim
none Details | Review
Send Range when offset is 0 (1.89 KB, patch)
2018-01-19 07:20 UTC, Paul Kim
none Details | Review

Description Paul Kim 2018-01-18 23:57:29 UTC
Cannot seek contents in a server and the reason is that the server sents different file headers when "Range" is included in request header or not.
For this reason, only gstreamer cannot do seek, but ffmpeg in chromium covers this case. To improve stability of gstreamer, should send "Range: 0-" for all cases.
Comment 1 Paul Kim 2018-01-18 23:59:30 UTC
Created attachment 367053 [details] [review]
Should send Range when offset is 0
Comment 2 Paul Kim 2018-01-19 07:20:26 UTC
Created attachment 367055 [details] [review]
Send Range when offset is 0
Comment 3 Sebastian Dröge (slomo) 2018-01-19 09:42:25 UTC
Please describe in more detail the situation when this happens, what the exact course of events is to have this happen and what the HTTP requests/responses are. Ideally a testcase, example HTTP streams, etc.

Always sending a Range header can cause problems with various server AFAIU (that reject such requests because they don't allow Range requests in general), and not sending one for starting from "the beginning" should be valid.
Comment 4 Sebastian Dröge (slomo) 2018-01-19 09:42:43 UTC
Also links to the relevant code in ffmpeg and chrome/firefox/etc would be useful.
Comment 5 Paul Kim 2018-01-20 23:46:30 UTC
Seek function is not working with LGE TVs because when "Range" field is not included in request header, "freevideo served by mod_h264_streaming" file header is transferred from serverside. However, when "Range" filed is included, "freevideo served by mod_h264_streaming" is not included in the mp4 file. Thus, the decoder can`t decode normally. 
The relevent code of ffmpeg is found https://github.com/FFmpeg/FFmpeg/blob/cbbb2067341d7c2d98f560f81c6fb103af33a490/libavformat/http.c 

 // Note: we send this on purpose even when s->off is 0 when we're probing,
 // since it allows us to detect more reliably if a (non-conforming)
 // server supports seeking by analysing the reply headers.
    if (!has_header(s->headers, "\r\nRange: ") && !post && (s->off > 0 || s->end_off || s->seekable == -1)) {
        len += av_strlcatf(headers + len, sizeof(headers) - len,
                           "Range: bytes=%"PRIu64"-", s->off);
        if (s->end_off)
            len += av_strlcatf(headers + len, sizeof(headers) - len,
                               "%"PRId64, s->end_off - 1);
        len += av_strlcpy(headers + len, "\r\n",
                          sizeof(headers) - len);
    }
Comment 6 Paul Kim 2018-01-20 23:55:45 UTC
Regarding the server behavior, it should ignore the range header if it does not want to see "Range" header in the request. The rfc spec is mentioning about that. https://tools.ietf.org/html/rfc7233#section-3.1
   A server MAY ignore the Range header field.  However, origin servers
   and intermediate caches ought to support byte ranges when possible,
   since Range supports efficient recovery from partially failed
   transfers and partial retrieval of large representations.  A server
   MUST ignore a Range header field received with a request method other
   than GET.
Comment 7 Jun Xie 2018-01-21 03:16:26 UTC
(In reply to Paul Kim from comment #5)
> Seek function is not working with LGE TVs because when "Range" field is not
> included in request header, "freevideo served by mod_h264_streaming" file
> header is transferred from serverside. However, when "Range" filed is
> included, "freevideo served by mod_h264_streaming" is not included in the
> mp4 file. Thus, the decoder can`t decode normally. 
> The relevent code of ffmpeg is found
> https://github.com/FFmpeg/FFmpeg/blob/
> cbbb2067341d7c2d98f560f81c6fb103af33a490/libavformat/http.c 
> 
>  // Note: we send this on purpose even when s->off is 0 when we're probing,
>  // since it allows us to detect more reliably if a (non-conforming)
>  // server supports seeking by analysing the reply headers.
>     if (!has_header(s->headers, "\r\nRange: ") && !post && (s->off > 0 ||
> s->end_off || s->seekable == -1)) {
>         len += av_strlcatf(headers + len, sizeof(headers) - len,
>                            "Range: bytes=%"PRIu64"-", s->off);
>         if (s->end_off)
>             len += av_strlcatf(headers + len, sizeof(headers) - len,
>                                "%"PRId64, s->end_off - 1);
>         len += av_strlcpy(headers + len, "\r\n",
>                           sizeof(headers) - len);
>     }

1. ffmpeg use range '0-' to verify if server accepts range GET, if yes, "Accept-Ranges" is processed and set URLContext->is_streamed to 0, as seekable.
Do we also need similar process in soupsrc?
2. what is exactly behaivor of mod_h264_streaming for non-range GET? is there any reference there?
Comment 8 Paul Kim 2018-01-21 04:04:42 UTC
@Jun,
1. I agree.
2. The seek index becomes different when we do seek so the decoder can`t decode normally. Sorry I thought I already posted the url. The issue can be observed if you play and do seek any contents in "http://video.eyny.com/index.php".
Comment 9 Sebastian Dröge (slomo) 2018-01-21 08:38:37 UTC
It could make sense to first do a request with "Range: 0-" and if that fails with any of the "range request not accepted" responses we could fall-back to doing a request without range header. That should cover both cases.

The "Accept-Ranges" header is already handled.
Comment 10 Sebastian Dröge (slomo) 2018-01-21 08:39:11 UTC
Do you want to provide a patch for that behaviour? First try with range, and if that fails consider the stream unseekable and do without range.
Comment 11 Jun Xie 2018-01-22 05:26:55 UTC
AFAIK, http server will ignore range if not supported, just response with whole requested resource.
It seems that it will be safe without fall-back for range '0-'
Comment 12 Paul Kim 2018-01-22 05:46:08 UTC
@Sebastian,
Like Jun mentioned and the spec I shared, server should ignore the range if not supported so the patch I uploaded without fallback works.
Comment 13 Jun Xie 2018-01-22 06:11:45 UTC
However, Paul, I just checked one of contents in "http://video.eyny.com/index.php".

"freevideo served by mod_h264_streaming" file header
--> does this mean a 'free' mp4 box?

if yes, why this useless box affect decoder?
Comment 14 Paul Kim 2018-01-22 06:25:15 UTC
If a range request is not included, a file containing "* freevideo served by mod_h264_streaming" is sent to the File Header, but if the Range request is included, the missing file is being sent. During the initial playback, HTTP request is sent with missing range request. As a result of this, when QT index is transmitted, * "freevideo served by mod_h264_streaming" is included in the header, so it is parsed by 42bytes index. , when attempting to seek, a request with a range request is included, and a 
file that does not contain "* freevideo served by mod_h264_streaming" is transmitted.
It can easily be observed if you try gst-play-1.0 and do seek contents in the website.
Comment 15 Jun Xie 2018-01-22 07:11:44 UTC
aha, I see, 'stco' is different between non-range and range '0-' .

I think server behavior needs to be consistent.
Comment 16 Paul Kim 2018-01-22 07:19:14 UTC
Yes, the server behavior is expected to be consistent, but chromium covers this case.
Comment 17 Sebastian Dröge (slomo) 2018-01-22 08:09:44 UTC
(In reply to Paul Kim from comment #12)
> @Sebastian,
> Like Jun mentioned and the spec I shared, server should ignore the range if
> not supported so the patch I uploaded without fallback works.

You're assuming that servers behave sane. The one you're currently fighting with is not already for example, but in a different way.
Comment 18 GStreamer system administrator 2018-11-03 15:25:29 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/433.