GNOME Bugzilla – Bug 607555
asfmux plugin generates data streams incompatible with WMSP (MMSH) and WMP
Last modified: 2010-01-25 22:57:51 UTC
Created attachment 151838 [details] [review] Patch to make WMP play muxed streams also through the WMSP (MMSH) protocol I've implemented a WMSP (also known as MMSH) server, but the gstreamer asfmux's data stream is incompatible with Windows Media Player. Attached is a patch for the asfmux plugin so that Windows Media Player now can understand gstreamer asfmuxed streams also when used through WMSP. Only one of all the changes are strictly needed to follow the official specification, the others are just tweaking the stream inside it's specification so that WMP understands it. Comic (or tragic) considering that the ones who wrote the specification can't even follow it themself. I'll happily explain any of the changes if needed.
Can you share the server's code for us to test your patch?
Created attachment 151928 [details] Small hack of a demo program for WMSP streaming Attached is a short demo program available that only does simple live streaming. It's a hack based on a HTTP-server hack, it must be restarted every time you restart streaming, it does not handle multiple simultaneous clients and it's so ugly written that I'll refuse to admit ownership if you blame me. But it works... as a test program for the asfmux. If you start it as ./wmsp_server 5000 you test it by starting windows media player and open the url: http://IP-address:5000/video
The server code has some problems and segfaults once the streaming is established, but it runs long enough to see WMP rejecting if data object size is 0 and trying to receive if it is the minimum size (even after applying the rest of your patch). IMHO we should add a property to change the initial size of the data object because the spec suggests using 0 with the broadcast flag on. This seems to be clearer from the receiver's point of view that the data object has an unknown length.
Ok, here are some comments from the different parts of the patch: - GST_WRITE_UINT64_LE (*buf + 16, 0); /* object size - needs updating */ + + /* Data object size. This is always >= ASF_DATA_OBJECT_SIZE. The standard + * specifically accepts the value 0 in live streams, but WMP is not accepting + * this while streaming using WMSP, so we default to minimum size also for + * live streams. Otherwise this field must be updated later on when we know + * the complete stream size. + */ + GST_WRITE_UINT64_LE (*buf + 16, ASF_DATA_OBJECT_SIZE); + The correct size for all live video streams are _always_ ASF_DATA_OBJECT_SIZE. As you know the specification explicitly allows 0 in live streams - telling demuxers to do a: if (data_object_size == 0) data_object_size = ASF_DATA_OBJECT_SIZE; To add a property to the muxer so that you can force it to 0 seems like overkill to me, but this is of course your decision. Without this change windows media player won't even try to play the stream. Shame on them. Next problem is that the windows media player breaks down after a certain time of buffering. The reason for this is that we don't send the correct data_packet size when we do padding. The part of the diff that fixes this is extracted below: - GST_WRITE_UINT32_LE (data + 2, asfmux->packet_size); .... + GST_WRITE_UINT32_LE (data + offset, asfmux->packet_size - size_left); We _must_ subtract size_left from the total length or we typically get packets with packet size 4803 bytes if we pad 3 bytes. This is a critical error and makes us violate the standard. With these two fixes there is one major problem left: Windows Media Player is buffering forever. It took me horribly many hours of tweaking the asfmux plugin to realize that it was due to the packet size and padding size in the parsing info header. WMP can for some unknown reason not accept 32 bit data & padding size. A quick fix would be to just always force 16 bit packet & padding sizes, we hardly ever want packet sizes above 64K, but it felt a bit hackish. So instead I created a function to check if the sizes are small enough to fit in 16 bit and adapt the parsing info header accordingly. On my PC the wmsp_server now finally streams correctly. All my work on the asfmux was done by analyzing the asf headers in NRK's (Norwegian Broadcasting Corporation) TV broadcast streams available on http://www.nrk.no/nett-tv/direkte/nrk1/ They use Microsoft's own streaming server solution and could easily be downloaded and re-streamed with wmsp_server. I hope this made things a bit clearer. And I find it strange that the wmsp_server didn't work for you. It shouldn't segfault while streaming - only if you restart the client without restarting the server. Oh, and to help you do debugging with WMP I can recommend that you download this web page and opens it in internet explorer: http://blogs.msdn.com/randomnumber/archive/2009/07/20/wmp-debug-page-to-dump-player-events.aspx It's not great - but better than working directly in WMP (just remember to allow activex plugins - or it won't work). I'm interested in getting this WMSP work out of the box in gstreamer - so please tell me more about how far you get with the test program and I'll see if I can assist you in any way.
(In reply to comment #4) > Ok, here are some comments from the different parts of the patch: > > - GST_WRITE_UINT64_LE (*buf + 16, 0); /* object size - needs updating */ > + > + /* Data object size. This is always >= ASF_DATA_OBJECT_SIZE. The standard > + * specifically accepts the value 0 in live streams, but WMP is not > accepting > + * this while streaming using WMSP, so we default to minimum size also for > + * live streams. Otherwise this field must be updated later on when we know > + * the complete stream size. > + */ > + GST_WRITE_UINT64_LE (*buf + 16, ASF_DATA_OBJECT_SIZE); > + > > The correct size for all live video streams are _always_ ASF_DATA_OBJECT_SIZE. > As you know the specification explicitly allows 0 in live streams - telling > demuxers to do a: > if (data_object_size == 0) > data_object_size = ASF_DATA_OBJECT_SIZE; > > To add a property to the muxer so that you can force it to 0 seems like > overkill to me, but this is of course your decision. Yes, other devels also agreed with you and convinced me, so no property :) > > Without this change windows media player won't even try to play the stream. > Shame on them. Next problem is that the windows media player breaks down after > a certain time of buffering. The reason for this is that we don't send the > correct data_packet size when we do padding. The part of the diff that fixes > this is extracted below: > > - GST_WRITE_UINT32_LE (data + 2, asfmux->packet_size); > .... > + GST_WRITE_UINT32_LE (data + offset, asfmux->packet_size - size_left); > > We _must_ subtract size_left from the total length or we typically get packets > with packet size 4803 bytes if we pad 3 bytes. This is a critical error and > makes us violate the standard. > > With these two fixes there is one major problem left: Windows Media Player is > buffering forever. It took me horribly many hours of tweaking the asfmux plugin > to realize that it was due to the packet size and padding size in the parsing > info header. WMP can for some unknown reason not accept 32 bit data & padding > size. A quick fix would be to just always force 16 bit packet & padding sizes, > we hardly ever want packet sizes above 64K, but it felt a bit hackish. So > instead I created a function to check if the sizes are small enough to fit in > 16 bit and adapt the parsing info header accordingly. On my PC the wmsp_server > now finally streams correctly. > > All my work on the asfmux was done by analyzing the asf headers in NRK's > (Norwegian Broadcasting Corporation) TV broadcast streams available on > http://www.nrk.no/nett-tv/direkte/nrk1/ > They use Microsoft's own streaming server solution and could easily be > downloaded and re-streamed with wmsp_server. > > I hope this made things a bit clearer. And I find it strange that the > wmsp_server didn't work for you. It shouldn't segfault while streaming - only > if you restart the client without restarting the server. > > Oh, and to help you do debugging with WMP I can recommend that you download > this web page and opens it in internet explorer: > http://blogs.msdn.com/randomnumber/archive/2009/07/20/wmp-debug-page-to-dump-player-events.aspx > It's not great - but better than working directly in WMP (just remember to > allow activex plugins - or it won't work). > > I'm interested in getting this WMSP work out of the box in gstreamer - so > please tell me more about how far you get with the test program and I'll see if > I can assist you in any way. Currently I'm tracking a regression introduced by your patch with a simple pipeline like: gst-launch videotestsrc num-buffers=500 ! ffenc_wmv2 ! asfmux ! filesink location=test.asf It doesn't play till the end in totem anymore (haven't tested other players yet). After solving this, I'll push your patch.
Review of attachment 151838 [details] [review]: 1549 /* Due to a limitation in WMP while streaming through WMSP we reduce the 1520 GST_WRITE_UINT32_LE (data + 6, size_left); /* padding size */ 1550 * packet & padding size to 16bit if theay are <= 65535 bytes 1551 */ 1552 if (asfmux->packet_size > 65535) { 1553 GST_WRITE_UINT32_LE (data + offset, asfmux->packet_size - size_left); 1554 offset += 4; 1555 } else { 1556 *data &= ~(ASF_FIELD_TYPE_MASK << 5); 1557 *data |= ASF_FIELD_TYPE_WORD << 5; 1558 GST_WRITE_UINT16_LE (data + offset, asfmux->packet_size - size_left); 1559 offset += 2; 1560 } Why are you using "packet_size - size_left" here?
(In reply to comment #6) > Review of attachment 151838 [details] [review]: > > 1549 /* Due to a limitation in WMP while streaming through WMSP we reduce > the > 1520 GST_WRITE_UINT32_LE (data + 6, size_left); /* padding size */ > 1550 * packet & padding size to 16bit if theay are <= 65535 bytes > 1551 */ > 1552 if (asfmux->packet_size > 65535) { > 1553 GST_WRITE_UINT32_LE (data + offset, asfmux->packet_size - > size_left); > 1554 offset += 4; > 1555 } else { > 1556 *data &= ~(ASF_FIELD_TYPE_MASK << 5); > 1557 *data |= ASF_FIELD_TYPE_WORD << 5; > 1558 GST_WRITE_UINT16_LE (data + offset, asfmux->packet_size - > size_left); > 1559 offset += 2; > 1560 } > > Why are you using "packet_size - size_left" here? Because we want to write the size of codec data - and _not_ the size of (codec data + padding). That's why we must remove size_left (= padding). The funny thing is that this bug also is found in the gstreamer's asfdemuxer. I'll attach a quick patch to fix the asfdemuxer as well. I don't know if we should open a new bug on gstreamer-ugly, but I guess you can just commit a fix when you find a solution you like.
Created attachment 151955 [details] [review] Quick fix for demuxing data with padding The demuxer forgot to check if the padding data was present. The fix is so small and easy it should be self explainable.
Where have you read/seen that the asf packet length is indeed the "media data length" and not the total packet length? I'd be surprised that asfdemux remained with this bug for all this time. This would possibly mean that the packet length field isn't used. That leads me to another question: does your WMSP stuff need the packet length field? I'm considering not setting it in asfmux.
(In reply to comment #9) > Where have you read/seen that the asf packet length is indeed the "media data > length" and not the total packet length? > > I'd be surprised that asfdemux remained with this bug for all this time. This > would possibly mean that the packet length field isn't used. That leads me to > another question: does your WMSP stuff need the packet length field? I'm > considering not setting it in asfmux. I agree that the specification can be a bit unclear at this point. The reason why I believe the packet length = total packet length - padding length is because I found this to be the case in Microsoft's own streaming solution. To test this try to stream the following stream in WMP: http://straumV.nrk.no/nrk_tv_webvid03_h while you use tcpdump. If you look at the network dump in wireshark you can see that the stream changes the packet size between 8000 and lower values (typically 7998) - indicating padding. To save you some time with wireshark: switch to hex dump, search for $D and skip 15 bytes. Then you'll see the packet size field - normally 40 1f => 8000. If you where right that packet length == total packet length, then this would be wrong. My thinking is that if Microsoft implements it this way - then that is the "correct way" to do it. I don't think we should ignore this field entirely though, as that would make us incompatible with the specification - strictly speaking.
> that the stream changes the packet size between 8000 and lower values > (typically 7998) - indicating padding. To save you some time with wireshark: > switch to hex dump, search for $D and skip 15 bytes. Then you'll see the packet > size field - normally 40 1f => 8000. If you where right that packet length == > total packet length, then this would be wrong. I can see that my previous post is a bit unclear, it should read: that the stream changes the packet size between 8000 and lower values (typically 7998) - indicating padding. To save you some time with wireshark: switch to hex dump, search for $D and skip 15 bytes. Then you'll see the packet size field - normally 40 1f => 8000, but sometimes less. If you where right that packet length == total packet length, then Microsoft implements their own protocol wrong when they set packet length to lower values. Here's the first MMS data packet I get from the NRK stream (which sets min/max packet size in the file prop. header to 8000). As you can see the packet size is set to 1661 bytes (7d 06). The ASF packet starts at position 0x09c. 00000090 24 44 85 06 9b ac 43 00 01 00 85 06 82 00 00 41 $D....C. .......A 000000A0 5d 7d 06 9c c3 e6 11 46 ]}.....F Here's the 3rd MMS data packet. As you can see is the packet size now is 8000 bytes (40 if). 00000268 24 44 48 1f 9f ac 43 00 01 04 48 1f 82 00 00 41 $DH...C. ..H....A 00000278 5d 40 1f 8d c4 e6 11 47 ]@.....G Also the WMSP specification suggests that the data size field should not include the padding size. From the WMSP specification, page 69: --- Payload (variable): This field MUST contain exactly one complete ASF packet. If the ASF packet contains a Padding Data field, as specified in [ASF] section 5.2.4, that field SHOULD be removed before encapsulating the ASF packet in the $D packet, except if the client identifies itself as "NSServer" with a version earlier than 5.0 on the User-Agent (section 2.2.1.8) header. If the Padding Data field is removed, the Padding Length field in the ASF payload parsing information section, as specified in [ASF] section 5.2.2, MUST be updated to indicate a nonexistent Padding Data field. --- When padding is removed from the packet it only says the the padding size should be updated and not the packet size. Thus packet size cannot be interpreted as the size including padding.
Module: gst-plugins-bad Branch: master Commit: 0a90963fcf91bd77be019cc813da16f4500f295c URL: http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/?id=0a90963fcf91bd77be019cc813da16f4500f295c Author: Håkon Skjelten <skjelten@pvv.org> Date: Thu Jan 21 10:51:16 2010 -0300 asfmux: Compatibility with WMSP in WMP Makes the asfmux content compatible with WMSP and does some hacks to make it playable in WMP, it doesn't accept data objects with 0 size indicating that we don't know its size, though the spec says it should be possible. Fixes #607555