GNOME Bugzilla – Bug 743687
playback: gstreamer-vaapi doesn't work with Totem master
Last modified: 2016-08-17 02:38:39 UTC
Using: - GStreamer (and co.) 1.4.5 - gstreamer-vaapi from git master (da6b88692f117537e61e607a831505aa4ef3d770) - Totem master - clutter-gst master (3.0.x) - libva-intel-driver-1.4.1-1.fc21.x86_64 I get "GStreamer encountered a general stream error." I would have expected video to start playing, and be hardware accelerated so as to eat less battery. I get the same problem with stock Fedora 21 components: - GStreamer (and co.) 1.4.5 - gstreamer-vaapi 0.5.9 - Totem 3.14 - clutter-gst-2.0.12 - libva-intel-driver-1.4.1-1.fc21.x86_64
Bastien, Could you try with the patch I posted in https://bugzilla.gnome.org/show_bug.cgi?id=741237 ??? With it totem works for me... :/
Does the issue is reproducible with any media sample?
Víctor's patches don't make any difference. * Files that are working: - Kung-fu film: Topology: container: Audio Video Interleave (AVI) audio: AC-3 (ATSC A/52) video: MPEG-4 Video Properties: Duration: 2:13:05.840000000 Seekable: yes Tags: title: Bodyguards.And.Assassins.TRUEFRENCH.DCVDRIP.XVID-FwD album: BODYGUARDS_AND_ASSASSINS_LOC encoder: Lavf52.33.0 container format: AVI audio codec: AC-3 (ATSC A/52) bitrate: 448000 video codec: MPEG-4 Video - Hard-rock episode: Topology: container: Audio Video Interleave (AVI) audio: MPEG-1 Layer 3 (MP3) video: MPEG-4 Video Properties: Duration: 0:11:48.457750000 Seekable: yes Tags: encoder: VirtualDubMod 1.5.4.1 (build 2178/release) container format: AVI audio codec: MPEG-1 Layer 3 (MP3) bitrate: 178000 has crc: false channel mode: joint-stereo video codec: MPEG-4 Video Files that don't work: - Swedish Robot episode: Topology: container: Matroska subtitles: Timed Text subtitles: Timed Text audio: MPEG-4 AAC audio: MPEG-4 AAC video: H.264 Properties: Duration: 0:58:27.181000000 Seekable: yes Tags: container format: Matroska audio codec: MPEG-4 AAC language code: fr video codec: H.264 minimum bitrate: 1723000 bitrate: 1265520 maximum bitrate: 1723000 - Austin High: Topology: container: Quicktime audio: MPEG-4 AAC video: H.264 Properties: Duration: 1:40:56.703333333 Seekable: yes Tags: datetime: 2013-10-11T14:41:51Z QT atom: 0000009b686e7469000000937274702073647020623d41533a333531340d0a613d782d636f707972696768743a204d50342f3347502046696c652068696e7465642077697468204750414320302e342e362d4445562d726576556e76657273696f6e6564206469726563746f727920284329323030302d32303035202d20687474703a2f2f677061632e736f75726365666f7267652e6e65740d0a:None:R3N0U2VnbWVudCwgZmxhZ3M9KEdzdFNlZ21lbnRGbGFncylHU1RfU0VHTUVOVF9GTEFHX05PTkUsIHJhdGU9KGRvdWJsZSkxLCBhcHBsaWVkLXJhdGU9KGRvdWJsZSkxLCBmb3JtYXQ9KEdzdEZvcm1hdClHU1RfRk9STUFUX1RJTUUsIGJhc2U9KGd1aW50NjQpMCwgb2Zmc2V0PShndWludDY0KTAsIHN0YXJ0PShndWludDY0KTAsIHN0b3A9KGd1aW50NjQpMTg0NDY3NDQwNzM3MDk1NTE2MTUsIHRpbWU9KGd1aW50NjQpMCwgcG9zaXRpb249KGd1aW50NjQpMCwgZHVyYXRpb249KGd1aW50NjQpMTg0NDY3NDQwNzM3MDk1NTE2MTU7AA__:YXBwbGljYXRpb24veC1nc3QtcXQtaG50aS10YWcsIHN0eWxlPShzdHJpbmcpaXNvOwA_ container format: ISO MP4/M4A audio codec: MPEG-4 AAC maximum bitrate: 133968 bitrate: 61552 video codec: H.264
I did some quick tests with gst-launch (gstremaer-1.4, gstreamer-vaapi_master and clutter-gst-3.0.4) and can't reproduce any issue. Can you please run with GST_DEBUG=4 ??
This is the problem with Totem master, reproduced with gst-launch-1.0 $ gst-launch-1.0 playbin uri=file:///home/hadess/bug743687.mkv video-sink=clutterautovideosink audio-filter=scaletempo Setting pipeline to PAUSED ... Pipeline is PREROLLING ... libva info: VA-API version 0.36.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib64/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_36 libva info: va_openDriver() returns 0 Got context from element 'vaapidecode0': gst.vaapi.Display=context, display=(GstVaapiDisplay)NULL; WARNING: from element /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vdbin/GstVideoConvert:vdconv: not negotiated Additional debug info: gstbasetransform.c(2138): gst_base_transform_handle_buffer (): /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vdbin/GstVideoConvert:vdconv: not negotiated ERROR: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: Internal data stream error. Additional debug info: gstmultiqueue.c(1524): gst_multi_queue_loop (): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: streaming stopped, reason not-negotiated ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... Freeing pipeline ... I can also reproduce the problem with clutter-gst2. bug743687.mkv is at: https://hadess.fedorapeople.org/bug743687.mkv
*** Bug 743977 has been marked as a duplicate of this bug. ***
In my case (bug 743977) it's with totem-3.14.2 , but works ok with gst-launch-1.4.5
I've tested with https://hadess.fedorapeople.org/bug743687.mkv (thanks Bastien) and found that in the negotiation of the renderer in clutter-gst-video-sink, the uploader to gl is set to a dummy one. I cooked a patch for clutter-gst3 for workaround the problem, but still, it needs a bit more dig to fix it for real. In the case of the video tested in bug 743977, using master (totem, clutter, gstreamer-vaapi) works without the workaround.
Created attachment 296095 [details] [review] video-sink: upload by software if there's no gl uploader
Cannot it be done for clutter-gst-2.0.x? If I don't misremember we cannot use 3.x yet with Gnome 3.14 Thanks
(In reply to comment #9) > Created an attachment (id=296095) [details] [review] > video-sink: upload by software if there's no gl uploader Doesn't make a difference in my usage I'm afraid, using gstreamer-vaapi from master. I double-checked and it also happens with libva, libva-intel-driver and gstreamer-vaapi all from master.
(In reply to comment #10) > Cannot it be done for clutter-gst-2.0.x? If I don't misremember we cannot use > 3.x yet with Gnome 3.14 Thanks That's what my jhbuild is using: the branch clutter-gst-3.0 (254d3bd) Also totem master (3859ad4), gstreamer-vaapi (d2e2784) and gstreamer/gst-plugins-* master too. Totem master cannot be built with clutter-gst-2.0
Just to be clear, with bug743687.mkv, the error that I see is this: 0:00:00.646562262 13003 0x1d59600 INFO cluttervideosink ./clutter-gst-video-sink.c:1889:clutter_gst_video_sink_parse_caps:<cluttergstvideosink0> found the I420 glsl renderer 0:00:00.646594387 13003 0x1d59600 INFO cluttervideosink ./clutter-gst-video-sink.c:1894:clutter_gst_video_sink_parse_caps:<cluttergstvideosink0> saving infos 0:00:00.646608542 13003 0x1d59600 DEBUG cluttervideosink ./clutter-gst-video-sink.c:1999:clutter_gst_source_dispatch:<cluttergstvideosink0> Trying to upload buffer 0x7f984c089880 with GL 0:00:00.646620471 13003 0x1d59600 WARN cluttervideosink ./clutter-gst-video-sink.c:2041:clutter_gst_source_dispatch:<cluttergstvideosink0> Failed to upload buffer 0:00:00.657282883 13003 0x7f984003f800 ERROR subtitleoverlay gstsubtitleoverlay.c:1605:gst_subtitle_overlay_src_proxy_chain:<suboverlay> Downstream chain error: error 0:00:01.688847059 13003 0x7f9850002a30 ERROR vaapidecode gstvaapidecode.c:384:gst_vaapidecode_push_decoded_frame: video sink rejected the video buffer (error -5) My explanation is for some reason, clutter-gst-videosink is choosing the I420 glsl renderer which doesn't have GL buffer uploader. But that doesn't seem to be the issue for Pacho or Bastien. I'm a bit lost.
Hi, Is the following of any help? https://github.com/gbeauchesne/gstreamer-vaapi/ ("egl" branch). Some work still needs to be done prior to integrating to mainstream but vaapidecode|vaapipostproc work equally well with glimagesink in { { X11, Wayland } x { GL, GLESv2 } x EGL, GLX } situations. i.e. if cluttervideosink correctly supports GstVideoGLTextureUploadMeta and provides a suitable GstGLContext, then it should also Just-Work there.
My logs are at https://bugzilla.gnome.org/show_bug.cgi?id=743977 , would you need that I run totem with higher debugging level? Maybe the problem lying on clutter-gst would explain why totem fails to play the file while plain gst-launch works ok :/ I am running with "normal" opengl+X11 setup. Thanks
(In reply to comment #15) > My logs are at https://bugzilla.gnome.org/show_bug.cgi?id=743977 , would you > need that I run totem with higher debugging level? Maybe the problem lying on > clutter-gst would explain why totem fails to play the file while plain > gst-launch works ok :/ If you're not using the gst-launch line from https://wiki.gnome.org/Apps/Videos/BugReporting or comment 5 (for newer versions of clutter-gst) then it's not testing the same pipeline as Totem's.
With "cluttersink" as wiki suggests I get no window at all, only this output: $ LC_ALL=C gst-launch-1.0 playbin uri=file:///home/pacho/Descargas/300.mp4 video-sink=cluttersink audio-filter=scaletempo Setting pipeline to PAUSED ... Pipeline is PREROLLING ... libva info: VA-API version 0.37.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib64/va/drivers/i965_drv_video.so libva info: Found init function __vaDriverInit_0_35 libva info: va_openDriver() returns 0 Got context from element 'vaapidecode0': gst.vaapi.Display=context, display=(GstVaapiDisplay)NULL; ^Chandling interrupt. Interrupt: Stopping pipeline ... ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... Freeing pipeline ... With the link from comment #5 it fails because that video-sink is not recognized: $ LC_ALL=C gst-launch-1.0 playbin uri=file:///home/pacho/Descargas/300.mp4 video-sink=clutterautovideosink audio-filter=scaletempo WARNING: erroneous pipeline: could not set property "video-sink" in element "playbin0" to "clutterautovideosink"
GST_DEBUG=*:9 output here: https://hadess.fedorapeople.org/bug743687.xz Watch out, it's pretty big.
(In reply to comment #14) > Hi, > > Is the following of any help? > https://github.com/gbeauchesne/gstreamer-vaapi/ ("egl" branch). Didn't work for me, debug output is above.
Finally I could reproduce the error that hit Bastien: it happens when using the 1.4 branch of gstreamer. If you switch it to master, it is fixed. The problem, afaiu, is that in 1.4 videoconvert, which inherits from basetransform: 0:00:00.487268539 4418 0x7f6724073590 WARN basetransform gstbasetransform.c:1397:gst_base_transform_setcaps:<vdconv> transform could not transform video/x-raw(meta:GstVideoGLTextureUploadMeta), format=(string)RGBA, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:1:0:0, framerate=(fraction)25/1 in anything we support Comparing logs, the videoconvert is set to passthrough in master meanwhile in 1.4 it is not. Hence, IMO, this bug is not related with gstreamer-vaapi but with gstreamer (basetransform) or gst-plugins-base (videoconvert).
Still, the patch for clutter-gst3 is required.
(In reply to comment #20) > Finally I could reproduce the error that hit Bastien: it happens when using the > 1.4 branch of gstreamer. If you switch it to master, it is fixed. I *did* mention I was using 1.4.5, didn't I :) > Hence, IMO, this bug is not related with gstreamer-vaapi but with gstreamer > (basetransform) or gst-plugins-base (videoconvert). Thanks, reassigning to gstreamer then.
(In reply to comment #21) > Still, the patch for clutter-gst3 is required. Mind filing a separate bug about it?
Could you have a look to see what patch(es) fixed it please, so we can pick them into 1.4?
(In reply to comment #23) > (In reply to comment #21) > > Still, the patch for clutter-gst3 is required. > > Mind filing a separate bug about it? Done: https://bugzilla.gnome.org/show_bug.cgi?id=744039
(In reply to comment #24) > Could you have a look to see what patch(es) fixed it please, so we can pick > them into 1.4? According to my bisects, these are the required patches: http://cgit.freedesktop.org/gstreamer/gst-plugins-base/commit/?id=994680b04eb390a116a9b5bda2abd647df233be3 http://cgit.freedesktop.org/gstreamer/gst-plugins-base/commit/?id=57999c28fd7720f5f3e9b3adf55528ddad21cc9c http://cgit.freedesktop.org/gstreamer/gst-plugins-base/commit/?id=11ef208736c5867046156e0bb6786f2eb56e6776
I can confirm, these are the commits that fix the issue. But there is still something else going wrong.
Removing NEEDINFO, question is answered in comment 26.
Thanks a lot for tracking down the specific commits needed. Unfortunately I don't think we can pick all of those into 1.4, that's too risky/invasive.
Just mentioning that, if we can't get the vaapi decoder to work properly with playbin, we can always deactivate the support for GL texture upload in clutter-gst by configuring it with --enable-gl-texture-upload=no. Just as a workaround.
I don't think we should backport these commits to 1.4, at least not the first 2. They required follow-up fixes in decodebin and elsewhere and are quite risky. Do these patches on top of 1.4.5 really solve anything? If so, can you get a debug log with and without the patches?
(In reply to Sebastian Dröge (slomo) from comment #31) > Do these patches on top of 1.4.5 really solve anything? If so, can you get a > debug log with and without the patches? They enable the negotiation. Otherwise the pipeline just stops. Though Bastien, on comment #18, posted the log without patches, I'll upload both.
Created attachment 296512 [details] debug log (no color) of the pipeline without patches (and fails to negotiate)
Created attachment 296513 [details] debug log (no color) of the pipeline with patches (and works)
What do you mean with "they enable negotiation"? Between which elements?
(In reply to Sebastian Dröge (slomo) from comment #35) > What do you mean with "they enable negotiation"? Between which elements? I mean, the pipeline links completely and goes to play state (attachment 296513 [details]) :) Without the patches, the pipeline never reaches the play state, fails as in comment #5 (and the attachment 296512 [details])
This is interesting, because those patches produce the opposite effect for me : https://bugzilla.gnome.org/show_bug.cgi?id=742924
*** Bug 739530 has been marked as a duplicate of this bug. ***
*** Bug 744523 has been marked as a duplicate of this bug. ***
As usual, I think Edward is right :) After debugging I found that part of the problem is in gstreamer-vaapi, which I state in bug #744618 : Currently, in vaapidecode, the src caps are set immediately after the sink caps are negotiated, but at that moment the pipeline may not be fully constructed and the video sink has not negotiated its supported caps. In this case, the forced feature is the least optimized one: memory:SystemMemory and I420 color format. Even more, the reconfiguration events in downstream are ignored. The first patch (attachment #297391 [details]) tries to solve this problem delaying the src caps negotiation until the first frame in handled. Also it checks if the caps in the src pad need to be reconfigured. Nevertheless, I found *another* problem, which is not related with gstreamer-vaapi in my opinion, but with the caps negotiation during the auto-plugin in the playback: vaapidecode always adds to its buffer pool the configuration option GST_BUFFER_POOL_OPTION_VIDEO_GL_TEXTURE_UPLOAD_META if the decide_allocation()'s query has GST_VIDEO_GL_TEXTURE_UPLOAD_META_API_TYPE. There are occasions where the query has the API type, but the negotiated src caps don't have the feature meta:GstVideoGLTextureUploadMeta but memory:SystemMemory. An example of this case occurs with https://hadess.fedorapeople.org/bug743687.mkv rendering ClutterAutoVideoSink: after several caps re-negotiations they are settle as memory:SystemMemory with I420 format, though during the decide_allocation() the vaapidecoder buffer pool enables the options GLTextureUplodMeta because the query has the API type. Under this contradiction, vaapidecode adds the GLTextureUploadMeta API to its buffer pool configuration, and adds its buffer's meta to each output buffer, even if the negotiated caps feature is memory:SystemMemory with I420 color format. As a result, ClutterAutoVideoSink doesn't render any buffer since its gl upload method for I420 buffers is a no-op. The second patch (attachment #297746 [details]) workarounds this situation by only adding the option GST_BUFFER_POOL_OPTION_VIDEO_GL_TEXTURE_UPLOAD_META to the vaapidecode buffer pool *if* the query has the API type *and* the negotiated caps feature is meta:GstVideoGLTextureUploadMeta. In my opinion I should open another bug for this issue in playback. What do you think?
(In reply to Víctor Manuel Jáquez Leal from comment #40) > There are occasions where the query has the API type, but the negotiated src > caps don't have the feature meta:GstVideoGLTextureUploadMeta but > memory:SystemMemory. That's a completely valid situation. The capsfeatures are only there to require during negotiation that the meta *must* be there. You can have arbitrary metas on sysmem caps as long as a) downstream supports them if they are required for interpretation of the memory (e.g. GstVideoMeta) and b) everything works like sysmem (i.e. you can use gst_buffer_map()). As discussed with Victor, everything else here is not really a bug and bug #742924 is the real problem.