After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 730997 - Unable to decode or display h264 high-10 files?
Unable to decode or display h264 high-10 files?
Status: RESOLVED FIXED
Product: gstreamer-vaapi
Classification: Other
Component: general
0.5.8
Other Linux
: Normal normal
: ---
Assigned To: gstreamer-vaapi maintainer(s)
gstreamer-vaapi maintainer(s)
Depends on: 732239
Blocks: 731852
 
 
Reported: 2014-05-30 14:03 UTC by Nirbheek Chauhan
Modified: 2014-10-29 14:58 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
$ GST_DEBUG=*:6 gst-play-1.0 test.mkv (1.44 MB, application/x-xz)
2014-05-30 14:09 UTC, Nirbheek Chauhan
  Details
$ GST_DEBUG=*:6 gst-launch-1.0 filesrc location=test.mkv ! matroskademux ! avdec_h264 ! autovideosink (941.48 KB, application/x-xz)
2014-05-30 14:10 UTC, Nirbheek Chauhan
  Details
vaapidecode: Expose the supported profiles as caps to upstream (1.64 KB, patch)
2014-10-21 14:17 UTC, sreerenj
none Details | Review

Description Nirbheek Chauhan 2014-05-30 14:03:07 UTC
It seems that gst-vaapi is unable to do caps negotiation properly when given high-10 h264 video files, and gives a stream error when the pipeline is played.

For the attached file, the format is: caps = video/x-h264, level=(string)5, profile=(string)high-10

Using playbin:

$ gst-play-1.0 test.mkv 
Now playing /home/nirbheek/test.mkv
libva info: VA-API version 0.34.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_34
libva info: va_openDriver() returns 0
ERROR GStreamer encountered a general stream error. for file:///home/nirbheek/test.mkv
ERROR debug information: matroska-demux.c(4671): gst_matroska_demux_loop (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0:
stream stopped, reason not-supported
Reached end of play list.


Using software decode and vaapisink:

$ gst-launch-1.0 filesrc location=test.mkv ! matroskademux ! avdec_h264 ! autovideosink 
Setting pipeline to PAUSED ...
libva info: VA-API version 0.34.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_34
libva info: va_openDriver() returns 0
Pipeline is PREROLLING ...
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: GStreamer encountered a general stream error.
Additional debug info:
matroska-demux.c(4671): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0:
stream stopped, reason not-negotiated
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...


Using software decode and xvimagesink works.

$ gst-launch-1.0 filesrc location=test.mkv ! matroskademux ! avdec_h264 ! videoconvert ! xvimagesink


Stream details:

/GstPipeline:pipeline0/avdec_h264:avdec_h264-0.GstPad:sink: caps = video/x-h264, level=(string)5, profile=(string)high-10, codec_data=(buffer)016e0032ffe1001c676e0032a6c520b014016e9a810100a000007d2000177011e306317001000768e8ae09cf23c000000800, stream-format=(string)avc, alignment=(string)au, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1000/1

/GstPipeline:pipeline0/avdec_h264:avdec_h264-0.GstPad:src: caps = video/x-raw, format=(string)I420_10LE, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, framerate=(fraction)1000/1


After talking to Tim and Arun, it seems that the problem *might be* that the high-10 profile isn't supported by vaapidecode[1], and that I420_10LE isn't supported by vaapisink(?)[2], but playbin should still do caps negotiation properly and do a fallback.

GST_DEBUG=*:6 logs for both the error conditions above are attached in later comments.


---------
1. # vainfo 
libva info: VA-API version 0.34.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_34
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.34 (libva 1.2.1)
vainfo: Driver version: Intel i965 driver - 1.2.2
vainfo: Supported profile and entrypoints
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Simple            :	VAEntrypointEncSlice
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice

2. # gst-inspect-1.0 vaapisink
[...]
  Version                  0.5.8
[...]
    Capabilities:
      video/x-raw(memory:VASurface)
                 format: { ENCODED, NV12, I420, YV12 }
[...]
      video/x-raw
                 format: { I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, NV16, NV24, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16, BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, I420_10LE, I420_10BE, I422_10LE, I422_10BE, Y444_10LE, Y444_10BE, GBR, GBR_10LE, GBR_10BE }
Comment 1 Nirbheek Chauhan 2014-05-30 14:07:19 UTC
I was unable to attach the mkv file because it was too large (3MB, 20 seconds). Making it any smaller causes gstreamer to not be able to finish playing the pipeline, and the whole thing just sits there without giving any errors. So, I've put it on my server:

http://nirbheek.in/files/test.mkv
Comment 2 Nirbheek Chauhan 2014-05-30 14:09:27 UTC
Created attachment 277552 [details]
$ GST_DEBUG=*:6 gst-play-1.0 test.mkv

GST_DEBUG log file with playbin (vaapidecode + vaapisink) and the h264-high10 file.
Comment 3 Nirbheek Chauhan 2014-05-30 14:10:47 UTC
Created attachment 277553 [details]
$ GST_DEBUG=*:6 gst-launch-1.0 filesrc location=test.mkv ! matroskademux ! avdec_h264 ! autovideosink

GST_DEBUG log file with avdec_h264 + vaapisink and the h264-high10 file.
Comment 4 Gwenole Beauchesne 2014-06-03 12:40:56 UTC
AFAIK, there is no hardware accelerated decoder for H.264 High-10 profiles out there. So decoding will fail. This also means that vaapisink will not support high bit depth YUV formats either at this time.

As to the h264parse element, I was also recently thinking about making it expose the underlying profile so that fallbacks could be more effective. That could also be useful to MVC so that to support base views if there is no native decoder for non-base-views.

I think vaapidecode already reports the exact set of supported profiles by the hardware. i.e. only h264parse would need to expose the parsed profile specs IMHO.
Comment 5 Nirbheek Chauhan 2014-06-03 14:22:30 UTC
(In reply to comment #4)
> As to the h264parse element, I was also recently thinking about making it
> expose the underlying profile so that fallbacks could be more effective. That
> could also be useful to MVC so that to support base views if there is no native
> decoder for non-base-views.
> 
> I think vaapidecode already reports the exact set of supported profiles by the
> hardware. i.e. only h264parse would need to expose the parsed profile specs
> IMHO.

That would be really helpful, because right now the mere act of installing the gstreamer-vaapi plugin causes seemingly-random h264 files to stop playing in Totem with an obscure "Internal stream error" message.
Comment 6 Gwenole Beauchesne 2014-07-03 08:33:41 UTC
A bug was created for h264parse, but it looks too late for GStreamer 1.4 release. So, that would be postponed to 1.5. Of course, your favourite OS vendor is free to pick and back-port the necessary support patches to GStreamer 1.2.
Comment 7 Nirbheek Chauhan 2014-07-05 04:03:14 UTC
Thank you for looking into this and keeping track! :)
Comment 8 sreerenj 2014-10-20 11:48:26 UTC
The h264parse issue has been fixed and it is here: https://bugzilla.gnome.org/show_bug.cgi?id=732239.

But one fix is needed in vaapidecode too :).I will get back to this soon.
Comment 9 sreerenj 2014-10-21 14:17:54 UTC
Created attachment 289041 [details] [review]
vaapidecode: Expose the supported profiles as caps to upstream

The attached patch plus the fix I have provided in #732239 will allows the S/W decoder fallback for profiles like h264-high-10 which are not supported by hardware decoder.

Again, in order to test this with gstreamer-vaapi master, we need to pull the upstream h264parser changes to vaapiparse_h264 too.
Comment 10 Gwenole Beauchesne 2014-10-29 06:00:47 UTC
Review of attachment 289041 [details] [review]:

OK, with the mentioned change. This will break 0.10 builds but I don't care, it's deprecated/obsolete already.

Note: I didn't realize it was so simple and that we already had the utility functions in place. :) I will merge the upstream h264parse changes to test another bug too (bug #739291).

::: gst/vaapi/gstvaapidecode.c
@@ +919,3 @@
         const gchar *media_type_name;
+        const gchar *profile_name;
+        GstStructure *st;

s/st/structure/ for consistency with the rest.
Comment 11 sreerenj 2014-10-29 14:38:35 UTC
commit b58bdd1a1239c2996c42b417db31632d596de9c4
Author: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
Date:   Wed Oct 29 15:46:47 2014 +0200

    vaapidecode: Expose the supported profiles as caps to upstream
    
    This will allows the playbin to fallback to Software Decoder
    if the Hardware Decoder does not support a particular profile.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=730997
Comment 12 sreerenj 2014-10-29 14:40:08 UTC
Please close this bug once you manged to pull the upstream h264parser changes to vaapiparse_h264.
Comment 13 Gwenole Beauchesne 2014-10-29 14:58:18 UTC
commit c472403bb4380b9bdb70ee9205758cd0cae570ed
Author: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Date:   Wed Oct 29 15:45:50 2014 +0100

    codecparsers: update to gst-vaapi-branch commit f9d3bde.
    
    2218b02 h264parse: expose parsed profile and level to downstream
    3dbfab4 h264parse: return flushing if we get chained while being set to READY
    d40fa8b h264: fix frame packing SEI parsing
    32d40be h264: Use proper bit_reader api while parsing buffering_period SEI
    b3e022e h264: initialize some fields of pic_timing structure
    a70661d vc1: fix expected level in sequence-layer parsing unit test
    6cee88d vc1: fix level values for simple/main profile
    356c189 vc1: add unit test for sequence-layer parsing
    ab9f641 vc1: take care of endianness when parsing sequence-layer
    8dc8e35 mpeg4: fix vlc table used for sprite trajectory