After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 745660 - [g45-h264] revert the use of I420 back to NV12
[g45-h264] revert the use of I420 back to NV12
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gstreamer-vaapi
unspecified
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2015-03-05 10:34 UTC by Ilya Gordeev
Modified: 2016-10-31 14:21 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
GST_DEBUG=5 firefox (1.37 MB, application/x-gzip)
2015-03-05 10:41 UTC, Ilya Gordeev
  Details
GST_DEBUG=5 firefox with gstreamer-vaapi-git-8f56b758b6a3d6e7 (1.20 MB, application/x-gzip)
2015-03-13 17:40 UTC, Ilya Gordeev
  Details
mozconfig (3.35 KB, text/plain)
2015-05-27 20:04 UTC, Ilya Gordeev
  Details
0.5.9-revert-exposing-I420-format.patch (3.19 KB, patch)
2015-05-27 20:06 UTC, Ilya Gordeev
none Details | Review
0.5.10-revert-exposing-I420-format.patch (3.96 KB, patch)
2015-05-27 20:07 UTC, Ilya Gordeev
none Details | Review
master-revert-exposing-I420-format.patch (4.75 KB, patch)
2015-05-27 20:07 UTC, Ilya Gordeev
none Details | Review

Description Ilya Gordeev 2015-03-05 10:34:31 UTC
If I try to play any H.264/MPEG-4 video in firefox then just green field is shown. And firefox prints this messages to console:
(firefox:21153): GStreamer-CRITICAL **: gst_mini_object_lock: assertion 'object != NULL' failed
(firefox:21153): GStreamer-CRITICAL **: gst_memory_unmap: assertion 'mem != NULL' failed
(firefox:21153): GStreamer-CRITICAL **: gst_memory_get_sizes: assertion 'mem != NULL' failed
(firefox:21153): GStreamer-CRITICAL **: gst_mini_object_lock: assertion 'object != NULL' failed
(firefox:21153): GStreamer-CRITICAL **: gst_memory_unmap: assertion 'mem != NULL' failed
(firefox:21153): GStreamer-CRITICAL **: gst_memory_get_sizes: assertion 'mem != NULL' failed
(firefox:21153): GStreamer-CRITICAL **: gst_mini_object_unlock: assertion 'object != NULL' failed
(firefox:21153): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed
... (and so on)

At the same time "gst-play-1.0 --videosink=vaapisink" works fine and can play any H.264/MPEG-4 video.
The same bug: https://bugzilla.redhat.com/show_bug.cgi?id=1126956

distribution: Gentoo Linux

packages versions:
glib-2.42.2
gstreamer-1.4.5
gstreamer-vaapi-0.5.10
libva-1.5.0
libva-intel-driver-1.5.0 (merged with "g45-h264" branch)
firefox-35.0

vainfo:
libva info: VA-API version 0.37.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/va/drivers/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_37
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.37 (libva 1.5.0)
vainfo: Driver version: Intel i965 driver for Intel(R) GM45 Express Chipset - 1.5.0
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointVLD


I tried to debug a little (but it turned up too complex for me).
When I run "GST_DEBUG=5 firefox" I get large debug output where I found this lines:
0:00:01.092433681  5077 0x7ffbb14fe6a0 DEBUG                  vaapi gstreamer-vaapi-0.5.10/gst-libs/gst/vaapi/gstvaapiimage.c:261:gst_vaapi_image_new: format I420, size 1280x720
0:00:01.092484595  5077 0x7ffbb14fe6a0 DEBUG                  vaapi gstreamer-vaapi-0.5.10/gst-libs/gst/vaapi/gstvaapiimage.c:206:gst_vaapi_image_create: image 0xa000000
...
0:00:01.092543122  5077 0x7ffbb14fe6a0 DEBUG                  vaapi gstreamer-vaapi-0.5.10/gst-libs/gst/vaapi/gstvaapiutils.c:53:vaapi_check_status: vaGetImage(): invalid VAImageFormat
0:00:01.092796366  5077 0x7ffbb14fe6a0 ERROR       vaapivideomemory gstreamer-vaapi-0.5.10/gst/vaapi/gstvaapivideomemory.c:453:gst_vaapi_video_memory_map: failed to make image current
0:00:01.092807471  5077 0x7ffbb14fe6a0 ERROR             GST_MEMORY gstreamer-1.4.5/gst/gstmemory.c:318:gst_memory_map: mem 0x7ffbafa20450: subclass map failed

So most likely problem appears after vaGetImage() returns VA_STATUS_ERROR_INVALID_IMAGE_FORMAT.
Here it is backtrace of this call:
  • #0 vaapi_check_status
  • #1 gst_vaapi_surface_get_image
    at gstreamer-vaapi-0.5.10/gst-libs/gst/vaapi/gstvaapisurface.c line 684
  • #2 ensure_image_is_current
    at gstreamer-vaapi-0.5.10/gst/vaapi/gstvaapivideomemory.c line 105
  • #3 gst_vaapi_video_memory_map
    at gstreamer-vaapi-0.5.10/gst/vaapi/gstvaapivideomemory.c line 405
  • #4 gst_memory_map
    at gstreamer-1.4.5/gst/gstmemory.c line 295
  • #5 gst_memory_make_mapped
    at gstreamer-1.4.5/gst/gstmemory.c line 236
  • #6 _get_mapped
    at gstreamer-1.4.5/gst/gstbuffer.c line 885
  • #7 gst_buffer_extract
    at gstreamer-1.4.5/gst/gstbuffer.c line 1712
  • #8 ??
    from /usr/lib64/firefox/libxul.so

This VA_STATUS_ERROR_INVALID_IMAGE_FORMAT comes from library libva-intel-driver, file i965_drv_video.c, function i965_sw_getimage, lines 3969-3970:
if (obj_surface->fourcc != obj_image->image.format.fourcc)
    return VA_STATUS_ERROR_INVALID_IMAGE_FORMAT;

I added debug print in this "if" and have got this output:
obj_surface->fourcc = NV12, obj_image->image.format.fourcc = I420

Maybe it will help to fix this problem.
Comment 1 Ilya Gordeev 2015-03-05 10:41:45 UTC
Created attachment 298619 [details]
GST_DEBUG=5 firefox

Full "GST_DEBUG=5 firefox" debug log
Comment 2 Víctor Manuel Jáquez Leal 2015-03-11 12:26:54 UTC
I have been poking at this bug. No news yet, just a couple comments:

In my Debian box, Firefox doesn't link with GStreamer. I downloaded the latest compiled tarball for x86_64, and it doesn't link to GStreamer either.

Then I grabbed the source code of the last release, and a wild grep tells me there's GStreamer options at compilation. By default it uses gstreamer-0.10, but flipping some switches, gstreamer-1.0 can be activated.

Nevertheless my compilation fails at a random point. I need to look deeper.
Comment 3 Víctor Manuel Jáquez Leal 2015-03-11 12:34:56 UTC
Ilya, out of curiosity, could you try with the current master of gstreamer-vaapi?
What's your hardware?
Comment 4 Víctor Manuel Jáquez Leal 2015-03-11 12:35:55 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #3)

> What's your hardware?

Oh, nevermind about this question: "GM45 Express Chipset"
Comment 5 Víctor Manuel Jáquez Leal 2015-03-11 16:44:01 UTC
Ok.

I just setup 

* gstreamer master (1.5)
* gstreamer-vaapi master 
* libva master - 0.37 (libva 1.5.1.pre1)
* libva-intel-driver master (without merging g45-h264) -  1.5.1.pre1 (1.3.2-209-gc9c4d24)

My hardware is a Haswell Mobile.

Finally I could compile today's firefox nightly build (firefox-37.0b4)

And... I can play a complete motion picture (mp4/h264) within my firefox compiled-from-scratch.

As I mentioned you, if you can compile gstreamer-vaapi, the current master branch, and verify if your firefox 35 works, would be great.
Comment 6 Ilya Gordeev 2015-03-13 17:36:29 UTC
Updated firefox to 36.0.1: nothing changed.

Installed newest gstreamer-vaapi (git commit 8f56b758b6a3d6e7).

firefox:
Playing html5 mp4 video: empty (black) field.
Playing local mp4 file: field for video isn't drawn, just audio player.
Nothing (strange) in console.
In GST_DEBUG=5 log:
0:00:01.235103430 10025 0x7fe7d0a40000 DEBUG               GST_CAPS gstreamer-1.4.5/gst/gstpad.c:5030:pre_eventfunc_check:<queue:sink> caps video/x-raw, format=(string)I420, width=(int)640, height=(int)368, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, framerate=(fraction)24/1 not accepted
0:00:01.235126129 10025 0x7fe7d0a40000 DEBUG               GST_PADS gstreamer-1.4.5/gst/gstpad.c:5235:gst_pad_send_event_unchecked:<queue:sink> pre event check failed
0:00:01.235138351 10025 0x7fe7d0a40000 DEBUG               GST_PADS gstreamer-1.4.5/gst/gstpad.c:3477:push_sticky:<vaapidecode:src> result not-negotiated, mark pending events
0:00:01.235153716 10025 0x7fe7d0a40000 ERROR            vaapidecode gstreamer-vaapi-0.5.10/gst/vaapi/gstvaapidecode.c:359:gst_vaapidecode_push_decoded_frame: video sink rejected the video buffer (error: not-negotiated [-4])
...
0:00:01.235238922 10025 0x7fe7d0a40000 ERROR            vaapidecode gstreamer-vaapi-0.5.10/gst/vaapi/gstvaapidecode.c:447:gst_vaapidecode_handle_frame: push loop error after decoding -4
0:00:01.235253030 10025 0x7fe7d0a40000 DEBUG           videodecoder gst-plugins-base-1.4.5/gst-libs/gst/video/gstvideodecoder.c:3008:gst_video_decoder_decode_frame:<vaapidecode> flow error not-negotiated

gst-play-1.0:
Now it immediately stops without playing anything (even audio) and prints to console:
---
ERROR GStreamer encountered a general stream error. for file:///tmp/test.mp4
ERROR debug information: gst-plugins-good-1.4.5/gst/isomp4/qtdemux.c(4434): gst_qtdemux_loop (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason not-negotiated
---
Run with GST_DEBUG=5 produces almost the same log.
Comment 7 Ilya Gordeev 2015-03-13 17:40:38 UTC
Created attachment 299344 [details]
GST_DEBUG=5 firefox with gstreamer-vaapi-git-8f56b758b6a3d6e7
Comment 8 Ilya Gordeev 2015-03-13 17:41:33 UTC
Also I installed ubuntu "vivid" (15.04) to chroot via debootstrap:
debootstrap --arch amd64 vivid ./rootfs/ http://archive.ubuntu.com/ubuntu/
(bind-mounted /dev and /tmp/.X11-unix so I can run any X-apps and use VAAPI)

packages versions:
gstreamer1.0-*-1.4.5
gstreamer1.0-vaapi-0.5.9, libgstreamer-vaapi1.0-0.5.9
libva1-1.5.0
i965-va-driver-1.5.0
firefox-36.0.1 (already has gstreamer1.0 support)

First of all with original i965-va-driver I can use VAAPI only for mpeg2.
vainfo:
libva info: VA-API version 0.37.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_37
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.37 (libva 1.5.0)
vainfo: Driver version: Intel i965 driver for Intel(R) GM45 Express Chipset - 1.5.0
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD

Firefox plays mp4 videos (because now it doesn't use VAAPI).

Playing mpeg2-file gst-play-1.0 shows only green screen. In GST_DEBUG=5 log:
0:00:01.117447476 17627 0x7f559c0474a0 DEBUG                  vaapi gst-libs/gst/vaapi/gstvaapiutils.c:53:vaapi_check_status: vaPutImage(): invalid VAImageFormat
0:00:01.117468638 17627 0x7f559c0474a0 ERROR       vaapivideomemory gst/vaapi/gstvaapivideomemory.c:257:gst_video_meta_unmap_vaapi_memory: failed to upload image
(maybe it isn't related to this issue, but I don't use mpeg2 anyway)

But with newest gstreamer-vaapi from git gst-play-1.0 stops and prints:
---
ERROR Internal data stream error. for file:///tmp/test.mpg
ERROR debug information: gstmpegdemux.c(2948): gst_flups_demux_loop (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMpegPSDemux:mpegpsdemux0:
stream stopped, reason not-negotiated
---
In GST_DEBUG=5 log is the same "not-negotiated" error.

========

Then I built and installed i965-va-driver_1.5.0-1_amd64.deb with g45-h264 branch merge.
Now vainfo adds H264 profiles:
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointVLD

Firefox shows green screen, doesn't print any errors to console. In GST_DEBUG=5 log:
0:00:04.659235020 10581 0x7f97d42610c0 DEBUG                  vaapi gst-libs/gst/vaapi/gstvaapiutils.c:53:vaapi_check_status: vaGetImage(): invalid VAImageFormat

gst-play-1.0 correctly plays mp4 file (mpeg2 - still green screen)

========

Then I installed gstreamer1.0-vaapi & libgstreamer-vaapi1.0 from git (commit 8f56b758b6a3d6e7)

Firefox shows empty field. No errors in console. In GST_DEBUG=5 log:
0:00:01.381423642  9425 0x7f31952634f0 DEBUG               GST_CAPS gstpad.c:5030:pre_eventfunc_check:<queue:sink> caps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, framerate=(fraction)24000/1001 not accepted
0:00:01.381449832  9425 0x7f31952634f0 DEBUG               GST_PADS gstpad.c:5235:gst_pad_send_event_unchecked:<queue:sink> pre event check failed
0:00:01.381463731  9425 0x7f31952634f0 DEBUG               GST_PADS gstpad.c:3477:push_sticky:<vaapidecode:src> result not-negotiated, mark pending events
0:00:01.381480353  9425 0x7f31952634f0 ERROR            vaapidecode gst/vaapi/gstvaapidecode.c:359:gst_vaapidecode_push_decoded_frame: video sink rejected the video buffer (error: not-negotiated [-4])
0:00:01.381496207  9425 0x7f31952634f0 DEBUG                default gstvideoutils.c:40:_gst_video_codec_frame_free: free frame 0x7f31988cbcc0
0:00:01.381518556  9425 0x7f31952634f0 DEBUG             GST_MEMORY gstmemory.c:87:_gst_memory_free: free memory 0x7f31a8a985c0
0:00:01.381540067  9425 0x7f31952634f0 DEBUG             GST_BUFFER gstbuffer.c:1201:gst_buffer_is_memory_range_writable: idx 0, length -1
0:00:01.381563464  9425 0x7f31952634f0 ERROR            vaapidecode gst/vaapi/gstvaapidecode.c:447:gst_vaapidecode_handle_frame: push loop error after decoding -4

Playing mp4-file gst-play-1.0 stops with error:
---
ERROR GStreamer encountered a general stream error. for file:///tmp/test.mp4
ERROR debug information: qtdemux.c(4434): gst_qtdemux_loop (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason not-negotiated
---
The same "not-negotiated" error in GST_DEBUG=5 log.

========

If you decide to test this issue in ubuntu environment I can provide any help to build ubuntu chroot.
Also I can post any GST_DEBUG=5 log if needed.
Comment 9 Víctor Manuel Jáquez Leal 2015-03-23 15:13:49 UTC
The only version of firefox that I could built is a 37.0 nightly release. With that I can play videos with gstreamer 1.4.5 and 1.5.

Can you try with firefox 37.0?

Where I can get you media test file?
Comment 10 Ilya Gordeev 2015-03-25 19:42:54 UTC
Tried firefox-37.0_beta6 - nothing changed.
Also I forget to describe that I disable firefox plugin "OpenH264 Video Codec provided by Cisco" to play videos exactly though gstreamer.
Any mpeg4 video lead to the same behavior, for example this: http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4
Comment 11 Víctor Manuel Jáquez Leal 2015-04-08 14:11:47 UTC
I'm still unable to replicate the error. 

I'm right now using firefox-37.0b4 under Debian testing, with its own gstreamer packages (even the gstreamer-vaapi 0.5.9) and works :/

How do you compile firefox?

This is my mozconfig file:

#export PATH="/usr/lib/ccache:${PATH}"
export CC=gcc
export CXX=g++
#export CCACHE_PREFIX=icecc
#export ICECC_DEBUG=1

# Normal build flags.  These make a debug browser build.
mk_add_options MOZ_MAKE_FLAGS="-s -j24 VERBOSE=1"
#mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../ff-dbg

ac_add_options --enable-application=browser
ac_add_options --disable-debug
ac_add_options --disable-optimize
ac_add_options --disable-libnotify
ac_add_options --disable-gnomevfs
ac_add_options --disable-gamepad
ac_add_options --enable-gstreamer=1.0
ac_add_options --with-ccache=/usr/bin/ccache
Comment 12 Víctor Manuel Jáquez Leal 2015-05-21 11:06:38 UTC
Is this bug verifiable?

(In reply to Víctor Manuel Jáquez Leal from comment #11)
> How do you compile firefox?


ping?
Comment 13 Víctor Manuel Jáquez Leal 2015-05-27 11:20:35 UTC
Closing this bug report as no further information has been provided. Please feel free to reopen this bug report if you can provide the information that was asked for in a previous comment.

Thanks!
Comment 14 Ilya Gordeev 2015-05-27 20:04:15 UTC
Unfortunately, I still have this issue.
I use Gentoo, so firefox is compiled according to the corresponding ebuild in portage tree. I'll add mozconfig to attachment. But as I told I got this issue in ubuntu chroot with its native firefox package.
Maybe you can't reproduce this bug because you have different video card or because I use libva-intel-driver merged with "g45-h264" branch.
Comment 15 Ilya Gordeev 2015-05-27 20:04:57 UTC
Created attachment 304103 [details]
mozconfig
Comment 16 Ilya Gordeev 2015-05-27 20:06:11 UTC
This bug appeared after I updated gstreamer-vaapi from 0.5.8 to 0.5.9 version. To find exactly what change leads to problem I used git-bisect. And I've got that issue shows up after commit dac20cecb4e2c2eb "plugins: expose I420 format for interop with SW elements". So I tried to revert this change and prepared patch for version 0.5.9 that reverts exposing I420 format. Then I rebased this patch with some changes to 0.5.10 and master. All this patches work for me. I'll put they to attachments.
Unfortunately preparing this patch for master was not so easy for me because code from master leads to another runtime error as you can see in my previous comments (/GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0: streaming stopped, reason not-negotiated) (like https://bugzilla.gnome.org/show_bug.cgi?id=724062). So I just disabled gstvaapidecodebin plugin and then gst-play and firefox worked fine.
Comment 17 Ilya Gordeev 2015-05-27 20:06:51 UTC
Created attachment 304104 [details] [review]
0.5.9-revert-exposing-I420-format.patch
Comment 18 Ilya Gordeev 2015-05-27 20:07:23 UTC
Created attachment 304105 [details] [review]
0.5.10-revert-exposing-I420-format.patch
Comment 19 Ilya Gordeev 2015-05-27 20:07:42 UTC
Created attachment 304106 [details] [review]
master-revert-exposing-I420-format.patch
Comment 20 Víctor Manuel Jáquez Leal 2015-06-01 14:31:17 UTC
It seems that the GM45 Express Chipset is not capable to handle I420 colorspace (using libva-intel-driver's branch g45-h264), which is the current default colorspace used in gstreamer-vaapi.

Reverting commit dac20cecb, returning to NV12 colorspace as default, seems to solve the issue.

I'm not sure why I420 was choosen as default (I need to verify if NV12 is well supported in gstreamer 1.2).
Comment 21 Víctor Manuel Jáquez Leal 2015-06-01 14:45:02 UTC
(In reply to Ilya Gordeev from comment #16)

> Unfortunately preparing this patch for master was not so easy for me because
> code from master leads to another runtime error as you can see in my
> previous comments
> (/GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/
> GstQTDemux:qtdemux0: streaming stopped, reason not-negotiated) (like
> https://bugzilla.gnome.org/show_bug.cgi?id=724062). So I just disabled
> gstvaapidecodebin plugin and then gst-play and firefox worked fine.

This is a known issue in gstreamer 1.4. See bug 749554.
Comment 22 sreerenj 2016-03-24 16:54:24 UTC
Moving to Product:GStreamer, Component:gstreamer-vaapi
Comment 23 Víctor Manuel Jáquez Leal 2016-04-26 11:18:44 UTC
Since commit fd04a7c046 NV12 is again the default color format.

So let's close this bug as already fixed.