After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 781759 - visualizations fail with vaapi
visualizations fail with vaapi
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gstreamer-vaapi
1.10.4
Other Linux
: Normal normal
: 1.12.1
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2017-04-26 11:49 UTC by Jussi Kukkonen
Modified: 2017-05-16 11:39 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
GST_DEBUG=vaapi:5,vaapisink:5 gst-launch-1.0 audiotestsrc ! goom ! videoconvert ! vaapisink (18.30 KB, text/plain)
2017-04-26 11:49 UTC, Jussi Kukkonen
  Details
plugins: handle pixel-aspect-ratio with value 0/1 (1.79 KB, patch)
2017-04-26 13:55 UTC, Víctor Manuel Jáquez Leal
committed Details | Review
plugins: remove par from caps negotiation (1011 bytes, patch)
2017-05-12 11:48 UTC, Víctor Manuel Jáquez Leal
committed Details | Review
image of broken visualization (105.98 KB, image/jpeg)
2017-05-16 08:13 UTC, Maxin B. John
  Details

Description Jussi Kukkonen 2017-04-26 11:49:18 UTC
Created attachment 350452 [details]
GST_DEBUG=vaapi:5,vaapisink:5 gst-launch-1.0 audiotestsrc ! goom ! videoconvert ! vaapisink

"gtk-play" from gst-player fails to play any audio files with vaapi installed because playbins autovideosink picks vaapisink as the sink and then visualizations fail and that makes the audio playback fail as well. I'm assuming the autovideosink decision to use vaapisink is reasonable so filed this bug on vaapi...

This seems to trigger the same issue:
    gst-launch-1.0 audiotestsrc ! goom ! videoconvert ! vaapisink

The same command with ximagesink or xvimagesink works. With vaapi I get a bunch of criticals like "gst_video_calculate_display_ratio: assertion 'num > 0' failed" and then a internal data stream error.

The issue seems to be a video pixel-aspect-ratio "0/1" (which I don't see with other sinks).

Versions:
 * gstreamer & gstreamer-vaapi 1.10.4
 * libva and va-intel-driver 1.7.3
Hardware is a skylake NUC.
Comment 1 Víctor Manuel Jáquez Leal 2017-04-26 13:16:10 UTC
Obviously, this can be workarounded with

gst-launch-1.0 audiotestsrc ! goom ! video/x-raw, pixel-aspect-ratio=\(fraction\)1/1 ! videoconvert ! vaapisink

but still, this should be an easy fix.
Comment 2 Víctor Manuel Jáquez Leal 2017-04-26 13:55:44 UTC
Created attachment 350478 [details] [review]
plugins: handle pixel-aspect-ratio with value 0/1

When downstream negotiates a pixel-aspect-ratio of 0/1, the
calculations for resizing and formatting in vaapipostproc and
vaapisink, respectively, failed, and thus the pipeline.

This patch handles this situation by converting p-a-r of 0/1 to
1/1. This is how other sinks, such as glimagesink, work.
Comment 3 Jussi Kukkonen 2017-04-26 16:17:28 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #1)
> Obviously, this can be workarounded with
> 
> gst-launch-1.0 audiotestsrc ! goom ! video/x-raw,
> pixel-aspect-ratio=\(fraction\)1/1 ! videoconvert ! vaapisink

Yep, the issue is apps (like gtk-play that we give to QA as testing tool). It's trickier to start injecting something into their playbin-based pipeline.

it's still a minor issue that we can work around, I mostly filed it in case it's a symptom of some more important bug.

> This patch handles this situation by converting p-a-r of 0/1 to
> 1/1. This is how other sinks, such as glimagesink, work.

Ugh, I looked at that code... I should have thought to do that. Thanks.
Comment 4 Jussi Kukkonen 2017-04-27 08:41:47 UTC
I had a chance to test this: It does fix the issue with the gst-launch-1.0 pipeline. Unfortunately something still goes wrong in the original pipeline that gtk-play uses :(

The issue looks related (and still only happens with gst-vaapi but maybe isn't actually a gst-vaapi issue). The playbin actually gets to PLAYING state but then

gstbasetransform.c:1417:gst_base_transform_setcaps:<conv> transform could not transform video/x-raw, format=(string)BGRx, width=(int)320, height=(int)200, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)0/1 in anything we support

and 

gstbaseparse.c:3608:gst_base_parse_loop:<mpegaudioparse0> error: Internal data stream error.

I'll try to figure it out myself and attach proper logs if I don't get it.
Comment 5 Víctor Manuel Jáquez Leal 2017-04-27 09:18:55 UTC
Comment on attachment 350478 [details] [review]
plugins: handle pixel-aspect-ratio with value 0/1

Attachment 350478 [details] pushed as 4820d2d - plugins: handle pixel-aspect-ratio with value 0/1
Comment 6 Víctor Manuel Jáquez Leal 2017-04-27 09:31:31 UTC
on branch 1.10

commit 1bd051c533426fa619b9d17ecb6f8db1c1a65798

    plugins: handle pixel-aspect-ratio with value 0/1
Comment 7 Víctor Manuel Jáquez Leal 2017-04-27 09:33:31 UTC
closing this bug, but I'll open another to fix goom which is the real culprit.
Comment 8 Víctor Manuel Jáquez Leal 2017-05-12 11:47:05 UTC
Reopening because I found I better approach, or the more common approach to avoid pars of value of 0/1: don't ask for p-a-r explicitly in caps.
Comment 9 Víctor Manuel Jáquez Leal 2017-05-12 11:48:02 UTC
Created attachment 351710 [details] [review]
plugins: remove par from caps negotiation
Comment 10 Hyunjun Ko 2017-05-12 12:41:45 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #9)
> Created attachment 351710 [details] [review] [review]
> plugins: remove par from caps negotiation

I like this :)
Comment 11 Víctor Manuel Jáquez Leal 2017-05-12 14:54:38 UTC
Attachment 351710 [details] pushed as b3c374e - plugins: remove par from caps negotiation
Comment 12 Víctor Manuel Jáquez Leal 2017-05-12 15:30:19 UTC
branch 1.10 

* 97007d1d plugins: remove par from caps negotiation
Comment 13 Víctor Manuel Jáquez Leal 2017-05-12 15:33:34 UTC
branch 1.12 

* 39636b6e plugins: remove par from caps negotiation
Comment 14 Maxin B. John 2017-05-16 08:13:47 UTC
Created attachment 351945 [details]
image of broken visualization

Hi,

goom seems to work with the patch. However, it looks broken (please see the image). Can we reopen this bug ?
Comment 15 Víctor Manuel Jáquez Leal 2017-05-16 08:43:53 UTC
> goom seems to work with the patch. However, it looks broken (please see the
> image). Can we reopen this bug ?

Are you using gstreamer-vaapi 1.10?
What version of the intel-va-driver are you using?

That's is fixed in gstreamer-vaapi 1.12 using a recent intel-va-driver (master at least)

The problem is that the color format pushed by goom is (in theory) handled by va display surface, it is not. Recently we applied a patch to reject those color spaces, so gstreamer-vaapi use VPP to transform the color space.
Comment 16 Maxin B. John 2017-05-16 09:00:32 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #15)
> > goom seems to work with the patch. However, it looks broken (please see the
> > image). Can we reopen this bug ?
> 
> Are you using gstreamer-vaapi 1.10?

gstreamer-vaapi version: 1.12

> What version of the intel-va-driver are you using?
 
libva-intel-driver version: 1.7.3

> That's is fixed in gstreamer-vaapi 1.12 using a recent intel-va-driver
> (master at least)

Ok.

> The problem is that the color format pushed by goom is (in theory) handled
> by va display surface, it is not. Recently we applied a patch to reject
> those color spaces, so gstreamer-vaapi use VPP to transform the color space.

Thank you! Will upgrade the intel-va-driver to the latest version and try again.
Comment 17 Maxin B. John 2017-05-16 11:39:16 UTC
Upgrading libva-intel-driver to 1.8.3.pre1 release fixed it. Thank you for the advice.