After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 779807 - Both videomixer and glvideomixer do not want to work properly with avdec_h264 decoder
Both videomixer and glvideomixer do not want to work properly with avdec_h264...
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-libav
1.10.x
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2017-03-09 14:27 UTC by Eugen Klim
Modified: 2018-11-03 12:58 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
scripts (681 bytes, application/x-compressed-tar)
2017-03-09 14:30 UTC, Eugen Klim
Details

Description Eugen Klim 2017-03-09 14:27:04 UTC
Hello. I want to build a graph which would mix pictures from a few h264 streams into one frame (so called ``picture in picture''). It seems that videomixer element is the right thing to do the trick. And videomixer works as expected with videotestsrc [graph_0.sh]. However, when I build graph with h264 decoder [graph_2.sh], it fails with an error: 

>ERROR: from element /GstPipeline:pipeline0/MpegTSParse2:mpegtsparse2-0: Internal data stream error.
>Additional debug info:
>mpegtsbase.c(1613): mpegts_base_loop (): /GstPipeline:pipeline0/MpegTSParse2:mpegtsparse2-0:
>streaming stopped, reason not-negotiated (-4)

But when I switch videomixer with glvideomixer [graph_3.sh] it works fine. However, if I add background via videotestsrc, it also fails [graph_3b.sh].

Graph with decodebin and glvideomixer also works [graph_4.sh], but graph with decodebin, glvideomixer and videotestsrc background [graph_4b.sh] -- does not.

But I've noticed that [graph_4b.sh] works perfectly with mpeg2 decoded ts, while failing with h264 stream.

I could not understand what's the problem, but it seems that the problem is avdec_h264 related, so I need some help. I'm not sure is it a bug or am i doing something wrong.

I've attached graph scripts and 2 test streams (mpeg2 and h264).

Thank you.
Comment 1 Eugen Klim 2017-03-09 14:30:37 UTC
Created attachment 347564 [details]
scripts
Comment 2 Eugen Klim 2017-03-09 14:35:46 UTC
Test streams are too large to be attached so I've posted them here:

https://yadi.sk/d/Iwpnwwxy3FMryR
Comment 3 Olivier Crête 2017-03-09 17:20:38 UTC
videomixer is being deprecated slowly, the replacement is "compositor", which uses the same baseclass as glvideomixer, so it should work the same, but you may want to test it too just in case.
Comment 4 Eugen Klim 2017-03-10 07:23:27 UTC
Yeah, thank you for reply. Compositor works for me even more worse than glvideomixer.

Pipeline

>gst-launch-1.0 \
>        filesrc location=$1 ! \
>        tsparse ! queue ! tee name=t \
>        compositor name=mix ! \
>        glimagesink \
>        videotestsrc pattern=0 ! video/x-raw,format=I420,width=1920,height=1080 ! mix.sink_0 \
>        t. ! queue ! tsdemux ! decodebin ! \
>        mix.sink_1 \
>        t. ! queue ! tsdemux ! decodebin ! \
>        mix.sink_2

fails with an error:

>ERROR: from element /GstPipeline:pipeline0/MpegTSParse2:mpegtsparse2-0: >Internal data stream error.
>Additional debug info:
>mpegtsbase.c(1613): mpegts_base_loop (): /GstPipeline:pipeline0/MpegTSParse2:mpegtsparse2-0:
>streaming stopped, reason not-negotiated (-4)
>Execution ended after 0:00:00.000194153

And 

>gst-launch-1.0 \
>        filesrc location=$1 ! \
>        tsparse ! tee name=t \
>        compositor name=mix ! \
>        xvimagesink \
>        t. ! queue ! tsdemux ! decodebin ! \
>        mix.sink_0 \
>        t. ! queue ! tsdemux ! decodebin ! \
>        mix.sink_1

works only once from four (literally, I start the graph, it fails, I start it agan, it works).

And in the pipeline with 

>compositor ! glimagesink

I have wrong colours (everything is green), seems like wrong colourspace conversion on the compositor's output, compositor ! xvimagesink works fine (but crushes once from four as I've said).

So glvideomixer seems works, but I still could not understand why

>gst-launch-1.0 \
>        filesrc location=$1 ! \
>        tsparse ! queue ! tee name=t \
>        glvideomixer name=mix ! \
>        glimagesink \
>        videotestsrc pattern=0 ! video/x-raw,format=I420,width=1920,height=1080 ! mix.sink_0 \
>        t. ! queue ! tsdemux ! decodebin ! \
>        mix.sink_1 \
>        t. ! queue ! tsdemux ! decodebin ! \
>        mix.sink_2

fails with my h264 stream while works perfectly with mpeg2 stream. Could you help me?
Comment 5 Eugen Klim 2017-03-10 08:17:54 UTC
It seems that the problem is in my h264 stream. It is valid, but it causes graph to fail. Other h264 streams (from youtube for example) works.
Comment 6 Eugen Klim 2017-03-10 08:29:23 UTC
Here is some additional info:

GST_DEBUG=1 gives

>0:00:00.726825969  6377 0x55a193b42de0 ERROR                  libav :0:: mmco: unref short failure
>0:00:00.726841208  6377 0x55a193b42de0 ERROR                  libav :0:: number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
>0:00:00.726870437  6377 0x55a193af6c50 ERROR                  libav :0:: mmco: unref short failure
>0:00:00.726888991  6377 0x55a193af6c50 ERROR                  libav :0:: number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one
>0:00:00.728964172  6377 0x55a193af6c50 ERROR        videoaggregator gstvideoaggregator.c:881:gst_video_aggregator_pad_sink_setcaps:<mixer:sink_1> got >input caps video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)720, height=(int)576, framerate=(fraction)25/1, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)16/11, texture-target=(string)2D, but current caps are video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)RGBA, texture-target=(string)2D, pixel-aspect-ratio=(fraction)1/1

My stream has pixel-aspect-ratio=16/11 while the working h264 streams have 1/1. Could it be the reason? Without videotestsrc everything works, but videotestsrc+avdec_h264+glvideomixer does not work.
Comment 7 Eugen Klim 2017-03-10 09:24:22 UTC
Ok, so h264 stream https://yadi.sk/d/Iwpnwwxy3FMryR does not work, while the stream decoded by ffmpeg as

>ffmpeg -i h264.ts -c:v h264 -c:a mp2 -f mpegts test.ts

works while

>ffmpeg -i h264.ts -c:v copy -c:a copy -f mpegts test.ts

does not.

The difference between those two are minor.

Transcoded stream (works):

>Input #0, mpegts, from './test.ts':
>  Duration: 00:00:26.81, start: 1.400000, bitrate: 1573 kb/s
>  Program 1 
>    Metadata:
>      service_name    : Service01
>      service_provider: FFmpeg
>    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 720x576 [SAR 16:11 DAR 20:11], 25 fps, 25 tbr, 90k tbn, 50 tbc
>    Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 384 kb/s

Copied stream (pipeline fails):

>Input #0, mpegts, from './test_copy.ts':
>  Duration: 00:00:26.78, start: 1.400000, bitrate: 3104 kb/s
>  Program 1 
>    Metadata:
>      service_name    : Service01
>      service_provider: FFmpeg
>    Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg), 720x576 [SAR 16:11 DAR 20:11], 25 fps, 25 tbr, 90k tbn, 50 tbc
>    Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
Comment 8 Eugen Klim 2017-03-10 10:12:43 UTC
Ok, so 

>gst-launch-1.0 \
>        filesrc name=h264.ts ! \
>        tsparse ! queue ! tee name=t \
>        glvideomixer name=mix ! \
>        glimagesink \
>        videotestsrc pattern=0 ! video/x-raw,format=RGBA,width=1920,height=1080 ! mix.sink_0 \
>        t. ! queue ! tsdemux ! decodebin !  \
>        mix.sink_1 \
>        t. ! queue ! tsdemux ! decodebin !  \
>        mix.sink_2

works fine with vaapi, bat fails with libav_h264.
Comment 9 GStreamer system administrator 2018-11-03 12:58:27 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-libav/issues/33.