GNOME Bugzilla – Bug 779807
Both videomixer and glvideomixer do not want to work properly with avdec_h264 decoder
Last modified: 2018-11-03 12:58:27 UTC
Hello. I want to build a graph which would mix pictures from a few h264 streams into one frame (so called ``picture in picture''). It seems that videomixer element is the right thing to do the trick. And videomixer works as expected with videotestsrc [graph_0.sh]. However, when I build graph with h264 decoder [graph_2.sh], it fails with an error: >ERROR: from element /GstPipeline:pipeline0/MpegTSParse2:mpegtsparse2-0: Internal data stream error. >Additional debug info: >mpegtsbase.c(1613): mpegts_base_loop (): /GstPipeline:pipeline0/MpegTSParse2:mpegtsparse2-0: >streaming stopped, reason not-negotiated (-4) But when I switch videomixer with glvideomixer [graph_3.sh] it works fine. However, if I add background via videotestsrc, it also fails [graph_3b.sh]. Graph with decodebin and glvideomixer also works [graph_4.sh], but graph with decodebin, glvideomixer and videotestsrc background [graph_4b.sh] -- does not. But I've noticed that [graph_4b.sh] works perfectly with mpeg2 decoded ts, while failing with h264 stream. I could not understand what's the problem, but it seems that the problem is avdec_h264 related, so I need some help. I'm not sure is it a bug or am i doing something wrong. I've attached graph scripts and 2 test streams (mpeg2 and h264). Thank you.
Created attachment 347564 [details] scripts
Test streams are too large to be attached so I've posted them here: https://yadi.sk/d/Iwpnwwxy3FMryR
videomixer is being deprecated slowly, the replacement is "compositor", which uses the same baseclass as glvideomixer, so it should work the same, but you may want to test it too just in case.
Yeah, thank you for reply. Compositor works for me even more worse than glvideomixer. Pipeline >gst-launch-1.0 \ > filesrc location=$1 ! \ > tsparse ! queue ! tee name=t \ > compositor name=mix ! \ > glimagesink \ > videotestsrc pattern=0 ! video/x-raw,format=I420,width=1920,height=1080 ! mix.sink_0 \ > t. ! queue ! tsdemux ! decodebin ! \ > mix.sink_1 \ > t. ! queue ! tsdemux ! decodebin ! \ > mix.sink_2 fails with an error: >ERROR: from element /GstPipeline:pipeline0/MpegTSParse2:mpegtsparse2-0: >Internal data stream error. >Additional debug info: >mpegtsbase.c(1613): mpegts_base_loop (): /GstPipeline:pipeline0/MpegTSParse2:mpegtsparse2-0: >streaming stopped, reason not-negotiated (-4) >Execution ended after 0:00:00.000194153 And >gst-launch-1.0 \ > filesrc location=$1 ! \ > tsparse ! tee name=t \ > compositor name=mix ! \ > xvimagesink \ > t. ! queue ! tsdemux ! decodebin ! \ > mix.sink_0 \ > t. ! queue ! tsdemux ! decodebin ! \ > mix.sink_1 works only once from four (literally, I start the graph, it fails, I start it agan, it works). And in the pipeline with >compositor ! glimagesink I have wrong colours (everything is green), seems like wrong colourspace conversion on the compositor's output, compositor ! xvimagesink works fine (but crushes once from four as I've said). So glvideomixer seems works, but I still could not understand why >gst-launch-1.0 \ > filesrc location=$1 ! \ > tsparse ! queue ! tee name=t \ > glvideomixer name=mix ! \ > glimagesink \ > videotestsrc pattern=0 ! video/x-raw,format=I420,width=1920,height=1080 ! mix.sink_0 \ > t. ! queue ! tsdemux ! decodebin ! \ > mix.sink_1 \ > t. ! queue ! tsdemux ! decodebin ! \ > mix.sink_2 fails with my h264 stream while works perfectly with mpeg2 stream. Could you help me?
It seems that the problem is in my h264 stream. It is valid, but it causes graph to fail. Other h264 streams (from youtube for example) works.
Here is some additional info: GST_DEBUG=1 gives >0:00:00.726825969 6377 0x55a193b42de0 ERROR libav :0:: mmco: unref short failure >0:00:00.726841208 6377 0x55a193b42de0 ERROR libav :0:: number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one >0:00:00.726870437 6377 0x55a193af6c50 ERROR libav :0:: mmco: unref short failure >0:00:00.726888991 6377 0x55a193af6c50 ERROR libav :0:: number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one >0:00:00.728964172 6377 0x55a193af6c50 ERROR videoaggregator gstvideoaggregator.c:881:gst_video_aggregator_pad_sink_setcaps:<mixer:sink_1> got >input caps video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)720, height=(int)576, framerate=(fraction)25/1, interlace-mode=(string)mixed, pixel-aspect-ratio=(fraction)16/11, texture-target=(string)2D, but current caps are video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)RGBA, texture-target=(string)2D, pixel-aspect-ratio=(fraction)1/1 My stream has pixel-aspect-ratio=16/11 while the working h264 streams have 1/1. Could it be the reason? Without videotestsrc everything works, but videotestsrc+avdec_h264+glvideomixer does not work.
Ok, so h264 stream https://yadi.sk/d/Iwpnwwxy3FMryR does not work, while the stream decoded by ffmpeg as >ffmpeg -i h264.ts -c:v h264 -c:a mp2 -f mpegts test.ts works while >ffmpeg -i h264.ts -c:v copy -c:a copy -f mpegts test.ts does not. The difference between those two are minor. Transcoded stream (works): >Input #0, mpegts, from './test.ts': > Duration: 00:00:26.81, start: 1.400000, bitrate: 1573 kb/s > Program 1 > Metadata: > service_name : Service01 > service_provider: FFmpeg > Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 720x576 [SAR 16:11 DAR 20:11], 25 fps, 25 tbr, 90k tbn, 50 tbc > Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 384 kb/s Copied stream (pipeline fails): >Input #0, mpegts, from './test_copy.ts': > Duration: 00:00:26.78, start: 1.400000, bitrate: 3104 kb/s > Program 1 > Metadata: > service_name : Service01 > service_provider: FFmpeg > Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg), 720x576 [SAR 16:11 DAR 20:11], 25 fps, 25 tbr, 90k tbn, 50 tbc > Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
Ok, so >gst-launch-1.0 \ > filesrc name=h264.ts ! \ > tsparse ! queue ! tee name=t \ > glvideomixer name=mix ! \ > glimagesink \ > videotestsrc pattern=0 ! video/x-raw,format=RGBA,width=1920,height=1080 ! mix.sink_0 \ > t. ! queue ! tsdemux ! decodebin ! \ > mix.sink_1 \ > t. ! queue ! tsdemux ! decodebin ! \ > mix.sink_2 works fine with vaapi, bat fails with libav_h264.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-libav/issues/33.