GNOME Bugzilla – Bug 774772
androidmedia: hardware encoder fails to encode stream, time stamps issue
Last modified: 2017-02-08 18:39:11 UTC
Created attachment 340394 [details] adb logcat details for amcvideoenc buffer timestamp issue Hello, Android app with latest cerbero build 1.10.1 fails to encode with hardware encoder. If I replace hardware(gstamcvideoenc/gstamcvideodec) encoder/decoder with software codecs avdec_h264/x264 pipeline working fine. Even previous build functional with gstamcvideoenc. This issue has been observed on OMX.Exynos.AVC.Encoder as well on OMX.qcom.video.encoder.avc MediaCodecs. Looks issue with Buffer timestamp adb logcat details: D/GStreamer+amcvideoenc(19440): 0:00:02.287604169 0x7a779b80 gstamcvideoenc.c:1059:gst_amc_video_enc_loop:<amcvideoenc-omxexynosavcencoder0> Finished frame: ok D/GStreamer+amcvideoenc(19440): 0:00:02.287664377 0x7a779b80 gstamcvideoenc.c:929:gst_amc_video_enc_loop:<amcvideoenc-omxexynosavcencoder0> Waiting for available output buffer W/GStreamer+qtmux(19440): 0:00:02.291733335 0x79ec7a60 gstqtmux.c:3395:gst_qt_mux_add_buffer:<mux> error: Buffer has no PTS. V/GStreamer+VideoTranscodeNative(19440): 0:00:02.292001752 0x79084460 jni/videoTranscodeGstNative.c:206:bus_message got message error E/GStreamer+VideoTranscodeNative(19440): 0:00:02.292194168 0x79084460 jni/videoTranscodeGstNative.c:216:bus_message ERROR from element mux: Could not multiplex stream. E/GStreamer+VideoTranscodeNative(19440): E/GStreamer+VideoTranscodeNative(19440): 0:00:02.292379668 0x79084460 jni/videoTranscodeGstNative.c:217:bus_message Debugging info: gstqtmux.c(3395): gst_qt_mux_add_buffer (): /GstPipeline:pipeline0/GstMP4Mux:mux: E/GStreamer+VideoTranscodeNative(19440): Buffer has no PTS. D/GStreamer+amcvideoenc(19440): 0:00:02.294446794 0x7a779b80 gstamcvideoenc.c:1017:gst_amc_video_enc_loop:<amcvideoenc-omxexynosavcencoder0> Got output buffer at index 2: size 968 time 33288 flags 0x00000000 E/GStreamer+amcvideoenc(19440): 0:00:02.294558377 0x7a779b80 gstamcvideoenc.c:905:gst_amc_video_enc_handle_output_frame:<amcvideoenc-omxexynosavcencoder0> No corresponding frame found D/GStreamer+amcvideoenc(19440): 0:00:02.294922210 0x7a779b80 gstamcvideoenc.c:1059:gst_amc_video_enc_loop:<amcvideoenc-omxexynosavcencoder0> Finished frame: error W/GStreamer+amcvideoenc(19440): 0:00:02.295026919 0x7a779b80 gstamcvideoenc.c:1132:gst_amc_video_enc_loop:<amcvideoenc-omxexynosavcencoder0> error: Internal data stream error. W/GStreamer+amcvideoenc(19440): 0:00:02.295074877 0x7a779b80 gstamcvideoenc.c:1132:gst_amc_video_enc_loop:<amcvideoenc-omxexynosavcencoder0> error: streaming stopped, reason error (-5) W/GStreamer+queue(19440): 0:00:02.295324127 0x7a779b80 gstqueue.c:989:gst_queue_handle_sink_event:<queue2> error: Internal data stream error. W/GStreamer+queue(19440): 0:00:02.295383835 0x7a779b80 gstqueue.c:989:gst_queue_handle_sink_event:<queue2> error: streaming stopped, reason error (-5) I/ACodec (19440): [OMX.Exynos.AVC.Encoder] Now Flushing Please find attached log for more details. Thanks in advance
"E/GStreamer+VideoTranscodeNative(19440): Buffer has no PTS." This to me is a perfectly fine reason to fail. Why don't you have PTS set on your buffers ? Where are the buffer from ?
I'm using gstreamer pipeline for my android app and PTS suppose to be set on output buffer of amcvideoenc and I can see in PTS configured properly at gstamcvideoenc.c GST_BUFFER_PTS (out_buf) = gst_util_uint64_scale (buffer_info->presentation_time_us, GST_USECOND, 1); Gstreamer pipeline: filesrc location=/storage/emulated/0/DCIM/Camera/VID_20161011_001420.mp4 ! qtdemux name=demux demux.audio_0 ! queue max-size-time=5000000000 ! audio/mpeg,mpegversion=4 ! mp4mux name=mux ! filesink name=sink location=/storage/emulated/0/DCIM/Camera/VID_20161011_001420_transcoded.mp4 demux.video_0 ! h264parse ! amcviddec-omxqcomvideodecoderavc ! queue ! video/x-raw,width=1920,height=1080,format=(string)NV12 ! amcvidenc-omxqcomvideoencoderavc bitrate=3811 i-frame-interval=2 ! video/x-h264 ! queue ! h264parse ! mux.video_0 and same pipeline working fine without any issues with previous gstreamer build 1.11.0.
Please provide additional traces that could show timestamp over the pipeline (e.g GST_DEBUG=GST_SCHEDULING:5).
Unless it's realted to: https://bugzilla.gnome.org/show_bug.cgi?id=774840 ?
Created attachment 340612 [details] gstqtmux log with hardware codecs
Created attachment 340613 [details] gstqtmux log with software codecs (avdec_h264 and x264)
Created attachment 340616 [details] amcvideoenc logs
(In reply to Nicolas Dufresne (stormer) from comment #4) > Unless it's realted to: https://bugzilla.gnome.org/show_bug.cgi?id=774840 ? I don't think so my issue related to https://bugzilla.gnome.org/show_bug.cgi?id=774840 because if qtmux fails in that case its fair assumption it must fails for all cases not only for hardware based codecs.
(In reply to Nicolas Dufresne (stormer) from comment #3) > Please provide additional traces that could show timestamp over the pipeline > (e.g GST_DEBUG=GST_SCHEDULING:5). I have attached three logs 1) gst_debug_amcvideoenc.log 2) gst_debug_gstqtmux_hwcodecs_pipeline.log This gstqtmux logs captured pipeline with hardware codecs 3) gst_debug_gstqtmux_swcodecs_pipeline.log This qtmux log captured pipeline with software codecs i.e avdec_h264/x264
Created attachment 340641 [details] amcvideodecoder log Looks like hardware decoder amcvideodec-omxexynosavcdecoder generating output buffer with right time stamps
Did any an experiment lunch pipeline with hardware decoder + x264 pipeline works fine. filesrc location=/storage/emulated/0/DCIM/Camera/1080p.mp4 ! qtdemux name=demux demux.audio_0 ! queue max-size-time=5000000000 ! audio/mpeg,mpegversion=4 ! mp4mux name=mux ! filesink name=sink location=/storage/emulated/0/DCIM/Camera/transcoded.mp4 demux.video_0 ! h264parse ! amcviddec-omxexynosavcdecoder ! queue ! x264enc pass=5 quantizer=23 speed-preset=2 psy-tune=4 vbv-buf-capacity=0 option-string="psnr" ! mux.video_0
The problem here seems to be that the encoder outputs more buffers than input frames, and we invent something in those cases. Will look what we can best do about that...
Part of the problem solved, now there's only something left that looks like it's in h264parse. commit aca89aeeaac3d86687b4680a3aca8f76382406a0 Author: Sebastian Dröge <sebastian@centricular.com> Date: Tue Feb 7 19:44:54 2017 +0200 amcvideoenc: Encoder output is generally not properly parsed Don't claim it is and let h264parse and other parsers do their job. https://bugzilla.gnome.org/show_bug.cgi?id=774772 commit 7817f857592c15cd5c5c28709bcdda3163cc8694 Author: Sebastian Dröge <sebastian@centricular.com> Date: Tue Feb 7 19:42:47 2017 +0200 amcvideoenc: Set timestamps on header buffers too https://bugzilla.gnome.org/show_bug.cgi?id=774772 commit c27091d67fe9435f8f57a1ee79786891978e4d32 Author: Sebastian Dröge <sebastian@centricular.com> Date: Tue Feb 7 19:42:00 2017 +0200 amcvideoenc: Encoded H264 is generally not AU aligned Claiming that it is, can cause h264parse to skip some parsing steps and the output stays unaligned. https://bugzilla.gnome.org/show_bug.cgi?id=774772
Actually it seems to be all working now, I can't reproduce the problem anymore.