GNOME Bugzilla – Bug 763401
Zero-copy video playback doesn't work on Android 4.x (QA 1.7.90)
Last modified: 2016-03-11 15:56:07 UTC
Created attachment 323557 [details] Log video not playing Video playback on Nexus 7 2013 Android 4.4.3 doesn't work if zero-copy (direct frame passing from decoder to GPU) is used. No video is displayed. Also, video is played if frame copy occurs between decoder and GPU. The exact same problem occurs on two TCL Alcatel OneTouch devices (phone and Hero8 tablet), both running 4.4.2. The same app, video, gstreamer-1.7.90 are working on Android 5.0 and 6.0.1. For information, Android 4.x still represents 35% of the installed park of Android worldwide. Attached debug log *:5 on the Alcatel phone.
This log doesn't contain any output from amcvideodec so: 1. make sure it's actually attempting to use androidmedia 2. add amc*:7 to the debug log categories when you're sure you have 1) sorted.
Can you send a new log with *:6? This one suggests that things just hang for some reason after the pipeline goes to READY. Nothing that even looks like mediacodec related things, not dataflow has happened yet.
Created attachment 323601 [details] log6
Attached from Nexus 7 2013 running 4.4.3. If frame is copied via NV12, working. If zero-copy, not working. Same app is working on Android 6.0.1.
Still nothing with amcvideodec in the log. It looks like all goes to READY state and nothing's wrong at this point. What is the change you do in your app to go from "If frame is copied via NV12, working" to "If zero-copy, not working"?
We might have two issues here. In both cases, with or without zero-copy, the pipeline is the same: filesrc ! decodebin ! glimagesink In case of direct copy, I intercept the autoplug-query of decodebin: g_signal_connect(G_OBJECT(ElementD), "autoplug-query", G_CALLBACK(autoplug_query_cb), data); For GST_QUERY_CAPS and GST_QUERY_CONTEXT, I mimic what playbin is doing. I have read my code and I might use string comparison that only works for 5.x/6.x. I need to check. If https://bugzilla.gnome.org/show_bug.cgi?id=742924 was fixed, I would not have a problem. Will that bug fix in the near future? Second part: if I update the pipeline to playbin, I start to see some decoding messages. Keep in mind once again that the same app is working on 6.x with zero copy.
Created attachment 323656 [details] Log with playbin
Just to make it clear, pipeline with playbin on Android 4.4 is not working, meaning that the screen stays black.
Errm gstamcvideodec.c:1250:gst_amc_video_dec_loop:<...> dequeueOutputBuffer() returned -2147483648 (0x80000000) gstamcvideodec.c:1293:gst_amc_video_dec_loop:<...> Failure dequeueing output buffer gstamcvideodec.c:1533:gst_amc_video_dec_loop:<...> error: Failed to get buffer 0 So something in gst_amc_codec_dequeue_output_buffer() returns G_MININT. Could you find out which failure case it's hitting (add some meaningful GST_ERROR()'s at each failure case) ? Side note: I just tried zero-copy on an android 4.4.2 device I have and things worked marvellously so it's not Android version specific.
Another data point to test is with some other video files as "width=(int)1280, height=(int)1440, framerate=(fraction)154/3" isn't exactly cheap and it may be baulking at the ~50fps frame rate.
Just as a reminder, the same app works on Android 6.0.1. This is the code that is hit: line 545 of gstamc.c if (ret == INFO_OUTPUT_BUFFERS_CHANGED || ret == INFO_OUTPUT_FORMAT_CHANGED || (ret >= 0 && !codec->output_buffers && !media_codec.get_output_buffer)) { if (!media_codec.get_output_buffer) { if (codec->output_buffers) gst_amc_jni_free_buffer_array (env, codec->output_buffers, codec->n_output_buffers); codec->output_buffers = gst_amc_codec_get_output_buffers (codec, &codec->n_output_buffers, err); if (!codec->output_buffers) { ret = G_MININT; goto done; } } Twice it hits this test and it never reaches line 560. Then, it fails in gstamcvideodec.c line 1293.
Problem also occurs with big bunny h264.
So which of the conditions is it? > ret == INFO_OUTPUT_BUFFERS_CHANGED || ret == INFO_OUTPUT_FORMAT_CHANGED || (ret >= 0 && !codec->output_buffers && media_codec.get_output_buffer) And what does err->message contain after gst_amc_codec_get_output_buffers() (I assume that's the one that fails?)?
Ok so the problem is in gst_amc_jni_get_buffer_array(), where it gets NULL for the first element of the output buffer array.
And the reason is: > The data processing is nearly identical to the ByteBuffer mode when using an > output Surface; however, the output buffers will not be accessible, and are > represented as null values. E.g. getOutputBuffer/Image(int) will return null > and getOutputBuffers() will return an array containing only null-s. Let's try a fix for this
For the records, it's ret == INFO_OUTPUT_FORMAT_CHANGED only. And the message is: 03-10 23:48:00.744: E/GStreamer+amc(3435): 0:00:12.279907226 0x7ee23920 gstamc.c:556:gst_amc_codec_dequeue_output_buffer: Failed to get buffer 0
Created attachment 323683 [details] [review] amcvideodec: getOutputBuffers() returns a NULL array when a surface was configured So don't error out if it does.
Created attachment 323684 [details] [review] amc: Correctly handle NULL input buffers
Created attachment 323685 [details] [review] amcvideodec: getOutputBuffers() returns a NULL array when a surface was configured So don't error out if it does.
Review of attachment 323684 [details] [review]: Looks sane.
Review of attachment 323685 [details] [review]: Looks sane. Maybe add a reference/copy verbatim the doc snippet somewhere (message, comment, ...)
playbin is now working on Android 4.4 with the patches 323684 and 323685. Also, it seems working when doing zero-copy through decodebin with the autoplugin-query mimic of playbin.
Attachment 323684 [details] pushed as 610705c - amc: Correctly handle NULL input buffers Attachment 323685 [details] pushed as 9132b72 - amcvideodec: getOutputBuffers() returns a NULL array when a surface was configured