GNOME Bugzilla – Bug 489275
[basesink] support audio/video sink using hardware (motorola contribution)
Last modified: 2010-03-23 13:34:42 UTC
On current mobilephone, more products are using hardware(DSP, GPU or others) to do decoding and rendering, and some of them combin the decoding and rendering together. For gstreamer, if we want to use audio/video sink to implement it, gstbasesink should not put preroll buffer into queue, because at this time, hardware needs buffers to do initialization when preroll. To resolve this issue, we add a new property "preroll-extension" for gstbasesink, add queue_preroll function in _GstBaseSinkClass to notify subclass preroll buffers. When set "preroll-extension" to TRUE, gstbasesink will not put preroll buffers into queue when prerolling, and just call queue_preroll to handle buffers, the queue_preroll needs to be implemented by subclass. at this time, the data sent to these sinks are encoded audio/video data, not raw audio/video data any more. Changed files: libs/gst/base/gstbasesink.c libs/gst/base/gstbasesink.h Motorola would like to contribute Gstreamer patch to the Gstreamer open source community project. Please find attached the patch applicable for Gstreamer. For any questions, please feel free to contact (Zhao Liang "e3423c@motorola.com", Shi Ling "w20230@motorola.com").
Created attachment 97698 [details] [review] gstbasesink.h.patch
Created attachment 97699 [details] [review] gstbasesink.c.patch
Is this a duplicate of Bug #342155?
Hi Wim, It is not, althought it is similar, we make another solution for this issue. This issue rises from our developing on mobilephone with hardware decoder and renderer. How about it?
Could you please give more details about what the issue is with the hardware initialization ? Your sink element is going to receive one buffer in the preroll method during preroll phase and then all the other buffers through the render method during playback phase. The internal queue in basesink should not really affect you. [This bug is part of Fluendo support for Motorola.]
our requirement is to display the first frame in PAUSED state. e.g, when first set pipeline as PAUSED, but not PLAYING, it should display the first video frame. Another example, when pipeline is PAUSED, do seek, and after seek, the first video frame at target position should be displayed. By current gstbasesink implementation, we could not do it.
The preroll vmethod in basesink is exactly made to display the first frame in PAUSED and is used by all other videosinks.
Our issue is we should preroll 2 buffers, and handle these buffers at once by hardware limitation. But if we use preroll vmethod, it can only be invoked once at prerolling. see the following code, In gst_base_sink_queue_object_unlocked() { .... if (length == 1) { ret = gst_base_sink_preroll_object (basesink, pad, obj); if (G_UNLIKELY (ret != GST_FLOW_OK)) goto preroll_failed; } ... }
If I understand correctly your sink element receives encoded data. What you say here is that a single buffer is not enough to initialize your element and you need n buffers of data to commit the PREROLLED state. If that's the case I believe this is completely a duplicate of bug #342155 Wim any comment on introducing a way for the preroll method to ask for more data ?
it's a dup AFAICS. It would still be nice to have this, like call preroll in a row while it returns a certain GstFlowReturn. Another alternative is to create a buffering element that groups the data before sending it to the sink.
As julien said, we really want to preroll more than 1 buffer, and I think that is a common issue for different hardware/software platform. I have checked bug #342155, it is different fix with ours. How do you think about this fix?
Marking as duplicate of the other bug. The patches here don't really look like the right approach to me. There shouldn't be any need for a new property, just a new vfunc at most, which the subclass either implements or not, and then basesink can do the right thing based on that. (Also, please provide patches in diff -u format next time.) *** This bug has been marked as a duplicate of bug 342155 ***