GNOME Bugzilla – Bug 720421
eglglessink: simultaneous eglglessinks not possible in iOS application
Last modified: 2014-01-02 19:01:34 UTC
iOS application crashes when trying to launch simultaneous video pipelines sinking to autovideosink. Each pipeline gets their own UIView (with overridden layerClass), used to set the window handle for video overlay. Current code based on Tutorial #3 at the link below, updated for GStreamer 1.2.1: http://docs.gstreamer.com/display/GstSDK/iOS+tutorial+3%3A+Video From what I can tell this is an issue of sharing an EAGLContext. I can currently work around this by starting one pipeline, issuing the call [EAGLContext setCurrentContext:nil], then starting the next pipeline. My workaround becomes more difficult when the UIView size is changed while in the PLAYING or PAUSED state. The correct EAGLContext must be set before changing the view size, since the surface setup process occurs on dimension change as well. This is easy enough to work around while changing the size of a single UIView, but I have been unable to change dimensions of multiple video display UIViews simultaneously, without restarting the pipeline to achieve fine-grained control over execution order. I would think a call for [EAGLContext setCurrentContext:] is necessary in the gst_egl_adaptation_create_surface() method, inside gst-plugins-bad/ext/eglgles/gstegladaptation_eagl.m. Building for iOS 7, using XCode 5.0, on OS X 10.8.5. Using gstreamer 1.2.1 binaries downloaded from: http://gstreamer.freedesktop.org/data/pkg/ios/1.2.1/
This is already happening in theory, make_current() is called by create_egl_context() already. I think the problem here is that we not only set the current context for the calling thread, but also for the main thread. I can imagine some problems here if multiple sinks are used :) Can you attach your example application that shows the problems? I have some ideas about how to fix it, but with a proper testcase that's easier to verify and as you already have one... :)
Created attachment 264327 [details] gstreamer ios simultaneous eglglessink issue sample code
Hi Sebastian, I am attaching two sample xcode ios projects, one showing my issue and the other my current work-around. Also, I noticed I cannot re-use a UIView when stopping/re-starting a pipeline. Is this intentional? Thank you, Patrick
Is there a reason for setting the EAGLContext on both the calling and main threads? Any problem with the sample code previously attached?
Thanks, will look at this now
One thing that is wrong in your code is that you set the window handle too late on the sink. You have to install a sink bus handler that will set the window handle on the sink when it requests that, see the original code of Tutorial 3. Without that the code is racy
This commit here most likely fixes it, at least it fixes something that could cause your problem ;) Please test provide a testcase that properly uses the GstVideoOverlay API or test with this commit yourself. The problem with setting the window handle only after setting the pipeline to PAUSED is that this might be too late already. The sink wants the window handle while it transitions to PAUSED, and it works in your "hack" version because of different timing. commit d9525c134bd424b1e1107e92ef13d69e248f70a7 Author: Sebastian Dröge <sebastian@centricular.com> Date: Thu Dec 19 21:36:50 2013 +0100 eglglessink: Don't set the EAGLContext of the main thread and don't reuse it either ... instead create a new context for every sink instance. https://bugzilla.gnome.org/show_bug.cgi?id=720421
For reference: http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/?id=d9525c134bd424b1e1107e92ef13d69e248f70a7
Hi Sebastian, I appreciate your prompt attention in the matter. Thank you for the note on timing for setting the window handle- my code should have been changing to GST_STATE_READY instead of GST_STATE_PAUSED. I patched in your change and built for iOS, however I'm running into a few different issues when attempting multiple video displays. Here are some behaviors I'm experiencing when the pipelines are launched: 1. Crash with EXC_BAD_ACCESS at glDrawElements() in gsteglglessink.c (line 1767) 2. If both displays appear without crashing, only one of them remains in the playing state. 3. Only one video display appears. The log entry "Can't create surface" appears for gstegladaptation.c:461:gst_egl_adaptation_init_surface. 4. Other various crashes (#1 is definitely most common). I am attaching a simplified test project for reproduction of the issues above. Thank you! Patrick
Created attachment 264659 [details] simplified multivideo tests after suggested patch to gstegladaptation_eagl.m
Confirmed here, 1 at least.
Are you sure you're running an eglglessink with that specific patch and relinked your application against that? How are you doing that? Because it works for me with that patch and your sample application. Previously I accidentially used the old version.
It's possible I did not build/relink correctly if the solution works for you. After cloning the 1.2 branch from git://anongit.freedesktop.org/gstreamer/cerbero I ran: './cerbero-uninstalled -c config/cross-ios-universal.cbc bootstrap' './cerbero-uninstalled -c config/cross-ios-universal.cbc package gstreamer-1.0' Two questions: 1. Should I need to patch the source for both arm and x86 versions, or is there a better place to apply the patch? 2. Which cerbero commands should I issue after patching to rebuild the installer with the changes? Thank you!
Try with the binaries here: http://gstreamer.freedesktop.org/data/pkg/ios/1.2.2/ They include this change.
Confirmed this is working now. Thank you!