After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 747707 - applemedia: expose GstCoreMediaMeta to app
applemedia: expose GstCoreMediaMeta to app
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other All
: Normal enhancement
: git master
Assigned To: Ilya Konstantinov
GStreamer Maintainers
Depends on: corevideomemory
Blocks:
 
 
Reported: 2015-04-11 18:06 UTC by Ilya Konstantinov
Modified: 2018-11-03 13:34 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
applemedia: refactor into library (80.87 KB, patch)
2015-05-03 01:22 UTC, Ilya Konstantinov
none Details | Review
applemedia: refactor into library (81.04 KB, patch)
2015-05-05 13:33 UTC, Ilya Konstantinov
none Details | Review
[1/3] applemedia: rename Core(Video,Media)BufferMeta fields (6.62 KB, patch)
2015-06-16 22:56 UTC, Ilya Konstantinov
none Details | Review
[2/3] applemedia: add Apple to GstCore(Media,Video)BufferMeta (22.36 KB, patch)
2015-06-16 22:57 UTC, Ilya Konstantinov
none Details | Review
[3/3] applemedia: refactor into library (83.83 KB, patch)
2015-06-16 23:01 UTC, Ilya Konstantinov
none Details | Review

Description Ilya Konstantinov 2015-04-11 18:06:22 UTC
An OS X and iOS app might use appsink to get resulting GstBuffers from a pipeline containing applemedia elements. Furthermore, it can wish to access (and retain) the underlying CMSampleBufferRef or CMPixelBufferRef, as they're needed:

 1) to work with AVFoundation, Core Video and Core Image 

 2) for GL rendering done by app

How can we expose some of the applemedia stuff (e.g. coremediabuffer.h) as API?
Comment 1 Sebastian Dröge (slomo) 2015-04-13 07:02:35 UTC
We should first of all merge the two metas into one, and I think these also shouldn't be metas but instead custom GstMemory.

Otherwise this is a really good idea, and we could add a libgstcoremedia library in gst-plugins-bad for that. It could also contain things like a GstClock based on the CoreMedia clock.
Comment 2 Ilya Konstantinov 2015-04-18 22:50:15 UTC
As discussed on IRC, GstCoreVideoMemory should not be the only kind of object retaining the Apple objects -- since:

  a) there are multiple GstCoreVideoMemory for planes -- which one you look into?

  b) when passing a CMSampleBuffer / CVPixelBuffer between supporting elements (determined via meta feature-caps), we can avoid populating the GstBuffer with GstCoreVideoMemory objects at all


We might unify the metas, possibly keeping a variant of GstCoreMediaMeta whose sample_buf can be NULL?
Comment 3 Ilya Konstantinov 2015-04-27 02:50:20 UTC
avfassetsrc:

 * output: CMSampleBuffer
           -> CMBlockBuffer (if audio)
           -> CVImageBuffer (if video)

avfvideosrc:

 * output: CMSampleBuffer
           -> CVImageBuffer

vtenc:

 * input:  CVImageBuffer
 * output: CMSampleBuffer

vtdec:

 * input:  CMSampleBuffer
 * output: CVImageBuffer

avsamplevideosink:

 * input:  CMSampleBuffer

- - -

CVImageBuffer, in addition to CVPixelBuffer, can also be CVOpenGLBuffer (Mac-only) and CVOpenGL(ES)Texture.

Since, we don't populate GstBuffer with GstMemory when configured with "meta:GstCoreVideoBuffer" caps, we don't actually care -- we just pass the CVImageBufferRef to the framework.
Comment 4 Ilya Konstantinov 2015-05-03 01:22:52 UTC
Created attachment 302783 [details] [review]
applemedia: refactor into library

Preliminary patch moving GstCoreMediaMeta, GstCoreVideoMeta and GstCoreVideoMemory into libgstapplemedia-1.0.

APPLYING:

Applies on top of bug 747216, i.e. https://github.com/ikonst/gst-plugins-bad/tree/corevideomemory

REMARKS:

As said before, the rationale for naming it "applemedia" (and not "coremedia") is the significance of Apple technologies. The words "core" and "media" are just too generic.

I've tried enabling it for GObject-Introspection, and that's something I'd like to do, but it wasn't trivial so it'll be a separate patch.

I can't think of other parts of "applemedia" which should turn into "API".
Comment 5 Ilya Konstantinov 2015-05-05 13:33:11 UTC
Created attachment 302927 [details] [review]
applemedia: refactor into library

UPDATE:

sys/applemedia/Makefile.am: remove headers from noinst_HEADERS
Comment 6 Ilya Konstantinov 2015-06-16 22:56:50 UTC
Created attachment 305436 [details] [review]
[1/3] applemedia: rename Core(Video,Media)BufferMeta fields

Unify field names between the two metas.
Comment 7 Ilya Konstantinov 2015-06-16 22:57:51 UTC
Created attachment 305437 [details] [review]
[2/3] applemedia: add Apple to GstCore(Media,Video)BufferMeta

Yes, this is a very noisy patch.
Comment 8 Ilya Konstantinov 2015-06-16 23:01:23 UTC
Created attachment 305438 [details] [review]
[3/3] applemedia: refactor into library
Comment 9 GStreamer system administrator 2018-11-03 13:34:07 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/238.