GNOME Bugzilla – Bug 732896
iOS: AudioUnit on iOS produces echo with in/out streams
Last modified: 2018-11-03 14:53:28 UTC
The osxaudiosrc plugin is Remote I/O unit it is not suited for VoIP purpose because no echo cancellator. According to the Apple documentation, The Voice-Processing I/O unit extends the Remote I/O unit by adding acoustic echo cancelation for use in a VoIP or voice-chat application. https://developer.apple.com/library/ios/documentation/MusicAudio/Conceptual/AudioUnitHostingGuide_iOS/AudioUnitHostingFundamentals/AudioUnitHostingFundamentals.html To use Voice-rocessing I/O unit I did these changes: in gstosxcoreaudioremoteio.c static gboolean gst_core_audio_open_impl (GstCoreAudio * core_audio) { return gst_core_audio_open_device (core_audio, kAudioUnitSubType_VoiceProcessingIO, "VoiceProcessingIO"); //return gst_core_audio_open_device (core_audio, kAudioUnitSubType_RemoteIO, // "RemoteIO"); } This patch was not enough to solve the issue because I received status = -50 after some calls of AudioUnitRender in getosxaudiosrc.c, it seems AudioUnitRender modifies buf->core_audio->recBufferList->mBuffers[0].mDataByteSize based on data returned, so before call AudioUnitRender it requires to set mDataByteSize today I’ve created mine osxaudio plugin with these changes, hardcoding "bytes per frame” and “channels” and solved my problem, I did’d check the buffer allocated because it’s big enough. buf->core_audio->recBufferList->mNumberBuffers=1; buf->core_audio->recBufferList->mBuffers[0].mDataByteSize=inNumberFrames*4/*where 4 is bytes per frame*/; buf->core_audio->recBufferList->mBuffers[0].mNumberChannels=1 /*num of channels*/; status = AudioUnitRender (buf->core_audio->audiounit, ioActionFlags, inTimeStamp, inBusNumber, inNumberFrames, buf->core_audio->recBufferList); if (status) { GST_WARNING_OBJECT (buf, "AudioUnitRender returned %d", (int) status); return status; }
Can you upload a patch for this?
Rob, I'd like to fix this. I've been considering a change to how osxaudio code is laid out. Now we have gstosxcoreaudio.c including *either* 1) gstcoreaudiohal.c (osx only) 2) gstcoreaudioremoteio.c (ios only) Yes, literally #include-ing. I've been considering either: 1) Build 3 implementations (hal, remote io, vp io) as separate compilation units, then call impl->start_processing(...) etc. (where impl is a pointer to a struct of func ptrs) 2) If Remote IO and HAL aren't that different, maybe we should unify the code and #ifdef out the parts that don't build on iOS (since iOS is the subset). What do you think? I don't want to rush into it without giving it a bit of thought from someone more experienced.
Oops, I was confused. HAL is only available on OS X. Remote IO and VoiceProcessing IO are only available on iOS. My little ambitious project is irrelevant. :) I'll go on to implement a parameter to trigger VPIO on iOS.
Short term, having a selector to pick VoiceProcessingIO would be good. Long term, I'd like to get rid of the #include'ing as well, but this is more a cleanup that a feature. What I looked up suggests that VoiceProcessingIO works on OS X and iOS, so would be good to keep that in mind while implementing this.
You seem to be right, VoiceProcessing IO is available on OS X: https://developer.apple.com/library/mac/documentation/AudioUnit/Reference/AUComponentServicesReference/ Funny enough, Apple's OS X documentation contains a copy-paste mistake: "An audio unit that interfaces to the audio inputs and outputs of iPhone OS devices ..." I'll keep that in mind.
VoiceProcessing is 10.7 or later, so should be ok to compile in unconditionally.
(In reply to Sebastian Dröge (slomo) from comment #6) > VoiceProcessing is 10.7 or later, so should be ok to compile in > unconditionally. The trouble is making gstosxcoreaudioremoteio.c build on OS X at all. Right now it's really "the iOS codepath".
Does it not work if in both cases we choose the VoiceProcessingIO subtype at device open time?
Are you still planning to work on this Ilya?
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/123.