GNOME Bugzilla – Bug 759389
fakesrc ! wasapisink and any other combination raises The stream is in the wrong format
Last modified: 2018-11-03 13:44:37 UTC
I found no way to use wasapisink from 1.6.1 on Windows 8. I tried all combinations of elements that came to my mind, but even basic fakesrc ! wasapisink does not work. $ LC_ALL=C GST_DEBUG=*:4 gst-launch-1.0.exe fakesrc ! wasapisink 0:00:00.000200057 3764 424000 INFO GST_INIT gst.c:510:init_pre: Initializing GStreamer Core Library version 1.6.1 0:00:00.000323701 3764 424000 INFO GST_INIT gst.c:511:init_pre: Using library installed in C:\msys64\mingw64\lib 0:00:00.000676201 3764 424000 INFO GST_INIT gstmessage.c:119:_priv_gst_message_initialize: init messages 0:00:00.001358161 3764 424000 INFO GST_INIT gstcontext.c:77:_priv_gst_context_initialize: init contexts 0:00:00.002192563 3764 424000 INFO GST_PLUGIN_LOADING gstplugin.c:316:_priv_gst_plugin_initialize: registering 0 static plugins 0:00:00.002389932 3764 424000 INFO GST_PLUGIN_LOADING gstplugin.c:224:gst_plugin_register_static: registered static plugin "staticelements" 0:00:00.002445994 3764 424000 INFO GST_PLUGIN_LOADING gstplugin.c:226:gst_plugin_register_static: added static plugin "staticelements", result: 1 0:00:00.013519014 3764 424000 INFO GST_REGISTRY gstregistry.c:1723:ensure_current_registry: reading registry cache: C:\Users\Marcin\AppData\Local\Microsoft\Windows\INetCache\gstreamer-1.0\registry.x86_64.bin 0:00:00.028065576 3764 424000 INFO GST_REGISTRY gstregistrybinary.c:619:priv_gst_registry_binary_read_cache: loaded C:\Users\Marcin\AppData\Local\Microsoft\Windows\INetCache\gstreamer-1.0\registry.x86_64.bin in 0.015000 seconds 0:00:00.028355870 3764 424000 INFO GST_REGISTRY gstregistry.c:1579:scan_and_update_registry: Validating plugins from registry cache: C:\Users\Marcin\AppData\Local\Microsoft\Windows\INetCache\gstreamer-1.0\registry.x86_64.bin 0:00:00.040341241 3764 424000 INFO GST_REGISTRY gstregistry.c:1681:scan_and_update_registry: Registry cache has not changed 0:00:00.040407671 3764 424000 INFO GST_REGISTRY gstregistry.c:1758:ensure_current_registry: registry reading and updating done, result = 1 0:00:00.040478325 3764 424000 INFO GST_INIT gst.c:720:init_post: GLib runtime version: 2.46.0 0:00:00.040519795 3764 424000 INFO GST_INIT gst.c:722:init_post: GLib headers version: 2.46.1 0:00:00.040565490 3764 424000 INFO GST_INIT gst.c:723:init_post: initialized GStreamer successfully 0:00:00.040623471 3764 424000 INFO GST_PIPELINE gstparse.c:323:gst_parse_launch_full: parsing pipeline description 'fakesrc ! wasapisink ' 0:00:00.041529296 3764 424000 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin "C:\msys64\mingw64\lib\gstreamer-1.0\libgstcoreelements.dll" loaded 0:00:00.041604173 3764 424000 INFO GST_ELEMENT_FACTORY gstelementfactory.c:364:gst_element_factory_create: creating element "fakesrc" 0:00:00.041852996 3764 424000 INFO GST_ELEMENT_PADS gstelement.c:646:gst_element_add_pad:<GstBaseSrc@00000000026ddf10> adding pad 'src' 0:00:00.043203861 3764 424000 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin "C:\msys64\mingw64\lib\gstreamer-1.0\libgstwasapi.dll" loaded 0:00:00.043277202 3764 424000 INFO GST_ELEMENT_FACTORY gstelementfactory.c:364:gst_element_factory_create: creating element "wasapisink" 0:00:00.043544841 3764 424000 INFO GST_ELEMENT_PADS gstelement.c:646:gst_element_add_pad:<GstBaseSink@00000000026f1100> adding pad 'sink' 0:00:00.043881213 3764 424000 INFO GST_ELEMENT_FACTORY gstelementfactory.c:364:gst_element_factory_create: creating element "pipeline" 0:00:00.044032888 3764 424000 INFO GST_PIPELINE grammar.y:571:gst_parse_perform_link: linking fakesrc0:(any) to wasapisink0:(any) (0/0) with caps "(NULL)" 0:00:00.044110837 3764 424000 INFO GST_ELEMENT_PADS gstutils.c:1571:gst_element_link_pads_full: trying to link element fakesrc0:(any) to element wasapisink0:(any) 0:00:00.044179187 3764 424000 INFO GST_PADS gstutils.c:932:gst_pad_check_link: trying to link fakesrc0:src and wasapisink0:sink 0:00:00.044250224 3764 424000 INFO GST_PADS gstutils.c:1444:prepare_link_maybe_ghosting: fakesrc0 and wasapisink0 in same bin, no need for ghost pads 0:00:00.044341229 3764 424000 INFO GST_PADS gstpad.c:2234:gst_pad_link_prepare: trying to link fakesrc0:src and wasapisink0:sink 0:00:00.044415722 3764 424000 INFO GST_PADS gstpad.c:2440:gst_pad_link_full: linked fakesrc0:src and wasapisink0:sink, successful 0:00:00.044479080 3764 424000 INFO GST_EVENT gstevent.c:1374:gst_event_new_reconfigure: creating reconfigure event 0:00:00.044540518 3764 424000 INFO GST_EVENT gstpad.c:5501:gst_pad_send_event_unchecked:<fakesrc0:src> Received event on flushing pad. Discarding Setting pipeline to PAUSED ... 0:00:00.044681441 3764 424000 INFO GST_STATES gstbin.c:2243:gst_bin_element_set_state:<wasapisink0> current NULL pending VOID_PENDING, desired next READY 0:00:00.047987181 3764 424000 INFO GST_STATES gstelement.c:2330:gst_element_continue_state:<wasapisink0> completed state change to READY 0:00:00.048047467 3764 424000 INFO GST_STATES gstelement.c:2235:_priv_gst_element_state_changed:<wasapisink0> notifying about state-changed NULL to READY (VOID_PENDING pending) 0:00:00.048107753 3764 424000 INFO GST_STATES gstbin.c:2707:gst_bin_change_state_func:<pipeline0> child 'wasapisink0' changed state to 2(READY) successfully 0:00:00.048155751 3764 424000 INFO GST_STATES gstbin.c:2243:gst_bin_element_set_state:<fakesrc0> current NULL pending VOID_PENDING, desired next READY 0:00:00.048194534 3764 424000 INFO GST_STATES gstelement.c:2330:gst_element_continue_state:<fakesrc0> completed state change to READY 0:00:00.048227173 3764 424000 INFO GST_STATES gstelement.c:2235:_priv_gst_element_state_changed:<fakesrc0> notifying about state-changed NULL to READY (VOID_PENDING pending) 0:00:00.048268259 3764 424000 INFO GST_STATES gstbin.c:2707:gst_bin_change_state_func:<pipeline0> child 'fakesrc0' changed state to 2(READY) successfully 0:00:00.048309730 3764 424000 INFO GST_STATES gstelement.c:2305:gst_element_continue_state:<pipeline0> committing state from NULL to READY, pending PAUSED, next PAUSED 0:00:00.048444125 3764 424000 INFO GST_STATES gstelement.c:2235:_priv_gst_element_state_changed:<pipeline0> notifying about state-changed NULL to READY (PAUSED pending) 0:00:00.048594264 3764 424000 INFO GST_STATES gstelement.c:2312:gst_element_continue_state:<pipeline0> continue state change READY to PAUSED, final PAUSED 0:00:00.048647254 3764 424000 INFO GST_STATES gstbin.c:2243:gst_bin_element_set_state:<wasapisink0> current READY pending VOID_PENDING, desired next PAUSED 0:00:00.048713300 3764 424000 INFO GST_STATES gstbin.c:2713:gst_bin_change_state_func:<pipeline0> child 'wasapisink0' is changing state asynchronously to PAUSED 0:00:00.048755922 3764 424000 INFO GST_STATES gstbin.c:2243:gst_bin_element_set_state:<fakesrc0> current READY pending VOID_PENDING, desired next PAUSED 0:00:00.048805841 3764 424000 INFO basesrc gstbasesrc.c:1339:gst_base_src_do_seek:<fakesrc0> seeking: bytes segment start=0, offset=0, stop=-1, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0, base=0, position 0, duration -1 0:00:00.048904525 3764 424000 INFO task gsttask.c:450:gst_task_set_lock: setting stream lock 00000000026de2c0 on task 00000000026e4050 0:00:00.048944460 3764 424000 INFO GST_PADS gstpad.c:5847:gst_pad_start_task:<fakesrc0:src> created task 00000000026e4050 0:00:00.049050440 3764 424000 INFO GST_STATES gstelement.c:2330:gst_element_continue_state:<fakesrc0> completed state change to PAUSED 0:00:00.049204035 3764 424000 INFO GST_STATES gstelement.c:2235:_priv_gst_element_state_changed:<fakesrc0> notifying about state-changed READY to PAUSED (VOID_PENDING pending) 0:00:00.049265088 3764 26b9c30 INFO GST_ELEMENT_PADS gstelement.c:894:gst_element_get_static_pad: no such pad 'sink' in element "fakesrc0" 0:00:00.049381436 3764 424000 INFO GST_STATES gstbin.c:2707:gst_bin_change_state_func:<pipeline0> child 'fakesrc0' changed state to 3(PAUSED) successfully 0:00:00.049392188 3764 26b9c30 FIXME default gstutils.c:3766:gst_pad_create_stream_id_internal:<fakesrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id Pipeline is PREROLLING ... 0:00:00.049647539 3764 26b9c30 INFO GST_EVENT gstevent.c:760:gst_event_new_segment: creating segment event bytes segment start=0, offset=0, stop=-1, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0, base=0, position 0, duration -1 0:00:00.049749679 3764 26b9c30 INFO basesrc gstbasesrc.c:2838:gst_base_src_loop:<fakesrc0> marking pending DISCONT 0:00:00.049837228 3764 26b9c30 WARN audiobasesink gstaudiobasesink.c:1213:gst_audio_base_sink_preroll:<wasapisink0> error: sink not negotiated. 0:00:00.049924393 3764 26b9c30 INFO GST_ERROR_SYSTEM gstelement.c:1837:gst_element_message_full:<wasapisink0> posting message: The stream is in the wrong format. 0:00:00.050003111 3764 26b9c30 INFO GST_ERROR_SYSTEM gstelement.c:1860:gst_element_message_full:<wasapisink0> posted error message: The stream is in the wrong format. 0:00:00.050066852 ERROR: from element /GstPipeline:pipeline0/GstWasapiSink:wasapisink0: The stream is in the wrong format. 3764 26b9c30 INFO GST_STATES gstelement.c:2205:gst_element_abort_state:<wasapisink0>Additional debug info: ../../../../gst-plugins-base-1.6.1/gst-libs/gst/audio/gstaudiobasesink.c(1213): gst_audio_base_sink_preroll (): /GstPipeline:pipeline0/GstWasapiSink:wasapisink0: sink not negotiated. aborting state from READY to PAUSED ERROR: pipeline doesn't want to preroll. 0:00:00.050139810 3764Setting pipeline to NULL ... 26b9c30 WARN basesrc gstbasesrc.c:2943:gst_base_src_loop:<fakesrc0> error: Internal data flow error. 0:00:00.050174369 3764 424000 INFO GST_STATES gstbin.c:2243:gst_bin_element_set_state:<wasapisink0> current READY pending PAUSED, desired next NULL 0:00:00.050199712 3764 26b9c30 WARN basesrc gstbasesrc.c:2943:gst_base_src_loop:<fakesrc0> error: streaming task paused, reason not-negotiated (-4) 0:00:00.050326811 3764 424000 INFO GST_STATES gstelement.c:2330:gst_element_continue_state:<wasapisink0> completed state change to NULL 0:00:00.050342555 3764 26b9c30 INFO GST_ERROR_SYSTEM gstelement.c:1837:gst_element_message_full:<fakesrc0> posting message: Internal data flow error. 0:00:00.050384025 3764 424000 INFO GST_STATES gstelement.c:2235:_priv_gst_element_state_changed:<wasapisink0> notifying about state-changed READY to NULL (VOID_PENDING pending) 0:00:00.050547987 3764 26b9c30 INFO GST_ERROR_SYSTEM gstelement.c:1860:gst_element_message_full:<fakesrc0> posted error message: Internal data flow error. 0:00:00.050619025 3764 424000 INFO GST_STATES gstbin.c:2707:gst_bin_change_state_func:<pipeline0> child 'wasapisink0' changed state to 1(NULL) successfully 0:00:00.050670863 3764 26b9c30 INFO GST_EVENT gstpad.c:5501:gst_pad_send_event_unchecked:<wasapisink0:sink> Received event on flushing pad. Discarding 0:00:00.050737293 3764 424000 INFO GST_STATES gstbin.c:2243:gst_bin_element_set_state:<fakesrc0> current PAUSED pending VOID_PENDING, desired next NULL 0:00:00.050788747 3764 26b9c30 INFO task gsttask.c:315:gst_task_func:<fakesrc0:src> Task going to paused 0:00:00.050917766 3764 26b9c30 INFO task gsttask.c:317:gst_task_func:<fakesrc0:src> Task resume from paused 0:00:00.050994564 3764 424000 INFO GST_STATES gstelement.c:2305:gst_element_continue_state:<fakesrc0> committing state from PAUSED to READY, pending NULL, next NULL 0:00:00.051062913 3764 424000 INFO GST_STATES gstelement.c:2235:_priv_gst_element_state_changed:<fakesrc0> notifying about state-changed PAUSED to READY (NULL pending) 0:00:00.051116287 3764 424000 INFO GST_STATES gstelement.c:2312:gst_element_continue_state:<fakesrc0> continue state change READY to NULL, final NULL 0:00:00.051175805 3764 424000 INFO GST_STATES gstelement.c:2330:gst_element_continue_state:<fakesrc0> completed state change to NULL 0:00:00.051231483 3764 424000 INFO GST_STATES gstelement.c:2235:_priv_gst_element_state_changed:<fakesrc0> notifying about state-changed READY to NULL (VOID_PENDING pending) 0:00:00.051282938 3764 424000 INFO GST_STATES gstbin.c:2707:gst_bin_change_state_func:<pipeline0> child 'fakesrc0' changed state to 1(NULL) successfully 0:00:00.051380470 3764 424000 INFO GST_STATES gstelement.c:2330:gst_element_continue_state:<pipeline0> completed state change to NULL 0:00:00.051432692 3764 424000 INFO GST_STATES gstelement.c:2235:_priv_gst_element_state_changed:<pipeline0> notifying about state-changed READY to NULL (VOID_PENDING pending) Freeing pipeline ... 0:00:00.051537137 3764 424000 INFO GST_ELEMENT_PADS gstpad.c:1991:gst_pad_unlink: unlinking fakesrc0:src(00000000026de250) and wasapisink0:sink(0000000000436930) 0:00:00.051619694 3764 424000 INFO GST_ELEMENT_PADS gstpad.c:2045:gst_pad_unlink: unlinked fakesrc0:src and wasapisink0:sink 0:00:00.051698795 3764 424000 INFO GST_PARENTAGE gstbin.c:1559:gst_bin_remove_func:<pipeline0> removed child "wasapisink0" 0:00:00.051782504 3764 424000 INFO GST_REFCOUNTING gstelement.c:2880:gst_element_dispose:<wasapisink0> dispose 0:00:00.051838182 3764 424000 INFO GST_ELEMENT_PADS gstelement.c:766:gst_element_remove_pad:<wasapisink0> removing pad 'sink' 0:00:00.051903076 3764 424000 INFO GST_REFCOUNTING gstelement.c:2924:gst_element_dispose:<wasapisink0> parent class dispose 0:00:00.052144219 3764 424000 INFO GST_REFCOUNTING gstelement.c:2955:gst_element_finalize:<wasapisink0> finalize 0:00:00.052204889 3764 424000 INFO GST_REFCOUNTING gstelement.c:2960:gst_element_finalize:<wasapisink0> finalize parent 0:00:00.052267479 3764 424000 INFO GST_PARENTAGE gstbin.c:1559:gst_bin_remove_func:<pipeline0> removed child "fakesrc0" 0:00:00.052330069 3764 424000 INFO GST_REFCOUNTING gstelement.c:2880:gst_element_dispose:<fakesrc0> dispose 0:00:00.052385747 3764 424000 INFO GST_ELEMENT_PADS gstelement.c:766:gst_element_remove_pad:<fakesrc0> removing pad 'src' 0:00:00.052439889 3764 424000 INFO GST_REFCOUNTING gstelement.c:2924:gst_element_dispose:<fakesrc0> parent class dispose 0:00:00.052560845 3764 424000 INFO GST_REFCOUNTING gstelement.c:2955:gst_element_finalize:<fakesrc0> finalize 0:00:00.052613067 3764 424000 INFO GST_REFCOUNTING gstelement.c:2960:gst_element_finalize:<fakesrc0> finalize parent 0:00:00.052661065 3764 424000 INFO GST_REFCOUNTING gstelement.c:2880:gst_element_dispose:<pipeline0> dispose 0:00:00.052710216 3764 424000 INFO GST_REFCOUNTING gstelement.c:2924:gst_element_dispose:<pipeline0> parent class dispose 0:00:00.052757062 3764 424000 INFO GST_REFCOUNTING gstelement.c:2955:gst_element_finalize:<pipeline0> finalize 0:00:00.052803524 3764 424000 INFO GST_REFCOUNTING gstelement.c:2960:gst_element_finalize:<pipeline0> finalize parent 0:00:00.052850371 3764 424000 INFO GST_INIT gst.c:952:gst_deinit: deinitializing GStreamer 0:00:00.055001079 3764 424000 INFO GST_INIT gst.c:1083:gst_deinit: deinitialized GStreamer
fakesrc will only work with fakesink. This is not a valid use case. Why do you want to do that ? What is the real issue you are facing ?
wasapisink will only handle content of the following type (caps): "audio/x-raw, format = (string) S16LE, layout = (string) interleaved, rate = (int) 44100, channels = (int) 2" You should look into the audio elements for one that creates this that suits your needs.
Try audiotestsrc ! wasapisink or audiotestsrc ! audioconvert ! wasapisink
None of them works.
Created attachment 317523 [details] GST_DEBUG=*:5 gst-launch-1.0.exe audiotestsrc ! audioconvert ! wasapisink
Created attachment 317524 [details] GST_DEBUG=*:5 gst-launch-1.0.exe audiotestsrc ! wasapisink
Yeah the problem with the wasapi elements is that no negotiation with the device is implemented. You need to force exactly the caps that it would accept, otherwise it will fail. Proper device probing and negotiation is the main missing feature in the wasapi elements.
wasapisrc and wasapisink are currently broken in git. trying to submit a patch soon
Created attachment 325467 [details] [review] fix basic stuff made wasapisrc and wasapisink working, but caps negotiation and device selection is still missing. Still need some love ;) please apply patch - dont know yet when I will be able to spend more time on it.
Review of attachment 325467 [details] [review]: In general please run "gst-indent" over your code first :) There are a few problems in the patch that have to be solved first. It doesn't help to add new bugs to the code even if that makes things work for you. Let's try to get any new changes looking good first ::: sys/wasapi/gstwasapisink.c @@ +80,3 @@ + gst_element_class_add_pad_template (gstelement_class, + gst_static_pad_template_get (&sink_template)); Why this change? It was intentionally changed just a while ago @@ +111,3 @@ + if (FAILED(hr)) { + GST_ELEMENT_ERROR(self, RESOURCE, OPEN_READ, + ("CoInitializeEx: %ld", hr), (NULL)); GST_ELEMENT_ERROR() does not work during init() @@ +144,3 @@ + + GST_DEBUG ("gst_wasapi_sink_get_caps filter: %s", gst_caps_to_string (filter)); + GST_DEBUG ("gst_wasapi_sink_get_caps: %s", gst_caps_to_string (caps)); You need to filter it against the filter caps if filter!=NULL @@ +155,3 @@ IAudioClient *client = NULL; + GST_DEBUG ("gst_wasapi_sink_open"); Use GST_DEBUG_OBJECT(sink, ...), also in all the other places @@ +225,3 @@ + GST_DEBUG ("gst_wasapi_sink_prepare - buffer time in us: %"G_GUINT64_FORMAT, spec->buffer_time ); + GST_DEBUG ("IAudioClient::GetBufferSize (number of audio frames that the buffer can hold) :%i", NumBufferFrames ); + newTime = NumBufferFrames / self->info.channels * self->info.rate / 1000; You should update spec->buffer_time @@ +341,2 @@ + //TODO: unsure if the calculation is correct + hr = IAudioClient_GetCurrentPadding ((IAudioClient*)self->client, &nbOfSamplesInQueue); Padding sounds wrong. We're interested in how many samples are currently queued in WASAPI but not played out yet ::: sys/wasapi/gstwasapisrc.c @@ +205,3 @@ + hr = IAudioClient_Initialize (self->client, AUDCLNT_SHAREMODE_SHARED, AUDCLNT_STREAMFLAGS_EVENTCALLBACK, + spec->buffer_time / 100, 0, (WAVEFORMATEX *) &format, NULL); In the sink you do * 10, here you do / 100. What is correct now? @@ +347,3 @@ + GST_DEBUG ("gst_wasapi_src_delay"); + + hr = IAudioCaptureClient_GetNextPacketSize (self->capture_client, &nsamples); Also doesn't seem to be the correct API. Maybe here you could return the difference between how much data you read and the audio clock position? See old get_time() function
Created attachment 325478 [details] [review] fix basic stuff - 2nd try + gst_element_class_add_pad_template (gstelement_class, + gst_static_pad_template_get (&sink_template)); Why this change? It was intentionally changed just a while ago -> otherwise I will get an warning "C:/mingw-w64/i686-4.8.1-posix-sjlj-rt_v3-rev2/mingw32/i686-w64-mingw32/include/gstreamer-1.0/gst/gstelement.h:660:25: note: expected 'struct GstPadTemplate *' but argument is of type 'struct GstStaticPadTemplat e *' void gst_element_class_add_pad_template (GstElementClass *klass, GstPadTemplate *templ);" @@ +111,3 @@ + if (FAILED(hr)) { + GST_ELEMENT_ERROR(self, RESOURCE, OPEN_READ, + ("CoInitializeEx: %ld", hr), (NULL)); GST_ELEMENT_ERROR() does not work during init() -> dont know the correct gst way of printing err msg then @@ +144,3 @@ + + GST_DEBUG ("gst_wasapi_sink_get_caps filter: %s", gst_caps_to_string (filter)); + GST_DEBUG ("gst_wasapi_sink_get_caps: %s", gst_caps_to_string (caps)); You need to filter it against the filter caps if filter!=NULL -> added comment TODO @@ +155,3 @@ IAudioClient *client = NULL; + GST_DEBUG ("gst_wasapi_sink_open"); Use GST_DEBUG_OBJECT(sink, ...), also in all the other places -> fixed that @@ +225,3 @@ + GST_DEBUG ("gst_wasapi_sink_prepare - buffer time in us: %"G_GUINT64_FORMAT, spec->buffer_time ); + GST_DEBUG ("IAudioClient::GetBufferSize (number of audio frames that the buffer can hold) :%i", NumBufferFrames ); + newTime = NumBufferFrames / self->info.channels * self->info.rate / 1000; You should update spec->buffer_time -> fixed that @@ +341,2 @@ + //TODO: unsure if the calculation is correct + hr = IAudioClient_GetCurrentPadding ((IAudioClient*)self->client, &nbOfSamplesInQueue); Padding sounds wrong. We're interested in how many samples are currently queued in WASAPI but not played out yet -> I know it sounds wrong, but for me description sounds correct https://msdn.microsoft.com/de-de/library/windows/desktop/dd370868%28v=vs.85%29.aspx ::: sys/wasapi/gstwasapisrc.c @@ +205,3 @@ + hr = IAudioClient_Initialize (self->client, AUDCLNT_SHAREMODE_SHARED, AUDCLNT_STREAMFLAGS_EVENTCALLBACK, + spec->buffer_time / 100, 0, (WAVEFORMATEX *) &format, NULL); In the sink you do * 10, here you do / 100. What is correct now? -> you're right - microsoft use hns (100ns) as unit @@ +347,3 @@ + GST_DEBUG ("gst_wasapi_src_delay"); + + hr = IAudioCaptureClient_GetNextPacketSize (self->capture_client, &nsamples); Also doesn't seem to be the correct API. Maybe here you could return the difference between how much data you read and the audio clock position? See old get_time() function -> I know it sounds wrong, but for me description sounds correct
(In reply to Thomas Roos from comment #11) > Created attachment 325478 [details] [review] [review] > fix basic stuff - 2nd try > > + gst_element_class_add_pad_template (gstelement_class, > + gst_static_pad_template_get (&sink_template)); > > Why this change? It was intentionally changed just a while ago > -> otherwise I will get an warning > "C:/mingw-w64/i686-4.8.1-posix-sjlj-rt_v3-rev2/mingw32/i686-w64-mingw32/ > include/gstreamer-1.0/gst/gstelement.h:660:25: note: expected 'struct > GstPadTemplate *' but argument is of type 'struct GstStaticPadTemplat > e *' > void gst_element_class_add_pad_template > (GstElementClass *klass, GstPadTemplate *templ);" It was gst_element_class_add_*static*_pad_template(), which was added with 1.8.0. That gives you a warning? > @@ +111,3 @@ > + if (FAILED(hr)) { > + GST_ELEMENT_ERROR(self, RESOURCE, OPEN_READ, > + ("CoInitializeEx: %ld", hr), (NULL)); > > GST_ELEMENT_ERROR() does not work during init() > > -> dont know the correct gst way of printing err msg then You could remember in an instance variable that things failed, and then during the state change from NULL->READY fail like that. Also it makes probably more sense to do the COM initialization in class_init() (i.e. once per class, not once per instance). > @@ +144,3 @@ > + > + GST_DEBUG ("gst_wasapi_sink_get_caps filter: %s", gst_caps_to_string > (filter)); > + GST_DEBUG ("gst_wasapi_sink_get_caps: %s", gst_caps_to_string (caps)); > > You need to filter it against the filter caps if filter!=NULL > -> added comment TODO It's not a TODO, it's a bug that will cause assertions. Alternatively you can just continue returning NULL here, then the base class should do the right thing. > @@ +341,2 @@ > + //TODO: unsure if the calculation is correct > + hr = IAudioClient_GetCurrentPadding ((IAudioClient*)self->client, > &nbOfSamplesInQueue); > > Padding sounds wrong. We're interested in how many samples are currently > queued > in WASAPI but not played out yet > -> I know it sounds wrong, but for me description sounds correct > https://msdn.microsoft.com/de-de/library/windows/desktop/dd370868%28v=vs. > 85%29.aspx Ok > ::: sys/wasapi/gstwasapisrc.c > @@ +205,3 @@ > > + hr = IAudioClient_Initialize (self->client, AUDCLNT_SHAREMODE_SHARED, > AUDCLNT_STREAMFLAGS_EVENTCALLBACK, > + spec->buffer_time / 100, 0, (WAVEFORMATEX *) &format, NULL); > > In the sink you do * 10, here you do / 100. What is correct now? > > -> you're right - microsoft use hns (100ns) as unit Ok :) Should you also check here afterwards if you got the requested buffer time, and update with the actual value returned? > @@ +347,3 @@ > + GST_DEBUG ("gst_wasapi_src_delay"); > + > + hr = IAudioCaptureClient_GetNextPacketSize (self->capture_client, > &nsamples); > > Also doesn't seem to be the correct API. Maybe here you could return the > difference between how much data you read and the audio clock position? See > old > get_time() function > > -> I know it sounds wrong, but for me description sounds correct ok :)
> Created attachment 325478 [details] [review] [review] [review] > fix basic stuff - 2nd try > > + gst_element_class_add_pad_template (gstelement_class, > + gst_static_pad_template_get (&sink_template)); > > Why this change? It was intentionally changed just a while ago > -> otherwise I will get an warning > "C:/mingw-w64/i686-4.8.1-posix-sjlj-rt_v3-rev2/mingw32/i686-w64-mingw32/ > include/gstreamer-1.0/gst/gstelement.h:660:25: note: expected 'struct > GstPadTemplate *' but argument is of type 'struct GstStaticPadTemplat > e *' > void gst_element_class_add_pad_template > (GstElementClass *klass, GstPadTemplate *templ);" It was gst_element_class_add_*static*_pad_template(), which was added with 1.8.0. That gives you a warning? -> you're probably right - Im currently using 1.4.5. will remove my change from patch > @@ +111,3 @@ > + if (FAILED(hr)) { > + GST_ELEMENT_ERROR(self, RESOURCE, OPEN_READ, > + ("CoInitializeEx: %ld", hr), (NULL)); > > GST_ELEMENT_ERROR() does not work during init() > > -> dont know the correct gst way of printing err msg then You could remember in an instance variable that things failed, and then during the state change from NULL->READY fail like that. Also it makes probably more sense to do the COM initialization in class_init() (i.e. once per class, not once per instance). -> ok > @@ +144,3 @@ > + > + GST_DEBUG ("gst_wasapi_sink_get_caps filter: %s", gst_caps_to_string > (filter)); > + GST_DEBUG ("gst_wasapi_sink_get_caps: %s", gst_caps_to_string (caps)); > > You need to filter it against the filter caps if filter!=NULL > -> added comment TODO It's not a TODO, it's a bug that will cause assertions. Alternatively you can just continue returning NULL here, then the base class should do the right thing. -> returning the actual caps is not an good idea? The way I've implemented it is a 100% copy of directsound src and sink!? > ::: sys/wasapi/gstwasapisrc.c > @@ +205,3 @@ > > + hr = IAudioClient_Initialize (self->client, AUDCLNT_SHAREMODE_SHARED, > AUDCLNT_STREAMFLAGS_EVENTCALLBACK, > + spec->buffer_time / 100, 0, (WAVEFORMATEX *) &format, NULL); > > In the sink you do * 10, here you do / 100. What is correct now? > > -> you're right - microsoft use hns (100ns) as unit Ok :) Should you also check here afterwards if you got the requested buffer time, and update with the actual value returned? -> I did that already in the recent patch version (second try)
Created attachment 325582 [details] [review] fix basic stuff - 3nd try
Review of attachment 325582 [details] [review]: Does not apply, something went wrong here with the patch: patch: **** malformed patch at line 58: @@ -93,6 +96,12 @@ gst_wasapi_sink_class_init (GstWasapiSinkClass * klass) ::: sys/wasapi/gstwasapisink.c @@ +100,3 @@ + hr = CoInitializeEx (NULL, COINIT_MULTITHREADED); + if (FAILED (hr)) { + // TODO print error You could make this a GST_ERROR() for now. Better than nothing @@ +142,3 @@ + caps = gst_pad_get_pad_template_caps (GST_BASE_SRC_PAD (bsink)); + + //TODO filter it against the filter caps if filter!=NULL Why don't you just do it then? That's as fast as writing that comment :P if (filter) { GstCaps *tmp = gst_caps_intersect_full (filter, caps, GST_CAPS_INTERSECT_FIRST); gst_caps_unref (caps); caps = tmp; } @@ +212,3 @@ hr = IAudioClient_Initialize (self->client, AUDCLNT_SHAREMODE_SHARED, AUDCLNT_STREAMFLAGS_EVENTCALLBACK, + spec->buffer_time * 10, 0, (WAVEFORMATEX *) & format, NULL); buffer_time / latency_time are in microseconds btw, not milliseconds. So if WASAPI wants it in 100ns units, you would have to divide by 10. @@ +232,3 @@ + "IAudioClient::GetBufferSize (number of audio frames that the buffer can hold) :%i", + NumBufferFrames); + newTime = NumBufferFrames / self->info.channels * self->info.rate / 1000; gst_util_uint64_scale() is usually helpful for these kind of things to prevent overflows while keeping precision
Thomas, are you planning to update the patch?
yes, I think tmr you will get an updated version.
Created attachment 328513 [details] [review] fix basic stuff - 4rd try tried to include all previous comments
I am doing some improvements in WASAPI (see bug 773638) and I would like to also handle this case, but I do not want to reinvent the wheel. I am mostly interested in fixing get_caps() so wasapisrc/wasapisink exposes caps that can be handled by the soundcard for sure. I've read Thomas patches maybe I don't understand something but I think the right approach to implement get_caps() would have been to call IAudioClient::GetMixFormat which returns internal mix format used by WASAPI in the shared mode and return it as caps. The second approach, that even might be better is to call IAudioClient::IsFormatSupported with given caps and return these caps if format matches or closest one (WASAPI returns information about closest matching format if there's no exact match). What do you think about that?
(In reply to Marcin Lewandowski from comment #19) > I've read Thomas patches maybe I don't understand something but I think the > right approach to implement get_caps() would have been to call > IAudioClient::GetMixFormat which returns internal mix format used by WASAPI > in the shared mode and return it as caps. Sounds reasonable > The second approach, that even might be better is to call > IAudioClient::IsFormatSupported with given caps and return these caps if > format matches or closest one (WASAPI returns information about closest > matching format if there's no exact match). That seems like it would require probing then. You would have to iterate over a set of known formats in get_caps() and check if they are supported (and ideally cache that information). Similar to what alsasink (or v4l2src) already does (so yes, that's also an option).
If you work on this, please either merge these patches in your branch or let us know with which of your patches they are obsolete
I won't rely on Thomas' work - it modifies too many things at the same time. I want to fix one method, so I will rely on master. Please tell me, what is the role of "filter" parameter passed to get_caps()? It's not documented. I've seen in some elements and default implementation of get_caps() that it if it's not NULL the actual caps are intersected with caps passed as filter. Should I do this?
See the documentation of the CAPS query. It exists to limit the size of the result, by only providing those caps that are actually relevant (only something that is a subset of the filter caps should be returned). It is also used for giving a preference by order from the caller, in general the order from the filter should be preferred unless the element has a very good reason not to do that
This has been fixed with https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/?id=1450851095915b6fd558d95a8d0aa4b98c043f3d and later commits. See also https://bugzilla.gnome.org/show_bug.cgi?id=792897. Note that the source and sinks now require F32LE because that's the only thing WASAPI will accept, so you will need to add audioconvert/audioresample depending on your pipeline. I would really appreciate it if someone could test this code and tell us if it works as well as I think it does. :)
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/337.