After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 795166 - Last DTLS handshake packets may not be sent to the wire at all
Last DTLS handshake packets may not be sent to the wire at all
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
1.14.0
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2018-04-11 15:14 UTC by Daniel F
Modified: 2018-11-03 14:21 UTC
See Also:
GNOME target: ---
GNOME version: ---


Description Daniel F 2018-04-11 15:14:35 UTC 
Sometimes FreeStreamer does not send last DTLS handshake packets to the wire at all, and starts sending SRTP/SRTCP. However receiver is not able to decode them, because handshake was not completed. DTLS handshake retry also fails, at least this happened during my tests.

I found this when I tested WebRTC connection between GStreamer and FreeSwitch, but this may be a generic problem (not related to WebRTC).

Logs and more details are here:
https://pastebin.com/E7HiAfd7
https://lists.freedesktop.org/archives/gstreamer-devel/2018-April/067556.html
Comment 1 Tim-Philipp Müller 2018-04-11 15:29:02 UTC 
So do I understand you right that this is a bug elsewhere but GStreamer should handle it more gracefully?
Comment 2 Daniel F 2018-04-11 18:33:03 UTC 
No, bug is in GStreamer itself - it looks that it creates DTLS handshake packets, but for some reason they are not sent to network.
Comment 3 Daniel F 2018-04-13 08:33:55 UTC 
Thread responsible for sending these missing packets is stuck here:

+ Trace 238554

  • #0 __lll_lock_wait
    from /lib64/libpthread.so.0
  • #1 _L_lock_870
    from /lib64/libpthread.so.0
  • #2 pthread_mutex_lock
    from /lib64/libpthread.so.0
  • #3 gst_funnel_sink_chain_object
    at gstfunnel.c line 325
  • #4 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #5 gst_pad_push_data
    at gstpad.c line 4535
  • #6 gst_pad_push
    at gstpad.c line 4654
  • #7 src_task_loop
    at gstdtlsenc.c line 461
  • #8 gst_task_func
    at gsttask.c line 332
  • #9 g_thread_pool_thread_proxy
    at gthreadpool.c line 307
  • #10 g_thread_proxy
    at gthread.c line 784
  • #11 start_thread
    from /lib64/libpthread.so.0
  • #12 clone
    from /lib64/libc.so.6 

Could you give me some hints how to fix this or if some workaround exists? This bug blocks my work and I am looking for any way to get around it.
Comment 4 Matthew Waters (ystreet00) 2018-04-15 14:06:35 UTC 
gstfunnel.c:325 is taking the stream lock.  You have to find which other thread has taken the stream lock and is not releasing it :)  There are a few blocking pad probes in there that you might want to watch out for.
Comment 5 Daniel F 2018-04-16 13:47:08 UTC 
Blocked thread is named dtlsenc0:src (in RTP part of pipeline). It is blocked by thread named queue1:src (queue between rtpopuspay and capsfilter).

I use following pipeline:

webrtcbin name=sendrecv 
audiotestsrc wave=sine freq=1000 ! queue ! opusenc ! rtpopuspay ! 
queue ! application/x-rtp,media=audio,encoding-name=OPUS,payload=97 ! sendrecv.

Queue thread is waiting on ppoll():

+ Trace 238560

  • #0 ppoll
    from /lib64/libc.so.6
  • #1 gst_poll_wait
    at gstpoll.c line 1370
  • #2 gst_system_clock_id_wait_jitter_unlocked
    at gstsystemclock.c line 730
  • #3 gst_clock_id_wait
    at gstclock.c line 543
  • #4 gst_base_sink_wait_clock
    at gstbasesink.c line 2208
  • #5 gst_base_sink_do_sync
    at gstbasesink.c line 2607
  • #6 gst_base_sink_chain_unlocked
    at gstbasesink.c line 3513
  • #7 gst_base_sink_chain_main
    at gstbasesink.c line 3672
  • #8 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #9 gst_pad_push_data
    at gstpad.c line 4535
  • #10 gst_pad_push
    at gstpad.c line 4654
  • #11 gst_proxy_pad_chain_default
    at gstghostpad.c line 127
  • #12 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #13 gst_pad_push_data
    at gstpad.c line 4535
  • #14 gst_pad_push
    at gstpad.c line 4654
  • #15 gst_funnel_sink_chain_object
    at gstfunnel.c line 339
  • #16 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #17 gst_pad_push_data
    at gstpad.c line 4535
  • #18 gst_pad_push
    at gstpad.c line 4654
  • #19 gst_srtp_enc_chain
    at gstsrtpenc.c line 1118
  • #20 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #21 gst_pad_push_data
    at gstpad.c line 4535
  • #22 gst_pad_push
    at gstpad.c line 4654
  • #23 gst_proxy_pad_chain_default
    at gstghostpad.c line 127
  • #24 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #25 gst_pad_push_data
    at gstpad.c line 4535
  • #26 gst_pad_push
    at gstpad.c line 4654
  • #27 gst_proxy_pad_chain_default
    at gstghostpad.c line 127
  • #28 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #29 gst_pad_push_data
    at gstpad.c line 4535
  • #30 gst_pad_push
    at gstpad.c line 4654
  • #31 gst_proxy_pad_chain_default
    at gstghostpad.c line 127
  • #32 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #33 gst_pad_push_data
    at gstpad.c line 4535
  • #34 gst_pad_push
    at gstpad.c line 4654
  • #35 gst_rtp_session_send_rtp
    at gstrtpsession.c line 1352
  • #36 source_push_rtp
    at rtpsession.c line 1380
  • #37 rtp_source_send_rtp
    at rtpsource.c line 1330
  • #38 rtp_session_send_rtp
    at rtpsession.c line 3016
  • #39 gst_rtp_session_chain_send_rtp_common
    at gstrtpsession.c line 2294
  • #40 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #41 gst_pad_push_data
    at gstpad.c line 4535
  • #42 gst_pad_push
    at gstpad.c line 4654
  • #43 gst_proxy_pad_chain_default
    at gstghostpad.c line 127
  • #44 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #45 gst_pad_push_data
    at gstpad.c line 4535
  • #46 gst_pad_push
    at gstpad.c line 4654
  • #47 gst_proxy_pad_chain_default
    at gstghostpad.c line 127
  • #48 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #49 gst_pad_push_data
    at gstpad.c line 4535
  • #50 gst_pad_push
    at gstpad.c line 4654
  • #51 gst_base_transform_chain
    at gstbasetransform.c line 2321
  • #52 gst_pad_chain_data_unchecked
    at gstpad.c line 4279
  • #53 gst_pad_push_data
    at gstpad.c line 4535
  • #54 gst_pad_push
    at gstpad.c line 4654
  • #55 gst_queue_push_one
    at gstqueue.c line 1384
  • #56 gst_queue_loop
    at gstqueue.c line 1537
  • #57 gst_task_func
    at gsttask.c line 332
  • #58 g_thread_pool_thread_proxy
    at gthreadpool.c line 307
  • #59 g_thread_proxy
    at gthread.c line 784
  • #60 start_thread
    from /lib64/libpthread.so.0
  • #61 clone
    from /lib64/libc.so.6 






Comment 6


Daniel F





          2018-04-18 08:24:10 UTC
        



Looks that DTLS thread gets starved with default thread scheduling/priority config. I have updated app to set FIFO realtime scheduler and priority 1 for threads, and everything started working smoothly.






Comment 7


GStreamer system administrator





          2018-11-03 14:21:14 UTC
        



-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/687.