[sr-dev] SIGSEGV on DMQ

Sergio Carvalho sergioffpc at gmail.com
Fri Sep 10 17:41:22 CEST 2021


Hi everyone,

I'm trying to use the DMQ module to share the user location info across our
Kamailio's cluster.  But when I start the service under docker I get a
segmentation fault violation.  After analysing the core dump with the *LLDB*
I get (if you want I can share the core dump with you using Google Drive):

> (lldb) bt
> * thread #1, name = 'kamailio', stop reason = signal SIGSEGV
>   * frame #0: 0x000000000071c63c kamailio`get_hdr_field(buf="", end="",
> hdr=0x0000000000000000) at msg_parser.c:144:5
>     frame #1: 0x0000000000575b02
> kamailio`resolve_select(s=0x00007ffd470c6da0) at select.c:384:3
>     frame #2: 0x00007fbbd7637fb9
>     frame #3: 0x00000000005ad785
> kamailio`clen_builder(msg=0x00007fbbd7f6e008, clen_len=0x0000000000000000,
> diff=0, body_only=0) at msg_translator.c:451:3
>     frame #4: 0x0000000000439aa7
> kamailio`_dns_hash_find(name=0x00007fbbd7f6e498, type=32765,
> h=0x0000000000000000, err=0x0000000000000000) at dns_cache.c:551:4
>     frame #5: 0x0000000000440cb2
> kamailio`dns_cache_do_request(name=0x0000000000000000, type=0) at
> dns_cache.c:1767:3
>     frame #6: 0x000000000043f8f6
> kamailio`dns_cache_mk_rd_entry(name=0x0000000000000000, type=0,
> rd_lst=0x0000000000000000) at dns_cache.c:1226:7
>     frame #7: 0x000000000046501e
> kamailio`print_action(t=0x0000000000000000) at route_struct.c:467:4
>     frame #8: 0x0000000000471f2f
> kamailio`xavp_print_list_content(head=0x00007ffd470c31a0, level=32699) at
> xavp.c:584:2
>     frame #9: 0x0000000000462047
> kamailio`print_action(t=0x0000000000000000) at route_struct.c:392:4
>     frame #10: 0x0000000000471f2f
> kamailio`xavp_print_list_content(head=0x00007ffd470c31a0, level=32699) at
> xavp.c:584:2
>     frame #11: 0x00000000004654dc
> kamailio`print_action(t=0x0000000000000000) at route_struct.c:470:4
>     frame #12: 0x0000000000471f2f
> kamailio`xavp_print_list_content(head=0x00007ffd470c31a0, level=32699) at
> xavp.c:584:2
>     frame #13: 0x0000000000465532
> kamailio`print_action(t=0x0000000003863810) at route_struct.c:470:4
>     frame #14: 0x0000000000471f2f
> kamailio`xavp_print_list_content(head=0x00007ffd470c31a0, level=32699) at
> xavp.c:584:2
>     frame #15: 0x0000000000462047
> kamailio`print_action(t=0x00000000006da9d3) at route_struct.c:392:4
>     frame #16: 0x0000000000471f2f
> kamailio`xavp_print_list_content(head=0x00007ffd470c31a0, level=32699) at
> xavp.c:584:2
>     frame #17: 0x0000000000472680
> kamailio`xavp_get_list_key_names(xavp=0x00007ffd470c6da0) at xavp.c:622:3
>     frame #18: 0x00007fbbb357da54
>     frame #19: 0x00007fbbb3580370
>     frame #20: 0x00007fbbb358317b
>     frame #21: 0x00007fbbb3583147
>     frame #22: 0x00007fbbb3585e8d
>     frame #23: 0x00007fbbaec3ef23
>     frame #24: 0x00007fbbaec3d5da
>     frame #25: 0x00007fbbaec4e158
>     frame #26: 0x00007fbbaec4aadd
>     frame #27: 0x00007fbbaec53f05
>     frame #28: 0x0000000000541563
> kamailio`handle_tcp_child(tcp_c=0x00000004b4481868, fd_i=136) at
> tcp_main.c:3477:1
>     frame #29: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x0000000000000000, fd_i=32699) at
> tcp_main.c:3470:5
>     frame #30: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00000000470c6da0, fd_i=0) at
> tcp_main.c:3470:5
>     frame #31: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00000000b4481868, fd_i=1863578990) at
> tcp_main.c:3470:5
>     frame #32: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00000000b4481868, fd_i=1863578990) at
> tcp_main.c:3470:5
>     frame #33: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00007fbbb4481868, fd_i=1863578990) at
> tcp_main.c:3470:5
>     frame #34: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00007ffd470c64d0, fd_i=32699) at
> tcp_main.c:3470:5
>     frame #35: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00007ffd470c658c, fd_i=32765) at
> tcp_main.c:3470:5
>     frame #36: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00000000b4481868, fd_i=1863578990) at
> tcp_main.c:3470:5
>     frame #37: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00007fbbb4481868, fd_i=1863578990) at
> tcp_main.c:3470:5
>     frame #38: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00007fbbd8059e38, fd_i=32699) at
> tcp_main.c:3470:5
>     frame #39: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00007ffd470c6770, fd_i=32765) at
> tcp_main.c:3470:5
>     frame #40: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x0000000000000000, fd_i=32765) at
> tcp_main.c:3470:5
>     frame #41: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00007fbbb5179a80, fd_i=0) at
> tcp_main.c:3470:5
>     frame #42: 0x0000000000541223
> kamailio`handle_tcp_child(tcp_c=0x00000065dbb52b93, fd_i=32765) at
> tcp_main.c:3470:5
>     frame #43: 0x00000000005418fa
> kamailio`handle_ser_child(p=0x00000065dbb52b93, fd_i=32765) at
> tcp_main.c:3513:3
>     frame #44: 0x0000000000425d6e kamailio`main_loop at main.c:1620:4
>     frame #45: 0x000000000042c6af kamailio`main(argc=13,
> argv=0x00007ffd470c6da8) at main.c:2568:3
>

Using *GDB* I get the following info:

> #22 0x00007efcca978e8d in request () from /usr/lib64/kamailio/modules/tm.so
> ---Type <return> to continue, or q <return> to quit---
> No symbol table info available.
> #23 0x00007efcc6031f23 in dmq_send_message (peer=0x7efccc5545c0,
> body=0x7efcef106ef8, node=0x7efccc5ded10, resp_cback=0x7efcc625d370
> <notification_callback>, max_forwards=2,
>     content_type=0x7efcc625d360 <notification_content_type>) at
> dmq_funcs.c:251
>         uac_r = {method = 0x7efcc625d3b0 <dmq_request_method>, headers =
> 0x7ffe13cb34b0, body = 0x7efcef106ef8, ssock = 0x7efcc625da20
> <dmq_server_socket>, dialog = 0x7efccc5e0918,
>           cb_flags = 1024, cb = 0x7efcc602eb7b <dmq_tm_callback>, cbp =
> 0x7efccc5def58, callid = 0x0}
>         str_hdr = {s = 0x7efcef1004e8 "Max-Forwards: 2\r\nContent-Type:
> text/plain\r\n", len = 43}
>         from = {s = 0x7efcef107248 "sip:notification_peer at 10.12.4.238:5090",
> len = 38}
>         to = {s = 0x7efcef11d100 "sip:notification_peer at 10.12.4.170:5090",
> len = 38}
>         cb_param = 0x7efccc5def58
>         result = 0
>         len = 43
>         __FUNCTION__ = "dmq_send_message"
> #24 0x00007efcc60305da in bcast_dmq_message1 (peer=0x7efccc5545c0,
> body=0x7efcef106ef8, except=0x0, resp_cback=0x7efcc625d370
> <notification_callback>, max_forwards=2,
>     content_type=0x7efcc625d360 <notification_content_type>,
> incl_inactive=1) at dmq_funcs.c:170
>         node = 0x7efccc5ded10
>         __FUNCTION__ = "bcast_dmq_message1"
> #25 0x00007efcc6041158 in request_nodelist (node=0x7efccc5deac8,
> forward=2) at notification_peer.c:573
>         body = 0x7efcef106ef8
>         ret = 0
>         __FUNCTION__ = "request_nodelist"
> #26 0x00007efcc603dadd in add_server_and_notify (paddr=0x7efcc625da30
> <dmq_notification_address>) at notification_peer.c:340
>         puri_data = "sip:10.12.5.29:5090\000\376\177\000\000\250I+\002\000\000\000\000\230L+\002\000\000\000\000\260;\313\023\376\177\000\000\250c+\002\000\000\000\000`;\313\023\376\177",
> '\000' <repeats 17 times>,
> "\200`?\313\023\376\177\000\000\325\222\255\307\374~\000\000\024=\313\023\376\177\000\000\030=\313\023\376\177\000\000\034=\313\023\376\177\000\000\260:\313\023\376\177\000\000*\000\000\000\000\000\000\000]s\f\000\000\000\000\000\001\000\000\000\000\000\000\000\244\201\000\000\204\352",
> '\000' <repeats 19 times>,
> "\340\f\000\000\000\000\000\000\020\000\000\000\000\000\000p\006\000\000\000\000\000\000"...
>         puri_list = {0x7ffe13cb3950 "sip:10.12.5.29:5090", 0x7ffe13cb3a50
> "sip:10.12.4.170:5090", 0x7ffe13cb3b50 "x\321-\002", 0x7ffe13cb3c50 "",
> 0x7ffe13cb3d50 " @\313\023\376\177",
>           0x7ffe13cb3e50 "", 0x7ffe13cb3f50 "", 0x7ffe13cb4050 "",
> 0x7ffe13cb4150 "2\247h", 0x7ffe13cb4250 "pB\313\023\376\177",
> 0x7ffe13cb4350 "\200S\237\312\374~",
>           0x7ffe13cb4450 "8\341\006\357\374~", 0x7ffe13cb4550
> "\320E\313\023\376\177", 0x7ffe13cb4650 "\210*,\002", 0x7ffe13cb4750
> "\330N\313\023\376\177", 0x7ffe13cb4850 "",
>           0x7ffe13cb4950 "", 0x7ffe13cb4a50 "\200J\313\023\376\177",
> 0x7ffe13cb4b50 "\035\362\067\037\003", 0x7ffe13cb4c50
> "\320L\313\023\376\177", 0x7ffe13cb4d50 "\177", 0x7ffe13cb4e50 "n",
>           0x7ffe13cb4f50 "1\325\\\002", 0x7ffe13cb5050
> "\260P\313\023\376\177", 0x7ffe13cb5150 "", 0x7ffe13cb5250
> "\320\343\270\307\374~", 0x7ffe13cb5350 "", 0x7ffe13cb5450 "xV,\002",
>           0x7ffe13cb5550 "2\247h", 0x7ffe13cb5650 "\320\235\005\306\374~"}
>         pfirst = 0x7efccc5deac8
>         pnode = 0x7efccc5ded10
>         host_cnt = 2
>         index = 2
>         puri = {{user = {s = 0x0, len = 0}, passwd = {s = 0x0, len = 0},
> host = {s = 0x7efceefdde8c "
> kamailio-kazoo.svc.cpaas.eu-west-1.aws.prv.talkdeskstg.com:5090", len =
> 58}, port = {
>               s = 0x7efceefddec7 "5090", len = 4}, params = {s = 0x0, len
> = 0}, sip_params = {s = 0x0, len = 0}, headers = {s = 0x0, len = 0},
> port_no = 5090, proto = 0, type = SIP_URI_T,
>             flags = (unknown: 0), transport = {s = 0x0, len = 0}, ttl = {s
> = 0x0, len = 0}, user_param = {s = 0x0, len = 0}, maddr = {s = 0x0, len =
> 0}, method = {s = 0x0, len = 0}, lr = {
>               s = 0x0, len = 0}, r2 = {s = 0x0, len = 0}, gr = {s = 0x0,
> len = 0}, transport_val = {s = 0x0, len = 0}, ttl_val = {s = 0x0, len = 0},
> user_param_val = {s = 0x0, len = 0},
>             maddr_val = {s = 0x0, len = 0}, method_val = {s = 0x0, len =
> 0}, lr_val = {s = 0x0, len = 0}, r2_val = {s = 0x0, len = 0}, gr_val = {s =
> 0x0, len = 0}}}
>         pstr = {{s = 0x7ffe13cb3a50 "sip:10.12.4.170:5090", len = 20}}
>         __FUNCTION__ = "add_server_and_notify"
> #27 0x00007efcc6046f05 in child_init (rank=0) at dmq.c:302
>         i = 4
>         newpid = 160
> ---Type <return> to continue, or q <return> to quit---
>         __FUNCTION__ = "child_init"
> #28 0x0000000000541563 in init_mod_child ()
> No symbol table info available.
> #29 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #30 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #31 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #32 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #33 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #34 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #35 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #36 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #37 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #38 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #39 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #40 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #41 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #42 0x0000000000541223 in init_mod_child ()
> No symbol table info available.
> #43 0x00000000005418fa in init_child ()
> No symbol table info available.
> #44 0x0000000000425d6e in main_loop ()
> No symbol table info available.
> #45 0x000000000042c6af in main ()
> No symbol table info available.
>

DMQ module parameters are:

> #!ifdef DMQ_ROLE
> loadmodule "dmq.so"
> loadmodule "dmq_usrloc.so"
>
> # ---- dmq params ----
> modparam("dmq", "server_address", "sip:DMQ_SERVER_ADDRESS")
> modparam("dmq", "notification_address", "sip:DMQ_NOTIFY_ADDRESS")
> modparam("dmq", "ping_interval", 5)
> modparam("dmq", "multi_notify", 1)
> modparam("dmq", "num_workers", 4)
> modparam("dmq", "worker_usleep", 0)
> modparam("dmq_usrloc", "enable", 1)
> #!endif
>

Kamailio version and flags are:

> version: kamailio 5.2.5 (x86_64/linux)
> flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS,
> DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC,
> Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX,
> FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR,
> USE_DST_BLACKLIST, HAVE_RESOLV_RES
> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144 MAX_URI_SIZE 1024,
> BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
> id: unknown
> compiled on 01:14:59 Nov 19 2019 with gcc 7.3.1
>

Do you have any idea what might be causing this *SEGV*?

Best regards,
Sergio Carvalho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-dev/attachments/20210910/695287be/attachment-0001.htm>


More information about the sr-dev mailing list