[sr-dev] [kamailio/kamailio] Random crash on startup Kamailio version (#2213)

Odorik.cz notifications at github.com
Thu Feb 6 14:36:41 CET 2020


<!--
Kamailio Project uses GitHub Issues only for bugs in the code or feature requests. Please use this template only for bug reports.

If you have questions about using Kamailio or related to its configuration file, ask on sr-users mailing list:

  * http://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

If you have questions about developing extensions to Kamailio or its existing C code, ask on sr-dev mailing list:

  * http://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev

Please try to fill this template as much as possible for any issue. It helps the developers to troubleshoot the issue.

If there is no content to be filled in a section, the entire section can be removed.

You can delete the comments from the template sections when filling.

You can delete next line and everything above before submitting (it is a comment).
-->

### Description

<!--
We use kamailio in productions about 11 thousand lines registered at any time. We have more open ports like 5060, 443, 6688 to avoid ALG and buggy router issues. Several TCP and TLS ports are open too.
Kamailio is able to run a year without a single issue. However when we restart it,  to make changes, it randomly crashes few seconds to few hours after the new start. If it survives few hours, it will never crash again till next restart.
We have found out that when there is no real traffic, there is no crash. So if the firewall is closed, there is no crash. So we made script which opens the firewall with delay. We also do not open all the ports at once, but one port/protocol at a time with few seconds delay.  It seems to help a bit.

all open ports:

socket_workers=4
listen=udp:81.31.45.51:5060
socket_workers=4
listen=udp:81.31.45.51:6688
socket_workers=1
listen=udp:81.31.45.51:443
socket_workers=1
listen=tcp:81.31.45.51:5060
socket_workers=1
listen=tcp:81.31.45.51:443
socket_workers=1
listen=tcp:81.31.45.51:6688
socket_workers=1
listen=udp:81.31.45.51:6699
socket_workers=1
listen=tcp:81.31.45.51:6699
socket_workers=1
listen=tls:81.31.45.51:5061
socket_workers=1
listen=tls:81.31.45.51:6689
socket_workers=1
listen=tls:81.31.45.51:6670
socket_workers=1
listen=udp:81.31.43.196:5060
socket_workers=1
listen=udp:81.31.43.196:53535
socket_workers=1
listen=tls:81.31.43.196:5061

-->

### Troubleshooting

#### Reproduction

<!--
Random after restart.
-->

#### Debugging Data

<!--
If you got a core dump, use gdb to extract troubleshooting data - full backtrace,
local variables and the list of the code at the issue location.

  gdb /path/to/kamailio /path/to/corefile
  bt full
  info locals
  list

If you are familiar with gdb, feel free to attach more of what you consider to
be relevant.
-->
We have 3 crash from last restart.
```
---
file: /core.kamailio.sig11.26294

[New LWP 26294]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid -f /usr/local/etc/kamailio/ka'.
Program terminated with signal 11, Segmentation fault.
#0  0x000000000062fc42 in destroy_avp_list (list=0xab6740) at usr_avp.c:635
635                     avp = avp->next;
#0  0x000000000062fc42 in destroy_avp_list (list=0xab6740) at usr_avp.c:635
        avp = 0x800000008
        foo = 0x800000008
        __FUNCTION__ = "destroy_avp_list"
#1  0x000000000062fd9b in reset_avps () at usr_avp.c:665
        i = 0
#2  0x000000000051a018 in ksr_msg_env_reset () at receive.c:355
No locals.
#3  0x0000000000519a6c in receive_msg (buf=0x2324380 "OPTIONS sip:sip.odorik.cz:5061 SIP/2.0\r\nVia: SIP/2.0/TLS 10.0.0.234:5060;branch=z9hG4bK2085676490;rport;alias\r\nFrom: <sips:748307 at sip.odorik.cz:5061>;tag=394217558\r\nTo: <sip:sip.odorik.cz:5061>\r\nCall-"..., len=523, rcv_info=0x7ff2d592f540) at receive.c:317
        msg = 0x7ff358aa5fe0
        ctx = {rec_lev = 35874256, run_flags = 552, last_retcode = 7921624, jmp_env = {{__jmpbuf = {146028888117, 17179869186, 515396075521, 146028888066, 0, 3600, 35874256, 2}, __mask_was_saved = 3, __saved_mask = {__val = {140680941992488, 7, 140680941992491, 140726632980624, 6344480, 21474836480, 140683200626657, 36848512, 0, 65552, 140683202956952, 140683202956968, 1580780093, 65536, 519695341616, 65584}}}}}
        ret = 65536
        inb = {s = 0x2324380 "OPTIONS sip:sip.odorik.cz:5061 SIP/2.0\r\nVia: SIP/2.0/TLS 10.0.0.234:5060;branch=z9hG4bK2085676490;rport;alias\r\nFrom: <sips:748307 at sip.odorik.cz:5061>;tag=394217558\r\nTo: <sip:sip.odorik.cz:5061>\r\nCall-"..., len = 523}
        netinfo = {data = {s = 0x0, len = -88}, dst = 0x7ffd78f81fc0, rcv = 0x7ff35c25a95c}
        __FUNCTION__ = "receive_msg"
#4  0x00000000006118c8 in receive_tcp_msg (tcpbuf=0x7ff2d592f820 "OPTIONS sip:sip.odorik.cz:5061 SIP/2.0\r\nVia: SIP/2.0/TLS 10.0.0.234:5060;branch=z9hG4bK2085676490;rport;alias\r\nFrom: <sips:748307 at sip.odorik.cz:5061>;tag=394217558\r\nTo: <sip:sip.odorik.cz:5061>\r\nCall-"..., len=523, rcv_info=0x7ff2d592f540, con=0x7ff2d592f528) at tcp_read.c:1306
        buf = 0x2324380 "OPTIONS sip:sip.odorik.cz:5061 SIP/2.0\r\nVia: SIP/2.0/TLS 10.0.0.234:5060;branch=z9hG4bK2085676490;rport;alias\r\nFrom: <sips:748307 at sip.odorik.cz:5061>;tag=394217558\r\nTo: <sip:sip.odorik.cz:5061>\r\nCall-"...
        bsize = 65535
        blen = 65535
        __FUNCTION__ = "receive_tcp_msg"
#5  0x000000000061390d in tcp_read_req (con=0x7ff2d592f528, bytes_read=0x7ffd78f82550, read_flags=0x7ffd78f82548) at tcp_read.c:1474
        bytes = 523
        total_bytes = 523
        resp = 1
        size = 24
        req = 0x7ff2d592f5a8
        dst = {send_sock = 0x7ffd78f82540, to = {s = {sa_family = 8, sa_data = "\000\000\000\000\000\000\001 \000\000(z\252X"}, sin = {sin_family = 8, sin_port = 0, sin_addr = {s_addr = 0}, sin_zero = "\001 \000\000(z\252X"}, sin6 = {sin6_family = 8, sin6_port = 0, sin6_flowinfo = 0, sin6_addr = {__in6_u = {__u6_addr8 = "\001 \000\000(z\252X\363\177\000\000\000\000\000", __u6_addr16 = {8193, 0, 31272, 22698, 32755, 0, 0, 0}, __u6_addr32 = {8193, 1487567400, 32755, 0}}}, sin6_scope_id = 7928126}}, id = 0, proto = 8 '\b', send_flags = {f = 0, blst_imask = 0}}
        c = 114 'r'
        ret = -1
        __FUNCTION__ = "tcp_read_req"
#6  0x000000000061734f in handle_io (fm=0x7ff358aa79f8, events=1, idx=-1) at tcp_read.c:1706
        ret = 8
        n = 0
        read_flags = 1
        con = 0x7ff2d592f528
        s = 17
        resp = 1
        t = 1361453701
        __FUNCTION__ = "handle_io"
#7  0x00000000006079e1 in io_wait_loop_epoll (h=0xa622e0, t=2, repeat=0) at io_wait.h:1065
        n = 1
        r = 0
        fm = 0x7ff358aa79f8
        revents = 1
        __FUNCTION__ = "io_wait_loop_epoll"
#8  0x00000000006186bb in tcp_receive_loop (unix_sock=58) at tcp_read.c:1818
        __FUNCTION__ = "tcp_receive_loop"
#9  0x00000000005fda26 in tcp_init_children () at tcp_main.c:4796
        r = 3
        i = -1
        reader_fd_1 = 58
        pid = 0
        si_desc = "tcp receiver (tls:81.31.45.51:5061)\000\000\177\000\000\377\017b\000\000\000\000\000\307\032y\000\000\000\000\000\001\000\000\000\377\377\377\377X\262 X\252f", '\000' <repeats 11 times>"\311, r\000\000\000\000\000\060\230A\000\000\000\000\000\200(\370x\000\000\000\000\310U\"X\363\177\000\000$\000\000\000%\000\000\000\030Zs\000\003\000\000"
        si = 0x0
        __FUNCTION__ = "tcp_init_children"
#10 0x00000000004b6cf1 in main_loop () at main.c:1717
        i = 1
        pid = 26281
        si = 0x0
        si_desc = "udp receiver child=0 sock=81.31.43.196:53535\000\000\000\000\"\000\000\000\000\000\000\000\a", '\000' <repeats 15 times>, "`\247+\\\363\177\000\000\060\000\000\000\060\000\000\000\230ьX\363\177\000\000P)\370x\375\177\000\000\003\000\000\000n\000\000\000\236\361t\000\000\000\000\000\305\337t\000\000\000\000"
        nrprocs = 1
        woneinit = 1
        __FUNCTION__ = "main_loop"
#11 0x00000000004bd357 in main (argc=10, argv=0x7ffd78f82d18) at main.c:2644
        cfg_stream = 0x2230010
        c = -1
        r = 0
        tmp = 0x7ffd78f83a0e ""
        tmp_len = 32755
        port = 1555726800
        proto = 0
        options = 0x743848 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:"
        ret = -1
        seed = 1597216416
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 0
        n_lst = 0x7ffd78f82c1e
        p = 0xf0b2ff <Address 0xf0b2ff out of bounds>
        st = {st_dev = 92, st_ino = 547488575, st_nlink = 2, st_mode = 16832, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1580722294, tv_nsec = 460542753}, st_mtim = {tv_sec = 1580779986, tv_nsec = 60599646}, st_ctim = {tv_sec = 1580779986, tv_nsec = 60599646}, __unused = {0, 0, 0}}
        __FUNCTION__ = "main"
avp = 0x800000008
foo = 0x800000008
__FUNCTION__ = "destroy_avp_list"
630
631             LM_DBG("destroying list %p\n", *list);
632             avp = *list;
633             while( avp ) {
634                     foo = avp;
635                     avp = avp->next;
636                     shm_free( foo );
637             }
638             *list = 0;
639     }

---
file: /core.kamailio.sig6.18855

[New LWP 18855]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid -f /usr/local/etc/kamailio/ka'.
Program terminated with signal 6, Aborted.
#0  0x00007f547478d125 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#0  0x00007f547478d125 in raise () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1  0x00007f54747903a0 in abort () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#2  0x000000000063fce7 in fm_free (qmp=0x7f53e6029000, p=0x7f53e5df9656, file=0x794a25 "core: usr_avp.c", func=0x796c20 "destroy_avp_list", line=636, mname=0x794a20 "core") at mem/f_malloc.c:601
        qm = 0x7f53e6029000
        f = 0x63cad0
        __FUNCTION__ = "fm_free"
#3  0x00000000006453b2 in fm_shm_free (qmp=0x7f53e6029000, p=0x7f53e5df9656, file=0x794a25 "core: usr_avp.c", func=0x796c20 "destroy_avp_list", line=636, mname=0x794a20 "core") at mem/f_malloc.c:1139
No locals.
#4  0x000000000062fc74 in destroy_avp_list (list=0xab6740) at usr_avp.c:636
        avp = 0xec81485354415541
        foo = 0x7f53e5df9656
        __FUNCTION__ = "destroy_avp_list"
#5  0x000000000062fd9b in reset_avps () at usr_avp.c:665
        i = 0
#6  0x000000000051a018 in ksr_msg_env_reset () at receive.c:355
No locals.
#7  0x0000000000519a6c in receive_msg (buf=0x2986380 "REGISTER sip:sip.odorik.cz:6670 SIP/2.0\r\nVia: SIP/2.0/TLS 192.168.10.17:6670;rport;branch=z9hG4bK599132626\r\nFrom: \"2N Helios IP Vario\" <sip:695897 at sip.odorik.cz:6670>;tag=810833466\r\nTo: \"2N Helios IP "..., len=512, rcv_info=0x7f53edef3af0) at receive.c:317
        msg = 0x7f547101f540
        ctx = {rec_lev = 42567120, run_flags = 541, last_retcode = 7921624, jmp_env = {{__jmpbuf = {8589934643, 17179869185, 515396075521, 146028888066, 0, 3600, 42567120, 2}, __mask_was_saved = 3, __saved_mask = {__val = {139998450892749, 7, 139998450892752, 140727828041200, 6344480, 21474836480, 140000709226465, 43541376, 0, 65552, 140000711556760, 140000711556776, 1580774571, 65536, 519695341616, 65584}}}}}
        ret = 65536
        inb = {s = 0x2986380 "REGISTER sip:sip.odorik.cz:6670 SIP/2.0\r\nVia: SIP/2.0/TLS 192.168.10.17:6670;rport;branch=z9hG4bK599132626\r\nFrom: \"2N Helios IP Vario\" <sip:695897 at sip.odorik.cz:6670>;tag=810833466\r\nTo: \"2N Helios IP "..., len = 512}
        netinfo = {data = {s = 0x0, len = -88}, dst = 0x7ffdc0334d20, rcv = 0x7f54747d595c}
        __FUNCTION__ = "receive_msg"
#8  0x00000000006118c8 in receive_tcp_msg (tcpbuf=0x7f53edef3dd0 "REGISTER sip:sip.odorik.cz:6670 SIP/2.0\r\nVia: SIP/2.0/TLS 192.168.10.17:6670;rport;branch=z9hG4bK599132626\r\nFrom: \"2N Helios IP Vario\" <sip:695897 at sip.odorik.cz:6670>;tag=810833466\r\nTo: \"2N Helios IP "..., len=512, rcv_info=0x7f53edef3af0, con=0x7f53edef3ad8) at tcp_read.c:1306
        buf = 0x2986380 "REGISTER sip:sip.odorik.cz:6670 SIP/2.0\r\nVia: SIP/2.0/TLS 192.168.10.17:6670;rport;branch=z9hG4bK599132626\r\nFrom: \"2N Helios IP Vario\" <sip:695897 at sip.odorik.cz:6670>;tag=810833466\r\nTo: \"2N Helios IP "...
        bsize = 65535
        blen = 65535
        __FUNCTION__ = "receive_tcp_msg"
#9  0x000000000061390d in tcp_read_req (con=0x7f53edef3ad8, bytes_read=0x7ffdc03352b0, read_flags=0x7ffdc03352a8) at tcp_read.c:1474
        bytes = 512
        total_bytes = 512
        resp = 1
        size = 24
        req = 0x7f53edef3b58
        dst = {send_sock = 0x7ffdc03352a0, to = {s = {sa_family = 8, sa_data = "\000\000\000\000\000\000\001 \000\000\200%\002q"}, sin = {sin_family = 8, sin_port = 0, sin_addr = {s_addr = 0}, sin_zero = "\001 \000\000\200%\002q"}, sin6 = {sin6_family = 8, sin6_port = 0, sin6_flowinfo = 0, sin6_addr = {__in6_u = {__u6_addr8 = "\001 \000\000\200%\002qT\177\000\000\000\000\000", __u6_addr16 = {8193, 0, 9600, 28930, 32596, 0, 0, 0}, __u6_addr32 = {8193, 1895966080, 32596, 0}}}, sin6_scope_id = 7928126}}, id = 0, proto = 8 '\b', send_flags = {f = 0, blst_imask = 0}}
        c = 13 '\r'
        ret = -1
        __FUNCTION__ = "tcp_read_req"
#10 0x000000000061734f in handle_io (fm=0x7f5471022580, events=1, idx=-1) at tcp_read.c:1706
        ret = 0
        n = 0
        read_flags = 1
        con = 0x7f53edef3ad8
        s = 15
        resp = 1
        t = 1933361819
        __FUNCTION__ = "handle_io"
#11 0x00000000006079e1 in io_wait_loop_epoll (h=0xa622e0, t=2, repeat=0) at io_wait.h:1065
        n = 1
        r = 0
        fm = 0x7f5471022580
        revents = 1
        __FUNCTION__ = "io_wait_loop_epoll"
#12 0x00000000006186bb in tcp_receive_loop (unix_sock=54) at tcp_read.c:1818
        __FUNCTION__ = "tcp_receive_loop"
#13 0x00000000005fda26 in tcp_init_children () at tcp_main.c:4796
        r = 1
        i = -1
        reader_fd_1 = 54
        pid = 0
        si_desc = "tcp receiver (tls:81.31.45.51:6670)\000\000\177\000\000\377\017b\000\000\000\000\000\307\032y\000\000\000\000\000\001\000\000\000\377\377\377\377Xbxp\235I", '\000' <repeats 11 times>"\311, r\000\000\000\000\000\060\230A\000\000\000\000\000\340U3\300\000\000\000\000\310\005zpT\177\000\000$\000\000\000%\000\000\000\030Zs\000\003\000\000"
        si = 0x0
        __FUNCTION__ = "tcp_init_children"
#14 0x00000000004b6cf1 in main_loop () at main.c:1717
        i = 1
        pid = 18844
        si = 0x0
        si_desc = "udp receiver child=0 sock=81.31.43.196:53535\000\000\000\000\"\000\000\000\000\000\000\000\a", '\000' <repeats 15 times>, "`W\203tT\177\000\000\060\000\000\000\060\000\000\000\300}\344pT\177\000\000\260V3\300\375\177\000\000\003\000\000\000n\000\000\000\236\361t\000\000\000\000\000\305\337t\000\000\000\000"
        nrprocs = 1
        woneinit = 1
        __FUNCTION__ = "main_loop"
#15 0x00000000004bd357 in main (argc=10, argv=0x7ffdc0335a78) at main.c:2644
        cfg_stream = 0x2892010
        c = -1
        r = 0
        tmp = 0x7ffdc0336a0e ""
        tmp_len = 32596
        port = 1964126672
        proto = 0
        options = 0x743848 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:"
        ret = -1
        seed = 2005620363
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 0
        n_lst = 0x7ffdc033597e
        p = 0xf0b2ff <Address 0xf0b2ff out of bounds>
        st = {st_dev = 92, st_ino = 547488575, st_nlink = 2, st_mode = 16832, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1580722294, tv_nsec = 460542753}, st_mtim = {tv_sec = 1580772409, tv_nsec = 370439321}, st_ctim = {tv_sec = 1580772409, tv_nsec = 370439321}, __unused = {0, 0, 0}}
        __FUNCTION__ = "main"
No symbol table info available.
1847            int proto;
1848            char *options;
1849            int ret;
1850            unsigned int seed;
1851            int rfd;
1852            int debug_save, debug_flag;
1853            int dont_fork_cnt;
1854            struct name_lst* n_lst;
1855            char *p;
1856            struct stat st = {0};

---
file: /core.kamailio.sig6.20507

[New LWP 20507]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid -f /usr/local/etc/kamailio/ka'.
Program terminated with signal 6, Aborted.
#0  0x00007f0982fd2125 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#0  0x00007f0982fd2125 in raise () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1  0x00007f0982fd53a0 in abort () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#2  0x000000000063fce7 in fm_free (qmp=0x7f08f486e000, p=0x33312e3031403137, file=0x7f09748d4364 "tls: tls_init.c", func=0x7f09748d7463 "ser_free", line=308, mname=0x7f09748d4360 "tls") at mem/f_malloc.c:601
        qm = 0x7f08f486e000
        f = 0x63cad0
        __FUNCTION__ = "fm_free"
#3  0x00000000006453b2 in fm_shm_free (qmp=0x7f08f486e000, p=0x33312e3031403137, file=0x7f09748d4364 "tls: tls_init.c", func=0x7f09748d7463 "ser_free", line=308, mname=0x7f09748d4360 "tls") at mem/f_malloc.c:1139
No locals.
#4  0x00007f09748991b0 in ser_free (ptr=0x33312e3031403137) at tls_init.c:308
        __FUNCTION__ = "ser_free"
#5  0x00007f0977521b9d in CRYPTO_free () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
No symbol table info available.
#6  0x00007f09775ab607 in sk_free () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
No symbol table info available.
#7  0x00007f0977523489 in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
No symbol table info available.
#8  0x00007f09778dff7d in SSL_free () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
No symbol table info available.
#9  0x00007f09748bdbc3 in tls_h_tcpconn_clean (c=0x7f08fc36bc48) at tls_server.c:637
        extra = 0x7f08fc350390
        __FUNCTION__ = "tls_h_tcpconn_clean"
#10 0x00000000005d6fa5 in _tcpconn_free (c=0x7f08fc36bc48) at tcp_main.c:1325
        __FUNCTION__ = "_tcpconn_free"
#11 0x00000000005e8167 in tcpconn_put_destroy (tcpconn=0x7f08fc36bc48) at tcp_main.c:2960
        __FUNCTION__ = "tcpconn_put_destroy"
#12 0x00000000005eec9e in handle_ser_child (p=0x7f08fb2a3e5c, fd_i=-1) at tcp_main.c:3520
        tcpconn = 0x7f08fc36bc48
        tmp = 0xffffffff82e1d240
        response = {139676567911496, -1}
        cmd = -1
        bytes = 16
        ret = 1
        fd = -1
        flags = 0
        t = 0
        con_lifetime = 0
        nxt_timeout = 57680
        __FUNCTION__ = "handle_ser_child"
#13 0x00000000005f7211 in handle_io (fm=0x7f097f867c10, ev=1, idx=-1) at tcp_main.c:4208
        ret = 0
        __FUNCTION__ = "handle_io"
#14 0x00000000005c70bd in io_wait_loop_epoll (h=0xa1e040, t=5, repeat=0) at io_wait.h:1065
        n = 1
        r = 0
        fm = 0x7f097f867c10
        revents = 1
        __FUNCTION__ = "io_wait_loop_epoll"
#15 0x00000000005fa1ba in tcp_main_loop () at tcp_main.c:4490
        si = 0x0
        r = 8
        __FUNCTION__ = "tcp_main_loop"
#16 0x00000000004b700c in main_loop () at main.c:1725
        i = 1
        pid = 0
        si = 0x0
        si_desc = "udp receiver child=0 sock=81.31.43.196:53535\000\000\000\000\"\000\000\000\000\000\000\000\a", '\000' <repeats 15 times>, "`\247\a\203\t\177\000\000\060\000\000\000\060\000\000\000\230\321h\177\t\177\000\000\320\325\341\202\375\177\000\000\003\000\000\000n\000\000\000\236\361t\000\000\000\000\000\305\337t\000\000\000\000"
        nrprocs = 1
        woneinit = 1
        __FUNCTION__ = "main_loop"
#17 0x00000000004bd357 in main (argc=10, argv=0x7ffd82e1d998) at main.c:2644
        cfg_stream = 0x17bd010
        c = -1
        r = 0
        tmp = 0x7ffd82e1ea0e ""
        tmp_len = 32521
        port = -2087288368
        proto = 0
        options = 0x743848 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:"
        ret = -1
        seed = 3320641579
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 0
        n_lst = 0x7ffd82e1d89e
        p = 0xf0b2ff <Address 0xf0b2ff out of bounds>
        st = {st_dev = 92, st_ino = 547488575, st_nlink = 2, st_mode = 16832, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 80, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1580722294, tv_nsec = 460542753}, st_mtim = {tv_sec = 1580774467, tv_nsec = 482431332}, st_ctim = {tv_sec = 1580774467, tv_nsec = 482431332}, __unused = {0, 0, 0}}
        __FUNCTION__ = "main"
No symbol table info available.
1847            int proto;
1848            char *options;
1849            int ret;
1850            unsigned int seed;
1851            int rfd;
1852            int debug_save, debug_flag;
1853            int dont_fork_cnt;
1854            struct name_lst* n_lst;
1855            char *p;
1856            struct stat st = {0};
```

#### Log Messages

<!--
Check the syslog file and if there are relevant log messages printed by Kamailio, add them next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->
Debug log messages for each crash is attached as file.
[18855.log](https://github.com/kamailio/kamailio/files/4165356/18855.log)
[20507.log](https://github.com/kamailio/kamailio/files/4165357/20507.log)
[26294.log](https://github.com/kamailio/kamailio/files/4165359/26294.log)

```
(paste your log messages here)
```

#### SIP Traffic

<!--
If the issue is exposed by processing specific SIP messages, grab them with ngrep or save in a pcap file, then add them next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->

```
(paste your sip traffic here)
```

### Possible Solutions

<!--
If you found a solution or workaround for the issue, describe it. Ideally, provide a pull request with a fix.
-->

### Additional Information

  * **Kamailio Version** - output of `kamailio -v`

```
version: kamailio 4.4.7 (x86_64/linux) 97f308-dirty
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 32, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 24MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 97f308 -dirty
compiled on 15:50:12 Jan 14 2019 with gcc 4.7.2

```

* **Operating System**:

<!--
Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...;
Kernel details (output of `uname -a`)
-->

```
Linux serZ 2.6.32-042stab113.21 #1 SMP Wed Mar 23 11:05:25 MSK 2016 x86_64 GNU/Linux
```


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2213
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-dev/attachments/20200206/0050d08d/attachment-0001.html>


More information about the sr-dev mailing list