Description

As per #3563 all other o/s & package details are the same

Troubleshooting

Reproduction

Debugging Data

           PID: 2110 (kamailio)
           UID: 990 (kamailio)
           GID: 988 (kamailio)
        Signal: 11 (SEGV)
     Timestamp: Wed 2023-09-13 23:39:27 UTC (14min ago)
  Command Line: /usr/sbin/kamailio -DD -P /run/kamailio/kamailio.pid -f /etc/kamailio/kamailio.cfg -m 256 -M 8 --atexit=no
    Executable: /usr/sbin/kamailio
 Control Group: /system.slice/kamailio.service
          Unit: kamailio.service
         Slice: system.slice
       Boot ID: 3fc4251c606f4c6183610573f44c46ba
    Machine ID: 16e913e686424f13b970f9b1044cc6b3
      Hostname: kamrtc01.voiceflex.com
       Storage: /var/lib/systemd/coredump/core.kamailio.990.3fc4251c606f4c6183610573f44c46ba.2110.1694648367000000.zst (present)
  Size on Disk: 3.6M
       Message: Process 2110 (kamailio) of user 990 dumped core.
                
                Stack trace of thread 2110:
                #0  0x00007fe7ffb9f707 EVP_PKEY_CTX_free (libcrypto.so.3 + 0x19f707)
                #1  0x00007fe7ffb796f9 EVP_MD_CTX_reset (libcrypto.so.3 + 0x1796f9)
                #2  0x00007fe7ffb79722 EVP_MD_CTX_free (libcrypto.so.3 + 0x179722)
                #3  0x00007fe80077faaa ssl3_free.part.0 (libssl.so.3 + 0x26aaa)
                #4  0x00007fe80079312e SSL_free (libssl.so.3 + 0x3a12e)
                #5  0x00007fe7ff3654a1 tls_h_tcpconn_clean_f (tls.so + 0x5f4a1)
                #6  0x0000000000689b32 _tcpconn_free (kamailio + 0x289b32)
                #7  0x0000000000689b87 _tcpconn_rm (kamailio + 0x289b87)
                #8  0x00000000006b1175 tcpconn_destroy_all (kamailio + 0x2b1175)
                #9  0x00000000006b3374 destroy_tcp (kamailio + 0x2b3374)
                #10 0x000000000041f273 cleanup (kamailio + 0x1f273)
                #11 0x0000000000420af1 shutdown_children (kamailio + 0x20af1)
                #12 0x0000000000423a97 handle_sigs (kamailio + 0x23a97)
                #13 0x00000000004308e1 main_loop (kamailio + 0x308e1)
                #14 0x0000000000439969 main (kamailio + 0x39969)
                #15 0x00007fe80103feb0 __libc_start_call_main (libc.so.6 + 0x3feb0)
                #16 0x00007fe80103ff60 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x3ff60)
                #17 0x000000000041dc65 _start (kamailio + 0x1dc65)
                ELF object binary architecture: AMD x86-64

GNU gdb (GDB) Red Hat Enterprise Linux 10.2-10.0.2.el9
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/kamailio...
Reading symbols from /usr/lib/debug/usr/sbin/kamailio-5.7.1-1.el9.x86_64.debug...

warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing
[New LWP 2110]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/sbin/kamailio -DD -P /run/kamailio/kamailio.pid -f /etc/kamailio/kamailio.'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at crypto/evp/pmeth_lib.c:415
415	    if (ctx->pmeth && ctx->pmeth->cleanup)
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.34-60.0.2.el9.x86_64 libgcc-11.3.1-4.3.0.1.el9.x86_64 libstdc++-11.3.1-4.3.0.1.el9.x86_64 sssd-client-2.8.2-3.0.2.el9_2.x86_64
(gdb) bt full
#0  0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at crypto/evp/pmeth_lib.c:415
No locals.
#1  0x00007fe7ffb796f9 in evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:86
No locals.
#2  evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:75
No locals.
#3  EVP_MD_CTX_reset (ctx=ctx@entry=0x7fe7ef9c5038) at crypto/evp/digest.c:101
No locals.
#4  0x00007fe7ffb79722 in EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:140
No locals.
#5  EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:135
No locals.
#6  0x00007fe80077faaa in ssl3_free_digest_list (s=0x7fe7f01ce9d0) at ssl/s3_enc.c:338
No locals.
#7  ssl3_free (s=0x7fe7f01ce9d0) at ssl/s3_lib.c:3364
No locals.
#8  0x00007fe80079312e in SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1249
        i = <optimized out>
        i = <optimized out>
#9  SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1172
        i = <optimized out>
#10 0x00007fe7ff3654a1 in tls_h_tcpconn_clean_f (c=0x7fe7f0308350) at tls_server.c:703
        extra = 0x7fe7f0253838
        __func__ = "tls_h_tcpconn_clean_f"
#11 0x0000000000689b32 in _tcpconn_free (c=0x7fe7f0308350) at core/tcp_main.c:1583
        __func__ = "_tcpconn_free"
#12 0x0000000000689b87 in _tcpconn_rm (c=0x7fe7f0308350) at core/tcp_main.c:1594
No locals.
#13 0x00000000006b1175 in tcpconn_destroy_all () at core/tcp_main.c:4778
        c = 0x7fe7f0308350
        next = 0x0
        h = 250
        fd = -1
#14 0x00000000006b3374 in destroy_tcp () at core/tcp_main.c:4964
        __func__ = "destroy_tcp"
#15 0x000000000041f273 in cleanup (show_status=1) at main.c:582
        memlog = 0
        __func__ = "cleanup"
#16 0x0000000000420af1 in shutdown_children (sig=15, show_status=1) at main.c:706
        __func__ = "shutdown_children"
#17 0x0000000000423a97 in handle_sigs () at main.c:804
        chld = 0
        chld_status = 139
        any_chld_stopped = 1
        memlog = 0
        __func__ = "handle_sigs"
#18 0x00000000004308e1 in main_loop () at main.c:1902
        i = 4
        pid = 2136
        si = 0x0
        si_desc = "udp receiver child=3 sock=172.16.128.55:5090\000\235\"r01Б\374\177\000\000\000\000\000\000\000\000\000\000\220\066Б\374\177\000\000\023e\204\000\000\000\000\000\033\000\000\000\000\000\000\000X_\212\000\350\177\000\000\000\000\000\020\000\000\000\000:|\024\001\350\177\000\000\060\000\000\000\060\000\000\000H2Б\374\177\000"
        nrprocs = 4
        woneinit = 1
        __func__ = "main_loop"
#19 0x0000000000439969 in main (argc=11, argv=0x7ffc91d037a8) at main.c:3086
        cfg_stream = 0x1bdf380
        c = -1
        r = 0
        tmp = 0x7ffc91d03ea4 ""
        tmp_len = 0
        port = 0
        proto = 0
        ahost = 0x0
        aport = 0
        options = 0x8322d8 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
        ret = -1
        seed = 4190893818
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 2
        n_lst = 0x200000064
        p = 0x0
        st = {st_dev = 24, st_ino = 891, st_nlink = 2, st_mode = 16832, st_uid = 990, st_gid = 988, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1694643229, tv_nsec = 729000000}, st_mtim = {tv_sec = 1694644781, tv_nsec = 493359816}, st_ctim = {tv_sec = 1694644781, tv_nsec = 493359816}, __glibc_reserved = {0, 0, 0}}
        tbuf = "\331\\M\001\350\177\000\000\070\033J\001\350\177\000\000 4Б\374\177\000\000\a\000\000\000\000\000\000\000\330\bJ\001\350\177\000\000\a\000\000\000#\000\000\000\060jM\001\350\177\000\000\245\066K\001\350\177\000\000\000\000\000\000\000\000\000\000\220IK\001\350\177\000\000\070\000\001\001\350\177\000\000\000zI\001\350\177\000\000\000\000\000\000\000\000\000\000\360\063Б\374\177\000\000\360^M\001\350\177\000\000\330\032J\001\350\177\000\000\330\032J\001\350\177\000\000\000\000\000\000\000\000\000\000؈I\001\350\177\000\000\360\063Б\374\177\000\000\220uM\001\350\177\000\000\000\000J\001\350\177\000\000\022\fJ\001\350\177\000\000\066\001K\001\350\177\000\000\344#\000\001\350\177\000\000\272\230L\001"...
        option_index = 12
        long_options = {{name = 0x834976 "help", has_arg = 0, flag = 0x0, val = 104}, {name = 0x82f4f4 "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x83497b "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x834981 "subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x834987 "substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x834990 "substdefs", has_arg = 1, flag = 0x0, val = 1027}, {
            name = 0x83499a "server-id", has_arg = 1, flag = 0x0, val = 1028}, {name = 0x8349a4 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name = 0x8349af "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x8349b8 "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x8349c3 "debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x8349c9 "cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {
            name = 0x8349d3 "atexit", has_arg = 1, flag = 0x0, val = 1034}, {name = 0x8349da "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}}
        __func__ = "main"
(gdb) 
#0  0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at crypto/evp/pmeth_lib.c:415
No locals.
#1  0x00007fe7ffb796f9 in evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:86
No locals.
#2  evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:75
No locals.
#3  EVP_MD_CTX_reset (ctx=ctx@entry=0x7fe7ef9c5038) at crypto/evp/digest.c:101
No locals.
#4  0x00007fe7ffb79722 in EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:140
No locals.
#5  EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:135
No locals.
#6  0x00007fe80077faaa in ssl3_free_digest_list (s=0x7fe7f01ce9d0) at ssl/s3_enc.c:338
No locals.
#7  ssl3_free (s=0x7fe7f01ce9d0) at ssl/s3_lib.c:3364
No locals.
#8  0x00007fe80079312e in SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1249
        i = <optimized out>
        i = <optimized out>
#9  SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1172
        i = <optimized out>
#10 0x00007fe7ff3654a1 in tls_h_tcpconn_clean_f (c=0x7fe7f0308350) at tls_server.c:703
        extra = 0x7fe7f0253838
        __func__ = "tls_h_tcpconn_clean_f"
#11 0x0000000000689b32 in _tcpconn_free (c=0x7fe7f0308350) at core/tcp_main.c:1583
        __func__ = "_tcpconn_free"
#12 0x0000000000689b87 in _tcpconn_rm (c=0x7fe7f0308350) at core/tcp_main.c:1594
No locals.
#13 0x00000000006b1175 in tcpconn_destroy_all () at core/tcp_main.c:4778
        c = 0x7fe7f0308350
        next = 0x0
        h = 250
        fd = -1
#14 0x00000000006b3374 in destroy_tcp () at core/tcp_main.c:4964
        __func__ = "destroy_tcp"
#15 0x000000000041f273 in cleanup (show_status=1) at main.c:582
        memlog = 0
        __func__ = "cleanup"
#16 0x0000000000420af1 in shutdown_children (sig=15, show_status=1) at main.c:706
        __func__ = "shutdown_children"
#17 0x0000000000423a97 in handle_sigs () at main.c:804
        chld = 0
        chld_status = 139
        any_chld_stopped = 1
        memlog = 0
        __func__ = "handle_sigs"
#18 0x00000000004308e1 in main_loop () at main.c:1902
        i = 4
        pid = 2136
        si = 0x0
        si_desc = "udp receiver child=3 sock=172.16.128.55:5090\000\235\"r01Б\374\177\000\000\000\000\000\000\000\000\000\000\220\066Б\374\177\000\000\023e\204\000\000\000\000\000\033\000\000\000\000\000\000\000X_\212\000\350\177\000\000\000\000\000\020\000\000\000\000:|\024\001\350\177\000\000\060\000\000\000\060\000\000\000H2Б\374\177\000"
        nrprocs = 4
        woneinit = 1
        __func__ = "main_loop"
#19 0x0000000000439969 in main (argc=11, argv=0x7ffc91d037a8) at main.c:3086
        cfg_stream = 0x1bdf380
        c = -1
        r = 0
        tmp = 0x7ffc91d03ea4 ""
        tmp_len = 0
        port = 0
        proto = 0
        ahost = 0x0
        aport = 0
        options = 0x8322d8 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
        ret = -1
        seed = 4190893818
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 2
        n_lst = 0x200000064
        p = 0x0
        st = {st_dev = 24, st_ino = 891, st_nlink = 2, st_mode = 16832, st_uid = 990, st_gid = 988, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1694643229, tv_nsec = 729000000}, st_mtim = {tv_sec = 1694644781, tv_nsec = 493359816}, st_ctim = {tv_sec = 1694644781, tv_nsec = 493359816}, __glibc_reserved = {0, 0, 0}}
        tbuf = "\331\\M\001\350\177\000\000\070\033J\001\350\177\000\000 4Б\374\177\000\000\a\000\000\000\000\000\000\000\330\bJ\001\350\177\000\000\a\000\000\000#\000\000\000\060jM\001\350\177\000\000\245\066K\001\350\177\000\000\000\000\000\000\000\000\000\000\220IK\001\350\177\000\000\070\000\001\001\350\177\000\000\000zI\001\350\177\000\000\000\000\000\000\000\000\000\000\360\063Б\374\177\000\000\360^M\001\350\177\000\000\330\032J\001\350\177\000\000\330\032J\001\350\177\000\000\000\000\000\000\000\000\000\000؈I\001\350\177\000\000\360\063Б\374\177\000\000\220uM\001\350\177\000\000\000\000J\001\350\177\000\000\022\fJ\001\350\177\000\000\066\001K\001\350\177\000\000\344#\000\001\350\177\000\000\272\230L\001"...
        option_index = 12
        long_options = {{name = 0x834976 "help", has_arg = 0, flag = 0x0, val = 104}, {name = 0x82f4f4 "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x83497b "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x834981 "subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x834987 "substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x834990 "substdefs", has_arg = 1, flag = 0x0, val = 1027}, {
            name = 0x83499a "server-id", has_arg = 1, flag = 0x0, val = 1028}, {name = 0x8349a4 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name = 0x8349af "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x8349b8 "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x8349c3 "debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x8349c9 "cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {
            name = 0x8349d3 "atexit", has_arg = 1, flag = 0x0, val = 1034}, {name = 0x8349da "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}}
        __func__ = "main"
(gdb) 
#0  0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at crypto/evp/pmeth_lib.c:415
No locals.
#1  0x00007fe7ffb796f9 in evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:86
No locals.
#2  evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:75
No locals.
#3  EVP_MD_CTX_reset (ctx=ctx@entry=0x7fe7ef9c5038) at crypto/evp/digest.c:101
No locals.
#4  0x00007fe7ffb79722 in EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:140
No locals.
#5  EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:135
No locals.
#6  0x00007fe80077faaa in ssl3_free_digest_list (s=0x7fe7f01ce9d0) at ssl/s3_enc.c:338
No locals.
#7  ssl3_free (s=0x7fe7f01ce9d0) at ssl/s3_lib.c:3364
No locals.
#8  0x00007fe80079312e in SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1249
        i = <optimized out>
        i = <optimized out>
#9  SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1172
        i = <optimized out>
#10 0x00007fe7ff3654a1 in tls_h_tcpconn_clean_f (c=0x7fe7f0308350) at tls_server.c:703
        extra = 0x7fe7f0253838
        __func__ = "tls_h_tcpconn_clean_f"
#11 0x0000000000689b32 in _tcpconn_free (c=0x7fe7f0308350) at core/tcp_main.c:1583
        __func__ = "_tcpconn_free"
#12 0x0000000000689b87 in _tcpconn_rm (c=0x7fe7f0308350) at core/tcp_main.c:1594
No locals.
#13 0x00000000006b1175 in tcpconn_destroy_all () at core/tcp_main.c:4778
        c = 0x7fe7f0308350
        next = 0x0
        h = 250
        fd = -1
#14 0x00000000006b3374 in destroy_tcp () at core/tcp_main.c:4964
        __func__ = "destroy_tcp"
#15 0x000000000041f273 in cleanup (show_status=1) at main.c:582
        memlog = 0
        __func__ = "cleanup"
#16 0x0000000000420af1 in shutdown_children (sig=15, show_status=1) at main.c:706
        __func__ = "shutdown_children"
#17 0x0000000000423a97 in handle_sigs () at main.c:804
        chld = 0
        chld_status = 139
        any_chld_stopped = 1
        memlog = 0
        __func__ = "handle_sigs"
#18 0x00000000004308e1 in main_loop () at main.c:1902
        i = 4
        pid = 2136
        si = 0x0
        si_desc = "udp receiver child=3 sock=172.16.128.55:5090\000\235\"r01Б\374\177\000\000\000\000\000\000\000\000\000\000\220\066Б\374\177\000\000\023e\204\000\000\000\000\000\033\000\000\000\000\000\000\000X_\212\000\350\177\000\000\000\000\000\020\000\000\000\000:|\024\001\350\177\000\000\060\000\000\000\060\000\000\000H2Б\374\177\000"
        nrprocs = 4
        woneinit = 1
        __func__ = "main_loop"
#19 0x0000000000439969 in main (argc=11, argv=0x7ffc91d037a8) at main.c:3086
        cfg_stream = 0x1bdf380
        c = -1
        r = 0
        tmp = 0x7ffc91d03ea4 ""
        tmp_len = 0
        port = 0
        proto = 0
        ahost = 0x0
        aport = 0
        options = 0x8322d8 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
        ret = -1
        seed = 4190893818
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 2
        n_lst = 0x200000064
        p = 0x0
        st = {st_dev = 24, st_ino = 891, st_nlink = 2, st_mode = 16832, st_uid = 990, st_gid = 988, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1694643229, tv_nsec = 729000000}, st_mtim = {tv_sec = 1694644781, tv_nsec = 493359816}, st_ctim = {tv_sec = 1694644781, tv_nsec = 493359816}, __glibc_reserved = {0, 0, 0}}
        tbuf = "\331\\M\001\350\177\000\000\070\033J\001\350\177\000\000 4Б\374\177\000\000\a\000\000\000\000\000\000\000\330\bJ\001\350\177\000\000\a\000\000\000#\000\000\000\060jM\001\350\177\000\000\245\066K\001\350\177\000\000\000\000\000\000\000\000\000\000\220IK\001\350\177\000\000\070\000\001\001\350\177\000\000\000zI\001\350\177\000\000\000\000\000\000\000\000\000\000\360\063Б\374\177\000\000\360^M\001\350\177\000\000\330\032J\001\350\177\000\000\330\032J\001\350\177\000\000\000\000\000\000\000\000\000\000؈I\001\350\177\000\000\360\063Б\374\177\000\000\220uM\001\350\177\000\000\000\000J\001\350\177\000\000\022\fJ\001\350\177\000\000\066\001K\001\350\177\000\000\344#\000\001\350\177\000\000\272\230L\001"...
        option_index = 12
        long_options = {{name = 0x834976 "help", has_arg = 0, flag = 0x0, val = 104}, {name = 0x82f4f4 "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x83497b "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x834981 "subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x834987 "substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x834990 "substdefs", has_arg = 1, flag = 0x0, val = 1027}, {
            name = 0x83499a "server-id", has_arg = 1, flag = 0x0, val = 1028}, {name = 0x8349a4 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name = 0x8349af "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x8349b8 "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x8349c3 "debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x8349c9 "cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {
            name = 0x8349d3 "atexit", has_arg = 1, flag = 0x0, val = 1034}, {name = 0x8349da "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}}
        __func__ = "main"
(gdb) 
#0  0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at crypto/evp/pmeth_lib.c:415
No locals.
#1  0x00007fe7ffb796f9 in evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:86
No locals.
#2  evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:75
No locals.
#3  EVP_MD_CTX_reset (ctx=ctx@entry=0x7fe7ef9c5038) at crypto/evp/digest.c:101
No locals.
#4  0x00007fe7ffb79722 in EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:140
No locals.
#5  EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:135
No locals.
#6  0x00007fe80077faaa in ssl3_free_digest_list (s=0x7fe7f01ce9d0) at ssl/s3_enc.c:338
No locals.
#7  ssl3_free (s=0x7fe7f01ce9d0) at ssl/s3_lib.c:3364
No locals.
#8  0x00007fe80079312e in SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1249
        i = <optimized out>
        i = <optimized out>
#9  SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1172
        i = <optimized out>
#10 0x00007fe7ff3654a1 in tls_h_tcpconn_clean_f (c=0x7fe7f0308350) at tls_server.c:703
        extra = 0x7fe7f0253838
        __func__ = "tls_h_tcpconn_clean_f"
#11 0x0000000000689b32 in _tcpconn_free (c=0x7fe7f0308350) at core/tcp_main.c:1583
        __func__ = "_tcpconn_free"
#12 0x0000000000689b87 in _tcpconn_rm (c=0x7fe7f0308350) at core/tcp_main.c:1594
No locals.
#13 0x00000000006b1175 in tcpconn_destroy_all () at core/tcp_main.c:4778
        c = 0x7fe7f0308350
        next = 0x0
        h = 250
        fd = -1
#14 0x00000000006b3374 in destroy_tcp () at core/tcp_main.c:4964
        __func__ = "destroy_tcp"
#15 0x000000000041f273 in cleanup (show_status=1) at main.c:582
        memlog = 0
        __func__ = "cleanup"
#16 0x0000000000420af1 in shutdown_children (sig=15, show_status=1) at main.c:706
        __func__ = "shutdown_children"
#17 0x0000000000423a97 in handle_sigs () at main.c:804
        chld = 0
        chld_status = 139
        any_chld_stopped = 1
        memlog = 0
        __func__ = "handle_sigs"
#18 0x00000000004308e1 in main_loop () at main.c:1902
        i = 4
        pid = 2136
        si = 0x0
        si_desc = "udp receiver child=3 sock=172.16.128.55:5090\000\235\"r01Б\374\177\000\000\000\000\000\000\000\000\000\000\220\066Б\374\177\000\000\023e\204\000\000\000\000\000\033\000\000\000\000\000\000\000X_\212\000\350\177\000\000\000\000\000\020\000\000\000\000:|\024\001\350\177\000\000\060\000\000\000\060\000\000\000H2Б\374\177\000"
        nrprocs = 4
        woneinit = 1
        __func__ = "main_loop"
#19 0x0000000000439969 in main (argc=11, argv=0x7ffc91d037a8) at main.c:3086
        cfg_stream = 0x1bdf380
        c = -1
        r = 0
        tmp = 0x7ffc91d03ea4 ""
        tmp_len = 0
        port = 0
        proto = 0
        ahost = 0x0
        aport = 0
        options = 0x8322d8 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
        ret = -1
        seed = 4190893818
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 2
        n_lst = 0x200000064
        p = 0x0
        st = {st_dev = 24, st_ino = 891, st_nlink = 2, st_mode = 16832, st_uid = 990, st_gid = 988, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1694643229, tv_nsec = 729000000}, st_mtim = {tv_sec = 1694644781, tv_nsec = 493359816}, st_ctim = {tv_sec = 1694644781, tv_nsec = 493359816}, __glibc_reserved = {0, 0, 0}}
        tbuf = "\331\\M\001\350\177\000\000\070\033J\001\350\177\000\000 4Б\374\177\000\000\a\000\000\000\000\000\000\000\330\bJ\001\350\177\000\000\a\000\000\000#\000\000\000\060jM\001\350\177\000\000\245\066K\001\350\177\000\000\000\000\000\000\000\000\000\000\220IK\001\350\177\000\000\070\000\001\001\350\177\000\000\000zI\001\350\177\000\000\000\000\000\000\000\000\000\000\360\063Б\374\177\000\000\360^M\001\350\177\000\000\330\032J\001\350\177\000\000\330\032J\001\350\177\000\000\000\000\000\000\000\000\000\000؈I\001\350\177\000\000\360\063Б\374\177\000\000\220uM\001\350\177\000\000\000\000J\001\350\177\000\000\022\fJ\001\350\177\000\000\066\001K\001\350\177\000\000\344#\000\001\350\177\000\000\272\230L\001"...
        option_index = 12
        long_options = {{name = 0x834976 "help", has_arg = 0, flag = 0x0, val = 104}, {name = 0x82f4f4 "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x83497b "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x834981 "subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x834987 "substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x834990 "substdefs", has_arg = 1, flag = 0x0, val = 1027}, {
            name = 0x83499a "server-id", has_arg = 1, flag = 0x0, val = 1028}, {name = 0x8349a4 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name = 0x8349af "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x8349b8 "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x8349c3 "debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x8349c9 "cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {
            name = 0x8349d3 "atexit", has_arg = 1, flag = 0x0, val = 1034}, {name = 0x8349da "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}}
        __func__ = "main"

Log Messages

This time it crashed immediately on startup after the crash reported in bug report #2

Sep 13 23:39:28 kamrtc01 kamailio[4142]: WARNING: <core> [core/daemonize.c:348]: daemonize(): pid file contains old pid, replacing pid
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: rr [../outbound/api.h:52]: ob_load_api(): unable to import bind_ob - maybe module is not loaded
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: rr [rr_mod.c:188]: mod_init(): outbound module not available
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: auth [auth_mod.c:345]: mod_init(): qop set, but nonce-count (nonce_count) support disabled
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_mod.c:408]: mod_init(): use OpenSSL version: 30000070
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_mod.c:410]: mod_init(): With ECDH-Support!
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_mod.c:413]: mod_init(): With Diffie Hellman
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: siptrace [siptrace.c:428]: mod_init(): can't load dlg api. Will not install dialog callbacks.
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: pike [pike.c:101]: pike_init(): PIKE - initializing
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: pike [ip_tree.c:81]: init_lock_set(): probing 256 set size
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: NOTICE: regex [regex_mod.c:168]: mod_init(): 'file' parameter is not set, group matching disabled
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: WARNING: tls [tls_init.c:949]: tls_h_mod_init_f(): openssl bug #1491 (crash/mem leaks on low memory) workaround enabled (on low memory tls operations will fail preemptively) with free memory thresholds 14155776 and 7077888 bytes
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [core/cfg/cfg_ctx.c:599]: cfg_set_now(): tls.low_mem_threshold1 has been changed to 14155776
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [core/cfg/cfg_ctx.c:599]: cfg_set_now(): tls.low_mem_threshold2 has been changed to 7077888
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [main.c:3063]: main(): processes (at least): 27 - shm size: 268435456 - pkg size: 8388608
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992 for fd 10
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [core/udp_server.c:207]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984 on fd 10
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992 for fd 11
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [core/udp_server.c:207]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984 on fd 11
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992 for fd 12
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [core/udp_server.c:207]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984 on fd 12
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:332]: ksr_tls_fill_missing(): TLSs<default>: tls_method=25
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:344]: ksr_tls_fill_missing(): TLSs<default>: certificate='/etc/kamailio/tls/wss.pem'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:351]: ksr_tls_fill_missing(): TLSs<default>: ca_list='(null)'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:358]: ksr_tls_fill_missing(): TLSs<default>: ca_path='(null)'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:365]: ksr_tls_fill_missing(): TLSs<default>: crl='(null)'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:368]: ksr_tls_fill_missing(): TLSs<default>: require_certificate=0
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:376]: ksr_tls_fill_missing(): TLSs<default>: cipher_list='(null)'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:383]: ksr_tls_fill_missing(): TLSs<default>: private_key='/etc/kamailio/tls/wss.pem'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:386]: ksr_tls_fill_missing(): TLSs<default>: verify_certificate=0
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:390]: ksr_tls_fill_missing(): TLSs<default>: verify_depth=9
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:393]: ksr_tls_fill_missing(): TLSs<default>: verify_client=0
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: NOTICE: tls [tls_domain.c:1148]: ksr_tls_fix_domain(): registered server_name callback handler for socket [:0], server_name='<default>' ...
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:754]: set_verification(): TLSs<default>: No client certificate required and no checks performed
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:332]: ksr_tls_fill_missing(): TLSc<default>: tls_method=23
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:344]: ksr_tls_fill_missing(): TLSc<default>: certificate='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:351]: ksr_tls_fill_missing(): TLSc<default>: ca_list='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:358]: ksr_tls_fill_missing(): TLSc<default>: ca_path='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:365]: ksr_tls_fill_missing(): TLSc<default>: crl='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:368]: ksr_tls_fill_missing(): TLSc<default>: require_certificate=1
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:376]: ksr_tls_fill_missing(): TLSc<default>: cipher_list='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:383]: ksr_tls_fill_missing(): TLSc<default>: private_key='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:386]: ksr_tls_fill_missing(): TLSc<default>: verify_certificate=1
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:390]: ksr_tls_fill_missing(): TLSc<default>: verify_depth=9
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:393]: ksr_tls_fill_missing(): TLSc<default>: verify_client=0
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:739]: set_verification(): TLSc<default>: Server MUST present valid certificate
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4143]: INFO: rtpengine [rtpengine.c:2960]: rtpp_test(): rtpengine instance <udp:127.0.0.1:2223> found, support for it enabled
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4159]: INFO: jsonrpcs [jsonrpcs_sock.c:460]: jsonrpc_dgram_process(): a new child 0/4159
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4160]: INFO: ctl [io_listener.c:213]: io_listen_loop(): io_listen_loop:  using epoll_lt io watch method (config)
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4166]: CRITICAL: <core> [core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f9973cc33f8), called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) - ignoring
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4164]: CRITICAL: <core> [core/mem/q_malloc.c:133]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f9973b78e70 (address 0x7f9973b78ea8) end overwritten (0, 0)! Memory allocator was called from tls: tls_init.c:405. Fragment marked by tls: tls_init.c:360. Exec from core/mem/q_malloc.c:511.
Sep 13 23:39:29 kamrtc01 systemd[1]: Started Process Core Dump (PID 4169/UID 0).
Sep 13 23:39:30 kamrtc01 systemd-coredump[4170]: Process 4164 (kamailio) of user 990 dumped core.#012#012Stack trace of thread 4164:#012#0  0x00007f99852a158c __pthread_kill_implementation (libc.so.6 + 0xa158c)#012#1  0x00007f9985254d46 raise (libc.so.6 + 0x54d46)#012#2  0x00007f99852287f3 abort (libc.so.6 + 0x287f3)#012#3  0x00000000007169ee qm_debug_check_frag (kamailio + 0x3169ee)#012#4  0x000000000071a703 qm_free (kamailio + 0x31a703)#012#5  0x0000000000725209 qm_shm_free (kamailio + 0x325209)#012#6  0x00007f998350d3d6 ser_free (tls.so + 0x313d6)#012#7  0x00007f9983d78b5c cleanup_old_md_data (libcrypto.so.3 + 0x178b5c)#012#8  0x00007f9983d7965f evp_md_ctx_clear_digest (libcrypto.so.3 + 0x17965f)#012#9  0x00007f9983d796d7 EVP_MD_CTX_reset (libcrypto.so.3 + 0x1796d7)#012#10 0x00007f9983d79722 EVP_MD_CTX_free (libcrypto.so.3 + 0x179722)#012#11 0x00007f9983e7c557 rsa_freectx (libcrypto.so.3 + 0x27c557)#012#12 0x00007f9983d9b2a0 evp_pkey_ctx_free_old_ops (libcrypto.so.3 + 0x19b2a0)#012#13 0x00007f9983d9f72a EVP_PKEY_CTX_free (libcrypto.so.3 + 0x19f72a)#012#14 0x00007f9983d796f9 EVP_MD_CTX_reset (libcrypto.so.3 + 0x1796f9)#012#15 0x00007f9983d79722 EVP_MD_CTX_free (libcrypto.so.3 + 0x179722)#012#16 0x00007f99849c2778 tls_construct_cert_verify (libssl.so.3 + 0x69778)#012#17 0x00007f99849bb181 state_machine.part.0 (libssl.so.3 + 0x62181)#012#18 0x00007f9983535584 tls_accept (tls.so + 0x59584)#012#19 0x00007f99835408be tls_h_read_f (tls.so + 0x648be)#012#20 0x00000000006c79bc tcp_read_headers (kamailio + 0x2c79bc)#012#21 0x00000000006cfb94 tcp_read_req (kamailio + 0x2cfb94)#012#22 0x00000000006d4cc3 handle_io (kamailio + 0x2d4cc3)#012#23 0x00000000006c271d io_wait_loop_epoll (kamailio + 0x2c271d)#012#24 0x00000000006d7a47 tcp_receive_loop (kamailio + 0x2d7a47)#012#25 0x00000000006b7689 tcp_init_children (kamailio + 0x2b7689)#012#26 0x000000000042fae9 main_loop (kamailio + 0x2fae9)#012#27 0x0000000000439969 main (kamailio + 0x39969)#012#28 0x00007f998523feb0 __libc_start_call_main (libc.so.6 + 0x3feb0)#012#29 0x00007f998523ff60 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x3ff60)#012#30 0x000000000041dc65 _start (kamailio + 0x1dc65)#012ELF object binary architecture: AMD x86-64


Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: <kamailio/kamailio/issues/3565@github.com>