14 Sep
2023
14 Sep
'23
2:58 a.m.
### Description
As per https://github.com/kamailio/kamailio/issues/3563 all other o/s & package
details are the same
### Troubleshooting
#### Reproduction
#### Debugging Data
<!--
If you got a core dump, use gdb to extract troubleshooting data - full backtrace,
local variables and the list of the code at the issue location.
gdb /path/to/kamailio /path/to/corefile
bt full
info locals
list
If you are familiar with gdb, feel free to attach more of what you consider to
be relevant.
-->
```
PID: 2110 (kamailio)
UID: 990 (kamailio)
GID: 988 (kamailio)
Signal: 11 (SEGV)
Timestamp: Wed 2023-09-13 23:39:27 UTC (14min ago)
Command Line: /usr/sbin/kamailio -DD -P /run/kamailio/kamailio.pid -f
/etc/kamailio/kamailio.cfg -m 256 -M 8 --atexit=no
Executable: /usr/sbin/kamailio
Control Group: /system.slice/kamailio.service
Unit: kamailio.service
Slice: system.slice
Boot ID: 3fc4251c606f4c6183610573f44c46ba
Machine ID: 16e913e686424f13b970f9b1044cc6b3
Hostname: kamrtc01.voiceflex.com
Storage:
/var/lib/systemd/coredump/core.kamailio.990.3fc4251c606f4c6183610573f44c46ba.2110.1694648367000000.zst
(present)
Size on Disk: 3.6M
Message: Process 2110 (kamailio) of user 990 dumped core.
Stack trace of thread 2110:
#0 0x00007fe7ffb9f707 EVP_PKEY_CTX_free (libcrypto.so.3 + 0x19f707)
#1 0x00007fe7ffb796f9 EVP_MD_CTX_reset (libcrypto.so.3 + 0x1796f9)
#2 0x00007fe7ffb79722 EVP_MD_CTX_free (libcrypto.so.3 + 0x179722)
#3 0x00007fe80077faaa ssl3_free.part.0 (libssl.so.3 + 0x26aaa)
#4 0x00007fe80079312e SSL_free (libssl.so.3 + 0x3a12e)
#5 0x00007fe7ff3654a1 tls_h_tcpconn_clean_f (tls.so + 0x5f4a1)
#6 0x0000000000689b32 _tcpconn_free (kamailio + 0x289b32)
#7 0x0000000000689b87 _tcpconn_rm (kamailio + 0x289b87)
#8 0x00000000006b1175 tcpconn_destroy_all (kamailio + 0x2b1175)
#9 0x00000000006b3374 destroy_tcp (kamailio + 0x2b3374)
#10 0x000000000041f273 cleanup (kamailio + 0x1f273)
#11 0x0000000000420af1 shutdown_children (kamailio + 0x20af1)
#12 0x0000000000423a97 handle_sigs (kamailio + 0x23a97)
#13 0x00000000004308e1 main_loop (kamailio + 0x308e1)
#14 0x0000000000439969 main (kamailio + 0x39969)
#15 0x00007fe80103feb0 __libc_start_call_main (libc.so.6 + 0x3feb0)
#16 0x00007fe80103ff60 __libc_start_main@@GLIBC_2.34 (libc.so.6 +
0x3ff60)
#17 0x000000000041dc65 _start (kamailio + 0x1dc65)
ELF object binary architecture: AMD x86-64
GNU gdb (GDB) Red Hat Enterprise Linux 10.2-10.0.2.el9
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/kamailio...
Reading symbols from /usr/lib/debug/usr/sbin/kamailio-5.7.1-1.el9.x86_64.debug...
warning: Can't open file /dev/zero (deleted) during file-backed mapping note
processing
[New LWP 2110]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/sbin/kamailio -DD -P /run/kamailio/kamailio.pid -f
/etc/kamailio/kamailio.'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at
crypto/evp/pmeth_lib.c:415
415 if (ctx->pmeth && ctx->pmeth->cleanup)
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.34-60.0.2.el9.x86_64
libgcc-11.3.1-4.3.0.1.el9.x86_64 libstdc++-11.3.1-4.3.0.1.el9.x86_64
sssd-client-2.8.2-3.0.2.el9_2.x86_64
(gdb) bt full
#0 0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at
crypto/evp/pmeth_lib.c:415
No locals.
#1 0x00007fe7ffb796f9 in evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at
crypto/evp/digest.c:86
No locals.
#2 evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:75
No locals.
#3 EVP_MD_CTX_reset (ctx=ctx@entry=0x7fe7ef9c5038) at crypto/evp/digest.c:101
No locals.
#4 0x00007fe7ffb79722 in EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:140
No locals.
#5 EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:135
No locals.
#6 0x00007fe80077faaa in ssl3_free_digest_list (s=0x7fe7f01ce9d0) at ssl/s3_enc.c:338
No locals.
#7 ssl3_free (s=0x7fe7f01ce9d0) at ssl/s3_lib.c:3364
No locals.
#8 0x00007fe80079312e in SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1249
i = <optimized out>
i = <optimized out>
#9 SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1172
i = <optimized out>
#10 0x00007fe7ff3654a1 in tls_h_tcpconn_clean_f (c=0x7fe7f0308350) at tls_server.c:703
extra = 0x7fe7f0253838
__func__ = "tls_h_tcpconn_clean_f"
#11 0x0000000000689b32 in _tcpconn_free (c=0x7fe7f0308350) at core/tcp_main.c:1583
__func__ = "_tcpconn_free"
#12 0x0000000000689b87 in _tcpconn_rm (c=0x7fe7f0308350) at core/tcp_main.c:1594
No locals.
#13 0x00000000006b1175 in tcpconn_destroy_all () at core/tcp_main.c:4778
c = 0x7fe7f0308350
next = 0x0
h = 250
fd = -1
#14 0x00000000006b3374 in destroy_tcp () at core/tcp_main.c:4964
__func__ = "destroy_tcp"
#15 0x000000000041f273 in cleanup (show_status=1) at main.c:582
memlog = 0
__func__ = "cleanup"
#16 0x0000000000420af1 in shutdown_children (sig=15, show_status=1) at main.c:706
__func__ = "shutdown_children"
#17 0x0000000000423a97 in handle_sigs () at main.c:804
chld = 0
chld_status = 139
any_chld_stopped = 1
memlog = 0
__func__ = "handle_sigs"
#18 0x00000000004308e1 in main_loop () at main.c:1902
i = 4
pid = 2136
si = 0x0
si_desc = "udp receiver child=3
sock=172.16.128.55:5090\000\235\"r01Б\374\177\000\000\000\000\000\000\000\000\000\000\220\066Б\374\177\000\000\023e\204\000\000\000\000\000\033\000\000\000\000\000\000\000X_\212\000\350\177\000\000\000\000\000\020\000\000\000\000:|\024\001\350\177\000\000\060\000\000\000\060\000\000\000H2Б\374\177\000"
nrprocs = 4
woneinit = 1
__func__ = "main_loop"
#19 0x0000000000439969 in main (argc=11, argv=0x7ffc91d037a8) at main.c:3086
cfg_stream = 0x1bdf380
c = -1
r = 0
tmp = 0x7ffc91d03ea4 ""
tmp_len = 0
port = 0
proto = 0
ahost = 0x0
aport = 0
options = 0x8322d8
":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
ret = -1
seed = 4190893818
rfd = 4
debug_save = 0
debug_flag = 0
dont_fork_cnt = 2
n_lst = 0x200000064
p = 0x0
st = {st_dev = 24, st_ino = 891, st_nlink = 2, st_mode = 16832, st_uid = 990,
st_gid = 988, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0,
st_atim = {tv_sec = 1694643229, tv_nsec = 729000000}, st_mtim = {tv_sec = 1694644781,
tv_nsec = 493359816}, st_ctim = {tv_sec = 1694644781, tv_nsec = 493359816},
__glibc_reserved = {0, 0, 0}}
tbuf = "\331\\M\001\350\177\000\000\070\033J\001\350\177\000\000
4Б\374\177\000\000\a\000\000\000\000\000\000\000\330\bJ\001\350\177\000\000\a\000\000\000#\000\000\000\060jM\001\350\177\000\000\245\066K\001\350\177\000\000\000\000\000\000\000\000\000\000\220IK\001\350\177\000\000\070\000\001\001\350\177\000\000\000zI\001\350\177\000\000\000\000\000\000\000\000\000\000\360\063Б\374\177\000\000\360^M\001\350\177\000\000\330\032J\001\350\177\000\000\330\032J\001\350\177\000\000\000\000\000\000\000\000\000\000؈I\001\350\177\000\000\360\063Б\374\177\000\000\220uM\001\350\177\000\000\000\000J\001\350\177\000\000\022\fJ\001\350\177\000\000\066\001K\001\350\177\000\000\344#\000\001\350\177\000\000\272\230L\001"...
option_index = 12
long_options = {{name = 0x834976 "help", has_arg = 0, flag = 0x0, val =
104}, {name = 0x82f4f4 "version", has_arg = 0, flag = 0x0, val = 118}, {name =
0x83497b "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x834981
"subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x834987
"substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x834990
"substdefs", has_arg = 1, flag = 0x0, val = 1027}, {
name = 0x83499a "server-id", has_arg = 1, flag = 0x0, val = 1028},
{name = 0x8349a4 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name =
0x8349af "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x8349b8
"log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x8349c3
"debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x8349c9
"cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {
name = 0x8349d3 "atexit", has_arg = 1, flag = 0x0, val = 1034},
{name = 0x8349da "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name =
0x0, has_arg = 0, flag = 0x0, val = 0}}
__func__ = "main"
(gdb)
#0 0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at
crypto/evp/pmeth_lib.c:415
No locals.
#1 0x00007fe7ffb796f9 in evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at
crypto/evp/digest.c:86
No locals.
#2 evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:75
No locals.
#3 EVP_MD_CTX_reset (ctx=ctx@entry=0x7fe7ef9c5038) at crypto/evp/digest.c:101
No locals.
#4 0x00007fe7ffb79722 in EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:140
No locals.
#5 EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:135
No locals.
#6 0x00007fe80077faaa in ssl3_free_digest_list (s=0x7fe7f01ce9d0) at ssl/s3_enc.c:338
No locals.
#7 ssl3_free (s=0x7fe7f01ce9d0) at ssl/s3_lib.c:3364
No locals.
#8 0x00007fe80079312e in SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1249
i = <optimized out>
i = <optimized out>
#9 SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1172
i = <optimized out>
#10 0x00007fe7ff3654a1 in tls_h_tcpconn_clean_f (c=0x7fe7f0308350) at tls_server.c:703
extra = 0x7fe7f0253838
__func__ = "tls_h_tcpconn_clean_f"
#11 0x0000000000689b32 in _tcpconn_free (c=0x7fe7f0308350) at core/tcp_main.c:1583
__func__ = "_tcpconn_free"
#12 0x0000000000689b87 in _tcpconn_rm (c=0x7fe7f0308350) at core/tcp_main.c:1594
No locals.
#13 0x00000000006b1175 in tcpconn_destroy_all () at core/tcp_main.c:4778
c = 0x7fe7f0308350
next = 0x0
h = 250
fd = -1
#14 0x00000000006b3374 in destroy_tcp () at core/tcp_main.c:4964
__func__ = "destroy_tcp"
#15 0x000000000041f273 in cleanup (show_status=1) at main.c:582
memlog = 0
__func__ = "cleanup"
#16 0x0000000000420af1 in shutdown_children (sig=15, show_status=1) at main.c:706
__func__ = "shutdown_children"
#17 0x0000000000423a97 in handle_sigs () at main.c:804
chld = 0
chld_status = 139
any_chld_stopped = 1
memlog = 0
__func__ = "handle_sigs"
#18 0x00000000004308e1 in main_loop () at main.c:1902
i = 4
pid = 2136
si = 0x0
si_desc = "udp receiver child=3
sock=172.16.128.55:5090\000\235\"r01Б\374\177\000\000\000\000\000\000\000\000\000\000\220\066Б\374\177\000\000\023e\204\000\000\000\000\000\033\000\000\000\000\000\000\000X_\212\000\350\177\000\000\000\000\000\020\000\000\000\000:|\024\001\350\177\000\000\060\000\000\000\060\000\000\000H2Б\374\177\000"
nrprocs = 4
woneinit = 1
__func__ = "main_loop"
#19 0x0000000000439969 in main (argc=11, argv=0x7ffc91d037a8) at main.c:3086
cfg_stream = 0x1bdf380
c = -1
r = 0
tmp = 0x7ffc91d03ea4 ""
tmp_len = 0
port = 0
proto = 0
ahost = 0x0
aport = 0
options = 0x8322d8
":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
ret = -1
seed = 4190893818
rfd = 4
debug_save = 0
debug_flag = 0
dont_fork_cnt = 2
n_lst = 0x200000064
p = 0x0
st = {st_dev = 24, st_ino = 891, st_nlink = 2, st_mode = 16832, st_uid = 990,
st_gid = 988, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0,
st_atim = {tv_sec = 1694643229, tv_nsec = 729000000}, st_mtim = {tv_sec = 1694644781,
tv_nsec = 493359816}, st_ctim = {tv_sec = 1694644781, tv_nsec = 493359816},
__glibc_reserved = {0, 0, 0}}
tbuf = "\331\\M\001\350\177\000\000\070\033J\001\350\177\000\000
4Б\374\177\000\000\a\000\000\000\000\000\000\000\330\bJ\001\350\177\000\000\a\000\000\000#\000\000\000\060jM\001\350\177\000\000\245\066K\001\350\177\000\000\000\000\000\000\000\000\000\000\220IK\001\350\177\000\000\070\000\001\001\350\177\000\000\000zI\001\350\177\000\000\000\000\000\000\000\000\000\000\360\063Б\374\177\000\000\360^M\001\350\177\000\000\330\032J\001\350\177\000\000\330\032J\001\350\177\000\000\000\000\000\000\000\000\000\000؈I\001\350\177\000\000\360\063Б\374\177\000\000\220uM\001\350\177\000\000\000\000J\001\350\177\000\000\022\fJ\001\350\177\000\000\066\001K\001\350\177\000\000\344#\000\001\350\177\000\000\272\230L\001"...
option_index = 12
long_options = {{name = 0x834976 "help", has_arg = 0, flag = 0x0, val =
104}, {name = 0x82f4f4 "version", has_arg = 0, flag = 0x0, val = 118}, {name =
0x83497b "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x834981
"subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x834987
"substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x834990
"substdefs", has_arg = 1, flag = 0x0, val = 1027}, {
name = 0x83499a "server-id", has_arg = 1, flag = 0x0, val = 1028},
{name = 0x8349a4 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name =
0x8349af "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x8349b8
"log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x8349c3
"debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x8349c9
"cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {
name = 0x8349d3 "atexit", has_arg = 1, flag = 0x0, val = 1034},
{name = 0x8349da "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name =
0x0, has_arg = 0, flag = 0x0, val = 0}}
__func__ = "main"
(gdb)
#0 0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at
crypto/evp/pmeth_lib.c:415
No locals.
#1 0x00007fe7ffb796f9 in evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at
crypto/evp/digest.c:86
No locals.
#2 evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:75
No locals.
#3 EVP_MD_CTX_reset (ctx=ctx@entry=0x7fe7ef9c5038) at crypto/evp/digest.c:101
No locals.
#4 0x00007fe7ffb79722 in EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:140
No locals.
#5 EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:135
No locals.
#6 0x00007fe80077faaa in ssl3_free_digest_list (s=0x7fe7f01ce9d0) at ssl/s3_enc.c:338
No locals.
#7 ssl3_free (s=0x7fe7f01ce9d0) at ssl/s3_lib.c:3364
No locals.
#8 0x00007fe80079312e in SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1249
i = <optimized out>
i = <optimized out>
#9 SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1172
i = <optimized out>
#10 0x00007fe7ff3654a1 in tls_h_tcpconn_clean_f (c=0x7fe7f0308350) at tls_server.c:703
extra = 0x7fe7f0253838
__func__ = "tls_h_tcpconn_clean_f"
#11 0x0000000000689b32 in _tcpconn_free (c=0x7fe7f0308350) at core/tcp_main.c:1583
__func__ = "_tcpconn_free"
#12 0x0000000000689b87 in _tcpconn_rm (c=0x7fe7f0308350) at core/tcp_main.c:1594
No locals.
#13 0x00000000006b1175 in tcpconn_destroy_all () at core/tcp_main.c:4778
c = 0x7fe7f0308350
next = 0x0
h = 250
fd = -1
#14 0x00000000006b3374 in destroy_tcp () at core/tcp_main.c:4964
__func__ = "destroy_tcp"
#15 0x000000000041f273 in cleanup (show_status=1) at main.c:582
memlog = 0
__func__ = "cleanup"
#16 0x0000000000420af1 in shutdown_children (sig=15, show_status=1) at main.c:706
__func__ = "shutdown_children"
#17 0x0000000000423a97 in handle_sigs () at main.c:804
chld = 0
chld_status = 139
any_chld_stopped = 1
memlog = 0
__func__ = "handle_sigs"
#18 0x00000000004308e1 in main_loop () at main.c:1902
i = 4
pid = 2136
si = 0x0
si_desc = "udp receiver child=3
sock=172.16.128.55:5090\000\235\"r01Б\374\177\000\000\000\000\000\000\000\000\000\000\220\066Б\374\177\000\000\023e\204\000\000\000\000\000\033\000\000\000\000\000\000\000X_\212\000\350\177\000\000\000\000\000\020\000\000\000\000:|\024\001\350\177\000\000\060\000\000\000\060\000\000\000H2Б\374\177\000"
nrprocs = 4
woneinit = 1
__func__ = "main_loop"
#19 0x0000000000439969 in main (argc=11, argv=0x7ffc91d037a8) at main.c:3086
cfg_stream = 0x1bdf380
c = -1
r = 0
tmp = 0x7ffc91d03ea4 ""
tmp_len = 0
port = 0
proto = 0
ahost = 0x0
aport = 0
options = 0x8322d8
":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
ret = -1
seed = 4190893818
rfd = 4
debug_save = 0
debug_flag = 0
dont_fork_cnt = 2
n_lst = 0x200000064
p = 0x0
st = {st_dev = 24, st_ino = 891, st_nlink = 2, st_mode = 16832, st_uid = 990,
st_gid = 988, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0,
st_atim = {tv_sec = 1694643229, tv_nsec = 729000000}, st_mtim = {tv_sec = 1694644781,
tv_nsec = 493359816}, st_ctim = {tv_sec = 1694644781, tv_nsec = 493359816},
__glibc_reserved = {0, 0, 0}}
tbuf = "\331\\M\001\350\177\000\000\070\033J\001\350\177\000\000
4Б\374\177\000\000\a\000\000\000\000\000\000\000\330\bJ\001\350\177\000\000\a\000\000\000#\000\000\000\060jM\001\350\177\000\000\245\066K\001\350\177\000\000\000\000\000\000\000\000\000\000\220IK\001\350\177\000\000\070\000\001\001\350\177\000\000\000zI\001\350\177\000\000\000\000\000\000\000\000\000\000\360\063Б\374\177\000\000\360^M\001\350\177\000\000\330\032J\001\350\177\000\000\330\032J\001\350\177\000\000\000\000\000\000\000\000\000\000؈I\001\350\177\000\000\360\063Б\374\177\000\000\220uM\001\350\177\000\000\000\000J\001\350\177\000\000\022\fJ\001\350\177\000\000\066\001K\001\350\177\000\000\344#\000\001\350\177\000\000\272\230L\001"...
option_index = 12
long_options = {{name = 0x834976 "help", has_arg = 0, flag = 0x0, val =
104}, {name = 0x82f4f4 "version", has_arg = 0, flag = 0x0, val = 118}, {name =
0x83497b "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x834981
"subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x834987
"substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x834990
"substdefs", has_arg = 1, flag = 0x0, val = 1027}, {
name = 0x83499a "server-id", has_arg = 1, flag = 0x0, val = 1028},
{name = 0x8349a4 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name =
0x8349af "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x8349b8
"log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x8349c3
"debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x8349c9
"cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {
name = 0x8349d3 "atexit", has_arg = 1, flag = 0x0, val = 1034},
{name = 0x8349da "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name =
0x0, has_arg = 0, flag = 0x0, val = 0}}
__func__ = "main"
(gdb)
#0 0x00007fe7ffb9f707 in EVP_PKEY_CTX_free (ctx=0x100000001) at
crypto/evp/pmeth_lib.c:415
No locals.
#1 0x00007fe7ffb796f9 in evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at
crypto/evp/digest.c:86
No locals.
#2 evp_md_ctx_reset_ex (keep_fetched=0, ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:75
No locals.
#3 EVP_MD_CTX_reset (ctx=ctx@entry=0x7fe7ef9c5038) at crypto/evp/digest.c:101
No locals.
#4 0x00007fe7ffb79722 in EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:140
No locals.
#5 EVP_MD_CTX_free (ctx=0x7fe7ef9c5038) at crypto/evp/digest.c:135
No locals.
#6 0x00007fe80077faaa in ssl3_free_digest_list (s=0x7fe7f01ce9d0) at ssl/s3_enc.c:338
No locals.
#7 ssl3_free (s=0x7fe7f01ce9d0) at ssl/s3_lib.c:3364
No locals.
#8 0x00007fe80079312e in SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1249
i = <optimized out>
i = <optimized out>
#9 SSL_free (s=0x7fe7f01ce9d0) at ssl/ssl_lib.c:1172
i = <optimized out>
#10 0x00007fe7ff3654a1 in tls_h_tcpconn_clean_f (c=0x7fe7f0308350) at tls_server.c:703
extra = 0x7fe7f0253838
__func__ = "tls_h_tcpconn_clean_f"
#11 0x0000000000689b32 in _tcpconn_free (c=0x7fe7f0308350) at core/tcp_main.c:1583
__func__ = "_tcpconn_free"
#12 0x0000000000689b87 in _tcpconn_rm (c=0x7fe7f0308350) at core/tcp_main.c:1594
No locals.
#13 0x00000000006b1175 in tcpconn_destroy_all () at core/tcp_main.c:4778
c = 0x7fe7f0308350
next = 0x0
h = 250
fd = -1
#14 0x00000000006b3374 in destroy_tcp () at core/tcp_main.c:4964
__func__ = "destroy_tcp"
#15 0x000000000041f273 in cleanup (show_status=1) at main.c:582
memlog = 0
__func__ = "cleanup"
#16 0x0000000000420af1 in shutdown_children (sig=15, show_status=1) at main.c:706
__func__ = "shutdown_children"
#17 0x0000000000423a97 in handle_sigs () at main.c:804
chld = 0
chld_status = 139
any_chld_stopped = 1
memlog = 0
__func__ = "handle_sigs"
#18 0x00000000004308e1 in main_loop () at main.c:1902
i = 4
pid = 2136
si = 0x0
si_desc = "udp receiver child=3
sock=172.16.128.55:5090\000\235\"r01Б\374\177\000\000\000\000\000\000\000\000\000\000\220\066Б\374\177\000\000\023e\204\000\000\000\000\000\033\000\000\000\000\000\000\000X_\212\000\350\177\000\000\000\000\000\020\000\000\000\000:|\024\001\350\177\000\000\060\000\000\000\060\000\000\000H2Б\374\177\000"
nrprocs = 4
woneinit = 1
__func__ = "main_loop"
#19 0x0000000000439969 in main (argc=11, argv=0x7ffc91d037a8) at main.c:3086
cfg_stream = 0x1bdf380
c = -1
r = 0
tmp = 0x7ffc91d03ea4 ""
tmp_len = 0
port = 0
proto = 0
ahost = 0x0
aport = 0
options = 0x8322d8
":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
ret = -1
seed = 4190893818
rfd = 4
debug_save = 0
debug_flag = 0
dont_fork_cnt = 2
n_lst = 0x200000064
p = 0x0
st = {st_dev = 24, st_ino = 891, st_nlink = 2, st_mode = 16832, st_uid = 990,
st_gid = 988, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0,
st_atim = {tv_sec = 1694643229, tv_nsec = 729000000}, st_mtim = {tv_sec = 1694644781,
tv_nsec = 493359816}, st_ctim = {tv_sec = 1694644781, tv_nsec = 493359816},
__glibc_reserved = {0, 0, 0}}
tbuf = "\331\\M\001\350\177\000\000\070\033J\001\350\177\000\000
4Б\374\177\000\000\a\000\000\000\000\000\000\000\330\bJ\001\350\177\000\000\a\000\000\000#\000\000\000\060jM\001\350\177\000\000\245\066K\001\350\177\000\000\000\000\000\000\000\000\000\000\220IK\001\350\177\000\000\070\000\001\001\350\177\000\000\000zI\001\350\177\000\000\000\000\000\000\000\000\000\000\360\063Б\374\177\000\000\360^M\001\350\177\000\000\330\032J\001\350\177\000\000\330\032J\001\350\177\000\000\000\000\000\000\000\000\000\000؈I\001\350\177\000\000\360\063Б\374\177\000\000\220uM\001\350\177\000\000\000\000J\001\350\177\000\000\022\fJ\001\350\177\000\000\066\001K\001\350\177\000\000\344#\000\001\350\177\000\000\272\230L\001"...
option_index = 12
long_options = {{name = 0x834976 "help", has_arg = 0, flag = 0x0, val =
104}, {name = 0x82f4f4 "version", has_arg = 0, flag = 0x0, val = 118}, {name =
0x83497b "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x834981
"subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x834987
"substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x834990
"substdefs", has_arg = 1, flag = 0x0, val = 1027}, {
name = 0x83499a "server-id", has_arg = 1, flag = 0x0, val = 1028},
{name = 0x8349a4 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name =
0x8349af "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x8349b8
"log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x8349c3
"debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x8349c9
"cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {
name = 0x8349d3 "atexit", has_arg = 1, flag = 0x0, val = 1034},
{name = 0x8349da "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name =
0x0, has_arg = 0, flag = 0x0, val = 0}}
__func__ = "main"
```
#### Log Messages
This time it crashed immediately on startup after the crash reported in bug report #2
```
Sep 13 23:39:28 kamrtc01 kamailio[4142]: WARNING: <core> [core/daemonize.c:348]:
daemonize(): pid file contains old pid, replacing pid
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: rr [../outbound/api.h:52]:
ob_load_api(): unable to import bind_ob - maybe module is not loaded
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: rr [rr_mod.c:188]: mod_init():
outbound module not available
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: auth [auth_mod.c:345]:
mod_init(): qop set, but nonce-count (nonce_count) support disabled
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_mod.c:408]: mod_init():
use OpenSSL version: 30000070
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_mod.c:410]: mod_init():
With ECDH-Support!
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_mod.c:413]: mod_init():
With Diffie Hellman
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: siptrace [siptrace.c:428]:
mod_init(): can't load dlg api. Will not install dialog callbacks.
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: pike [pike.c:101]: pike_init():
PIKE - initializing
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: pike [ip_tree.c:81]:
init_lock_set(): probing 256 set size
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: NOTICE: regex [regex_mod.c:168]:
mod_init(): 'file' parameter is not set, group matching disabled
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: WARNING: tls [tls_init.c:949]:
tls_h_mod_init_f(): openssl bug #1491 (crash/mem leaks on low memory) workaround enabled
(on low memory tls operations will fail preemptively) with free memory thresholds 14155776
and 7077888 bytes
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core>
[core/cfg/cfg_ctx.c:599]: cfg_set_now(): tls.low_mem_threshold1 has been changed to
14155776
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core>
[core/cfg/cfg_ctx.c:599]: cfg_set_now(): tls.low_mem_threshold2 has been changed to
7077888
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core> [main.c:3063]:
main(): processes (at least): 27 - shm size: 268435456 - pkg size: 8388608
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core>
[core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992 for fd
10
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core>
[core/udp_server.c:207]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984 on fd 10
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core>
[core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992 for fd
11
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core>
[core/udp_server.c:207]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984 on fd 11
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core>
[core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992 for fd
12
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: <core>
[core/udp_server.c:207]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984 on fd 12
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:332]:
ksr_tls_fill_missing(): TLSs<default>: tls_method=25
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:344]:
ksr_tls_fill_missing(): TLSs<default>:
certificate='/etc/kamailio/tls/wss.pem'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:351]:
ksr_tls_fill_missing(): TLSs<default>: ca_list='(null)'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:358]:
ksr_tls_fill_missing(): TLSs<default>: ca_path='(null)'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:365]:
ksr_tls_fill_missing(): TLSs<default>: crl='(null)'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:368]:
ksr_tls_fill_missing(): TLSs<default>: require_certificate=0
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:376]:
ksr_tls_fill_missing(): TLSs<default>: cipher_list='(null)'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:383]:
ksr_tls_fill_missing(): TLSs<default>:
private_key='/etc/kamailio/tls/wss.pem'
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:386]:
ksr_tls_fill_missing(): TLSs<default>: verify_certificate=0
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:390]:
ksr_tls_fill_missing(): TLSs<default>: verify_depth=9
Sep 13 23:39:28 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:393]:
ksr_tls_fill_missing(): TLSs<default>: verify_client=0
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: NOTICE: tls [tls_domain.c:1148]:
ksr_tls_fix_domain(): registered server_name callback handler for socket [:0],
server_name='<default>' ...
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:754]:
set_verification(): TLSs<default>: No client certificate required and no checks
performed
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:332]:
ksr_tls_fill_missing(): TLSc<default>: tls_method=23
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:344]:
ksr_tls_fill_missing(): TLSc<default>: certificate='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:351]:
ksr_tls_fill_missing(): TLSc<default>: ca_list='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:358]:
ksr_tls_fill_missing(): TLSc<default>: ca_path='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:365]:
ksr_tls_fill_missing(): TLSc<default>: crl='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:368]:
ksr_tls_fill_missing(): TLSc<default>: require_certificate=1
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:376]:
ksr_tls_fill_missing(): TLSc<default>: cipher_list='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:383]:
ksr_tls_fill_missing(): TLSc<default>: private_key='(null)'
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:386]:
ksr_tls_fill_missing(): TLSc<default>: verify_certificate=1
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:390]:
ksr_tls_fill_missing(): TLSc<default>: verify_depth=9
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:393]:
ksr_tls_fill_missing(): TLSc<default>: verify_client=0
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4142]: INFO: tls [tls_domain.c:739]:
set_verification(): TLSc<default>: Server MUST present valid certificate
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4143]: INFO: rtpengine [rtpengine.c:2960]:
rtpp_test(): rtpengine instance <udp:127.0.0.1:2223> found, support for it enabled
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4159]: INFO: jsonrpcs [jsonrpcs_sock.c:460]:
jsonrpc_dgram_process(): a new child 0/4159
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4160]: INFO: ctl [io_listener.c:213]:
io_listen_loop(): io_listen_loop: using epoll_lt io watch method (config)
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4166]: CRITICAL: <core>
[core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f9973cc33f8),
called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) -
ignoring
Sep 13 23:39:29 kamrtc01 /usr/sbin/kamailio[4164]: CRITICAL: <core>
[core/mem/q_malloc.c:133]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f9973b78e70 (address
0x7f9973b78ea8) end overwritten (0, 0)! Memory allocator was called from tls:
tls_init.c:405. Fragment marked by tls: tls_init.c:360. Exec from
core/mem/q_malloc.c:511.
Sep 13 23:39:29 kamrtc01 systemd[1]: Started Process Core Dump (PID 4169/UID 0).
Sep 13 23:39:30 kamrtc01 systemd-coredump[4170]: Process 4164 (kamailio) of user 990
dumped core.#012#012Stack trace of thread 4164:#012#0 0x00007f99852a158c
__pthread_kill_implementation (libc.so.6 + 0xa158c)#012#1 0x00007f9985254d46 raise
(libc.so.6 + 0x54d46)#012#2 0x00007f99852287f3 abort (libc.so.6 + 0x287f3)#012#3
0x00000000007169ee qm_debug_check_frag (kamailio + 0x3169ee)#012#4 0x000000000071a703
qm_free (kamailio + 0x31a703)#012#5 0x0000000000725209 qm_shm_free (kamailio +
0x325209)#012#6 0x00007f998350d3d6 ser_free (tls.so + 0x313d6)#012#7 0x00007f9983d78b5c
cleanup_old_md_data (libcrypto.so.3 + 0x178b5c)#012#8 0x00007f9983d7965f
evp_md_ctx_clear_digest (libcrypto.so.3 + 0x17965f)#012#9 0x00007f9983d796d7
EVP_MD_CTX_reset (libcrypto.so.3 + 0x1796d7)#012#10 0x00007f9983d79722 EVP_MD_CTX_free
(libcrypto.so.3 + 0x179722)#012#11 0x00007f9983e7c557 rsa_freectx (libcrypto.so.3 +
0x27c557)#012#12 0x00007f9983d9b2a0 evp_pkey_ctx_free_old_ops (libcrypto.so.3 +
0x19b2a0)#012#13 0x00007f9983d9f72a EVP_PKEY_CTX_free (libcrypto.so.3 + 0x19f72a)#012#14
0x00007f9983d796f9 EVP_MD_CTX_reset (libcrypto.so.3 + 0x1796f9)#012#15 0x00007f9983d79722
EVP_MD_CTX_free (libcrypto.so.3 + 0x179722)#012#16 0x00007f99849c2778
tls_construct_cert_verify (libssl.so.3 + 0x69778)#012#17 0x00007f99849bb181
state_machine.part.0 (libssl.so.3 + 0x62181)#012#18 0x00007f9983535584 tls_accept (tls.so
+ 0x59584)#012#19 0x00007f99835408be tls_h_read_f (tls.so + 0x648be)#012#20
0x00000000006c79bc tcp_read_headers (kamailio + 0x2c79bc)#012#21 0x00000000006cfb94
tcp_read_req (kamailio + 0x2cfb94)#012#22 0x00000000006d4cc3 handle_io (kamailio +
0x2d4cc3)#012#23 0x00000000006c271d io_wait_loop_epoll (kamailio + 0x2c271d)#012#24
0x00000000006d7a47 tcp_receive_loop (kamailio + 0x2d7a47)#012#25 0x00000000006b7689
tcp_init_children (kamailio + 0x2b7689)#012#26 0x000000000042fae9 main_loop (kamailio +
0x2fae9)#012#27 0x0000000000439969 main (kamailio + 0x39969)#012#28 0x00007f998523feb0
__libc_start_call_main (libc.so.6 + 0x3feb0)#012#29 0x00007f998523ff60
__libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x3ff60)#012#30 0x000000000041dc65 _start
(kamailio + 0x1dc65)#012ELF object binary architecture: AMD x86-64
```
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/3565
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/3565(a)github.com>