Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
First, I apologize for the lack of detail here. I am unable to get a core dump and as a production box, I was unable to run in debug mode.
Using Kamailio 5.8.4 on Debian 12, every 60-80 minutes, the software would crash, blaming tls_wolfssl, such as:
2024-12-16T14:58:27.554497+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9084]: CRITICAL: <core> [core/mem/q_malloc.c:535]: qm_free(): BUG: bad pointer 0x1 (out of memory block!) called from tls_wolfssl: tls_init.c: ser_free(240) - ignoring
2024-12-16T14:58:27.554555+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9084]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f3440ba7560 (address 0x7f3440ba75a0) beginning overwritten (23)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by :29. Exec from core/mem/q_malloc.c:546.
2024-12-16T14:58:27.616558+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9089]: CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 103
2024-12-16T14:58:27.617684+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[8992]: ALERT: <core> [main.c:805]: handle_sigs(): child process 9084 exited by a signal 6
2024-12-16T14:58:27.618339+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[8992]: ALERT: <core> [main.c:809]: handle_sigs(): core was not generated
2024-12-16T14:58:27.618477+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[8992]: INFO: <core> [main.c:832]: handle_sigs(): terminating due to SIGCHLD
2024-12-16T14:58:27.618589+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9088]: INFO: <core> [main.c:888]: sig_usr(): signal 15 received
2024-12-16T14:57:52.029277+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21605]: CRITICAL: <core> [core/mem/q_malloc.c:535]: qm_free(): BUG: bad pointer 0x1 (out of memory block!) called from tls_wolfssl: tls_init.c: ser_free(240) - ignoring
2024-12-16T14:57:52.029347+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21605]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f085996b500 (address 0x7f085996b540) beginning overwritten (0)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from core/mem/q_malloc.c:546.
2024-12-16T14:57:52.048597+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21611]: CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 101
2024-12-16T14:57:52.048921+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21512]: ALERT: <core> [main.c:805]: handle_sigs(): child process 21605 exited by a signal 6
2024-12-16T14:57:52.048953+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21512]: ALERT: <core> [main.c:809]: handle_sigs(): core was not generated
2024-12-16T14:57:52.048975+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21512]: INFO: <core> [main.c:832]: handle_sigs(): terminating due to SIGCHLD
2024-12-16T14:57:52.048997+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21609]: INFO: <core> [main.c:888]: sig_usr(): signal 15 received
2024-12-16T16:21:35.967576+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[10591]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f442778d110 (address 0x7f442778d150) beginning overwritten (0)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from core/mem/q_malloc.c:546.
2024-12-16T13:31:55.995222+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7602]: CRITICAL: <core> [core/mem/q_malloc.c:555]: qm_free(): BUG: freeing already freed pointer (0x7f8e55830a70), called from tls_wolfssl: tls_init.c: ser_free(240), first free tls_wolfssl: tls_init.c: ser_malloc(228) - ignoring
2024-12-16T13:32:40.093954+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7603]: CRITICAL: <core> [core/mem/q_malloc.c:535]: qm_free(): BUG: bad pointer 0x7a100000000 (out of memory block!) called from tls_wolfssl: tls_init.c: ser_free(240) - ignoring
2024-12-16T13:32:40.094047+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7603]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f8e55762720 (address 0x7f8e55762760) beginning overwritten (0)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from core/mem/q_malloc.c:546.
2024-12-16T13:32:40.260913+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7504]: CRITICAL: <core> [core/mem/q_malloc.c:555]: qm_free(): BUG: freeing already freed pointer (0x7f8e55761a70), called from core: core/tcp_main.c: _tcpconn_free(1666), first free tls_wolfssl: tls_init.c: ser_free(240) - ignoring
2024-12-16T13:32:40.263520+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7504]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f8e55762720 (address 0x7f8e55762760) beginning overwritten (0)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from core/mem/q_malloc.c:546.
The system was handling ~900 TLS registrations with ~2700 presence SUBSCRIBE events.
Running the same config with tls.so does not result in crash.
version: kamailio 5.8.4 (x86_64/linux) 598105-dirty
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 598105 -dirty
compiled on 04:31:34 Dec 16 2024 with gcc 12.2.0
Distributor ID: Debian
Description: Debian GNU/Linux 12 (bookworm)
Release: 12
Codename: bookworm
Linux ip-172-16-0-52 6.1.0-28-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.119-1 (2024-11-22) x86_64 GNU/Linux
children=16
disable_tcp=no
tcp_connection_lifetime=3605
tcp_accept_no_cl=yes
tcp_max_connections=8192
tcp_defer_accept=yes
tcp_accept_unique=0
tcp_connection_lifetime=3605
tcp_connection_match=1
tcp_connect_timeout=10
tcp_crlf_ping=yes
tcp_linger2=1
tcp_keepalive=yes
tcp_reuse_port=yes
tcp_keepidle=20
tcp_keepintvl=15
tcp_rd_buf_size=65536
tcp_msg_data_timeout=40
tcp_msg_read_timeout=40
tcp_check_timer=20
tls_threads_mode=2
tls_max_connections=8192
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.