You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
Experiencing an issue with QMalloc when Kamailio receives a lot of subscriptions (dialog-info) in a short period of time. The following is reported in logs and the TCP (TLS) connection fails:
Apr 4 18:06:04 ua-proxy-01 /usr/local/sbin/kamailio[28456]: ERROR: <core> [tcp_main.c:959]: tcpconn_new(): mem. allocation failure
Apr 4 18:06:04 ua-proxy-01 /usr/local/sbin/kamailio[28456]: ERROR: <core> [tcp_main.c:3977]: handle_new_connect(): tcpconn_new failed, closing socket
A snapshot of SHM usage from around the time of the connection is as follows:
shmem:fragments = 39604
shmem:free_size = 49473400
shmem:max_used_size = 67108696
shmem:real_used_size = 17635464
shmem:total_size = 67108864
shmem:used_size = 9423760
Kamailio -I output is as follows:
Print out of kamailio internals
Version: kamailio 4.3.4 (x86_64/linux) b2843e
Default config: /usr/local/etc/kamailio/kamailio.cfg
Default paths to modules: /usr/local/lib64/kamailio/modules
Compile flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
MAX_RECV_BUFFER_SIZE=262144
MAX_LISTEN=16
MAX_URI_SIZE=1024
BUF_SIZE=65535
DEFAULT PKG_SIZE=8MB
DEFAULT SHM_SIZE=64MB
ADAPTIVE_WAIT_LOOPS=1024
TCP poll methods: poll, epoll_lt, epoll_et, sigio_rt, select
Source code revision ID: b2843e
Compiled with: gcc 4.7.2
Compiled on: 17:56:04 Apr 4 2016
Thank you for flying kamailio!
Once it has gotten in this state, TCP connections will no longer work, regardless of how low the SHM gets. Other connections (UDP) and subscriptions (dialog-info) work correctly at this time. There are no other OOM errors in the logs. The only thing that will resolve the issue is to restart Kamailio.
Compiling to use FMalloc (MEMMNG=0 MEMDBG=1 make cfg) appears to resolve the issue. Kamailio -I output as follows:
Print out of kamailio internals
Version: kamailio 4.3.4 (x86_64/linux) b2843e
Default config: /usr/local/etc/kamailio/kamailio.cfg
Default paths to modules: /usr/local/lib64/kamailio/modules
Compile flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, DBG_F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
MAX_RECV_BUFFER_SIZE=262144
MAX_LISTEN=16
MAX_URI_SIZE=1024
BUF_SIZE=65535
DEFAULT PKG_SIZE=8MB
DEFAULT SHM_SIZE=64MB
ADAPTIVE_WAIT_LOOPS=1024
TCP poll methods: poll, epoll_lt, epoll_et, sigio_rt, select
Source code revision ID: b2843e
Compiled with: gcc 4.7.2
Compiled on: 18:07:19 Apr 4 2016
Thank you for flying kamailio!
At the time the TCP connection works correctly, the SHM usage is:
shmem:fragments = 40669
shmem:free_size = 51558136
shmem:max_used_size = 67107816
shmem:real_used_size = 15550728
shmem:total_size = 67108864
shmem:used_size = 10045520
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub