[SR-Users] Memory Leaks with Kamailio using SEAS module
ziad habchi
ziad.habchi at jinnysoftware.com
Tue Dec 3 08:42:28 CET 2013
Hi ,
Yes it generated a core file; the debuginfos are missing, how can I complie
kamailio without stripping them?
below is the output of bt full:
#0 0x00359bc1 in __strlen_ia32 () from /lib/libc.so.6
No symbol table info available.
#1 0x003244df in vfprintf () from /lib/libc.so.6
No symbol table info available.
#2 0x003beea6 in __vsyslog_chk () from /lib/libc.so.6
No symbol table info available.
#3 0x003bf027 in syslog () from /lib/libc.so.6
No symbol table info available.
#4 0x0819f0a2 in qm_status (qm=0xf52e8000) at mem/q_malloc.c:761
f = 0xf54ccfbc
i = 56
j = <value optimized out>
h = <value optimized out>
memlog = 1
mem_summary = <value optimized out>
__FUNCTION__ = "qm_status"
#5 0x081a1555 in qm_debug_frag (qm=0xf52e8000, f=<value optimized out>) at
mem/q_malloc.c:160
__FUNCTION__ = "qm_debug_frag"
#6 0x081a2820 in qm_free (qm=0xf52e8000, p=0xf54ccfd4, file=0xea2ac9 "kex:
pkg_stats.c", func=0xea2b60 "pkg_proc_stats_destroy", line=111) at
mem/q_malloc.c:462
f = 0xf54ccfbc
size = <value optimized out>
next = <value optimized out>
prev = <value optimized out>
__FUNCTION__ = "qm_free"
#7 0x00ea1ab6 in pkg_proc_stats_destroy () at pkg_stats.c:111
__FUNCTION__ = "pkg_proc_stats_destroy"
#8 0x00e9e157 in destroy () at kex_mod.c:170
No locals.
#9 0x0813aeac in destroy_modules () at sr_module.c:790
t = 0xf73287ec
foo = 0xf73287ec
__FUNCTION__ = "destroy_modules"
#10 0x080b4af0 in cleanup (show_status=1) at main.c:573
memlog = <value optimized out>
__FUNCTION__ = "cleanup"
#11 0x080b5af9 in shutdown_children (sig=<value optimized out>,
show_status=1) at main.c:715
__FUNCTION__ = "shutdown_children"
#12 0x080b5ff3 in handle_sigs () at main.c:745
chld = <value optimized out>
chld_status = <value optimized out>
memlog = <value optimized out>
__FUNCTION__ = "handle_sigs"
#13 0x080b7f57 in main_loop () at main.c:1767
i = <value optimized out>
pid = <value optimized out>
si = <value optimized out>
si_desc = "udp receiver child=3
sock=193.100.200.18:5070\000\063\367\030\311\341\001\001\000\000\000\234\266
L\365жL\365\v\020\000\000l\302\061\365\270\266L\365\270q\364\377\000\000\000
\000X\243\360\b\000\000\000\000\2---Type <return> to continue, or q <return>
to quit---
03\000\000\000X\243\360\b\001", '\000' <repeats 23 times>"\270, q\364\377"
nrprocs = 4
__FUNCTION__ = "main_loop"
#14 0x080bb57e in main (argc=5, argv=0xfff47364) at main.c:2566
cfg_stream = 0x8ebf008
c = <value optimized out>
r = 70188
tmp = 0x80512dd "__libc_start_main"
tmp_len = 4657636
port = 136523065
proto = -757064
ret = <value optimized out>
seed = 3203921318
rfd = 4
debug_save = 136559068
debug_flag = 0
dont_fork_cnt = 0
n_lst = 0x823b9dc
p = 0x804aea8 "E\005"
__FUNCTION__ = "main"
-----Original Message-----
From: sr-users-bounces at lists.sip-router.org
[mailto:sr-users-bounces at lists.sip-router.org] On Behalf Of
Daniel-Constantin Mierla
Sent: Monday, December 2, 2013 7:40 PM
To: Kamailio (SER) - Users Mailing List
Subject: Re: [SR-Users] Memory Leaks with Kamailio using SEAS module
Hello,
did you get a core file? If yes, send here the output for 'bt full'
taken in gdb.
Cheers,
Daniel
On 12/2/13 5:14 PM, zhabchi wrote:
> the following is the error i got when the client connection to SEAS exited
:
> tm [t_reply.c:604]: _reply_light(): ERROR: _reply_light: cannot
> allocate shmem buffer
>
> i recompiled Kamailio with MEMDBG=1 and here what i got
>
> <core> [mem/q_malloc.c:159]: qm_debug_frag(): BUG: qm_*: prev. fragm.
> tail overwritten(c0c0c000, abcdefed)[0xf7336d18:0xf7336d30]!
>
>
>
> I am running Kamailio with -m 2048 paramter
>
>
> zhabchi wrote
>> Dear Support,
>>
>>
>>
>> I am having an "out of memory" problem while using kamailio with SEAS
>> module.
>>
>> After using Kamailio with the SEAS module for few hours with a high
>> load , I am getting "out of memory" error that I believe is caused
>> from a memory leak.
>>
>> I would like your help interpreting this error.
>>
>>
>>
>> The output of kamailio -v is :
>>
>>
>>
>> version: kamailio 4.0.4 (i386/linux) cabe58
>>
>> flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS,
>> USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM,
>> SHM_MMAP, PKG_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT,
>> USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST,
>> HAVE_RESOLV_RES
>>
>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
>> MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB
>>
>> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
>>
>> id: cabe58
>>
>> compiled on 14:23:27 Dec 2 2013 with gcc 4.1.2
>>
>>
>>
>> kamctl ps :
>>
>>
>>
>> Process:: ID=0 PID=31191 Type=attendant
>>
>> Process:: ID=1 PID=31192 Type=udp receiver child=0
>> sock=193.100.200.18:5070
>>
>> Process:: ID=2 PID=31193 Type=udp receiver child=1
>> sock=193.100.200.18:5070
>>
>> Process:: ID=3 PID=31194 Type=udp receiver child=2
>> sock=193.100.200.18:5070
>>
>> Process:: ID=4 PID=31195 Type=udp receiver child=3
>> sock=193.100.200.18:5070
>>
>> Process:: ID=5 PID=31196 Type=slow timer
>>
>> Process:: ID=6 PID=31197 Type=timer
>>
>> Process:: ID=7 PID=31198 Type=ctl handler
>>
>> Process:: ID=8 PID=31199 Type=MI FIFO
>>
>> Process:: ID=9 PID=31200 Type=SEAS
>>
>> Process:: ID=10 PID=31201 Type=tcp receiver (generic) child=0
>>
>> Process:: ID=11 PID=31202 Type=tcp receiver (generic) child=1
>>
>> Process:: ID=12 PID=31203 Type=tcp receiver (generic) child=2
>>
>> Process:: ID=13 PID=31204 Type=tcp receiver (generic) child=3
>>
>> Process:: ID=14 PID=31205 Type=tcp main process
>>
>>
>>
>>
>>
>> kamcmd pkg.stats:
>>
>> {
>>
>> entry: 0
>>
>> pid: 31191
>>
>> rank: 0
>>
>> used: 72480
>>
>> free: 4091840
>>
>> real_used: 102448
>>
>> }
>>
>> {
>>
>> entry: 1
>>
>> pid: 31192
>>
>> rank: 1
>>
>> used: 80624
>>
>> free: 4083696
>>
>> real_used: 110592
>>
>> }
>>
>> {
>>
>> entry: 2
>>
>> pid: 31193
>>
>> rank: 2
>>
>> used: 80624
>>
>> free: 4083696
>>
>> real_used: 110592
>>
>> }
>>
>> {
>>
>> entry: 3
>>
>> pid: 31194
>>
>> rank: 3
>>
>> used: 80624
>>
>> free: 4083696
>>
>> real_used: 110592
>>
>> }
>>
>> {
>>
>> entry: 4
>>
>> pid: 31195
>>
>> rank: 4
>>
>> used: 80624
>>
>> free: 4083696
>>
>> real_used: 110592
>>
>> }
>>
>> {
>>
>> entry: 5
>>
>> pid: 31196
>>
>> rank: -1
>>
>> used: 2276560
>>
>> free: 4083744
>>
>> real_used: 17770256
>>
>> }
>>
>> {
>>
>> entry: 6
>>
>> pid: 31197
>>
>> rank: -1
>>
>> used: 1787296
>>
>> free: 4083744
>>
>> real_used: 17280992
>>
>> }
>>
>> {
>>
>> entry: 7
>>
>> pid: 31198
>>
>> rank: -2
>>
>> used: 77760
>>
>> free: 4086544
>>
>> real_used: 107744
>>
>> }
>>
>> {
>>
>> entry: 8
>>
>> pid: 0
>>
>> rank: 0
>>
>> used: 89920
>>
>> free: 4074352
>>
>> real_used: 119936
>>
>> }
>>
>> {
>>
>> entry: 9
>>
>> pid: 0
>>
>> rank: 0
>>
>> used: 0
>>
>> free: 0
>>
>> real_used: 0
>>
>> }
>>
>> {
>>
>> entry: 10
>>
>> pid: 31201
>>
>> rank: 5
>>
>> used: 140240
>>
>> free: 4023728
>>
>> real_used: 170560
>>
>> }
>>
>> {
>>
>> entry: 11
>>
>> pid: 31202
>>
>> rank: 6
>>
>> used: 140240
>>
>> free: 4023712
>>
>> real_used: 170576
>>
>> }
>>
>> {
>>
>> entry: 12
>>
>> pid: 31203
>>
>> rank: 7
>>
>> used: 140240
>>
>> free: 4023856
>>
>> real_used: 170432
>>
>> }
>>
>> {
>>
>> entry: 13
>>
>> pid: 31204
>>
>> rank: 8
>>
>> used: 140240
>>
>> free: 4023712
>>
>> real_used: 170576
>>
>> }
>>
>> {
>>
>> entry: 14
>>
>> pid: 31205
>>
>> rank: -4
>>
>> used: 3877344
>>
>> free: 4030176
>>
>> real_used: 19269056
>>
>> }
>>
>>
>>
>> As you can see above , the output of the SEAS module is not showing
>> (pid
>> 31200)
>>
>> Moreover , I can see that the real_used value in the kamcmd
>> core.shmmem keep on increasing , and free decrease until I finally
>> get an error "out of memory"
>>
>>
>>
>> When my SEAS client exit the output of kamcmd core.shmmem :
>>
>> {
>>
>> total: 33554432
>>
>> free: 16273440
>>
>> used: 1787296
>>
>> real_used: 17280992
>>
>> max_used: 33553520
>>
>> fragments: 8098
>>
>> }
>>
>>
>>
>> The traffic is a simple SIP MESSAGE from a seagull simulator, with a
>> 200OK reply from my SEAS client.
>>
>>
>>
>> I appreciate your support,
>>
>> Thank you in advance,
>>
>> Ziad Habchi
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing
>> list sr-users at .sip-router
>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>
>
>
>
> --
> View this message in context:
> http://sip-router.1086192.n5.nabble.com/Memory-Leaks-with-Kamailio-usi
> ng-SEAS-module-tp123410p123417.html
> Sent from the Users mailing list archive at Nabble.com.
>
> _______________________________________________
> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing
> list sr-users at lists.sip-router.org
> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users at lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2014.0.4259 / Virus Database: 3629/6886 - Release Date: 12/02/13
More information about the sr-users
mailing list