[SR-Users] Memory leak in 3.3.4

David Cunningham dcunningham at voisonics.com
Thu Oct 17 04:23:54 CEST 2013


Hi Daniel,

Configuration sent to you privately. Thanks.



On 16 October 2013 20:28, Daniel-Constantin Mierla <miconda at gmail.com>wrote:

>  Hello,
>
> indeed, it looks like a system memory leak, for what so ever reason the
> discussion was misled to pkg.
>
> Can you send me the list of modules you load (loadmodule lines) as well as
> the functions from your perl script that are related to kamailio internal
> functions or variables? You can send directly to me, without mailing list
> if there is something you want to protect from public eyes.
>
> Cheers,
> Daniel
>
>
> On 10/15/13 5:12 AM, David Cunningham wrote:
>
>   Hi Daniel,
>
>  Thanks for the reply again. Looking at the email history, I'm not sure
> how we decided it was definitely a pkg memory problem. What we see is the
> output of "ps aux" as follows:
>
> root at sip0-test:~# ps aux | egrep -i "kamailio|mem"
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> root      6794  0.0  0.0  22480  1868 ?        Ss   Oct02   0:12
> /opt/testuser/current/sbin/testuser_safe_kamailio
> testuser  6822  0.0  0.2 556528  4580 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6824  0.3  8.7 825552 180244 ?       S    Oct02  56:40
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6825  0.3  8.7 825536 180776 ?       S    Oct02  56:20
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6826  0.3  8.7 825912 180296 ?       S    Oct02  55:54
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6827  0.3  8.7 825744 180580 ?       S    Oct02  56:19
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6828  0.3  8.7 825536 180092 ?       S    Oct02  56:25
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6829  0.3  8.7 825536 180632 ?       S    Oct02  56:21
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6830  0.3  8.7 825472 180968 ?       S    Oct02  56:37
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6831  0.3  8.7 825276 180272 ?       S    Oct02  56:41
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6832  0.0  0.0 556528  1324 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6833  0.0  0.0 556528  1324 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6834  0.0  0.0 556528  1324 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6835  0.0  0.0 556528  1324 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6836  0.0  0.0 556528  1324 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6837  0.0  0.0 556528  1324 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6838  0.0  0.0 556528  1324 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6839  0.0  0.0 556528  1324 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6840  0.0  0.0 556528  1776 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6841  0.0  0.0 556528  1780 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6842  0.0  0.0 556528  1780 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6843  0.0  0.0 556528  1328 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6844  0.0  0.0 556528  1780 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6845  0.0  0.0 556528  1328 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6846  0.0  0.0 556528  1328 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6847  0.0  0.0 556528  1328 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6848  0.0  0.0 556528  1676 ?        S    Oct02   0:02
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6849  0.0  0.1 556528  3568 ?        S    Oct02   5:28
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6850  0.0  0.0 556612  1600 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6851  0.0  0.0 556532  1188 ?        S    Oct02   0:00
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
> testuser  6852  0.0  0.0 556528  1360 ?        S    Oct02   0:02
> /sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
>
>  You'll see for example that process 6824 is using 8.7% of memory, which
> is much more than it was using 5 days ago. Yet if I run the same sercmd
> again I get (exactly!) the same numbers:
>
> root at sip0-test:~# sercmd pkg.stats pid 6824
> {
>     entry: 1
>     pid: 6824
>     rank: 1
>     used: 209836
>     free: 3704080
>     real_used: 490224
> }
>
>  Any ideas?
>
>
>
> On 12 October 2013 00:23, Daniel-Constantin Mierla <miconda at gmail.com>wrote:
>
>> Hi David,
>>
>>
>> On 10/10/13 11:36 PM, David Cunningham wrote:
>>
>>> Hi Daniel,
>>>
>>>  Thanks for the reply. Perhaps what we're seeing is normal, and the
>>> memory use is meant to increase as time progresses. Would you expect to see
>>> an ongoing memory use increase, or when should it stop increasing?
>>>
>>>
>>>  private memory (pkg) should stay rather constant. It increases when
>> there is a sip message processed, but once is sent out, it should come back
>> around the average.
>>
>> There are couple of functions that can fill the private memory and keep
>> it up, such as doing an sql_query() that returns a big data and the result
>> is not freed (sql_result_free()). It is not actually a leak as the next
>> sql_query() will free previous result, but in case you have such query for
>> some corner case that is not executed frequently, then the memory can stay
>> filled in. Another example will be storing very large value in a $var(...)
>> (e.g., $var(x) ).
>>
>> This is private memory, per process, which is meant for temporary
>> operations. Shared memory (shm) can increase over the time, being the place
>> where the dynamic data required at runtime is stored (e.g., location
>> records, hash tables, transactions) - so as you get more traffic or more
>> phones using kamailio, more shm is used. But your problem was reported for
>> pkg.
>>
>> Anyhow, keep an eye on the pkg.stats and if you see constant increase
>> which is substantial, then get a mem log dump.
>>
>> Cheers,
>> Daniel
>>
>>
>> --
>> Daniel-Constantin Mierla - http://www.asipto.com
>> http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
>> Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
>>   - more details about Kamailio trainings at http://www.asipto.com -
>>
>>
>
>
> --
> David Cunningham, Voisonics
> http://voisonics.com/
> USA: +1 213 221 1092
> UK: +44 (0) 20 3298 1642
> Australia: +61 (0) 2 8063 9019
>
>
> --
> Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
> Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
>   - more details about Kamailio trainings at http://www.asipto.com -
>
>


-- 
David Cunningham, Voisonics
http://voisonics.com/
USA: +1 213 221 1092
UK: +44 (0) 20 3298 1642
Australia: +61 (0) 2 8063 9019
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20131017/000a1ff4/attachment-0001.html>


More information about the sr-users mailing list