[SR-Users] Memory leak in 3.3.4
Daniel-Constantin Mierla
miconda at gmail.com
Wed Oct 2 09:17:06 CEST 2013
Hello,
ahh, right, forgot that it was disabled because it was not safe to use
at runtime for worker processes. You have to use:
kamcmd cfg.set_now_int core mem_dump_pkg <pid>
then wait for a bit so that process is receiving a SIP message.
Or, if you restart, the stats are printed in syslog for all processes.
Another alternative is to call pkg_status() or pkg_summary() from
cfgutils module in config file, like:
if(src_ip==127.0.0.1 && is_method("OPTIONS") && $rU=="pkgstats") {
send_reply("200", "OK");
pkg_summary();
exit;
}
You can use sipsak to send an options request to "sip:pkgstats at ..."
Cheers,
Daniel
On 10/2/13 2:30 AM, David Cunningham wrote:
> Hi Daniel,
>
> When I send SIGUSR1 to a worker process nothing is logged. Any ideas
> there? The following is the output of "sercmd pkg.stats".
>
> {
> entry: 0
> pid: 14570
> rank: 0
> used: 191100
> free: 3731504
> real_used: 462800
> }
> {
> entry: 1
> pid: 14571
> rank: 1
> used: 202656
> free: 3714236
> real_used: 480068
> }
> {
> entry: 2
> pid: 14572
> rank: 2
> used: 202720
> free: 3714220
> real_used: 480084
> }
> {
> entry: 3
> pid: 14573
> rank: 3
> used: 202664
> free: 3714324
> real_used: 479980
> }
> {
> entry: 4
> pid: 14574
> rank: 4
> used: 202744
> free: 3714100
> real_used: 480204
> }
> {
> entry: 5
> pid: 14575
> rank: 5
> used: 201908
> free: 3715176
> real_used: 479128
> }
> {
> entry: 6
> pid: 14576
> rank: 6
> used: 202572
> free: 3714368
> real_used: 479936
> }
> {
> entry: 7
> pid: 14577
> rank: 7
> used: 202740
> free: 3714488
> real_used: 479816
> }
> {
> entry: 8
> pid: 14578
> rank: 8
> used: 199760
> free: 3717084
> real_used: 477220
> }
> {
> entry: 9
> pid: 14579
> rank: 9
> used: 199440
> free: 3722828
> real_used: 471476
> }
> {
> entry: 10
> pid: 14580
> rank: 10
> used: 199440
> free: 3722828
> real_used: 471476
> }
> {
> entry: 11
> pid: 14581
> rank: 11
> used: 199440
> free: 3722828
> real_used: 471476
> }
> {
> entry: 12
> pid: 14582
> rank: 12
> used: 199440
> free: 3722828
> real_used: 471476
> }
> {
> entry: 13
> pid: 14583
> rank: 13
> used: 199440
> free: 3722828
> real_used: 471476
> }
> {
> entry: 14
> pid: 14584
> rank: 14
> used: 199440
> free: 3722828
> real_used: 471476
> }
> {
> entry: 15
> pid: 14585
> rank: 15
> used: 199440
> free: 3722828
> real_used: 471476
> }
> {
> entry: 16
> pid: 14586
> rank: 16
> used: 199440
> free: 3722828
> real_used: 471476
> }
> {
> entry: 17
> pid: 14587
> rank: -1
> used: 199400
> free: 3722868
> real_used: 471436
> }
> {
> entry: 18
> pid: 14588
> rank: -1
> used: 2105844
> free: 3722868
> real_used: 2337944
> }
> {
> entry: 19
> pid: 14589
> rank: -2
> used: 198552
> free: 3723236
> real_used: 471068
> }
> {
> entry: 20
> pid: 0
> rank: 0
> used: 209756
> free: 3712368
> real_used: 481936
> }
> {
> entry: 21
> pid: 14591
> rank: -4
> used: 253660
> free: 3668704
> real_used: 525600
> }
>
>
>
>
> On 1 October 2013 22:45, Daniel-Constantin Mierla <miconda at gmail.com
> <mailto:miconda at gmail.com>> wrote:
>
> Hello,
>
> send SIGUSR1 to one of the worker processes (e.g., UDP listener).
> The main process of kamailio doesn't process SIP messages, so it
> is not exposed to leaks.
>
> You can used 'kamcmd pkg.stats" to see statistics about usage of
> pkg per each process (older versions have sercmd).
>
> 'kamctl ps' gives the list of processes.
>
> Cheers,
> Daniel
>
>
> On 10/1/13 2:35 PM, David Cunningham wrote:
>> Hi Daniel,
>>
>> What I copied and pasted is all that's listed when I do a kill
>> -SIGUSR1 on the Kamailio master process. How do I get fuller logs?
>>
>> Thanks for your help.
>>
>>
>>
>> On 1 October 2013 21:57, Daniel-Constantin Mierla
>> <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>>
>> Hello,
>>
>> you gave only the first part of memory log output. The chunks
>> listed there are from startup, thus not part of a leak,
>> because they are not allocated again at runtime.
>>
>> Can you give the full logs with messages from qm_status? The
>> leaks are typically listed towards the end of those messages.
>>
>> Yesterday someone else reported a potential leak when using
>> perl module, so it might be something else there. I found
>> one, which is related to an error case, but perhaps you don't
>> get that, otherwise you will notice the error message.
>>
>> Cheers,
>> Daniel
>>
>>
>> On 10/1/13 12:03 PM, David Cunningham wrote:
>>
>> Hello,
>>
>> Following up on this after a long break. It seems we are
>> still having a memory leak. We've added the memory
>> debugging options as per the documentation with the
>> following output.
>> I'm not sure how to read it though - can anyone help?
>> Thank you.
>>
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> <core> [main.c:754]: Memory status (pkg):
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: (0x4019b008):
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: heap size= 4194304
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: used= 191100, used+overhead=462800, free=3731504
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: max used (+overhead)= 480224
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: dumping all alloc'ed. fragments:
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 0. N address=0x401cf414 frag=0x401cf3fc
>> size=512 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: str_hash.h:
>> str_hash_alloc(69)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 1. N address=0x401cf644 frag=0x401cf62c
>> size=128 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: str_hash.h:
>> str_hash_alloc(69)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 2. N address=0x401cf6f4 frag=0x401cf6dc
>> size=256 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: counters.c:
>> cnt_hash_add(331)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 3. N address=0x401cf824 frag=0x401cf80c
>> size=64 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: counters.c:
>> init_counters(121)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 4. N address=0x401cf894 frag=0x401cf87c
>> size=8 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: route.c:
>> init_rlist(180)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 5. N address=0x401cf8cc frag=0x401cf8b4
>> size=64 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: str_hash.h:
>> str_hash_alloc(69)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 6. N address=0x401cf93c frag=0x401cf924
>> size=24 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: route.c:
>> route_add(159)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 7. N address=0x401cf984 frag=0x401cf96c
>> size=8 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: route.c:
>> init_rlist(180)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 8. N address=0x401cf9bc frag=0x401cf9a4
>> size=64 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: str_hash.h:
>> str_hash_alloc(69)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 9. N address=0x401cfa2c frag=0x401cfa14
>> size=24 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: route.c:
>> route_add(159)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 10. N address=0x401cfa74 frag=0x401cfa5c
>> size=8 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: db_id.c:
>> dupl_string(50)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 11. N address=0x401cfaac frag=0x401cfa94
>> size=64 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: str_hash.h:
>> str_hash_alloc(69)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 12. N address=0x401cfb1c frag=0x401cfb04
>> size=24 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: route.c:
>> route_add(159)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 13. N address=0x401cfb64 frag=0x401cfb4c
>> size=8 used=1
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: alloc'd from <core>: route.c:
>> init_rlist(180)
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: start check=f0f0f0f0, end check=
>> c0c0c0c0, abcdefed
>> Oct 1 05:58:48 sip0-test /sbin/kamailio[14570]: NOTICE:
>> qm_status: 14. N address=0x401cfb9c frag=0x401cfb84
>> size=64 used=1
>>
>>
>>
>> --
>> Daniel-Constantin Mierla - http://www.asipto.com
>> http://twitter.com/#!/miconda
>> <http://twitter.com/#%21/miconda> -
>> http://www.linkedin.com/in/miconda
>> Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov
>> 18-20, 2013
>> - more details about Kamailio trainings at
>> http://www.asipto.com -
>>
>>
>>
>>
>> --
>> David Cunningham, Voisonics
>> http://voisonics.com/
>> USA: +1 213 221 1092 <tel:%2B1%20213%20221%201092>
>> UK: +44 (0) 20 3298 1642 <tel:%2B44%20%280%29%2020%203298%201642>
>> Australia: +61 (0) 2 8063 9019
>> <tel:%2B61%20%280%29%202%208063%209019>
>
> --
> Daniel-Constantin Mierla -http://www.asipto.com
> http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> -http://www.linkedin.com/in/miconda
> Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
> - more details about Kamailio trainings athttp://www.asipto.com -
>
>
>
>
> --
> David Cunningham, Voisonics
> http://voisonics.com/
> USA: +1 213 221 1092
> UK: +44 (0) 20 3298 1642
> Australia: +61 (0) 2 8063 9019
--
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
- more details about Kamailio trainings at http://www.asipto.com -
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20131002/828d789f/attachment-0001.html>
More information about the sr-users
mailing list