Hello,

do you get errors like "pv_sprintf: ERROR: wrong format ..." from perl module? It looks like a leak in this error case.

If not, can you compile with MEMDBG=1, then set memlog in config less that debug and run/test it for a while?

Restart it and look at the logs at shut down to spot any memory leak. You can send it to me if you want me to analyze them.

I recommend you set children=2 or 4 to make the leak faster visible.

Also, you can decide what types of stats you want via:

http://www.kamailio.org/wiki/cookbooks/4.0.x/core#mem_summary

You can use value 12 to make the output more compact.

Cheers,
Daniel

On 9/30/13 4:25 PM, Dragos Oancea wrote:
Hi Daniel,


We are running Kamailio 4.0.3 . So it appears that yes, we have this commit.

Regards,
Dragos


From: Daniel-Constantin Mierla <miconda@gmail.com>
To: Dragos Oancea <droancea@yahoo.com>; Kamailio (SER) - Users Mailing List <sr-users@lists.sip-router.org>
Sent: Monday, September 30, 2013 4:06 PM
Subject: Re: [SR-Users] potential memory leak

Hello,

what version are you running? Is it after the commit:

http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=341f810dca0cc0596e22f2ac1bca86de0b8d142d ?

Cheers,
Daniel

On 9/30/13 3:59 PM, Dragos Oancea wrote:
Hello


We've put Kamailio on a machine with a SIP-Proxy role, we replaced another SIP proxy implementation.
We started with the default 4Mb of PKG Memory and 1024 Mb of Shared Memory but we had messages like these in the logs:

2013-09-26 13:48:26,939  ERROR: app_perl [kamailioxs.xs:371]: pv_sprintf(): pv_sprintf: Memory exhausted!
2013-09-26 13:48:50,267  ERROR: app_perl [kamailioxs.xs:371]: pv_sprintf(): pv_sprintf: Memory exhausted!
2013-09-26 13:48:51,822  ERROR: <core> [data_lump.c:197]: insert_subst_lump_after(): ERROR: insert_new_lump_after: out of memory
2013-09-26 13:48:51,823  ERROR: <core> [parser/contact/contact.c:194]: parse_contacts(): parse_contacts(): No memory left
2013-09-26 13:48:51,823  ERROR: memcached [../../parser/../ut.h:736]: pkg_str_dup(): could not allocate private memory from available pool
2013-09-26 13:48:51,823  ERROR: <core> [data_lump.c:373]: anchor_lump(): ERROR: anchor_lump: out of memory
2013-09-26 13:48:51,823  ERROR: <core> [dset.c:525]: rewrite_uri(): No memory left to rewrite r-uri
2013-09-26 13:48:51,823  ERROR: <core> [action.c:810]: do_action(): ERROR: do_action: memory allocation  failure
2013-09-26 13:48:51,823  ERROR: <core> [msg_translator.c:2164]: build_res_buf_from_sip_req(): ERROR: build_res_buf_from_sip_req: out of memory  ; needs 357
2013-09-26 13:48:51,823  ERROR: <core> [msg_translator.c:2164]: build_res_buf_from_sip_req(): ERROR: build_res_buf_from_sip_req: out of memory  ; needs 376


Then we started kamailio like below, with 1024Mb of private memory (per process) :

/opt/kamailio/sbin/kamailio -f /opt/kamailio/etc/kamailio//kamailio.cfg -w /opt/kamailio/core_dir/corefiles -P /opt/kamailio/var/run//kamailio.pid -m 1024 -M 1024 -u kamailio -g kamailio

Then after about 4 days we realized that we have a steady increase of used memory ( in the output of free command the '-/+ buffers/cache:' row  ), and we simply restarted the kamailio process.

The machine has 12 Gb of RAM , we run with children=32 .  1gb of memory per process x 32 child processes means that we are over the physical capacity of the machine.
The question is , are we in any danger of running out of memory on this box where we only run kamailio ? We don't want to reach the point where the kernel starts killing processes.
Shall we suspect some memory leak in the app_perl module ? (we do some http queries over SSL from a perl script)


Thank you.

Regards,
Dragos Oancea



_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- 
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
  - more details about Kamailio trainings at http://www.asipto.com -



-- 
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
  - more details about Kamailio trainings at http://www.asipto.com -