[SR-Users] Memory Leak on DB Errors?

[Daniel-Constantin Mierla]

Hello,

On 10/4/11 12:27 PM, Klaus Darilion wrote:
> Meanwhile the server was restarted and the DB problems were fixed. As 
> it is a production server I can not reproduce anymore.

So, once it started it didn't recovered, continued always with that 
error? How much of shm did you configure?

You can try to attach from time to time to one process (can be even the 
main one to avoid blocking a sip worker) and walk through the shm 
allocated chunks, in order to see if there are some unexpected 
repetitions of allocation from same place in sources.

I posted the gdb script for walking through pkg at some point, the 
difference will be to start from the head of shm list (i.e., starting 
with shm_block->first_frag instead of mem_block->first_frag):

http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory#walking_through_pkg_with_gdb

You should go as much as possible to the end of the allocated list.

Another option is to add shm_status() (see cfgutils module) in your 
config, executed on a special request you can send with 
sipsak/udp_flood/sipp . There are other options if you load the cfg_rpc 
module and send some rpc commands with sercmd.

Cheers,
Daniel

>
> Sorry
> Klaus
>
> On 04.10.2011 12:24, Daniel-Constantin Mierla wrote:
>> Hello,
>>
>> sqlops is using pkg and tm shm, so they should not be directly related,
>> but maybe in the way config file works.
>>
>> Can you run it again with memlog lower than debug and see where the
>> allocated (not-freed) chunks were done? It should appear soon, not
>> waiting for out of mem message.
>>
>> If you haven't restarted, there is a way to attach with gdb and walk
>> through shm allocated chunks to spot the occurences.
>>
>> Cheers,
>> Daniel
>>
>>
>> On 10/4/11 12:08 PM, Klaus Darilion wrote:
>>> Hi!
>>>
>>> I recently had a problem with Kamailio 3.1.4 (provided Debian 
>>> packages):
>>>
>>> I had some DB problems (missing tables). Thus, the timer module failed
>>> to insert the statistics (for siremis):
>>>
>>> ERROR: db_mysql [km_dbase.c:120]: driver error on query: Table
>>> 'kamailio.statistics_tmx' doesn't exist
>>> ERROR: <core> [db_query.c:130]: error while submitting query
>>> ERROR: sqlops [sql_api.c:217]: cannot do the query
>>>
>>>
>>> This happened for some time (weeks?), other DB queries were unaffected.
>>>
>>> Then, suddenly Kamailio ran out of memory:
>>>
>>> ERROR: <core> [sip_msg_clone.c:506]: ERROR: sip_msg_cloner: cannot
>>> allocate memory
>>> ERROR: tm [t_lookup.c:1338]: ERROR: new_t: out of mem:
>>> ERROR: tm [t_lookup.c:1478]: ERROR: t_newtran: new_t failed
>>> ERROR: sl [sl_funcs.c:282]: ERROR: sl_reply_error used: I'm terribly
>>> sorry, server error occurred (1/SL)
>>>
>>>
>>> It do not think it is a load problem as the server is more or less
>>> idle. May it be a memory leak due to wrong error handling?
>>>
>>> Where there any fixes recently?
>>>
>>> Thanks
>>> Klaus
>>>
>>> _______________________________________________
>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
>>> sr-users at lists.sip-router.org
>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>
>
> _______________________________________________
> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
> sr-users at lists.sip-router.org
> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- 
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20111004/f2501ce5/attachment.htm>