Hello,

On 10/4/11 12:27 PM, Klaus Darilion wrote:

Meanwhile the server was restarted and the DB problems were fixed. As it is a production server I can not reproduce anymore.

So, once it started it didn't recovered, continued always with that error? How much of shm did you configure?

You can try to attach from time to time to one process (can be even the main one to avoid blocking a sip worker) and walk through the shm allocated chunks, in order to see if there are some unexpected repetitions of allocation from same place in sources.

I posted the gdb script for walking through pkg at some point, the difference will be to start from the head of shm list (i.e., starting with shm_block->first_frag instead of mem_block->first_frag):

http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory#walking_through_pkg_with_gdb

You should go as much as possible to the end of the allocated list.

Another option is to add shm_status() (see cfgutils module) in your config, executed on a special request you can send with sipsak/udp_flood/sipp . There are other options if you load the cfg_rpc module and send some rpc commands with sercmd.

Cheers,
Daniel

Sorry
Klaus

On 04.10.2011 12:24, Daniel-Constantin Mierla wrote:

Hello,

sqlops is using pkg and tm shm, so they should not be directly related,
but maybe in the way config file works.

Can you run it again with memlog lower than debug and see where the
allocated (not-freed) chunks were done? It should appear soon, not
waiting for out of mem message.

If you haven't restarted, there is a way to attach with gdb and walk
through shm allocated chunks to spot the occurences.

Cheers,
Daniel

On 10/4/11 12:08 PM, Klaus Darilion wrote:

Hi!

I recently had a problem with Kamailio 3.1.4 (provided Debian packages):

I had some DB problems (missing tables). Thus, the timer module failed
to insert the statistics (for siremis):

ERROR: db_mysql [km_dbase.c:120]: driver error on query: Table
'kamailio.statistics_tmx' doesn't exist
ERROR: <core> [db_query.c:130]: error while submitting query
ERROR: sqlops [sql_api.c:217]: cannot do the query

This happened for some time (weeks?), other DB queries were unaffected.

Then, suddenly Kamailio ran out of memory:

ERROR: <core> [sip_msg_clone.c:506]: ERROR: sip_msg_cloner: cannot
allocate memory
ERROR: tm [t_lookup.c:1338]: ERROR: new_t: out of mem:
ERROR: tm [t_lookup.c:1478]: ERROR: t_newtran: new_t failed
ERROR: sl [sl_funcs.c:282]: ERROR: sl_reply_error used: I'm terribly
sorry, server error occurred (1/SL)

It do not think it is a load problem as the server is more or less
idle. May it be a memory leak due to wrong error handling?

Where there any fixes recently?

Thanks
Klaus

_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- 
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda