Hi,
I have a Kamailio 3.3.3 server that currently reports the following errors upon execution of t_relay() for certain initial INVITEs.
daemon.err /usr/sbin/kamailio[26916]: ERROR: <core> [sip_msg_clone.c:505]: ERROR: sip_msg_cloner: cannot allocate memory
daemon.err /usr/sbin/kamailio[26916]: ERROR: tm [t_lookup.c:1338]: ERROR: new_t: out of mem:
daemon.err /usr/sbin/kamailio[26917]: ERROR: tm [t_lookup.c:1478]: ERROR: t_newtran: new_t failed
I understand this error to be related to shared memory, based on [1]. However, sercmd reports normal shared memory utilization:
# sercmd core.shmmem
{
total: 67108864
free: 43189728
used: 20505784
real_used: 23919136
max_used: 65948568
fragments: 36887
}
These figures are quite similar to another server with the same configuration and Kamailio version, which is not reporting out-of-memory errors.
# sercmd core.shmmem
{
total: 67108864
free: 40219536
used: 23241952
real_used: 26889328
max_used: 66306288
fragments: 37832
}
I recently saw similar symptoms on a Kamailio 4.0.4 server, but Kamailio unfortunately crashed before I could get further details. In this case, I can consistently duplicate the problem with certain INVITEs. It seems that the failure is somehow related to the size of the SIP headers. For example, I have an INVITE from a Polycom phone which t_relay() routes successfully with 741 bytes of headers (total SIP length 1039 bytes). However, it seems that if I add one character to any of its SIP headers (742 bytes of headers; total SIP length 1039 bytes) the t_relay() will consistently fail.
This binary doesn't have DBG_QM_MALLOC configured (I am working on getting it re-compiled). However, since I don't know what caused the problem, I would still like to get as much information as I can out of the running system. I have attached the output of mi_statistics, as well as mem_dump_pkg for the external UDP receivers and the ctl handler. Are there any other diagnostics that could be useful?
-Andrew
[1] http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:faq#qwhat_does_errorsip_msg_clonercannot_allocate_memory_and_errornew_tout_of_memmean_and_how_do_i_fix_them
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users