[SR-Users] Kamailio consumes memory

Martin Tichatschek tichatschek at web.de
Fri Mar 2 13:01:39 CET 2012


Hi everyone,

I have a problem with Kamailio consuming memory. I started testing with 
Kamailio 3.2.1 upgraded to 3.2.2 and also tested 3.1.5, everywhere the 
same problem. Whereas Kamailio 1.5.0 is working fine.
Kamailio -V shows the following:

	kamailio at test:~$ kamailio -V
	version: kamailio 3.2.2 (i386/linux) 98ba92
	flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
	ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB
	poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
	id: 98ba92
	compiled on 17:51:30 Feb 29 2012 with gcc 4.2.4
	kamailio at test:~$

My test scenario consists of two SIPp servers (a UAC and a UAS) and the 
Kamailio in the middle proxying the SIP traffic.
The UAC sends 300 calls per second (INVITE, 100, 180, 200, ACK) and 
releases the calls (BYE, 200) after 20 sec call duration. UAS is simply 
sending back the right responds. Protocol is UDP.

After some minutes the Kamailio floods the syslog file with error 
messages indicating that he is running out of memory. The SIPp shows 
retransmissions and also timeouts. When I stop the test and do a single 
call I get some lines of error messages. I found three different sets of 
error messages for a single call so far:

	Mar  2 09:35:44 test P-CSCFtest[21251]: ERROR:<core>  [parser/msg_parser.c:172]: get_hdr_field: out of memory
	Mar  2 09:35:44 test P-CSCFtest[21251]: INFO:<core>  [parser/msg_parser.c:353]: ERROR: bad header field [To:<sip:00498111111]
	Mar  2 09:35:44 test P-CSCFtest[21251]: ERROR: tm [t_lookup.c:1118]: ERROR: reply cannot be parsed
	Mar  2 09:35:44 test P-CSCFtest[21251]: ERROR:<core>  [msg_translator.c:1884]: ERROR: build_res_buf_from_sip_res: out of mem
	Mar  2 09:35:44 test P-CSCFtest[21251]: ERROR:<core>  [forward.c:794]: ERROR: forward_reply: building failed
	Mar  2 09:35:44 test P-CSCFtest[21228]: ERROR:<core>  [msg_translator.c:1884]: ERROR: build_res_buf_from_sip_res: out of mem
	Mar  2 09:35:44 test P-CSCFtest[21228]: ERROR: tm [t_reply.c:1786]: ERROR: relay_reply: no mem for outbound reply buffer
	Mar  2 09:35:44 test P-CSCFtest[21228]: ERROR:<core>  [msg_translator.c:2036]: ERROR: build_res_buf_from_sip_req: out of memory  ; needs 311

	Mar  2 10:19:43 test P-CSCFtest[21346]: ERROR:<core>  [msg_translator.c:1884]: ERROR: build_res_buf_from_sip_res: out of mem
	Mar  2 10:19:43 test P-CSCFtest[21346]: ERROR: tm [t_reply.c:1786]: ERROR: relay_reply: no mem for outbound reply buffer
	Mar  2 10:19:43 test P-CSCFtest[21346]: ERROR:<core>  [msg_translator.c:2036]: ERROR: build_res_buf_from_sip_req: out of memory  ; needs 311

	Mar  2 10:20:16 test P-CSCFtest[21328]: ERROR:<core>  [action.c:852]: ERROR: do_action: memory allocation  failure
	Mar  2 10:20:16 test P-CSCFtest[21328]: ERROR:<core>  [action.c:1568]: run action error at: /usr/local/etc/kamailio//kamailio.cfg:28
	Mar  2 10:20:16 test P-CSCFtest[21328]: ERROR:<core>  [action.c:852]: ERROR: do_action: memory allocation  failure
	Mar  2 10:20:16 test P-CSCFtest[21328]: ERROR:<core>  [action.c:1568]: run action error at: /usr/local/etc/kamailio//kamailio.cfg:32
	Mar  2 10:20:16 test P-CSCFtest[21328]: ERROR:<core>  [parser/parse_from.c:71]: ERROR:parse_from_header: out of pkg_memory
	Mar  2 10:20:16 test P-CSCFtest[21328]: ERROR: tm [t_lookup.c:1326]: ERROR: new_t: no valid From in INVITE
	Mar  2 10:20:16 test P-CSCFtest[21328]: ERROR: tm [t_lookup.c:1478]: ERROR: t_newtran: new_t failed
	Mar  2 10:20:16 test P-CSCFtest[21328]: WARNING:<core>  [receive.c:209]: WARNING: receive_msg: error while trying script

I repeated the test various times, it always generates a memory problem. 
The "top" command shows that the free memory is decreasing constantly 
during the test. Stopping the test and waiting for some time (1 hour) 
doesn't increase the free memory showed by "top". Only a restart of the 
Kamailio frees the memory again.
I executed the test with 30 calls per second too and got the same 
behaviour (after some more minutes).

I finally simplified the Kamailio config as far as possible and came to 
this minimum:

	#!KAMAILIO
	listen=10.10.10.10:5060
	children=30
	user="kamailio"
	group="kamailio"
	debug=2
	disable_core_dump=yes
	log_facility=LOG_LOCAL0
	log_name="P-CSCFtest"
	server_signature=no
	sip_warning=0
	syn_branch=0
	disable_tcp=yes

	####### Modules Section #######
	mpath="/usr/local/lib/kamailio/modules_k/:/usr/local/lib/kamailio/modules/"
	loadmodule "tm.so"
	
	####### Main Routing Logic ########
	route{
	    if (src_ip==1.2.3.4) {
	       rewritehost("5.6.7.8");
	    } else {
	       rewritehost("1.2.3.4");
	    };
	    rewriteport("5061");
	    t_relay();
	}

This config still generates the memory problem. After replacing the 
"t_relay()" with the stateless "forward()" the problem is gone.
I played around with the number of child processes too, but no difference.

Is this a memory leak of the TM module or is there a different explanation?

Regards Fred

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20120302/c3e90717/attachment-0001.htm>


More information about the sr-users mailing list