Hi,
I am doing load testing with of Kamailio. I have sipp (version 3.3) as the load generator,Kamailio(4.2.7) as Load Balancer and Freeswitch (1.6.5) as Gateways.
I am using the following command to generate load ./sipp -sf uac.xml -d 2000 -m 4000 -l 1000 -r 200 -trace_stat -i SIPP_IP -p 15060 --trace_err -trace_error_codes -trace_calldebug -trace_screen KAM_IP
When the rate of generation is below 80 (-r 80), Kamailio behaves fine.But if we go beyond 80, Kamailio starts sending "500 Service Unavailable"
2016-01-12 11:18:15.375103 1452577695.375103: Aborting call on unexpected message for Call-Id '96-32493@SIPP_IP': while expecting '180' (index 2), received 'SIP/2.0 500 Service Unavailable^M Via: SIP/2.0/UDP SIPP_IP:15060;branch=z9hG4bK-32493-96-0^M From: sipp sip:sipp@SIPP_IP:15060;tag=96^M To: sut sip:service@KAM_IP :5060;tag=55f576f507e822fa6633cf4bc22740e6-6660^M Call-ID: 96-32493@SIPP_IP^M CSeq: 1 INVITE^M Server: kamailio (4.2.7 (x86_64/linux))^M Content-Length: 0^M
With the above command, sipp output is
----------------------------- Statistics Screen ------- [1-9]: Change Screen -- Start Time | 2016-01-12 11:26:07.723764 1452578167.723764 Last Reset Time | 2016-01-12 11:27:16.280901 1452578236.280901 Current Time | 2016-01-12 11:27:16.282737 1452578236.282737 -------------------------+---------------------------+-------------------------- Counter Name | Periodic value | Cumulative value -------------------------+---------------------------+-------------------------- Elapsed Time | 00:00:00:001000 | 00:01:08:558000 Call Rate | 0.000 cps | 58.345 cps -------------------------+---------------------------+-------------------------- Incoming call created | 0 | 0 OutGoing call created | 0 | 4000 Total Call created | | 4000 Current Call | 0 | -------------------------+---------------------------+-------------------------- Successful call | 0 | 1192 Failed call | 0 | 2808 -------------------------+---------------------------+-------------------------- Response Time 1 | 00:00:00:000000 | 00:00:15:102000 Call Length | 00:00:00:000000 | 00:00:10:614000 ------------------------------ Test Terminated --------------------------------
Kamailio is run with command /usr/local/sbin/kamailio -f /usr/local/etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 512 -M 8 -u kamailio -g kamailio
version: kamailio 4.2.7 (x86_64/linux) 727746 flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 727746 compiled on 15:59:03 Jan 11 2016 with gcc 4.6.3
System : Ubuntu 12.04.4 LTS Linux devops10-60-20-169 3.11.0-15-generic #25~precise1-Ubuntu SMP Thu Jan 30 17:39:31 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
I am not seeing any errors in Kamailio Logs.
Any pointers on how to get it to serve more calls/sec
Regards, Sandeep
The most profound statements are often said in silence. -Lynn Johnston
Hi,
If you are using dispatcher module or similar, then the bottle-neck is not Kamailio, but Freeswitch. I guess, Kamailio simply doesn't get any answer from the AS and thus replies with an error as no AS is available. Kamailio by default/minimal configuration won't send any errors, it depends on your configuration.
Thanks, Carsten
2016-01-12 7:27 GMT+01:00 sandeep goje sandeep.goje@gmail.com:
Hi,
I am doing load testing with of Kamailio. I have sipp (version 3.3) as the load generator,Kamailio(4.2.7) as Load Balancer and Freeswitch (1.6.5) as Gateways.
I am using the following command to generate load ./sipp -sf uac.xml -d 2000 -m 4000 -l 1000 -r 200 -trace_stat -i SIPP_IP -p 15060 --trace_err -trace_error_codes -trace_calldebug -trace_screen KAM_IP
When the rate of generation is below 80 (-r 80), Kamailio behaves fine.But if we go beyond 80, Kamailio starts sending "500 Service Unavailable"
2016-01-12 11:18:15.375103 1452577695.375103: Aborting call on unexpected message for Call-Id '96-32493@SIPP_IP': while expecting '180' (index 2), received 'SIP/2.0 500 Service Unavailable^M Via: SIP/2.0/UDP SIPP_IP:15060;branch=z9hG4bK-32493-96-0^M From: sipp sip:sipp@SIPP_IP:15060;tag=96^M To: sut sip:service@KAM_IP:5060;tag=55f576f507e822fa6633cf4bc22740e6-6660^M Call-ID: 96-32493@SIPP_IP^M CSeq: 1 INVITE^M Server: kamailio (4.2.7 (x86_64/linux))^M Content-Length: 0^M
With the above command, sipp output is
----------------------------- Statistics Screen ------- [1-9]: Change Screen
Start Time | 2016-01-12 11:26:07.723764 1452578167.723764 Last Reset Time | 2016-01-12 11:27:16.280901 1452578236.280901 Current Time | 2016-01-12 11:27:16.282737 1452578236.282737 -------------------------+---------------------------+-------------------------- Counter Name | Periodic value | Cumulative value -------------------------+---------------------------+-------------------------- Elapsed Time | 00:00:00:001000 | 00:01:08:558000 Call Rate | 0.000 cps | 58.345 cps -------------------------+---------------------------+-------------------------- Incoming call created | 0 | 0 OutGoing call created | 0 | 4000 Total Call created | | 4000 Current Call | 0 | -------------------------+---------------------------+-------------------------- Successful call | 0 | 1192 Failed call | 0 | 2808 -------------------------+---------------------------+-------------------------- Response Time 1 | 00:00:00:000000 | 00:00:15:102000 Call Length | 00:00:00:000000 | 00:00:10:614000
------------------------------ Test Terminated
Kamailio is run with command /usr/local/sbin/kamailio -f /usr/local/etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 512 -M 8 -u kamailio -g kamailio
version: kamailio 4.2.7 (x86_64/linux) 727746 flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 727746 compiled on 15:59:03 Jan 11 2016 with gcc 4.6.3
System : Ubuntu 12.04.4 LTS Linux devops10-60-20-169 3.11.0-15-generic #25~precise1-Ubuntu SMP Thu Jan 30 17:39:31 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
I am not seeing any errors in Kamailio Logs.
Any pointers on how to get it to serve more calls/sec
Regards, Sandeep
The most profound statements are often said in silence. -Lynn Johnston
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hello,
as an extra hint on top of Carsten's remarks: search your configuration for sl_send_reply("500", "Service Unavailable") or t_reply/send_reply with same parameters. Then you can identify better why that reply is sent. As Carsten pointed, such reply code and reason text is not coming from source code, but config.
Cheers, Daniel
On 12/01/16 08:13, Carsten Bock wrote:
Hi,
If you are using dispatcher module or similar, then the bottle-neck is not Kamailio, but Freeswitch. I guess, Kamailio simply doesn't get any answer from the AS and thus replies with an error as no AS is available. Kamailio by default/minimal configuration won't send any errors, it depends on your configuration.
Thanks, Carsten
2016-01-12 7:27 GMT+01:00 sandeep goje sandeep.goje@gmail.com:
Hi,
I am doing load testing with of Kamailio. I have sipp (version 3.3) as the load generator,Kamailio(4.2.7) as Load Balancer and Freeswitch (1.6.5) as Gateways.
I am using the following command to generate load ./sipp -sf uac.xml -d 2000 -m 4000 -l 1000 -r 200 -trace_stat -i SIPP_IP -p 15060 --trace_err -trace_error_codes -trace_calldebug -trace_screen KAM_IP
When the rate of generation is below 80 (-r 80), Kamailio behaves fine.But if we go beyond 80, Kamailio starts sending "500 Service Unavailable"
2016-01-12 11:18:15.375103 1452577695.375103: Aborting call on unexpected message for Call-Id '96-32493@SIPP_IP': while expecting '180' (index 2), received 'SIP/2.0 500 Service Unavailable^M Via: SIP/2.0/UDP SIPP_IP:15060;branch=z9hG4bK-32493-96-0^M From: sipp sip:sipp@SIPP_IP:15060;tag=96^M To: sut sip:service@KAM_IP:5060;tag=55f576f507e822fa6633cf4bc22740e6-6660^M Call-ID: 96-32493@SIPP_IP^M CSeq: 1 INVITE^M Server: kamailio (4.2.7 (x86_64/linux))^M Content-Length: 0^M
With the above command, sipp output is
----------------------------- Statistics Screen ------- [1-9]: Change Screen
Start Time | 2016-01-12 11:26:07.723764 1452578167.723764 Last Reset Time | 2016-01-12 11:27:16.280901 1452578236.280901 Current Time | 2016-01-12 11:27:16.282737 1452578236.282737 -------------------------+---------------------------+-------------------------- Counter Name | Periodic value | Cumulative value -------------------------+---------------------------+-------------------------- Elapsed Time | 00:00:00:001000 | 00:01:08:558000 Call Rate | 0.000 cps | 58.345 cps -------------------------+---------------------------+-------------------------- Incoming call created | 0 | 0 OutGoing call created | 0 | 4000 Total Call created | | 4000 Current Call | 0 | -------------------------+---------------------------+-------------------------- Successful call | 0 | 1192 Failed call | 0 | 2808 -------------------------+---------------------------+-------------------------- Response Time 1 | 00:00:00:000000 | 00:00:15:102000 Call Length | 00:00:00:000000 | 00:00:10:614000
------------------------------ Test Terminated
Kamailio is run with command /usr/local/sbin/kamailio -f /usr/local/etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 512 -M 8 -u kamailio -g kamailio
version: kamailio 4.2.7 (x86_64/linux) 727746 flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 727746 compiled on 15:59:03 Jan 11 2016 with gcc 4.6.3
System : Ubuntu 12.04.4 LTS Linux devops10-60-20-169 3.11.0-15-generic #25~precise1-Ubuntu SMP Thu Jan 30 17:39:31 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
I am not seeing any errors in Kamailio Logs.
Any pointers on how to get it to serve more calls/sec
Regards, Sandeep
The most profound statements are often said in silence. -Lynn Johnston
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hello.
I had similar problems while load test kamailio in front of yate. Yate have rate limiting enabled by default. May be you have it on Freeswitch?
I have made loadtest with sipp, kamailio 4.2.6, rtpengines and 2 Asterisk servers as backend. Test was success with 8000 concurrent calls and call-rate 200 cps. -- Best regards, Sergey Basov e-mail: sergey.v.basov@gmail.com
tel: (+38067) 403-62-54
2016-01-13 22:56 GMT+02:00 Daniel-Constantin Mierla miconda@gmail.com:
Hello,
as an extra hint on top of Carsten's remarks: search your configuration for sl_send_reply("500", "Service Unavailable") or t_reply/send_reply with same parameters. Then you can identify better why that reply is sent. As Carsten pointed, such reply code and reason text is not coming from source code, but config.
Cheers, Daniel
On 12/01/16 08:13, Carsten Bock wrote:
Hi,
If you are using dispatcher module or similar, then the bottle-neck is not Kamailio, but Freeswitch. I guess, Kamailio simply doesn't get any answer from the AS and thus replies with an error as no AS is available. Kamailio by default/minimal configuration won't send any errors, it depends on your configuration.
Thanks, Carsten
2016-01-12 7:27 GMT+01:00 sandeep goje sandeep.goje@gmail.com:
Hi,
I am doing load testing with of Kamailio. I have sipp (version 3.3) as the load generator,Kamailio(4.2.7) as Load Balancer and Freeswitch (1.6.5) as Gateways.
I am using the following command to generate load ./sipp -sf uac.xml -d 2000 -m 4000 -l 1000 -r 200 -trace_stat -i SIPP_IP -p 15060 --trace_err -trace_error_codes -trace_calldebug -trace_screen KAM_IP
When the rate of generation is below 80 (-r 80), Kamailio behaves fine.But if we go beyond 80, Kamailio starts sending "500 Service Unavailable"
2016-01-12 11:18:15.375103 1452577695.375103: Aborting call on unexpected message for Call-Id '96-32493@SIPP_IP': while expecting '180' (index 2), received 'SIP/2.0 500 Service Unavailable^M Via: SIP/2.0/UDP SIPP_IP:15060;branch=z9hG4bK-32493-96-0^M From: sipp sip:sipp@SIPP_IP:15060;tag=96^M To: sut sip:service@KAM_IP:5060;tag=55f576f507e822fa6633cf4bc22740e6-6660^M Call-ID: 96-32493@SIPP_IP^M CSeq: 1 INVITE^M Server: kamailio (4.2.7 (x86_64/linux))^M Content-Length: 0^M
With the above command, sipp output is
----------------------------- Statistics Screen ------- [1-9]: Change Screen
Start Time | 2016-01-12 11:26:07.723764 1452578167.723764 Last Reset Time | 2016-01-12 11:27:16.280901 1452578236.280901 Current Time | 2016-01-12 11:27:16.282737 1452578236.282737 -------------------------+---------------------------+-------------------------- Counter Name | Periodic value | Cumulative value -------------------------+---------------------------+-------------------------- Elapsed Time | 00:00:00:001000 | 00:01:08:558000 Call Rate | 0.000 cps | 58.345 cps -------------------------+---------------------------+-------------------------- Incoming call created | 0 | 0 OutGoing call created | 0 | 4000 Total Call created | | 4000 Current Call | 0 | -------------------------+---------------------------+-------------------------- Successful call | 0 | 1192 Failed call | 0 | 2808 -------------------------+---------------------------+-------------------------- Response Time 1 | 00:00:00:000000 | 00:00:15:102000 Call Length | 00:00:00:000000 | 00:00:10:614000
------------------------------ Test Terminated
Kamailio is run with command /usr/local/sbin/kamailio -f /usr/local/etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 512 -M 8 -u kamailio -g kamailio
version: kamailio 4.2.7 (x86_64/linux) 727746 flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 727746 compiled on 15:59:03 Jan 11 2016 with gcc 4.6.3
System : Ubuntu 12.04.4 LTS Linux devops10-60-20-169 3.11.0-15-generic #25~precise1-Ubuntu SMP Thu Jan 30 17:39:31 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
I am not seeing any errors in Kamailio Logs.
Any pointers on how to get it to serve more calls/sec
Regards, Sandeep
The most profound statements are often said in silence. -Lynn Johnston
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Book: SIP Routing With Kamailio - http://www.asipto.com http://miconda.eu
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-
Carsten Bock -
Daniel-Constantin Mierla -
sandeep goje -
Sergey Basov