Re: [SR-Users] More than TCP 800 concurrent calls, slows Kamailio (LUA FILE API REQUESTS)
Hello, please keep the list in CC, that others can comment as well. Beside from the obvious (improving the performance of the lua API call), you could use indeed asynchronous processing. If you just use the lua code to emit the REST request, using a native http client module would be probably also faster as doing it over lua. Cheers, Henning -- Henning Westerholt – https://skalatan.de/blog/ Kamailio services – https://gilawa.com<https://gilawa.com/> From: Giovanni Jose <gio.jose@gmail.com> Sent: Wednesday, March 30, 2022 5:17 PM To: Henning Westerholt <hw@gilawa.com> Subject: Re: [SR-Users] More than TCP 800 concurrent calls, slows Kamailio (LUA FILE API REQUESTS) Hello Henning, Thanks so much for the fast response I just commented the loop and I was hitting the backend and it's slow anyway maybe it's because, for this type of implementation when the destination of the call is decided by the API backend, I need to use the http_async_client module? I am noticing that for high volume calls kamailio slows the traffic and shows errors because there's a delay that is been added when the backend responds for every call. On Tue, Mar 29, 2022 at 11:53 PM Henning Westerholt <hw@gilawa.com<mailto:hw@gilawa.com>> wrote: Hello, why are you sleeping for 1s in the loop? This will cause Kamailio to block processing. I also do not see any stop condition on the loop, so it will loop until the limit. Cheers, Henning -- Henning Westerholt – https://skalatan.de/blog/ Kamailio services – https://gilawa.com<https://gilawa.com/> From: sr-users <sr-users-bounces@lists.kamailio.org<mailto:sr-users-bounces@lists.kamailio.org>> On Behalf Of Giovanni Jose Sent: Tuesday, March 29, 2022 9:46 PM To: sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org> Subject: [SR-Users] More than TCP 800 concurrent calls, slows Kamailio (LUA FILE API REQUESTS) Hello Guys I am having this issue where kamailio is receiving requests from SIPp (we are currently testing the platform and doing QA) but if we do more than 800 concurrent calls kamailio starts failing and slowing the requests, The scenario is the following: I am sending 5000 calls at 100 CPS on TCP and I am having this route in the configuration file: # Handle the calls to api route[CALL_API] { xlog("L_NOTICE"," Call request $var(call_request) method: $rm \n"); $var(loop_true)=1; while ($var(loop_true)) { if(!lua_run("call_request","$var(call_request)","$sht(token=>new_token)")) { xlog("L_NOTICE", "SCRIPT: failed to execute lua function!\n"); } if ($var(loop_true)){ sleep("1"); } } xlog("L_NOTICE", "SCRIPT: Sucess to execute lua function!\n"); } So I am executing a LUA script for the requests and the API response will tell the destination of the call. At 800 concurrent calls, I can notice SLOWLINESS in the SIPp testing tool, and then I will start receiving 408 timeouts. This sounds like a load issue or something because I believe there's a time that it's been added every time we hit the backend and wait for the response. any help will be appreciated Regards Gio
Hello again, Ok I was able to resolve the load issue and yes asynchronous processing is improving this A LOT, but now I am having issues with the ACK's as soon I am getting an ACK request I am having this error 106(112) ERROR: * cfgtrace:request_route=[RELAY] c=[/etc/kamailio/ccm-next-main.cfg] l=141 a=24 n=t_relay 106(112) INFO: tm [t_lookup.c:1333]: t_newtran(): attempt to create transaction for a faked request - try to avoid it 106(112) WARNING: tm [t_reply.c:1583]: t_retransmit_reply(): WARNING: t_retransmit_reply: no resolved dst to retransmit 106(112) ERROR: * cfgtrace:request_route=[RELAY] c=[/etc/kamailio/ccm-next-main.cfg] l=144 a=2 n=exit I think the problem could be because the t_newtran() is now working properly with the ACKs so, as soon this is going to t_relay() fails. route[RELAY_API] { $http_req(all) = $null; $http_req(suspend) = 0; $http_req(timeout) = 3000; $http_req(method) = "POST"; $http_req(hdr) = "Content-Type: application/json"; $http_req(hdr) = "Authorization: Bearer $sht(token=>new_token)"; jansson_set("obj", "network", '{"srcIP":"$fd"}', "$var(http_routing_query)"); jansson_set("string", "callID", $ci, "$var(http_routing_query)"); jansson_set("string", "sip", $mb, "$var(http_routing_query)"); xlog("L_INFO","$rm API ASYNC ROUTING REQUEST: $var(http_routing_query)\n"); $http_req(body) = $var(http_routing_query); if (is_method("INVITE")) { xlog("L_INFO","BEFORE $rm INIT_CALL_API $var(call_request): $var(http_routing_query)\n"); if (!t_newtran()) { sl_reply_error(); exit(); } http_async_query("http:// API_END_POINT /init", "INIT_RELAY_API_RESPONSE"); } if ( is_method("ACK|BYE|INFO") ) { xlog("L_INFO","BEFORE $rm DIALOG_CALL_API $var(call_request): $var(http_routing_query)\n"); if (!t_newtran()) { sl_reply_error(); exit(); } http_async_query("http://API_END_POINT", "RELAY"); } } On Wed, Mar 30, 2022 at 8:19 AM Henning Westerholt <hw@gilawa.com> wrote:
Hello,
please keep the list in CC, that others can comment as well.
Beside from the obvious (improving the performance of the lua API call), you could use indeed asynchronous processing. If you just use the lua code to emit the REST request, using a native http client module would be probably also faster as doing it over lua.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* Giovanni Jose <gio.jose@gmail.com> *Sent:* Wednesday, March 30, 2022 5:17 PM *To:* Henning Westerholt <hw@gilawa.com> *Subject:* Re: [SR-Users] More than TCP 800 concurrent calls, slows Kamailio (LUA FILE API REQUESTS)
Hello Henning, Thanks so much for the fast response
I just commented the loop and I was hitting the backend and it's slow anyway maybe it's because, for this type of implementation when the destination of the call is decided by the API backend, I need to use the http_async_client module? I am noticing that for high volume calls kamailio slows the traffic and shows errors because there's a delay that is been added when the backend responds for every call.
On Tue, Mar 29, 2022 at 11:53 PM Henning Westerholt <hw@gilawa.com> wrote:
Hello,
why are you sleeping for 1s in the loop? This will cause Kamailio to block processing. I also do not see any stop condition on the loop, so it will loop until the limit.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *Giovanni Jose *Sent:* Tuesday, March 29, 2022 9:46 PM *To:* sr-users@lists.kamailio.org *Subject:* [SR-Users] More than TCP 800 concurrent calls, slows Kamailio (LUA FILE API REQUESTS)
Hello Guys I am having this issue where kamailio is receiving requests from SIPp (we are currently testing the platform and doing QA) but if we do more than 800 concurrent calls kamailio starts failing and slowing the requests, The scenario is the following:
I am sending 5000 calls at 100 CPS on TCP and I am having this route in the configuration file:
# Handle the calls to api route[CALL_API] { xlog("L_NOTICE"," Call request $var(call_request) method: $rm \n"); $var(loop_true)=1; while ($var(loop_true)) {
if(!lua_run("call_request","$var(call_request)","$sht(token=>new_token)")) { xlog("L_NOTICE", "SCRIPT: failed to execute lua function!\n"); } if ($var(loop_true)){ sleep("1"); } } xlog("L_NOTICE", "SCRIPT: Sucess to execute lua function!\n"); }
So I am executing a LUA script for the requests and the API response will tell the destination of the call.
At 800 concurrent calls, I can notice SLOWLINESS in the SIPp testing tool, and then I will start receiving 408 timeouts.
This sounds like a load issue or something because I believe there's a time that it's been added every time we hit the backend and wait for the response.
any help will be appreciated
Regards
Gio
participants (2)
-
Giovanni Jose -
Henning Westerholt