[SR-Users] Seeing Large Waits

[Mack Hendricks]

Hey Daniel,

Yes, in this case we are using the cfg_rpc module:

This is what we were using:

 $ru = "sip:" + $rU + "@" + $sel(cfg_get.teleblock.gw_ip) + ":" + $sel(cfg_get.teleblock.gw_port);
        
Now, I will hardcode it to see if it make a difference

$ru = "sip:" + $rU + "@" + “66.203.XX.XXX:5033”;

I turned on the general log and slow log in mysql and we hard hitting the database except during startup and subscriber lookup, which isn’t in play here.


Mack Hendricks / Head of Support / dOpenSource
web: http://dopensource.com <http://dopensource.com/>
support: +888-907-2085
dSIPRouter <http://dsiprouter.org/> - GUI focused on implementing Kamailio to provide SIP Trunking and PBX Hosting Services

> On May 3, 2018, at 2:11 AM, Daniel-Constantin Mierla <miconda at gmail.com> wrote:
> 
> Hello,
> 
> 
> On 03.05.18 06:48, Mack Hendricks wrote:
>> Hey Alex,
>> 
>> Thanks for the insight. I remember reading your article before and it was super helpful. I think we disabled all of the heavy database operations, but I will double check.
>> 
>> We are building the RURI using the cfg_rpc module,
> 
> are you really using cfg_rpc module to build the r-uri? Or did you mean
> someting like jsonrpcc?
> 
> Cheers,
> Daniel
> 
>> so we aren’t hitting a database.   We have a failure route that causes another RURI to be created, which sends the call to a Freeswitch server to play a message.
>> 
>> Let me know if you see any issue with this, I will double check the database calls to be safe.
>> 
>> Sent from my iPhone
>> 
>>> On May 2, 2018, at 11:36 PM, Alex Balashov <abalashov at evaristesys.com> wrote:
>>> 
>>> Hi Mack,
>>> 
>>> What sort of end-to-end delays are you seeing, from receipt of request
>>> until immediately before t_relay()?
>>> 
>>> You can profile this with the benchmark module:
>>> 
>>>  https://kamailio.org/docs/modules/5.1.x/modules/benchmark.html
>>> 
>>> Or my preferred method, the hard way:
>>> 
>>>       # At the practical start of request processing.
>>> 
>>>       $avp(proc_start) = $TV(Sn);
>>> 
>>>       # At the end, right before calling t_relay() or forward().
>>> 
>>>       $var(cur_time) = $TV(Sn);
>>> 
>>>       $var(proc_diff) = (
>>>               (((
>>>                 $(var(cur_time){s.select,0,.}{s.int}) -
>>>                 $(avp(proc_start){s.select,0,.}{s.int})
>>>               ) * 1000000)
>>>               +
>>>               (
>>>                $(var(cur_time){s.select,1,.}{s.int}) -
>>>                $(avp(proc_start){s.select,1,.}{s.int})
>>>               ) / 1000) mod 1000
>>>       );
>>> 
>>>       xlog("L_INFO", "[R-MAIN:$ci] -> Request processing delay: $var(proc_diff) ms\n"););
>>> 
>>> Also, for general background on this topic, you might consider my
>>> article on the subject of Kamailio performance characteristics:
>>> 
>>>  http://blog.csrpswitch.com/tuning-kamailio-for-high-throughput-and-performance/
>>> 
>>> If I had to guess, shooting from the hip, you've got some really slow
>>> database queries somewhere.
>>> 
>>> -- Alex
>>> 
>>>> On Wed, May 02, 2018 at 06:19:00PM -0400, Mack Hendricks wrote:
>>>> 
>>>> Hey All,
>>>> 
>>>> We are load testing Kamailio 4.2.3 and we are seeing the waiting within the transaction manager increasing.   I’ve increased the children from 8 to 16 to 24, but we are still seeing a large amount of waiting.  I don’t see an errors in the logs.   Can anyone give me a clue on where to look. 
>>>> 
>>>> 05-02-18_03:07:38 PM
>>>> tm.stats
>>>> {
>>>>       current: 390
>>>>       waiting: 226
>>>>       total: 2361
>>>>       total_local: 0
>>>>       replied_locally: 1406
>>>>       6xx: 0
>>>>       5xx: 0
>>>>       4xx: 88
>>>>       3xx: 0
>>>>       2xx: 3180
>>>>       created: 2361
>>>>       freed: 1971
>>>>       delayed_free: 0
>>>> }
>>>> 
>>>> Mack Hendricks / Head of Support / dOpenSource
>>>> web: http://dopensource.com <http://dopensource.com/>
>>>> support: +888-907-2085
>>>> dSIPRouter <http://dsiprouter.org/> - GUI focused on implementing Kamailio to provide SIP Trunking and PBX Hosting Services
>>>> 
>>>> _______________________________________________
>>>> Kamailio (SER) - Users Mailing List
>>>> sr-users at lists.kamailio.org
>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>> 
>>> -- 
>>> Alex Balashov | Principal | Evariste Systems LLC
>>> 
>>> Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) 
>>> Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org <mailto:sr-users at lists.kamailio.org>
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
> 
> -- 
> Daniel-Constantin Mierla
> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
> Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com <http://www.kamailioworld.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20180503/c5d21743/attachment.html>