[sr-dev] [SR-Users] Processes blocking - troubleshooting slow actions

[Julien Chavanton]

It is running in docker, the interval is not stable ~30sec.
I know this is not related to DNS slow queries since I am monitoring them
as well.
I wonder what else can be intermittently blocking ...
I am going to look at the TM locks and the callbacks.

Not many modules used on this one, maybe Dialog module using a DB local
sqlite

modparam("dialog", "db_update_period", 60)
modparam("dialog", "db_mode", 3)


On Wed, Nov 13, 2019 at 10:43 PM Daniel-Constantin Mierla <miconda at gmail.com>
wrote:

> Is this a virtual machine? How often does it happen?
>
> Sometime the VM manager "freezes" the VM for different purposes, like
> backup snapshots, ...
>
> Cheers,
> Daniel
> On 13.11.19 23:40, Julien Chavanton wrote:
>
> I guess action type 16 IF_T is accounting for the entire scope block since
> it is recursive.
>
> I wonder why var assignment in the main route scope would do the same :
>
> 2019-11-13T21:58:04.018631614Z 23(42) WARNING: <core>
> [core/action.c:1586]: run_actions(): alert - action [corefunc (16)] cfg
> [/etc/kamailio/kamailio.cfg:115] took too long [2579486 us]
> $var(reply) = 0; // I wonder how this can endup blocking ?
>
> On Wed, Nov 13, 2019 at 2:22 PM Julien Chavanton <jchavanton at gmail.com>
> wrote:
>
>> I am troubleshooting intermittent latency on a server,  several processes
>> will report slow action at the same time every now and then,
>> something odd it that some of the actions are most likely not using any
>> mutexes, I can not imagine how they can end up blocking unless the process
>> is actually suspended by the OS, however the CPU usage and load average is
>> quite low on this server.
>>
>> Is there any reason why assigning to a PKG var could be delayed 3 seconds.
>>
>> 2019-11-13T21:58:04.018631614Z 23(42) WARNING: <core>
>> [core/action.c:1586]: run_actions(): alert - action [corefunc (16)] cfg
>> [/etc/kamailio/kamailio.cfg:115] took too long [2579486 us]
>> $var(reply) = 0; // I wonder how this can endup blocking ?
>>
>> 2019-11-13T21:58:04.018748710Z 42(61) WARNING: <core>
>> [core/action.c:1586]: run_actions(): alert - action [corefunc (16)] cfg
>> [/etc/kamailio/kamailio.cfg:221] took too long [2222010 us]
>> } // Samething here, simply exiting a scope
>>
>> 2019-11-13T21:58:04.015675095Z 11(30) WARNING: <core>
>> [core/action.c:1586]: run_actions(): alert - action [t_relay_cancel (24)]
>> cfg [/etc/kamailio/kamailio.cfg:86] took too long [2588818 us]
>> } if (!t_relay_cancel()) { // Indeed a t_relay_cancel
>>
>> 2019-11-13T21:58:04.018647988Z 26(45) WARNING: <core>
>> [core/action.c:1586]: run_actions(): alert - action [t_relay (24)] cfg
>> [/etc/kamailio/kamailio.cfg:289] took too long [2739366 us]
>>
>> t_relay(); // Indeed a t_relay
>>
>> 2019-11-13T21:58:04.018651243Z 15(34) WARNING: <core>
>> [core/action.c:1586]: run_actions(): alert - action [sl_send_reply (26)]
>> cfg [/etc/kamailio/kamailio.cfg:112] took too long [2587812 us]
>> sl_send_reply("100", "Trying"); // Indeed a sl_send_reply
>>
>>
>> Not sure which mailing list is best for this question
>> Regards
>> Julien
>>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing Listsr-users at lists.kamailio.orghttps://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
> --
> Daniel-Constantin Mierla -- www.asipto.comwww.twitter.com/miconda -- www.linkedin.com/in/miconda
> Kamailio World Conference - April 27-29, 2020, in Berlin -- www.kamailioworld.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-dev/attachments/20191113/f2b54004/attachment-0001.html>