[SR-Users] Kamailio IMS heavy load
Henning Westerholt
hw at gilawa.com
Tue Apr 12 10:57:52 CEST 2022
Hello,
checking CPU/I/O performance of the database during the load test would be a start. You can also monitor database access e.g. with a network debugging tools.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com<https://gilawa.com/>
From: Евгений Мастратов <dgeka25594 at gmail.com>
Sent: Tuesday, April 12, 2022 10:03 AM
To: Henning Westerholt <hw at gilawa.com>
Subject: Re: [SR-Users] Kamailio IMS heavy load
Hey Henning!
Thanks for the advice, I understand about logging, I will make edits and re-check. And how can I check the fact that what are the gaps when reading / writing the database? I am using MySQL.
вт, 12 апр. 2022 г. в 10:52, Henning Westerholt <hw at gilawa.com<mailto:hw at gilawa.com>>:
Hello,
as mentioned, I would start by monitoring my dependencies, in particular related to I/O, like database or monitoring.
Just to be sure, you are not running the tests with enabled debug log level? This would certainly cause problems.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com<https://gilawa.com/>
From: sr-users <sr-users-bounces at lists.kamailio.org<mailto:sr-users-bounces at lists.kamailio.org>> On Behalf Of ??????? ?????????
Sent: Monday, April 11, 2022 5:23 PM
To: sr-users at lists.kamailio.org<mailto:sr-users at lists.kamailio.org>
Subject: [SR-Users] Kamailio IMS heavy load
Hi!
Previously, I created a ticket on github https://github.com/kamailio/kamailio/issues/3071 and tried to deal with the problem there, but they sent me here for advice.
I set up Kamailio IMS , in the end everything turned out great, calls are made, and I decided to check how Kamailio copes with the load. I am creating a load using the sipp application. There is no problem with a low call generation rate, but if you raise the call generation to more than 20 calls per second, some of the calls fail.
I compared the calls and saw that the ICSCF does not send a 180 Ringing message towards the S-CSCF
[..]
As I found out further and suggested, the problem is that some calls are cut on the S-CSCF and this is probably due to some kind of timer, but I could not figure out which one.
Full logs and dump are attached to the issue on github, below is a log sample by bad dialog 0x7fc133ce61d8
```
Apr 5 08:24:33 ims-core /usr/local/sbin/kamailio[26061]: DEBUG: ims_dialog [dlg_handlers.c:284]: dlg_iuid_sfree(): freeing dlg iuid [465:10905] (0x7fc133ce61d8)
Apr 5 08:24:35 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:1025]: link_dlg(): ref dlg 0x7fc133ce61d8 with 1 -> 1
Apr 5 08:24:35 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_cb.c:251]: run_create_callbacks(): dialog=0x7fc133ce61d8
Apr 5 08:24:35 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:887]: lookup_dlg(): ref dlg 0x7fc133ce61d8 with 1 -> 2
Apr 5 08:24:35 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_handlers.c:789]: dlg_onreq(): dialog [0x7fc133ce61d8] added to tm callbacks
Apr 5 08:24:35 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:1066]: unref_dlg(): unref dlg 0x7fc133ce61d8 with 1 -> 1
Apr 5 08:24:35 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:887]: lookup_dlg(): ref dlg 0x7fc133ce61d8 with 1 -> 2
Apr 5 08:24:35 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:1066]: unref_dlg(): unref dlg 0x7fc133ce61d8 with 1 -> 1
Apr 5 08:24:35 ims-core /usr/local/sbin/kamailio[26059]: DEBUG: ims_dialog [dlg_hash.c:887]: lookup_dlg(): ref dlg 0x7fc133ce61d8 with 1 -> 2
Apr 5 08:24:35 ims-core /usr/local/sbin/kamailio[26059]: DEBUG: ims_dialog [dlg_hash.c:1066]: unref_dlg(): unref dlg 0x7fc133ce61d8 with 1 -> 1
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:887]: lookup_dlg(): ref dlg 0x7fc133ce61d8 with 1 -> 2
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:1066]: unref_dlg(): unref dlg 0x7fc133ce61d8 with 1 -> 1
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:887]: lookup_dlg(): ref dlg 0x7fc133ce61d8 with 1 -> 2
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:1066]: unref_dlg(): unref dlg 0x7fc133ce61d8 with 1 -> 1
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:887]: lookup_dlg(): ref dlg 0x7fc133ce61d8 with 1 -> 2
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:1293]: next_state_dlg(): dialog 0x7fc133ce61d8 changed from state 1 to state 6, due event 4
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_handlers.c:1489]: dlg_onreply(): dialog 0x7fc133ce61d8 failed (negative reply)
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_cb.c:277]: run_dlg_callbacks(): dialog=0x7fc133ce61d8, type=4
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:1066]: unref_dlg(): unref dlg 0x7fc133ce61d8 with 1 -> 1
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:1066]: unref_dlg(): unref dlg 0x7fc133ce61d8 with 1 -> 0
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:1066]: unref_dlg(): ref <=0 for dialog 0x7fc133ce61d8
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:196]: destroy_dlg(): destroying dialog 0x7fc133ce61d8
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_hash.c:207]: destroy_dlg(): removed timer for dlg 0x7fc133ce61d8 [3791:1419] with clid '43-3057631 at 10.2.16.63<mailto:43-3057631 at 10.2.16.63>' and tags '78885556600'
Apr 5 08:24:38 ims-core /usr/local/sbin/kamailio[26056]: DEBUG: ims_dialog [dlg_cb.c:277]: run_dlg_callbacks(): dialog=0x7fc133ce61d8, type=8192
```
What could be the reasons for this behavior and how to diagnose them? I did not observe problems with MEM or CPU during the tests.
I will be grateful for any advice.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20220412/e55db8c8/attachment.htm>
More information about the sr-users
mailing list