[SR-Users] Log levels severely impacts performance

[Henning Westerholt]

Hello,

additionally, try to see if you have some general I/O issues, e.g., by observing io-wait CPU, looking to I/O transaction times with monitoring tools etc..
Maybe there are other services also causing excessive I/O.

Cheers,

Henning

--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com<https://gilawa.com/>

From: sr-users <sr-users-bounces at lists.kamailio.org> On Behalf Of Daniel-Constantin Mierla
Sent: Wednesday, September 14, 2022 2:25 PM
To: Kamailio (SER) - Users Mailing List <sr-users at lists.kamailio.org>; Amit <amit at brytecall.com>
Subject: Re: [SR-Users] Log levels severely impacts performance


Hello,

excessive logging is known to impact the performances. If you have many log messages printed from config, try to reduce them.

Performance could be also a matter of how many child processes you created, via the children global parameter.

Cheers,
Daniel
On 14.09.22 04:18, Amit wrote:
Hello,

We’ve been running Kamailio 5.5.0 for a while now and recently ran into an issue where the application buffer would get full and Kamailio has a lot of difficulty keeping up with REG requests. This severely impacted Kamailio’s ability to perform and resulted in many REG timeouts.

Running ‘watch netstat -ulpn’ revealed the buffer issue.

We usually run the production system at log level INFO (2) and had not ran into any issues in the past.

When the buffer issue occurred and after some testing, we discovered that changing the log level to NOTICE (1) was sufficient to bring the system back into operational status.

For example, at log level INFO (2) we seem to achieve ~150-200 REG/s while at log level NOTICE (1) we reach ~500-700 REG/s.

I did find this article in the wiki and tried using the “-“ in front of the log path but this didn’t seem to help.
http://www.kamailio.org/wiki/tutorials/3.2.x/syslog

Even writing the logs directly to our external log server and preventing them from being written to the local disk doesn’t seem to alleviate the issue when the log level is set to INFO (2).

Does anyone have any experience with this or have any suggestions? We are finding it difficult to scale as a result of this issue.

Thank you in advance.
Amit




__________________________________________________________

Kamailio - Users Mailing List - Non Commercial Discussions

  * sr-users at lists.kamailio.org<mailto:sr-users at lists.kamailio.org>

Important: keep the mailing list in the recipients, do not reply only to the sender!

Edit mailing list options or unsubscribe:

  * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

--

Daniel-Constantin Mierla -- www.asipto.com<http://www.asipto.com>

www.twitter.com/miconda<http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda<http://www.linkedin.com/in/miconda>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20220915/35295ee7/attachment.htm>