[SR-Users] Log levels severely impacts performance

Alex Balashov abalashov at evaristesys.com
Thu Sep 15 23:20:47 CEST 2022


Agree, have had some very high-volume systems with prodigious logging output for every call and no problems. 

Have also had these types of load problems on other systems with much less logging. The problem was either that asynchronous logging was not turned on, or the I/O subsystem was abnormally slow for other reasons, such as other heavy demand on storage, or just intrinsically slow.

— Alex

> On Sep 15, 2022, at 5:18 PM, Henning Westerholt <hw at gilawa.com> wrote:
> 
> Hello,
> 
> Kamailio is using the system interface for syslog, this is also a code which is quite stable and has not seen a lot of changes in the last years.
> People are using Kamailio with a lot of logging in high load situation usually without problems, there has not been a lot of reports about that in the past.
> 
> Maybe it something related to a specific distribution version? I checked, the rsyslog is using asynchronous writing as default since a long time. What logging engine do you use?
> 
> Cheers,
> 
> Henning
> 
> -- 
> Henning Westerholt - https://skalatan.de/blog/
> Kamailio services - https://gilawa.com
> 
> -----Original Message-----
> From: sr-users <sr-users-bounces at lists.kamailio.org> On Behalf Of Jeremy Kister
> Sent: Thursday, September 15, 2022 9:59 PM
> To: sr-users at lists.kamailio.org
> Subject: Re: [SR-Users] Log levels severely impacts performance
> 
> On 9/13/2022 10:18 PM, Amit wrote:
>> Does anyone have any experience with this or have any suggestions? We 
>> are finding it difficult to scale as a result of this issue.
> 
> I was just dealing with this behavior. I was logging to a local rsyslog and rsyslog was just writing to a file on a local ssd (unsynced).
> 
> It's a ton of log.  But meh.
> 
> I disabled syslog entirely and enabled log_stderror = yes
> 
> Then I just captured stdout+stderr via multilog and write it to a file on the disk.  The problem went away.
> 
> I think there is some huge overhead in the way kam packages up the log and sends it out to syslog.
> 
> -- 
> 
> Jeremy Kister
> https://jeremy.kister.net./
> 
> __________________________________________________________
> Kamailio - Users Mailing List - Non Commercial Discussions
>  * sr-users at lists.kamailio.org
> Important: keep the mailing list in the recipients, do not reply only to the sender!
> Edit mailing list options or unsubscribe:
>  * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
> 
> __________________________________________________________
> Kamailio - Users Mailing List - Non Commercial Discussions
>  * sr-users at lists.kamailio.org
> Important: keep the mailing list in the recipients, do not reply only to the sender!
> Edit mailing list options or unsubscribe:
>  * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

-- 
Alex Balashov | Principal | Evariste Systems LLC

Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/




More information about the sr-users mailing list