[sr-dev] kamailio gets frozen when rsyslog-mysql cannot send the SQL query

[Olle E. Johansson]

28 jul 2010 kl. 17.29 skrev Iñaki Baz Castillo:

> Hi, finally I've discovered the cause of important issues with my
> Kamailio deployments which sometimes get frozen "with no reason":
> 
> - We use Debian Lenny with rsyslog writting the logs in local files
> (/var/log/kamailio.log) and also rsyslog-mysql extension to send the
> logs to a remote MySQL server using mysql TCP protocol.
> 
> - For "external" reason (external to me) such MySQL server sometimes
> fails or it's down.
> 
> - Then kamailio tries to log to syslog. rsyslog receives the request
> and tries to send it via MySQL. Unfortunatelly it gets frozen as the
> DB doesn't work, and it neither writes the log to the local log file
> !!!  (this is really surprising to me).
> 
> - Worse is the fact the rsyslog replies nothing to kamailio process so
> the kamailio process gets 100% *blocked* waiting for the response
> (this is what I suspect).
> 
> - As a dirty workaround, stopping the local rsyslog server makes
> kamailio to work again.
> 
> 
> So, I think this is mostly a bug in rsyslog-mysql as it shouldn't get
> frozen if the MySQL server doesn't work. However I also think that
> Kamailio should not depend on it, and perhaps it shouldn't wait
> indefinitely for the rsyslog response. What do you think?
> Would make sense a "ping" mechanism like with RtpProxy servers in
> order to enable/dissable the syslog logging?
> 
Syslog is defined in http://tools.ietf.org/html/rfc5424, but I can't find
anything there about this situation. It's a simplex protocol, so the implementation
should propably handle this situation, as you say, rsyslog is at fault here.

If it does, we need to figure out a way so that Kamailio doesn't lock up
if this happens, not logging in the same process or thread or something...

/O