[SR-Users] possible TCP deadlock (tls again?) // pike module not releasing IPs

Aymeric Moizard amoizard at gmail.com
Fri Dec 13 16:39:36 CET 2019


Hi List,

History:
* In the past, I had deadlock which was, most probably, related to ssl1.1.
  We have discussed this issue, and a fix is supposed to workaround the
issue that was detected.
* With latest 5.2.X, I have experienced ONCE a similar behavior with TCP
and TLS being mostly stuck. I have not been using this version much, but
the fix was supposed to be in the core of kamailio.

The status of the server this night:
* I'm today running version: kamailio 5.3.1 (x86_64/linux),
* Installed on stretch using http://deb.kamailio.org/kamailio53 repository.
* This versions use libssl1.1
* A user reported that he can't connect with TCP
* An average of 5000 IPs per 10 minutes are being banned by the pike module
   (could be twice the same)
Yesterday/Today:
* at the end of the outage, I had 2479 IP in my ipban htable. (which is
equivalent to my statistics showing 2 bans/IP every 10 minutes = 5000)
* looking at my logs, it appears that most (ALL?) ip being banned... are my
regular users.
* looking at my logs, I can't understand why pike would block them.

This is a graph for statistics on my service for the last 24 hours:
https://www.antisip.com/sip-antisip-com-register/status2.html

Yesterday, at 22:18:39, kamailio started to BAN some IPs. 52 IPs were
banned in a period of 10 minutes. I can confirm this from my logs.

My pike configuration is this one:

modparam("pike", "sampling_time_unit", 2)
modparam("pike", "reqs_density_per_unit", 64)
modparam("pike", "remove_latency", 4)

When detecting the issue, this morning, I typed:

$> sudo kamctl stats
$> sudo kamcmd htable.dump ipban
//FAILURE (answer too large...)
$> sudo kamctl trap

Then, I started an agent with TCP and it worked...???
Then, a few seconds, may be a minute after:

$> sudo kamcmd htable.dump ipban
//SUCCESS and shows 2479 banned ip.

and... everything is back to normal in a few minutes.

I haven't restarted kamailio, and all statistics are as expected, as usual.

Thus, it looks that " sudo kamctl trap" has triggered something. I already
experienced a similar behavior -when testing my ssl1.1 deadlock last year-.

2 questions:
1/ I beleive my "pike" configuration should not ban users. Is my pike
configuration wrong?
As an example, pike has banned an IP sending one message/second. I believe
my configuration should accept that?

2/ Could there still be a TLS issue with libssl1.1?

This is the result of the "kamctl trap":

https://sip.antisip.com/kamailio-pike-or-tls-issue-13-12-2019.kamctl-trap

Sorry for the long story & hoping to find a long term solution or at least
a workaround!

Regards
Aymeric

-- 
Antisip - http://www.antisip.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20191213/6dc6b1de/attachment.html>


More information about the sr-users mailing list