<div dir="ltr"><div dir="ltr">Hi Daniel,<div><br></div><div>Tks a lot for lookint at it.</div><div><br></div><div>$ ldd /usr/lib/x86_64-linux-gnu/kamailio/modules/tls.so<br> linux-vdso.so.1 (0x00007fff997dd000)<br> libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007fe40b53c000)<br> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe40b19d000)<br> libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007fe40ad03000)<br> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe40aaff000)<br> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe40a8e2000)<br> /lib64/ld-linux-x86-64.so.2 (0x00007fe40ba4a000)<br><div><br></div><div>$ /usr/sbin/kamailio -I<br>Print out of kamailio internals<br> Version: kamailio 5.3.1 (x86_64/linux)<br> Default config: /etc/kamailio/kamailio.cfg<br> Default paths to modules: /usr/lib/x86_64-linux-gnu/kamailio/modules<br> Compile flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES<br> MAX_RECV_BUFFER_SIZE=262144<br> MAX_URI_SIZE=1024<br> BUF_SIZE=65535<br> DEFAULT PKG_SIZE=8MB<br> DEFAULT SHM_SIZE=64MB<br> ADAPTIVE_WAIT_LOOPS=1024<br> TCP poll methods: poll, epoll_lt, epoll_et, sigio_rt, select<br> Source code revision ID: unknown<br> Compiled with: gcc 6.3.0<br> Compiled architecture: x86_64<br> Compiled on:<br>Thank you for flying kamailio!<br></div></div></div><div><br></div><div>Additional note:</div><div>I have tried to better understand the pike module and after reading the "end" of the module documentation,</div><div>I do better understand the "Tree of IP" and settings.</div><div><br></div>The pike documentation, for each settins and description, should refer to the section "Chapter 3. Developer Guide",<div>otherwise, the parameters cannot be understood. Also, it's not possible to understand, according to me, the real time</div><div>for removing an IP from the tree (removing it 100% or only last node of IP)<br><div><br></div><div>Looking again at my statistics,
I feel the first graph is definitly showing an issue. This graph is showing</div><div>"$stat(location-users)" and "$stat(location-contacts)". During the 10 hours, many users are banned, unregistred, etc..<br></div><div>so it is really not expected that the number of registred users is maintained. From what I understand, the fact</div><div>that the stats went down when deadlock dissapeared obviouly means kamailio threads was in a bad state for the</div><div>last 10 hours...</div><div><br></div><div><a href="https://www.antisip.com/sip-antisip-com-register/status2.htm">https://www.antisip.com/sip-antisip-com-register/status2.htm</a> <br><br></div><div></div><div>If you need more information, let me know...</div><div>Regards</div><div>Aymeric</div><div><br></div><div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le lun. 16 déc. 2019 à 08:22, Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com">miconda@gmail.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello,</p>
<p>can you provide output of ldd for tls.so and output of "kamailio
-I" (that's an uppercase i)?</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 13.12.19 16:39, Aymeric Moizard
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi List,
<div><br>
</div>
<div>History:</div>
<div>* In the past, I had deadlock which was, most probably,
related to ssl1.1.</div>
<div> We have discussed this issue, and a fix is supposed to
workaround the issue that was detected.</div>
<div>* With latest 5.2.X, I have experienced ONCE a similar
behavior with TCP and TLS being mostly stuck. I have not been
using this version much, but the fix was supposed to be in the
core of kamailio.</div>
<div><br>
</div>
<div>The status of the server this night:</div>
<div>* I'm today running version: kamailio 5.3.1
(x86_64/linux), </div>
<div>* Installed on stretch using <a href="http://deb.kamailio.org/kamailio53" target="_blank">http://deb.kamailio.org/kamailio53</a>
repository.</div>
<div>* This versions use libssl1.1</div>
<div>* A user reported that he can't connect with TCP</div>
<div>* An average of 5000 IPs per 10 minutes are being banned by
the pike module</div>
<div> (could be twice the same)</div>
<div>Yesterday/Today:</div>
<div>* at the end of the outage, I had 2479 IP in my ipban
htable. (which is equivalent to my statistics showing 2
bans/IP every 10 minutes = 5000)</div>
<div>* looking at my logs, it appears that most (ALL?) ip being
banned... are my regular users.</div>
<div>* looking at my logs, I can't understand why pike would
block them.</div>
<div><br>
</div>
<div>This is a graph for statistics on my service for the last
24 hours:</div>
<div><a href="https://www.antisip.com/sip-antisip-com-register/status2.html" target="_blank">https://www.antisip.com/sip-antisip-com-register/status2.html</a> <br clear="all">
<div><br>
</div>
<div>Yesterday, at 22:18:39, kamailio started to BAN some IPs.
52 IPs were banned in a period of 10 minutes. I can confirm
this from my logs.</div>
<div><br>
</div>
<div>My pike configuration is this one:</div>
<div><br>
</div>
<div>modparam("pike", "sampling_time_unit", 2)<br>
modparam("pike", "reqs_density_per_unit", 64)<br>
modparam("pike", "remove_latency", 4)<br>
<br>
</div>
<div>When detecting the issue, this morning, I typed:</div>
<div><br>
</div>
<div>$> sudo kamctl stats<br>
</div>
<div>$> sudo kamcmd htable.dump ipban<br>
</div>
<div>//FAILURE (answer too large...)</div>
<div>
$> sudo kamctl trap<br>
</div>
<div><br>
</div>
<div>Then, I started an agent with TCP and it worked...???</div>
<div>
Then, a few seconds, may be a minute after:</div>
<div><br>
</div>
<div>$> sudo kamcmd htable.dump ipban<br>
</div>
<div>//SUCCESS and shows 2479 banned ip.</div>
<div><br>
</div>
<div>and... everything is back to normal in a few minutes.</div>
<div><br>
</div>
<div>I haven't restarted kamailio, and all statistics are as
expected, as usual.</div>
<div><br>
</div>
<div>Thus, it looks that "
sudo kamctl trap" has triggered something. I already</div>
<div>experienced a similar behavior -when testing my ssl1.1
deadlock last year-.</div>
<div><br>
</div>
<div>2 questions:</div>
<div>1/ I beleive my "pike" configuration should not ban
users. Is my pike configuration wrong?</div>
<div>As an example, pike has banned an IP sending one
message/second. I believe my configuration should accept
that?</div>
<div><br>
</div>
<div>2/ Could there still be a TLS issue with libssl1.1?</div>
<div><br>
</div>
<div>This is the result of the "kamctl trap":</div>
<div><br>
</div>
<div><a href="https://sip.antisip.com/kamailio-pike-or-tls-issue-13-12-2019.kamctl-trap" target="_blank">https://sip.antisip.com/kamailio-pike-or-tls-issue-13-12-2019.kamctl-trap</a><br>
</div>
<div><br>
</div>
<div>Sorry for the long story & hoping to find a long term
solution or at least a workaround!</div>
<div><br>
</div>
<div>Regards</div>
<div>Aymeric</div>
<div><br>
</div>
-- <br>
<div dir="ltr"><img src="http://sip.antisip.com/am48.png">Antisip - <a href="http://www.antisip.com" target="_blank">http://www.antisip.com</a><br>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Kamailio (SER) - Users Mailing List
<a href="mailto:sr-users@lists.kamailio.org" target="_blank">sr-users@lists.kamailio.org</a>
<a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
</blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank">www.linkedin.com/in/miconda</a>
Kamailio World Conference - April 27-29, 2020, in Berlin -- <a href="http://www.kamailioworld.com" target="_blank">www.kamailioworld.com</a></pre>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><img src="http://sip.antisip.com/am48.png">Antisip - <a href="http://www.antisip.com" target="_blank">http://www.antisip.com</a><br></div></div></div></div>