<div dir="ltr"><div>You have many problems,</div><div><br></div><div>1. You do not seem to have TCP keepalives setup. So idle TCP connections may be disrupted, closed or reset by any router on the internet, forcing SIP client to re-register. As a result many more tcp connections are created by a single SIP end point then expected. You said you have 5K active users with 8K connection limit, which is way too low. The limit should be at least more then twice the active users (12.5K in your case). I usually just set it to maximum i.e. 64K.</div><div><br></div><div>2. TCP connection life time of 3605 seconds is way too high. There will be many many hack attempts on your server, some users will enter wrong password, some users change their network (and thus need to re-register), so allowing 3605 seconds life time for ALL of them is a HUGE mistake, and you will run out of connection limit pretty quick. So keep the life time as small as possible 5 or 10 seconds (approx. max time the server takes to process sip register x 20 e.g. 250ms x 20 = 5000 ms = 5 seconds). Once the SIP client successfully registers, you can increase the TCP life time using,<br></div><div><br></div><div>
<a href="https://kamailio.org/docs/modules/4.4.x/modules/tcpops.html#tcpops.f.tcp_set_connection_lifetime">https://kamailio.org/docs/modules/4.4.x/modules/tcpops.html#tcpops.f.tcp_set_connection_lifetime</a>
</div><div><br></div><div>3. Similarly setup TCP keepalives according to SIP Register success,</div><div><br></div><div>
<a href="https://kamailio.org/docs/modules/4.4.x/modules/tcpops.html#tcpops.f.tcp_keepalive_enable">https://kamailio.org/docs/modules/4.4.x/modules/tcpops.html#tcpops.f.tcp_keepalive_enable</a> <br></div><div>
<a href="https://kamailio.org/docs/modules/4.4.x/modules/tcpops.html#tcpops.f.tcp_keepalive_disable">https://kamailio.org/docs/modules/4.4.x/modules/tcpops.html#tcpops.f.tcp_keepalive_disable</a>
</div><div><br></div><div>4. If average SIP packet processing time is too high e.g. due to slow db connection for authentication etc. push notification setup, web queries etc. then use ASYNC for all time consuming tasks. So TCP listener processes get freed quickly and can process more new connections.<br></div><div><br></div><div>5. It's good to monitor and analyze TCP connection states and events, especially when TCP connections closes and why? See the TCPOPS module for more help.<br></div><div><br></div><div>Hope this helps.<br></div><div><br></div><div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 4, 2020 at 5:30 PM Tomas Zanet <<a href="mailto:tzanet@came.com">tzanet@came.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="IT">
<div class="gmail-m_7490540919437293444WordSection1">
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">As you suggested I increased the debug level from -1 to 3 and I got this Kamailio log:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:21 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [io_wait.h:602]: io_watch_del(): DBG: io_watch_del
(0xa20dc0, 3433, -1, 0x0) fd_no=2922 called<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:21 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:4139]: handle_tcpconn_ev(): sending
to child, events 1<u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:21 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (30)<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:21 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3821]: send2child(): selected
tcp worker 8 36(29409) for activity on [tls:X.X.X.X:5061], 0x7f482c2c2bc0<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:21 server-xip-01 /usr/local/sbin/kamailio[29409]: DEBUG: <core> [tcp_read.c:1576]: handle_io(): received
n=8 con=0x7f482c2c2bc0, fd=13<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:21 server-xip-01 /usr/local/sbin/kamailio[29409]: DEBUG: tls [tls_server.c:197]: tls_complete_init(): completing
tls connection initialization<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:21 server-xip-01 /usr/local/sbin/kamailio[29409]: DEBUG: tls [tls_server.c:226]: tls_complete_init(): Using
initial TLS domain TLSs<default> (dom 0x7f481b22ba50 ctx 0x7f481b6fbf20 sn [])<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:21 server-xip-01 /usr/local/sbin/kamailio[29409]: DEBUG: tls [tls_domain.c:715]: sr_ssl_ctx_info_callback():
SSL handshake started<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Looking for this warning into the log, I realized that there are a lot of these messages:
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">sudo tail -f /var/log/kamailio.log | grep "no free tcp receiver"
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:23 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (20)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:23 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (20)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:23 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (21)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:23 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (20)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:23 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (21)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (14)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (15)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (16)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (16)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (16)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (17)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (17)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (17)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (16)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (17)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (18)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (18)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (18)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (18)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (18)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Mar 4 15:40:24 server-xip-01 /usr/local/sbin/kamailio[29417]: DEBUG: <core> [tcp_main.c:3816]: send2child(): WARNING:
no free tcp receiver, connection passed to the least busy one (18)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">…..<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">….<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">As far as I know, I should increase the tcp_children parameter, right?.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Now, it’s set to 16 but it’s pretty clear which is not enough. I increased it step by step till 48 but that’s still not
enough. <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">If I set it to 64 then Kamailio failed to start.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">What would you recommend for a server which has more or less 5K users?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">Many thanks.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB">T.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri",sans-serif;color:rgb(31,73,125)" lang="EN-GB"><u></u> <u></u></span></p>
<p style="font-family:arial;font-size:10pt"><span style="font-size:10pt" id="gmail-m_7490540919437293444c1-id-7"><strong style="font-size:10pt" id="gmail-m_7490540919437293444c1-id-8">Tomas Zanet</strong><br style="font-size:10pt" id="gmail-m_7490540919437293444c1-id-9">
R&D Software Designer<br style="font-size:10pt" id="gmail-m_7490540919437293444c1-id-10">
<a href="mailto:tzanet@came.com" target="_blank">tzanet@came.com</a><br>
</span><br id="gmail-m_7490540919437293444c1-id-12">
<a id="gmail-m_7490540919437293444c1-id-14" title="CAME" href="http://www.came.com" name="m_7490540919437293444_c1-id-14" target="_blank"></a><img src="cid:170a6d595c46510e87b1" border="0"><br>
<br id="gmail-m_7490540919437293444c1-id-16">
<span style="font-size:8pt" id="gmail-m_7490540919437293444c1-id-17"><strong id="gmail-m_7490540919437293444c1-id-18"><span style="font-size:10pt">CAME S.p.A.</span></strong><br style="font-size:10pt">
<span style="font-size:10pt"><strong>Sede legale e operativa</strong><br>
</span></span><span style="font-size:10pt">Via Martiri della Libertà 15<br style="font-size:10pt">
31030 Dosson di Casier (TV) / Italy<br style="font-size:10pt" id="gmail-m_7490540919437293444c1-id-20">
<strong><font color="#00b0ed">T</font></strong> +39 0422 4940 <font style="font-size:10pt" color="#02b2ee">
<strong style="font-size:10pt">F</strong></font> +39 0422 4941<br>
<strong><font color="#02b2ee">P.IVA</font></strong> 03481280265<br>
Cod. dest. fatturazione elettronica: UHWULCH</span><span style="font-size:10pt"><br>
</span></p>
<p style="font-family:arial;font-size:10pt"><span style="font-size:8pt" id="gmail-m_7490540919437293444c1-id-24"><span style="font-size:10pt"><strong>Sede operativa</strong></span><br style="font-size:10pt">
<span style="font-size:10pt">Via Cornia 1/b,c</span><br style="font-size:10pt">
<span style="font-size:10pt">33079 Sesto al Reghena (PN) / Italy</span><br style="font-size:10pt" id="gmail-m_7490540919437293444c1-id-25">
<span style="font-size:10pt"><a href="http://www.came.com" target="_blank"><font color="#02b2ee">www.came.com</font></a><br>
</span><br style="font-size:10pt">
<span style="font-size:10pt">Follow us on</span>: <a style="font-size:10pt" href="https://www.facebook.com/CAME-Italia-1604586846431438/" target="_blank">Facebook</a> / <a style="font-size:10pt" href="https://twitter.com/CameGlobal" target="_blank">Twitter</a> /
<a style="font-size:10pt" href="https://www.youtube.com/user/CameChannel" target="_blank">Youtube</a> /
<a style="font-size:10pt" href="https://www.linkedin.com/company/came" target="_blank">Linkedin</a></span></p>
<p style="font-family:arial;font-size:10pt"><span style="font-size:7pt">Le informazioni contenute in questa e-mail e negli eventuali allegati sono riservate e confidenziali e ne è vietata la diffusione e distribuzione in qualunque modo. Qualora non ne fosse
il legittimo destinatario, è invitato a non diffonderle e ad eliminarle, dandone gentilmente informazione al mittente.
<br>
</span><span style="font-size:7pt"><br>
The information included in this e-mail and any attachments are confidential and can not be disclosed, duplicated or redistributed by any means. If you are not the correct recipient, you are kindly requested to cancel it and notify the sender immediately
</span></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:"Calibri",sans-serif" lang="EN-GB">Da:</span></b><span style="font-size:11pt;font-family:"Calibri",sans-serif" lang="EN-GB"> sr-users [mailto:<a href="mailto:sr-users-bounces@lists.kamailio.org" target="_blank">sr-users-bounces@lists.kamailio.org</a>]
<b>Per conto di </b>Denys Pozniak<br>
<b>Inviato:</b> mercoledì 4 marzo 2020 15:00<br>
<b>A:</b> Kamailio (SER) - Users Mailing List <<a href="mailto:sr-users@lists.kamailio.org" target="_blank">sr-users@lists.kamailio.org</a>><br>
<b>Oggetto:</b> Re: [SR-Users] Kamailio connection issue<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<div>
<p class="MsoNormal"><span lang="EN-GB">Hello!<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span lang="EN-GB">You can try to increase the debug printing level (like debug=2 or even debug=3) and check syslog when Kamailio gets stuck.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
<div>
<div>
<p class="MsoNormal">ср<span lang="EN-GB">, 4 </span>мар<span lang="EN-GB">. 2020
</span>г<span lang="EN-GB">. </span>в<span lang="EN-GB"> 13:12, Tomas Zanet <</span><a href="mailto:tzanet@came.com" target="_blank"><span lang="EN-GB">tzanet@came.com</span></a><span lang="EN-GB">>:<u></u><u></u></span></p>
</div>
<blockquote style="border-color:currentcolor currentcolor currentcolor rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<p class="MsoNormal"><span lang="EN-GB">Hi All,<br>
<br>
In these days I'm facing an issue or a misconfiguration, I don't know, with our remote apps (SIP clients) and our cloud server in which there's a Kamailio version 4.4.4.<br>
Sometimes our app takes some seconds (about 10-12sec) to connect to the Kamailio socket (sips on 5061) and our app is stuck on connect sytem call.<br>
<br>
To debug this behaviour I put a rule in the iptable like this:<br>
sudo iptables -I INPUT -p tcp -m tcp --dport 5061 -m state --state NEW -j LOG --log-level 1 --log-prefix "New Connection "<br>
<br>
when everything goes well, there is one row which describes the new incoming connection Mar 4 07:08:40 server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00 TTL=45
ID=0 DF PROTO=TCP SPT=15819 DPT=5061 WINDOW=65535 RES=0x00 CWR ECE SYN URGP=0<br>
<br>
when our app gets stuck in the connect, there are many attempts and seems that Kamailio is not ready to accept this connection:<br>
Mar 4 07:11:54 server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061 WINDOW=65535 RES=0x00 CWR ECE SYN URGP=0 Mar 4 07:11:54
server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061 WINDOW=65535 RES=0x00 SYN URGP=0 Mar 4 07:11:55 server-xip-01 kernel:
New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061 WINDOW=65535 RES=0x00 SYN URGP=0 Mar 4 07:11:55 server-xip-01 kernel: New Connection IN=eth0
OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061 WINDOW=65535 RES=0x00 SYN URGP=0 Mar 4 07:11:55 server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00
SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061 WINDOW=65535 RES=0x00 SYN URGP=0 Mar 4 07:11:56 server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y
LEN=64 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061 WINDOW=65535 RES=0x00 SYN URGP=0 Mar 4 07:11:56 server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00
TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061 WINDOW=65535 RES=0x00 SYN URGP=0 Mar 4 07:11:56 server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP
SPT=52900 DPT=5061 WINDOW=65535 RES=0x00 SYN URGP=0 Mar 4 07:11:58 server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061
WINDOW=65535 RES=0x00 SYN URGP=0 Mar 4 07:12:00 server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=64 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061 WINDOW=65535 RES=0x00
SYN URGP=0 Mar 4 07:12:05 server-xip-01 kernel: New Connection IN=eth0 OUT= MAC=bc:76:4e:20:1f:fb:e8:ed:f3:3a:52:41:08:00 SRC=X.X.X.X DST=Y.Y.Y.Y LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=52900 DPT=5061 WINDOW=65535 RES=0x00 SYN URGP=0 When it
happens our app takes more than 10 sec to connect to the Kamailio socket.<br>
<br>
I tried to increase the children configuration parameter in the cfg file because I thought there were not enough tcp receiver but with no luck.<br>
Here are some configuration parameters:<br>
<br>
children = 16 // was 8 before found this issue<br>
<br>
# life time of TCP connection when there is no traffic # - a bit higher than registration expires to cope with UA behind NAT<br>
tcp_connection_lifetime=3605<br>
<br>
open_files_limit=8192<br>
<br>
tcp_max_connections=8192<br>
tls_max_connections=8192<br>
<br>
Here is the output of the ulimit command:<br>
ulimit -a<br>
core file size (blocks, -c) 0<br>
data seg size (kbytes, -d) unlimited<br>
scheduling priority (-e) 0<br>
file size (blocks, -f) unlimited<br>
pending signals (-i) 31393<br>
max locked memory (kbytes, -l) 64<br>
max memory size (kbytes, -m) unlimited<br>
open files (-n) 65535<br>
pipe size (512 bytes, -p) 8<br>
POSIX message queues (bytes, -q) 819200<br>
real-time priority (-r) 0<br>
stack size (kbytes, -s) 8192<br>
cpu time (seconds, -t) unlimited<br>
max user processes (-u) 31393<br>
virtual memory (kbytes, -v) unlimited<br>
file locks (-x) unlimited<br>
In the kamailio log file I don't see anything wrong, I don't see messages like "[tcp_main.c:3913]: handle_new_connect(): maximum number of tls connections exceeded".<br>
<br>
Here is the output of kamctl ps command<br>
Process:: ID=0 PID=11725 Type=main process - attendant<br>
Process:: ID=1 PID=11727 Type=udp receiver child=0 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=2 PID=11728 Type=udp receiver child=1 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=3 PID=11729 Type=udp receiver child=2 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=4 PID=11730 Type=udp receiver child=3 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=5 PID=11731 Type=udp receiver child=4 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=6 PID=11732 Type=udp receiver child=5 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=7 PID=11733 Type=udp receiver child=6 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=8 PID=11734 Type=udp receiver child=7 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=9 PID=11735 Type=udp receiver child=8 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=10 PID=11736 Type=udp receiver child=9 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=11 PID=11737 Type=udp receiver child=10 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=12 PID=11738 Type=udp receiver child=11 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=13 PID=11739 Type=udp receiver child=12 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=14 PID=11740 Type=udp receiver child=13 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=15 PID=11741 Type=udp receiver child=14 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=16 PID=11742 Type=udp receiver child=15 sock=</span><a href="http://10.208.194.26:5061" target="_blank"><span lang="EN-GB">10.208.194.26:5061</span></a><span lang="EN-GB"><br>
Process:: ID=17 PID=11743 Type=slow timer<br>
Process:: ID=18 PID=11744 Type=timer<br>
Process:: ID=19 PID=11745 Type=secondary timer<br>
Process:: ID=20 PID=11746 Type=MI FIFO<br>
Process:: ID=21 PID=11747 Type=USRLOC Timer<br>
Process:: ID=22 PID=11748 Type=USRLOC Timer<br>
Process:: ID=23 PID=11749 Type=USRLOC Timer<br>
Process:: ID=24 PID=11750 Type=USRLOC Timer<br>
Process:: ID=25 PID=11751 Type=ctl handler<br>
Process:: ID=26 PID=11752 Type=RTIMER SEC EXEC<br>
Process:: ID=27 PID=11753 Type=TIMER NH<br>
Process:: ID=28 PID=11754 Type=tcp receiver (generic) child=0<br>
Process:: ID=29 PID=11755 Type=tcp receiver (generic) child=1<br>
Process:: ID=30 PID=11756 Type=tcp receiver (generic) child=2<br>
Process:: ID=31 PID=11757 Type=tcp receiver (generic) child=3<br>
Process:: ID=32 PID=11758 Type=tcp receiver (generic) child=4<br>
Process:: ID=33 PID=11759 Type=tcp receiver (generic) child=5<br>
Process:: ID=34 PID=11760 Type=tcp receiver (generic) child=6<br>
Process:: ID=35 PID=11761 Type=tcp receiver (generic) child=7<br>
Process:: ID=36 PID=11762 Type=tcp receiver (generic) child=8<br>
Process:: ID=37 PID=11763 Type=tcp receiver (generic) child=9<br>
Process:: ID=38 PID=11764 Type=tcp receiver (generic) child=10<br>
Process:: ID=39 PID=11765 Type=tcp receiver (generic) child=11<br>
Process:: ID=40 PID=11766 Type=tcp receiver (generic) child=12<br>
Process:: ID=41 PID=11767 Type=tcp receiver (generic) child=13<br>
Process:: ID=42 PID=11768 Type=tcp receiver (generic) child=14<br>
Process:: ID=43 PID=11769 Type=tcp receiver (generic) child=15<br>
Process:: ID=44 PID=11770 Type=tcp main process<br>
<br>
<br>
Can anyone address me to find out this happens, please?<br>
Please let me know if you need further information...<br>
<br>
Thanks in advance,<br>
Best Regards<br>
Tomas<br>
<br>
<br>
<br>
Tomas Zanet<br>
R&D Software Designer<br>
</span><a href="mailto:tzanet@came.com" target="_blank"><span lang="EN-GB">tzanet@came.com</span></a><span lang="EN-GB"><br>
CAME S.p.A.<br>
<br>
_______________________________________________<br>
Kamailio (SER) - Users Mailing List<br>
</span><a href="mailto:sr-users@lists.kamailio.org" target="_blank"><span lang="EN-GB">sr-users@lists.kamailio.org</span></a><span lang="EN-GB"><br>
</span><a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank"><span lang="EN-GB">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</span></a><span lang="EN-GB"><u></u><u></u></span></p>
</blockquote>
</div>
<p class="MsoNormal"><span lang="EN-GB"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span lang="EN-GB"><u></u> <u></u></span></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">BR,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Denys Pozniak<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
Kamailio (SER) - Users Mailing List<br>
<a href="mailto:sr-users@lists.kamailio.org" target="_blank">sr-users@lists.kamailio.org</a><br>
<a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" rel="noreferrer" target="_blank">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a><br>
</blockquote></div>