[SR-Users] TCP connection lifetime
Aymeric Moizard
amoizard at gmail.com
Tue Mar 26 17:16:04 CET 2019
Hi Again,
Here is an issue with TCP connection being kept for more:
Yesterday, I have discovered that a User-Agent (<Avaya IP Phone 1120E
(SIP1120e.04.04.30.00)> tried to register a lot. It was sending REGISTER
over new established TCP socket *every 2 seconds*.
All the REGISTER was rejected with 401. (may be the device was
misconfigured? or not receiving any of my answer? I can't tell)
NOTE: You can see the expires header was very large: 86400, ie: 24 hours...
I was checking the TCP/TLS connections on my server and discovered more
than 1000 TCP established connection to that user/ip, and thus, I have
tried to understand what happened.
Checking the logs, I received 4855 REGISTER from this device from "Mar 25
03:47:09" to "Mar 25 07:56:13" which is a rate of approx one new TCP
connection every 2.5 seconds...
Today, I decided to check it again around 11am.
jack at sip:~$ sudo kamctl stats tcp
{
"jsonrpc": "2.0",
"result": [
"tcp:con_reset = 1857",
"tcp:con_timeout = 35927",
"tcp:connect_failed = 25",
"tcp:connect_success = 2",
"tcp:current_opened_connections = 2291",
"tcp:current_write_queue_size = 0",
"tcp:established = 80778",
"tcp:local_reject = 0",
"tcp:passive_open = 80776",
"tcp:send_timeout = 2",
"tcp:sendq_full = 0"
],
"id": 7305
}
There was still A LOT of established connections. And the connections have
been established more than 24 hours ago.
At 11H16:
$> lsof -n -l | grep kamailio | grep TCP | grep 41.234.242.69 | grep ESTA |
wc -l
1161
At 11H22:
$> lsof -n -l | grep kamailio | grep TCP | grep 41.234.242.69 | grep ESTA |
wc -l
1018
At 11H35:
$> lsof -n -l | grep kamailio | grep TCP | grep 41.234.242.69 | grep ESTA |
wc -l
655
At 13H
$> lsof -n -l | grep kamailio | grep TCP | grep 41.234.242.69 | grep ESTA |
wc -l
0
So the established connections are all gone now.
Between 11h16 and 11H35, I was seeing the server regularly sending [FIN,
ACK] over each TCP established connection, with retransmissions for all of
them. (no incoming trafic)
I do not have numbers/capture/stats, but I think that kamailio was already
closing some
connection yesterday. I don't know when kamailio started to try closing
those connections.
I'm now back with this status:
At 13pm:
jack at sip:~$ sudo kamctl stats tcp
{
"jsonrpc": "2.0",
"result": [
"tcp:con_reset = 1896",
"tcp:con_timeout = 38042",
"tcp:connect_failed = 26",
"tcp:connect_success = 2",
"tcp:current_opened_connections = 939",
"tcp:current_write_queue_size = 0",
"tcp:established = 81950",
"tcp:local_reject = 0",
"tcp:passive_open = 81948",
"tcp:send_timeout = 2",
"tcp:sendq_full = 0"
],
"id": 12734
}
With around 155 registration entries using TCP and TLS in my location
database.
As you can see, tcp:current_opened_connections = 939 is still pretty high
compared to
my currently registred users.
I have "modparam("registrar", "max_expires", 86400)", because I'm keeping
contact entries (even with TCP connection down) for push notifications.
I have "tcp_connection_lifetime=3600" configured.
Question 1
With "tcp_connection_lifetime=3600", I would expect kamailio to close the
established connection after 3600 seconds without traffic. It is pretty
obvious that no data has been exchanged over the 4855 established
connection during a day.
Despite the issue with the Avaya phones is solved automatically after a
day, I guess similar stuff or happening, at a different rate, for other
users as well. (because current_opened_connections is way higher than
registred TCP/TLS users)
Question 2
I can list TLS connection with "kamctl rpc tls.list"
Can I get a similar list for TCP? (lsof returns a lot of duplicates...)
Tks
Aymeric
--
Antisip - http://www.antisip.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20190326/6fd5326e/attachment.html>
More information about the sr-users
mailing list