<!--
Kamailio Project uses GitHub Issues only for bugs in the code or feature requests. Please use this template only for bug reports.
If you have questions about using Kamailio or related to its configuration file, ask on sr-users mailing list:
* https://lists.kamailio.org/mailman3/postorius/lists/sr-users.lists.kamailio…
If you have questions about developing extensions to Kamailio or its existing C code, ask on sr-dev mailing list:
* https://lists.kamailio.org/mailman3/postorius/lists/sr-dev.lists.kamailio.o…
Please try to fill this template as much as possible for any issue. It helps the developers to troubleshoot the issue.
If there is no content to be filled in a section, the entire section can be removed.
You can delete the comments from the template sections when filling.
You can delete next line and everything above before submitting (it is a comment).
-->
### Description
<!--
Explain what you did, what you expected to happen, and what actually happened.
-->
Kamailio does not properly release connected WebSocket TCP connections, leading to a situation where the maximum number of allowable connections is reached and new connection requests are rejected.
### Troubleshooting
When examining the logs, we identified an issue where Kamailio is failing to release connected WebSocket TCP connections, causing a backlog of TIME_WAIT connections. The logs show a warning message indicating that the maximum number of allowable sockets per IP has been reached, leading to the rejection of further WebSocket requests.
#### Reproduction
Establish WebSocket connections with Kamailio.
Monitor the number of TIME_WAIT connections using netstat.
Observe the warning message in Kamailio logs when the maximum allowable sockets per IP are reached.
<!--
If the issue can be reproduced, describe how it can be done.
-->
#### Debugging Data
<!--
If you got a core dump, use gdb to extract troubleshooting data - full backtrace,
local variables and the list of the code at the issue location.
gdb /path/to/kamailio /path/to/corefile
bt full
info locals
list
If you are familiar with gdb, feel free to attach more of what you consider to
be relevant.
-->
```
(paste your debugging data here)
```
#### Log Messages
<!--
Check the syslog file and if there are relevant log messages printed by Kamailio, add them next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->
```
2023-10-03T13:19:33.699043+00:00 apps001 kamailio[23781]: WARNING: |<null>|websockets-role.cfg:224 (websocket_request) 223.233.86.93 is at the maximum 50 allowable sockets per IP, rejecting request for another websocket
netstat -punta | grep 223.233.86.93 | grep 5065
tcp 0 0 45.79.166.214:5065 223.233.86.93:52158 TIME_WAIT -
tcp 0 0 45.79.166.214:5065 223.233.86.93:52195 TIME_WAIT -
tcp 0 0 45.79.166.214:5065 223.233.86.93:52292 TIME_WAIT -
tcp 0 0 45.79.166.214:5065 223.233.86.93:52248 TIME_WAIT -
tcp 0 0 45.79.166.214:5065 223.233.86.93:52108 TIME_WAIT -
tcp 0 0 45.79.166.214:5065 223.233.86.93:52071 TIME_WAIT -
```
#### SIP Traffic
<!--
If the issue is exposed by processing specific SIP messages, grab them with ngrep or save in a pcap file, then add them next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->
```
(paste your sip traffic here)
```
### Possible Solutions
<!--
If you found a solution or workaround for the issue, describe it. Ideally, provide a pull request with a fix.
-->
restarting kamailio solves the problem.
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
```
version: kamailio 5.7.2 (x86_64/linux)
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled on 14:39:54 Sep 27 2023 with gcc 7.3.1
```
* **Operating System**:
<!--
Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...;
Kernel details (output of `lsb_release -a` and `uname -a`)
-->
```
CentOS Linux release 7.9.2009 (Core)
```
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/3589
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/3589(a)github.com>
Hello all
we are seeing these kind of logs in a Debian GNU/Linux 11 using kamailio 5.5.6
```Sep 21 17:41:13 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155702]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[164.152.22.248:5060](http://164.152.22.248:5060/))
Sep 21 17:41:13 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155698]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[208.74.138.184:5060](http://208.74.138.184:5060/))
Sep 21 17:41:13 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155695]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[208.74.138.181:5060](http://208.74.138.181:5060/))
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155693]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[38.102.250.60:5060](http://38.102.250.60:5060/))
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155707]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[87.1.1.27:5060](http://87.1.1.27:5060/))
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155707]: ERROR: tm [ut.h:302]: uri2dst2(): no corresponding socket for "87.1.1.27" af 2
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155707]: ERROR: tm [t_fwd.c:470]: prepare_new_uac(): can't fwd to af 2, proto 1 (no corresponding listening socket)
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155697]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[208.74.138.184:5060](http://208.74.138.184:5060/))
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155694]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[192.40.216.97:5060](http://192.40.216.97:5060/))
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155694]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[208.74.138.180:5060](http://208.74.138.180:5060/))
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155714]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[87.1.1.27:5060](http://87.1.1.27:5060/))
```
The kamailio instance we are using is receiving around 1500 calls per second in average when we do start seeing these errors more frequently
In this instance we are using multihomed
the listen address list is
``` children=14
socket_workers=1
listen=udp:[192.168.99.70:5081](http://192.168.99.70:5081/)
listen=udp:[192.168.99.81:5060](http://192.168.99.81:5060/)
listen=udp:[87.1.1.27:5060](http://87.1.1.27:5060/)
listen=tcp:[192.168.96.105:8095](http://192.168.96.105:8095/)
listen=tcp:[87.1.1.27:5060](http://87.1.1.27:5060/)
tcp_children=6
port=5060
```
We use the first worker only to perform the OPTIONS requests from the dispatcher module.
The sockets
listen=udp:[192.168.99.81:5060](http://192.168.99.81:5060/)
listen=udp:[87.1.1.27:5060](http://87.1.1.27:5060/)
are used to communicate both A and B legs
so when a message is received at [192.168.99.81:5060](http://192.168.99.81:5060/) we send it to [87.1.1.27:5060](http://87.1.1.27:5060/) and viceversa
We were using $fs variable before doing the t_relay() function.
This way is working, but when load increases, seems sometimes kamailio doesn't get the proper socket to forward the reply.
I think the errors mainly are related to responses which are being forwarded, like if the function get_sock_info_list was not able to retrieve the listen interfaces
we have tried to set in the onreply routes the commands
set_recv_socket("udp:[192.168.99.81:5060](http://192.168.99.81:5060/)");
set_send_socket("udp:[87.1.1.27:5060](http://87.1.1.27:5060/)"); (when reply goes from private to public domain)
and it seems it reduces the number of
```
Sep 21 17:41:13 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155702]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[164.152.22.248:5060](http://164.152.22.248:5060/))
Sep 21 17:41:13 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155698]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[208.74.138.184:5060](http://208.74.138.184:5060/))
Sep 21 17:41:13 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155695]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[208.74.138.181:5060](http://208.74.138.181:5060/))
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155693]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:[38.102.250.60:5060](http://38.102.250.60:5060/))
errors
```
But i honestly don't know why the
```Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155707]: ERROR: tm [ut.h:302]: uri2dst2(): no corresponding socket for "87.1.1.27" af 2
Sep 21 17:41:14 lax-dedge-1 /usr/local/kamailio/sbin/kamailio[3155707]: ERROR: tm [t_fwd.c:470]: prepare_new_uac(): can't fwd to af 2, proto 1 (no corresponding listening socket)
appear
```
We also tried to increase the kernel buffers rmem and wmem from 208KB (as we had by default) to 4MB
Do you know a reason which could cause these logs to appear?
Any setup that could mitigate them?
thanks a lot and regards
david escartin
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/3583
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/3583(a)github.com>