<p></p>
<h3>Description</h3>
<p>I found a memory leak which is causing some issues for us in our production. I think I was able to reproduce it and determine that this is happening when we set up an MSRP connection, and that connection is closed due to inactivity (for example when <code>tcp_connection_lifetime</code> is triggered).</p>
<p>This issue is the same one than <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1020124308" data-permission-text="Title is private" data-url="https://github.com/kamailio/kamailio/issues/2875" data-hovercard-type="issue" data-hovercard-url="/kamailio/kamailio/issues/2875/hovercard" href="https://github.com/kamailio/kamailio/issues/2875">#2875</a> but with latest supported Kamailio version.</p>
<h3>Troubleshooting</h3>
<h4>Reproduction</h4>
<p>In this environment we have a WebRTC SIP client, connected to Kamailio and a second party using SIP, both sending messages each via MSRP protocol. Web Client and SIP client establish a session normally, all fine here, they can communicate. At this point we have this status:</p>
<pre><code> TESTER|centos8.4|tester21 [2021-10-11][12:49:24] [/home/vagrant]$ sudo netstat -natp | grep 10.22.22
tcp 0 0 10.22.22.21:5060 0.0.0.0:* LISTEN 14714/kamailio
tcp 0 0 10.22.22.21:10000 0.0.0.0:* LISTEN 14714/kamailio
tcp 0 0 10.22.22.21:8080 0.0.0.0:* LISTEN 14714/kamailio
tcp 0 0 10.22.22.21:10001 0.0.0.0:* LISTEN 14714/kamailio
tcp 0 0 10.22.22.21:10002 0.0.0.0:* LISTEN 14714/kamailio
tcp 0 0 10.22.22.21:8080 10.22.22.1:58530 ESTABLISHED 14653/kamailio
tcp 0 0 10.22.22.21:10000 10.22.22.190:42530 ESTABLISHED 14714/kamailio
tcp 0 0 10.22.22.21:8080 10.22.22.1:58532 ESTABLISHED 14704/kamailio
TESTER|centos8.4|tester21 [2021-10-11][12:49:39] [/home/vagrant]$ kamcmd -s udp:127.0.0.1:2046 mod.stats core all | grep tcp
tcpconn_new(1134): 199392
init_tcp(4962): 8192
init_tcp(4956): 32768
init_tcp(4947): 8
init_tcp(4940): 8
init_tcp(4933): 8
init_tcp(4927): 8
init_tcp(4915): 8
</code></pre>
<p>In order to trigger this failure much faster, I set <code>tcp_connection_lifetime=60</code>. Then we just wait 60 seconds to let the TCP connections closed. Then we finish the session, and stop our WebClient registering, so all connections are cleaned up.</p>
<pre><code> TESTER|centos8.4|tester21 [2021-10-11][12:50:33] [/home/vagrant]$ sudo netstat -natp | grep 10.22.22
tcp 0 0 10.22.22.21:5060 0.0.0.0:* LISTEN 14714/kamailio
tcp 0 0 10.22.22.21:10000 0.0.0.0:* LISTEN 14714/kamailio
tcp 0 0 10.22.22.21:8080 0.0.0.0:* LISTEN 14714/kamailio
tcp 0 0 10.22.22.21:10001 0.0.0.0:* LISTEN 14714/kamailio
tcp 0 0 10.22.22.21:10002 0.0.0.0:* LISTEN 14714/kamailio
TESTER|centos8.4|tester21 [2021-10-11][12:51:34] [/home/vagrant]$ kamcmd -s udp:127.0.0.1:2046 mod.stats core all | grep tcp
tcpconn_new(1134): 66464
init_tcp(4962): 8192
init_tcp(4956): 32768
init_tcp(4947): 8
init_tcp(4940): 8
init_tcp(4933): 8
init_tcp(4927): 8
init_tcp(4915): 8
</code></pre>
<p>As you can see there are no connections but there is memory allocated by <code>tcpconn_new</code>. I know this is related to the MSRP because we noticed our SIP Web Client is not closing the socket, so we just wait for Kamailio to close it.</p>
<h4>Debugging Data</h4>
<pre><code>(paste your debugging data here)
</code></pre>
<h4>Log Messages</h4>
<pre><code>2021-07-25T23:54:41Z ERROR ERROR: <core> [mem/f_malloc.c:415]: fm_search_defrag(): fm_search_defrag(0x7f39a284c000, 66288); Free fragment not found!
2021-07-25T23:54:41Z ERROR ERROR: <core> [mem/f_malloc.c:498]: fm_malloc(): fm_malloc(0x7f39a284c000, 66288) called from core: tcp_main.c: tcpconn_new(957), module: core; Free fragment not found!
2021-07-25T23:54:41Z ERROR ERROR: <core> [tcp_main.c:959]: tcpconn_new(): mem. allocation failure
2021-07-25T23:54:41Z ERROR ERROR: <core> [tcp_main.c:3985]: handle_new_connect(): tcpconn_new failed, closing socket
</code></pre>
<h4>SIP Traffic</h4>
<pre><code>(paste your sip traffic here)
</code></pre>
<h3>Possible Solutions</h3>
<h3>Additional Information</h3>
<ul>
<li><strong>Kamailio Version</strong> - output of <code>kamailio -v</code></li>
</ul>
<pre><code>version: intcc 5.5.2 (x86_64/linux) 55e232
flags: , EXTRA_DEBUGUSE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 55e232
compiled on 12:15:14 Oct 11 2021 with gcc 8.4.1
</code></pre>
<ul>
<li><strong>Operating System</strong>:</li>
</ul>
<pre><code> TESTER|centos8.4|tester21 [2021-10-11][13:52:47] [/home/vagrant]$ uname -a
Linux tester21 4.18.0-305.19.1.el8_4.x86_64 #1 SMP Wed Sep 15 15:39:39 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
centos 8
</code></pre>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/kamailio/kamailio/issues/2880">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/ABO7UZOURHC6VQ62QEZMRKLUGLGFLANCNFSM5FYB2A2A">unsubscribe</a>.<br />Triage notifications on the go with GitHub Mobile for <a href="https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675">iOS</a> or <a href="https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub">Android</a>.
<img src="https://github.com/notifications/beacon/ABO7UZMSBLN33BVBDCUCDODUGLGFLA5CNFSM5FYB2A2KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4PHT26SA.gif" height="1" width="1" alt="" /></p>
<script type="application/ld+json">[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/kamailio/kamailio/issues/2880",
"url": "https://github.com/kamailio/kamailio/issues/2880",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]</script>