<h3>Description</h3>
<p>When SIP message fails to be sent due to no socket found, apparently the TM module is not deleting the transaction and so it is causing a memory leak.</p>
<h3>Troubleshooting</h3>
<h4>Reproduction</h4>
<p>The easiest way to reproduce the problem is to setup a dispatcher destination with SIP ping enabled. Kamailio should not be listening to the interface which is used to reach that destination. So this will cause Kamailio to complain about not finding a socket to send the OPTIONS.<br>
After this happens, it is possible to see from the <strong>kamcmd tm.stats</strong> command that the "created" has increased by 1, freed has not changed and "delayed_free" also increased by 1.<br>
Problem is that the delayed_free never gets freed and after checking the <strong>kamcmd core.shmmem</strong>, for every failed OPTIONS request under this condition, 5808 bits are leaked...<br>
This goes on until there is no shared memory left.....</p>
<h4>Debugging Data</h4>
<h4>Log Messages</h4>
<pre><code> /usr/sbin/kamailio[20755]: DEBUG: dispatcher [dispatch.c:2664]: ds_ping_set(): probing set #2, URI sip:10.11.12.13:6060
/usr/sbin/kamailio[20755]: DEBUG: tm [uac.c:427]: t_uac_prepare(): next_hop=<sip:10.11.12.13:6060>
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/resolve.c:1225]: srv_sip_resolvehost(): 10.11.12.13:6060 proto=0
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/resolve.c:1346]: srv_sip_resolvehost(): returning 0xada520 (10.11.12.13:6060 proto=1)
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/forward.c:174]: get_out_socket(): socket determined: 0x7f33189ee298
/usr/sbin/kamailio[20755]: DEBUG: tm [uac.c:152]: dlg2hash(): hashid 8554
/usr/sbin/kamailio[20755]: DEBUG: tm [uac.c:238]: t_run_local_req(): executing event_route[tm:local-request]
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:600]: parse_msg(): SIP Request:
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:602]: parse_msg(): method: <OPTIONS>
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:604]: parse_msg(): uri: <sip:10.11.12.13:6060>
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:606]: parse_msg(): version: <SIP/2.0>
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/parse_via.c:1303]: parse_via_param(): Found param type 232, <branch> = <z9hG4bKa612.f9317e75000000000000000000000000.0>; state=16
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/parse_via.c:2639]: parse_via(): end of header reached, state=5
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:491]: parse_headers(): Via found, flags=2
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:493]: parse_headers(): this is the first via
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/parse_addr_spec.c:864]: parse_addr_spec(): end of header reached, state=10
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:171]: get_hdr_field(): <To> [25]; uri=[sip:10.11.12.13:6060]
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:172]: get_hdr_field(): to body [<sip:10.11.12.13:6060>#015#012]
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:152]: get_hdr_field(): cseq <CSeq>: <10> <OPTIONS>
/usr/sbin/kamailio[20755]: DEBUG: pv [pv_trans.c:564]: tr_eval_string(): i=0 j=10
/usr/sbin/kamailio[20755]: NOTICE: <script>: [479309551f] OPTIONS: Local Request to sip:10.11.12.13:6060 - sip:10.11.12.13:6060
/usr/sbin/kamailio[20755]: DEBUG: dialog [dlg_cseq.c:115]: dlg_cseq_prepare_new_msg(): prepare new msg for cseq update operations
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:600]: parse_msg(): SIP Request:
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:602]: parse_msg(): method: <OPTIONS>
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:604]: parse_msg(): uri: <sip:10.11.12.13:6060>
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:606]: parse_msg(): version: <SIP/2.0>
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/parse_via.c:1303]: parse_via_param(): Found param type 232, <branch> = <z9hG4bKa612.f9317e75000000000000000000000000.0>; state=16
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/parse_via.c:2639]: parse_via(): end of header reached, state=5
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:491]: parse_headers(): Via found, flags=2
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:493]: parse_headers(): this is the first via
/usr/sbin/kamailio[20755]: DEBUG: dialog [dlg_cseq.c:59]: dlg_cseq_prepare_msg(): prepare msg for cseq update operations
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/parse_addr_spec.c:864]: parse_addr_spec(): end of header reached, state=10
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:171]: get_hdr_field(): <To> [25]; uri=[sip:10.11.12.13:6060]
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:172]: get_hdr_field(): to body [<sip:10.11.12.13:6060>#015#012]
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/msg_parser.c:152]: get_hdr_field(): cseq <CSeq>: <10> <OPTIONS>
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/parse_addr_spec.c:185]: parse_to_param(): add param: tag=744e9d847dad9acc2d5178ac65520eba-8cc2
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/parser/parse_addr_spec.c:864]: parse_addr_spec(): end of header reached, state=29
/usr/sbin/kamailio[20755]: DEBUG: dialog [dlg_cseq.c:382]: dlg_cseq_msg_sent(): traking cseq updates
/usr/sbin/kamailio[20755]: DEBUG: dialog [dlg_hash.c:766]: internal_get_dlg(): no dialog callid='479309551fdf88bf-20755@55.66.77.88' found
/usr/sbin/kamailio[20755]: DEBUG: dialog [dlg_hash.c:797]: get_dlg(): no dialog callid='479309551fdf88bf-20755@55.66.77.88' found
/usr/sbin/kamailio[20755]: DEBUG: dialog [dlg_handlers.c:1202]: dlg_lookup_msg_dialog(): dlg with callid '479309551fdf88bf-20755@55.66.77.88' not found
/usr/sbin/kamailio[20755]: DEBUG: dialog [dlg_cseq.c:394]: dlg_cseq_msg_sent(): no dialog for this request
/usr/sbin/kamailio[20755]: DEBUG: tm [uac.c:638]: send_prepared_request_impl(): uac: 0x7f331184c5f8 branch: 0 to 10.11.12.13:6060
/usr/sbin/kamailio[20755]: DEBUG: tm [../../core/onsend.h:69]: run_onsend(): required parameters are not available - ignoring
/usr/sbin/kamailio[20755]: DEBUG: dispatcher [dispatch.c:2664]: ds_ping_set(): probing set #1, URI sip:12.34.56.78:6060
/usr/sbin/kamailio[20755]: DEBUG: tm [uac.c:427]: t_uac_prepare(): next_hop=<sip:12.34.56.78:6060>
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/resolve.c:1225]: srv_sip_resolvehost(): 12.34.56.78:6060 proto=0
/usr/sbin/kamailio[20755]: DEBUG: <core> [core/resolve.c:1346]: srv_sip_resolvehost(): returning 0xada520 (12.34.56.78:6060 proto=1)
/usr/sbin/kamailio[20755]: ERROR: <core> [core/forward.c:181]: get_out_socket(): no socket found
/usr/sbin/kamailio[20755]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:12.34.56.78:6060)
/usr/sbin/kamailio[20755]: ERROR: tm [ut.h:317]: uri2dst2(): no corresponding socket found for "12.34.56.78" af 2 (udp:12.34.56.78:6060)
/usr/sbin/kamailio[20755]: ERROR: tm [uac.c:449]: t_uac_prepare(): no socket found
/usr/sbin/kamailio[20755]: ERROR: dispatcher [dispatch.c:2683]: ds_ping_set(): unable to ping [sip:12.34.56.78:6060]
</code></pre>
<p>Real IPs were replaced, this is their usage:<br>
10.11.12.13 = Kamailio listening IP in local network interface<br>
12.34.56.78 = destination SIP trunk which does not have a matching listened socket by Kamailio<br>
55.66.77.88 = Kamailio listening IP in public network interface</p>
<h4>SIP Traffic</h4>
<h3>Possible Solutions</h3>
<h3>Additional Information</h3>
<ul>
<li><strong>Kamailio Version</strong> - output of <code>kamailio -v</code></li>
</ul>
<pre><code># kamailio -v
version: kamailio 5.0.6 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled on 04:19:46 Mar 30 2018 with gcc 4.8.5
</code></pre>
<ul>
<li><strong>Operating System</strong>:</li>
</ul>
<pre><code># cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
# uname -a
Linux dev1-sbcaio-01 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
</code></pre>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/kamailio/kamailio/issues/1554">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AF36ZWIOS5iQBg8rY5SzJn4ZwcJN3XKFks5t50E0gaJpZM4Ub5GR">mute the thread</a>.<img src="https://github.com/notifications/beacon/AF36Zc7aeLRw6VBte4OOi-R9CBjpbmvFks5t50E0gaJpZM4Ub5GR.gif" height="1" width="1" alt="" /></p>
<script type="application/ld+json">{"@context":"http://schema.org","@type":"EmailMessage","potentialAction":{"@type":"ViewAction","target":"https://github.com/kamailio/kamailio/issues/1554","url":"https://github.com/kamailio/kamailio/issues/1554","name":"View Issue"},"description":"View this Issue on GitHub","publisher":{"@type":"Organization","name":"GitHub","url":"https://github.com"}}</script>
<script type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/kamailio/kamailio","title":"kamailio/kamailio","subtitle":"GitHub repository","main_image_url":"https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/kamailio/kamailio"}},"updates":{"snippets":[{"icon":"DESCRIPTION","message":"Memory leak after SIP message fails to be sent due to no socket found (#1554)"}],"action":{"name":"View Issue","url":"https://github.com/kamailio/kamailio/issues/1554"}}}</script>
<script type="application/ld+json">{
"@type": "MessageCard",
"@context": "http://schema.org/extensions",
"hideOriginalBody": "false",
"originator": "37567f93-e2a7-4e2a-ad37-a9160fc62647",
"title": "Memory leak after SIP message fails to be sent due to no socket found (#1554)",
"sections": [
{
"text": "",
"activityTitle": "**pwakano**",
"activityImage": "https://assets-cdn.github.com/images/email/message_cards/avatar.png",
"activitySubtitle": "@pwakano",
"facts": [
{
"name": "Repository: ",
"value": "kamailio/kamailio"
},
{
"name": "Issue #: ",
"value": 1554
}
]
}
],
"potentialAction": [
{
"name": "Add a comment",
"@type": "ActionCard",
"inputs": [
{
"isMultiLine": true,
"@type": "TextInput",
"id": "IssueComment",
"isRequired": false
}
],
"actions": [
{
"name": "Comment",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n\"commandName\": \"IssueComment\",\n\"repositoryFullName\": \"kamailio/kamailio\",\n\"issueId\": 1554,\n\"IssueComment\": \"{{IssueComment.value}}\"\n}"
}
]
},
{
"name": "Close issue",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n\"commandName\": \"IssueClose\",\n\"repositoryFullName\": \"kamailio/kamailio\",\n\"issueId\": 1554\n}"
},
{
"targets": [
{
"os": "default",
"uri": "https://github.com/kamailio/kamailio/issues/1554"
}
],
"@type": "OpenUri",
"name": "View on GitHub"
},
{
"name": "Unsubscribe",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n\"commandName\": \"MuteNotification\",\n\"threadId\": 342856081\n}"
}
],
"themeColor": "26292E"
}</script>