<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello,</p>
<p>for the sake of completion: the autoexpire should clean the items
if they are not used during the expiration interval. If you want
to get them deleted after first expiration interval always, see
the updateexpire attribute for htable modparam.</p>
<p>Also, note that replication should be done only between Kamailio
instances with same major version, because there can be internal
differences between major versions that can lead to unexpected
behaviour. In other words, if you replicate, doing between two
kamailio with version 5.5.x or between two kamailio with version
5.4.x, but not between a kamailkio 5.5.x and a kamailio 5.4.x.</p>
<p>The total amount of used memory in the stats file for 5.5 does
not seem to be high as a rough estimation. The highest by module
is in htable, but it is around 20MB. Maybe you took the stats too
early, quickly after a restart?<br>
</p>
<div class="moz-cite-prefix">Cheers,<br>
Daniel</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 30.06.21 17:20, George
Diamantopoulos wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAPcKEYMzPMUwNfWP4j7T2Ozk8fVovkm1-iQWH+N3EMZNtTzZJQ@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div>Hello Daniel,</div>
<div><br>
</div>
<div>Thanks for the feedback. I think I might have been too
quick to blame htable for this behaviour. In fact, version 5.4
seems to consume more memory than 5.5 (175129776 bytes vs
20581096), which makes sense since it has been running for
longer (I missed the extra digit previously).</div>
<div><br>
</div>
<div>So I'm not sure htable is to blame. On the other hand, I
don't see any other modules using up too much of shmem either,
so maybe memory stats can't provide the answer here?</div>
<div><br>
</div>
<div>To answer your question, though, I do use DMQ and both
tables that use it have autoexpire set to the same value on
both 5.4 and 5.5:</div>
<div><br>
</div>
<div><span style="font-family:monospace"><span
style="color:rgb(0,0,0);background-color:rgb(255,255,255)">/etc/kamailio#
grep dmq kamailio-module-params.cfg
</span><br>
modparam("dmq", "server_address", "sip:<a
href="http://172.30.43.1:5090" moz-do-not-send="true">172.30.43.1:5090</a>")
<br>
modparam("dmq", "notification_address", "sip:<a
href="http://dmq.services.mydomain.com:5090"
moz-do-not-send="true">dmq.services.mydomain.com:5090</a>")
<br>
modparam("dmq", "multi_notify", 1)
<br>
modparam("htable", "enable_dmq", 1)
<br>
modparam("htable", "htable",
'cid2hi=>size=8;autoexpire=600;dmqreplicate=1')
<br>
modparam("htable", "htable",
'xcid2count=>size=8;autoexpire=600;dmqreplicate=1')<br>
</span></div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, 30 Jun 2021 at 17:43,
Daniel-Constantin Mierla <<a
href="mailto:miconda@gmail.com" moz-do-not-send="true">miconda@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello,</p>
<p>do you replicate items in the htable via dmq? Does the
htable have autoexpire value set?</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 30.06.21 13:54, George Diamantopoulos wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Forwarding my reply to the list, using
gmail's reply button set Henning as the sole recipient
:-\<br>
<div><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">----------
Forwarded message ---------<br>
From: <b class="gmail_sendername" dir="auto">George
Diamantopoulos</b> <span dir="auto"><<a
href="mailto:georgediam@gmail.com"
target="_blank" moz-do-not-send="true">georgediam@gmail.com</a>></span><br>
Date: Sat, 26 Jun 2021 at 02:25<br>
Subject: Re: [SR-Users] Possible memory leak on
5.5.x (new)?<br>
To: Henning Westerholt <<a
href="mailto:hw@skalatan.de" target="_blank"
moz-do-not-send="true">hw@skalatan.de</a>><br>
</div>
<br>
<br>
<div dir="ltr">
<div>Hello Henning,</div>
<div><br>
</div>
<div>Thanks for your reply. Here's what has come
up after a few hours:</div>
<div><br>
</div>
<div>shm55: <a
href="https://pastebin.com/h9JCePmc"
target="_blank" moz-do-not-send="true">https://pastebin.com/h9JCePmc</a></div>
<div>shm54: <a
href="https://pastebin.com/Nx5xEEnA"
target="_blank" moz-do-not-send="true">https://pastebin.com/Nx5xEEnA</a></div>
<div><br>
</div>
<div>It seems to me htable is the culprit? Are you
seeing anything different? 54 has been running
for 77020 seconds, 55 for 28521 (significantly
less).</div>
<div><br>
</div>
<div>I'm going to turn it off until we figure
something out...</div>
<div><br>
</div>
<div>BR,</div>
<div>George<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, 25 Jun
2021 at 18:17, Henning Westerholt <<a
href="mailto:hw@skalatan.de" target="_blank"
moz-do-not-send="true">hw@skalatan.de</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div lang="DE">
<div>
<p class="MsoNormal"><span lang="EN-GB">Hello,</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">Good
observation. Please run the memory
statistics CLI commands to get more
hints about the module that might cause
it (as per below link). Then please
report more details. If you can point to
a particular module, you can also open
an issue on our tracker.</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB"><a
href="https://www.kamailio.org/wiki/tutorials/troubleshooting/memory"
target="_blank" moz-do-not-send="true">https://www.kamailio.org/wiki/tutorials/troubleshooting/memory</a></span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">Cheers,</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">Henning</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<div style="border-color:rgb(225,225,225)
currentcolor
currentcolor;border-style:solid none
none;border-width:1pt medium
medium;padding:3pt 0cm 0cm">
<p class="MsoNormal"
style="margin-left:35.4pt"><b>From:</b>
sr-users <<a
href="mailto:sr-users-bounces@lists.kamailio.org"
target="_blank" moz-do-not-send="true">sr-users-bounces@lists.kamailio.org</a>>
<b>On Behalf Of </b>George
Diamantopoulos<br>
<b>Sent:</b> Friday, June 25, 2021 4:53
PM<br>
<b>To:</b> Kamailio (SER) - Users
Mailing List <<a
href="mailto:sr-users@lists.kamailio.org"
target="_blank" moz-do-not-send="true">sr-users@lists.kamailio.org</a>><br>
<b>Subject:</b> [SR-Users] Possible
memory leak on 5.5.x (new)?</p>
</div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
<div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Hello all,</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">I'm still
investigating the (most likely
non-kamailio-related) memory leak of
my previous message to the list, there
have been no developments so far. I'll
update if anything changes.</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">This
concerns a new finding which seems to
affect kamailio 5.5.x. I have two
kamailio instances receiving the same
traffic via round-robin. I upgraded
only one of them to 5.5.1 and left the
other to 5.4.6 as I feared of any
issues arising. I was lucky to do so,
because with identical configuration,
5.5.x seems to run out of SHM very
quickly. Here are links to graphs
produced by our monitoring system:</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Old
kamailio (no memory leak): <a
href="https://pasteboard.co/K8fVBiD.png"
target="_blank"
moz-do-not-send="true">
https://pasteboard.co/K8fVBiD.png</a></p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">New
kamailio (possible leak): <a
href="https://pasteboard.co/K8fVS9N.png"
target="_blank"
moz-do-not-send="true">
https://pasteboard.co/K8fVS9N.png</a></p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">The
configuration uses mtree, htable, vars
and vns extensively. Has anyone come
across anything similar? Let me know
if I can provide any further
information to help disect this.
Thanks!</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">BR,</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">George</p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
* <a href="mailto:sr-users@lists.kamailio.org" target="_blank" moz-do-not-send="true">sr-users@lists.kamailio.org</a>
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
* <a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank" moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
</blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank" moz-do-not-send="true">www.linkedin.com/in/miconda</a></pre>
</div>
</blockquote>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla -- <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/miconda">www.twitter.com/miconda</a> -- <a class="moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda">www.linkedin.com/in/miconda</a></pre>
</body>
</html>