[SR-Users] Fwd: Possible memory leak on 5.5.x (new)?
Daniel-Constantin Mierla
miconda at gmail.com
Wed Jun 30 18:20:23 CEST 2021
Hello,
for the sake of completion: the autoexpire should clean the items if
they are not used during the expiration interval. If you want to get
them deleted after first expiration interval always, see the
updateexpire attribute for htable modparam.
Also, note that replication should be done only between Kamailio
instances with same major version, because there can be internal
differences between major versions that can lead to unexpected
behaviour. In other words, if you replicate, doing between two kamailio
with version 5.5.x or between two kamailio with version 5.4.x, but not
between a kamailkio 5.5.x and a kamailio 5.4.x.
The total amount of used memory in the stats file for 5.5 does not seem
to be high as a rough estimation. The highest by module is in htable,
but it is around 20MB. Maybe you took the stats too early, quickly after
a restart?
Cheers,
Daniel
On 30.06.21 17:20, George Diamantopoulos wrote:
> Hello Daniel,
>
> Thanks for the feedback. I think I might have been too quick to blame
> htable for this behaviour. In fact, version 5.4 seems to consume more
> memory than 5.5 (175129776 bytes vs 20581096), which makes sense since
> it has been running for longer (I missed the extra digit previously).
>
> So I'm not sure htable is to blame. On the other hand, I don't see any
> other modules using up too much of shmem either, so maybe memory stats
> can't provide the answer here?
>
> To answer your question, though, I do use DMQ and both tables that use
> it have autoexpire set to the same value on both 5.4 and 5.5:
>
> /etc/kamailio# grep dmq kamailio-module-params.cfg
> modparam("dmq", "server_address", "sip:172.30.43.1:5090
> <http://172.30.43.1:5090>")
> modparam("dmq", "notification_address",
> "sip:dmq.services.mydomain.com:5090
> <http://dmq.services.mydomain.com:5090>")
> modparam("dmq", "multi_notify", 1)
> modparam("htable", "enable_dmq", 1)
> modparam("htable", "htable",
> 'cid2hi=>size=8;autoexpire=600;dmqreplicate=1')
> modparam("htable", "htable",
> 'xcid2count=>size=8;autoexpire=600;dmqreplicate=1')
>
> On Wed, 30 Jun 2021 at 17:43, Daniel-Constantin Mierla
> <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>
> Hello,
>
> do you replicate items in the htable via dmq? Does the htable have
> autoexpire value set?
>
> Cheers,
> Daniel
>
> On 30.06.21 13:54, George Diamantopoulos wrote:
>> Forwarding my reply to the list, using gmail's reply button set
>> Henning as the sole recipient :-\
>>
>> ---------- Forwarded message ---------
>> From: *George Diamantopoulos* <georgediam at gmail.com
>> <mailto:georgediam at gmail.com>>
>> Date: Sat, 26 Jun 2021 at 02:25
>> Subject: Re: [SR-Users] Possible memory leak on 5.5.x (new)?
>> To: Henning Westerholt <hw at skalatan.de <mailto:hw at skalatan.de>>
>>
>>
>> Hello Henning,
>>
>> Thanks for your reply. Here's what has come up after a few hours:
>>
>> shm55: https://pastebin.com/h9JCePmc <https://pastebin.com/h9JCePmc>
>> shm54: https://pastebin.com/Nx5xEEnA <https://pastebin.com/Nx5xEEnA>
>>
>> It seems to me htable is the culprit? Are you seeing anything
>> different? 54 has been running for 77020 seconds, 55 for 28521
>> (significantly less).
>>
>> I'm going to turn it off until we figure something out...
>>
>> BR,
>> George
>>
>> On Fri, 25 Jun 2021 at 18:17, Henning Westerholt <hw at skalatan.de
>> <mailto:hw at skalatan.de>> wrote:
>>
>> Hello,
>>
>>
>>
>> Good observation. Please run the memory statistics CLI
>> commands to get more hints about the module that might cause
>> it (as per below link). Then please report more details. If
>> you can point to a particular module, you can also open an
>> issue on our tracker.
>>
>>
>>
>> https://www.kamailio.org/wiki/tutorials/troubleshooting/memory
>> <https://www.kamailio.org/wiki/tutorials/troubleshooting/memory>
>>
>>
>>
>> Cheers,
>>
>>
>>
>> Henning
>>
>>
>>
>> *From:* sr-users <sr-users-bounces at lists.kamailio.org
>> <mailto:sr-users-bounces at lists.kamailio.org>> *On Behalf Of
>> *George Diamantopoulos
>> *Sent:* Friday, June 25, 2021 4:53 PM
>> *To:* Kamailio (SER) - Users Mailing List
>> <sr-users at lists.kamailio.org
>> <mailto:sr-users at lists.kamailio.org>>
>> *Subject:* [SR-Users] Possible memory leak on 5.5.x (new)?
>>
>>
>>
>> Hello all,
>>
>>
>>
>> I'm still investigating the (most likely
>> non-kamailio-related) memory leak of my previous message to
>> the list, there have been no developments so far. I'll update
>> if anything changes.
>>
>>
>>
>> This concerns a new finding which seems to affect kamailio
>> 5.5.x. I have two kamailio instances receiving the same
>> traffic via round-robin. I upgraded only one of them to 5.5.1
>> and left the other to 5.4.6 as I feared of any issues
>> arising. I was lucky to do so, because with identical
>> configuration, 5.5.x seems to run out of SHM very quickly.
>> Here are links to graphs produced by our monitoring system:
>>
>>
>>
>> Old kamailio (no memory leak):
>> https://pasteboard.co/K8fVBiD.png
>> <https://pasteboard.co/K8fVBiD.png>
>>
>> New kamailio (possible leak):
>> https://pasteboard.co/K8fVS9N.png
>> <https://pasteboard.co/K8fVS9N.png>
>>
>>
>>
>> The configuration uses mtree, htable, vars and vns
>> extensively. Has anyone come across anything similar? Let me
>> know if I can provide any further information to help disect
>> this. Thanks!
>>
>>
>>
>> BR,
>>
>> George
>>
>>
>> __________________________________________________________
>> Kamailio - Users Mailing List - Non Commercial Discussions
>> * sr-users at lists.kamailio.org <mailto:sr-users at lists.kamailio.org>
>> Important: keep the mailing list in the recipients, do not reply only to the sender!
>> Edit mailing list options or unsubscribe:
>> * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>
> --
> Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com>
> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
>
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20210630/5353f486/attachment.htm>
More information about the sr-users
mailing list