<div dir="ltr">Hello,<div>Please find out the logs in the following link:</div><div><ul><li><a href="https://pastebin.com/h2UNHwYC">https://pastebin.com/h2UNHwYC</a><br></li></ul><div><div>Let me know if you need something else.</div><div><br></div><div>Cheers<br></div><div>José Seabra</div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com">miconda@gmail.com</a>> escreveu no dia quarta, 14/08/2019 à(s) 10:59:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hello,</p>
<p>that's memory status, printing details for each allocated chunk.
I could spot a lot of chunks from the parser and get header field
functions, but to be sure there is nothing else, can you set
global parameter:</p>
<p>mem_summary=12</p>
<p>and then reproduce again. The logs printed in syslog should have
smaller size, giving the summary of allocated chunks - how many
times and the size for each location in the code.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div class="gmail-m_-4136311559141794882moz-cite-prefix">On 14.08.19 11:51, José Seabra wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hello Daniel,
<div>Please find out the attached zip file with the output
pkg_summary.</div>
<div><br>
</div>
<div>Let me know if you need something else.</div>
<div><br>
</div>
<div>Thank you for your support</div>
<div><br>
</div>
<div>Cheers</div>
<div>José Seabra</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>>
escreveu no dia quarta, 14/08/2019 à(s) 10:03:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hello,</p>
<p>can you enable pkg summary and generate it once the out
of memory happens?</p>
<p>Set memlog to lower value than debug, when the memory
happens, do:</p>
<p><a href="https://www.kamailio.org/docs/modules/devel/modules/corex.html#corex.rpc.pkg_summary" target="_blank">https://www.kamailio.org/docs/modules/devel/modules/corex.html#corex.rpc.pkg_summary</a></p>
<p>Then send some traffic to that process and watch the
syslog.</p>
<p>Alternative, enable memory summary via core parameter and
stop kamailio when the errors happen, but you will get
summary for all processes, so it might be harder to
extract those log messages related the evapi process.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-cite-prefix">On
14.08.19 10:37, José Seabra wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hello there,
<div>I'm using kamailio 5.2.1 with evapi module to send
and receive events from an external application.</div>
<div> </div>
<div>During my stress tests to kamailio i have been
noticing that at some point kamailio is getting out of
memory:</div>
<div><br>
</div>
<div>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]ERROR:
ESP_LOG: <null>: <core>
[core/mem/q_malloc.c:291]: qm_find_free():
qm_find_free(0x7fb8730b6010, 64); Free fragment not
found!</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
ERROR: ESP_LOG: <null>: <core>
[core/mem/q_malloc.c:425]: qm_malloc():
qm_malloc(0x7fb8730b6010, 64) called from core:
core/parser/msg_parser.c: parse_headers(325),
module: core; Free fragment not found!</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
ERROR: ESP_LOG: <null>: <core>
[core/parser/msg_parser.c:328]: parse_headers():
memory allocation error</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
ERROR: ESP_LOG: <null>: pv [pv_core.c:704]:
pv_get_callid(): cannot parse Call-Id header</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
INFO: ESP_LOG: <null>:
[evapi:connection-received] EVAPI msg received ->
{"aor":<a class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-txt-link-rfc2396E">"sip:FUZECI1-x1516@uc.fuze.domain"</a>,"contact":"<a class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-txt-link-rfc2396E"><sip:FUZECI1-x1516@10.225.121.206:5060;transport=UDP></a>"}</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
ERROR: ESP_LOG: <null>: <core>
[core/mem/q_malloc.c:291]: qm_find_free():
qm_find_free(0x7fb8730b6010, 64); Free fragment not
found!</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
ERROR: ESP_LOG: <null>: <core>
[core/mem/q_malloc.c:425]: qm_malloc():
qm_malloc(0x7fb8730b6010, 64) called from core:
core/parser/msg_parser.c: parse_headers(325),
module: core; Free fragment not found!</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><br>
</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">The
process id 20565 is running out of memory as you can
see bellow:</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><br>
</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">output
of kamcmd pkg.stats </p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><br>
</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>entry:
14</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>pid:<span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-converted-space"> </span></p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>rank:
-2</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>used:
6058696</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>free:
824</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>real_used:
8387784</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>total_size:
8388608</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"> </p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>total_frags:
30</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><br>
</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">This
process ID is related with Evapi Dispatcher</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><br>
</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><br>
</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">kamcmd ps | grep
20565</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"><br>
</p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1"> </p>
<p class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-p1">20565<span class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>EvAPI
Dispatcher</p>
<div><br>
</div>
<div>After stop the sipp that was generating traffic
to kamailio the evapi process still have all the
memory consumed</div>
<div><br>
</div>
<div>Can it be some memory leak in Evapi module?</div>
<div><br>
</div>
<div>Best Regards</div>
<div>José Seabra</div>
-- <br>
<div dir="ltr" class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598gmail_signature">Cumprimentos
<div>José Seabra</div>
</div>
</div>
</div>
<br>
<fieldset class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598mimeAttachmentHeader"></fieldset>
<pre class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-quote-pre">_______________________________________________
Kamailio (SER) - Users Mailing List
<a class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-txt-link-abbreviated" href="mailto:sr-users@lists.kamailio.org" target="_blank">sr-users@lists.kamailio.org</a>
<a class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-txt-link-freetext" href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
</blockquote>
<pre class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-signature" cols="72">--
Daniel-Constantin Mierla -- <a class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-txt-link-abbreviated" href="http://www.asipto.com" target="_blank">www.asipto.com</a>
<a class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-txt-link-abbreviated" href="http://www.twitter.com/miconda" target="_blank">www.twitter.com/miconda</a> -- <a class="gmail-m_-4136311559141794882gmail-m_-4220985776146207598moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda" target="_blank">www.linkedin.com/in/miconda</a></pre>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr" class="gmail-m_-4136311559141794882gmail_signature">Cumprimentos
<div>José Seabra</div>
</div>
</blockquote>
<pre class="gmail-m_-4136311559141794882moz-signature" cols="72">--
Daniel-Constantin Mierla -- <a class="gmail-m_-4136311559141794882moz-txt-link-abbreviated" href="http://www.asipto.com" target="_blank">www.asipto.com</a>
<a class="gmail-m_-4136311559141794882moz-txt-link-abbreviated" href="http://www.twitter.com/miconda" target="_blank">www.twitter.com/miconda</a> -- <a class="gmail-m_-4136311559141794882moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda" target="_blank">www.linkedin.com/in/miconda</a></pre>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">Cumprimentos<div>José Seabra</div></div>