<div dir="ltr">Hello Daniel,<div>Please find out the attached zip file with the output pkg_summary.</div><div><br></div><div>Let me know if you need something else.</div><div><br></div><div>Thank you for your support</div><div><br></div><div>Cheers</div><div>José Seabra</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com">miconda@gmail.com</a>> escreveu no dia quarta, 14/08/2019 à(s) 10:03:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF">
    <p>Hello,</p>
    <p>can you enable pkg summary and generate it once the out of memory
      happens?</p>
    <p>Set memlog to lower value than debug, when the memory happens,
      do:</p>
    <p><a href="https://www.kamailio.org/docs/modules/devel/modules/corex.html#corex.rpc.pkg_summary" target="_blank">https://www.kamailio.org/docs/modules/devel/modules/corex.html#corex.rpc.pkg_summary</a></p>
    <p>Then send some traffic to that process and watch the syslog.</p>
    <p>Alternative, enable memory summary via core parameter and stop
      kamailio when the errors happen, but you will get summary for all
      processes, so it might be harder to extract those log messages
      related the evapi process.</p>
    <p>Cheers,<br>
      Daniel<br>
    </p>
    <div class="gmail-m_-4220985776146207598moz-cite-prefix">On 14.08.19 10:37, José Seabra wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">Hello there,
        <div>I'm using kamailio 5.2.1 with evapi module to send and
          receive events from an external application.</div>
        <div> </div>
        <div>During my stress tests to kamailio i have been noticing
          that at some point kamailio is getting out of memory:</div>
        <div><br>
        </div>
        <div>
          <p class="gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]ERROR:
            ESP_LOG: <null>: <core>
            [core/mem/q_malloc.c:291]: qm_find_free():
            qm_find_free(0x7fb8730b6010, 64); Free fragment not found!</p>
          <p class="gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
            ERROR: ESP_LOG: <null>: <core>
            [core/mem/q_malloc.c:425]: qm_malloc():
            qm_malloc(0x7fb8730b6010, 64) called from core:
            core/parser/msg_parser.c: parse_headers(325), module: core;
            Free fragment not found!</p>
          <p class="gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
            ERROR: ESP_LOG: <null>: <core>
            [core/parser/msg_parser.c:328]: parse_headers(): memory
            allocation error</p>
          <p class="gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
            ERROR: ESP_LOG: <null>: pv [pv_core.c:704]:
            pv_get_callid(): cannot parse Call-Id header</p>
          <p class="gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]: INFO:
            ESP_LOG: <null>: [evapi:connection-received] EVAPI msg
            received ->
{"aor":<a class="gmail-m_-4220985776146207598moz-txt-link-rfc2396E">"sip:FUZECI1-x1516@uc.fuze.domain"</a>,"contact":"<a class="gmail-m_-4220985776146207598moz-txt-link-rfc2396E"><sip:FUZECI1-x1516@10.225.121.206:5060;transport=UDP></a>"}</p>
          <p class="gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
            ERROR: ESP_LOG: <null>: <core>
            [core/mem/q_malloc.c:291]: qm_find_free():
            qm_find_free(0x7fb8730b6010, 64); Free fragment not found!</p>
          <p class="gmail-m_-4220985776146207598gmail-p1">edge-sip-proxy[20565]:
            ERROR: ESP_LOG: <null>: <core>
            [core/mem/q_malloc.c:425]: qm_malloc():
            qm_malloc(0x7fb8730b6010, 64) called from core:
            core/parser/msg_parser.c: parse_headers(325), module: core;
            Free fragment not found!</p>
          <p class="gmail-m_-4220985776146207598gmail-p1"><br>
          </p>
          <p class="gmail-m_-4220985776146207598gmail-p1">The process id 20565 is
            running out of memory as you can see bellow:</p>
          <p class="gmail-m_-4220985776146207598gmail-p1"><br>
          </p>
          <p class="gmail-m_-4220985776146207598gmail-p1">output of kamcmd pkg.stats </p>
          <p class="gmail-m_-4220985776146207598gmail-p1"><br>
          </p>
          <p class="gmail-m_-4220985776146207598gmail-p1" style="margin:0px;font:12px "Helvetica Neue";color:rgb(69,69,69)"><span class="gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>entry:
            14</p>
          <p class="gmail-m_-4220985776146207598gmail-p1" style="margin:0px;font:12px "Helvetica Neue";color:rgb(69,69,69)"><span class="gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>pid:<span class="gmail-m_-4220985776146207598gmail-Apple-converted-space"> </span></p>
          <p class="gmail-m_-4220985776146207598gmail-p1" style="margin:0px;font:12px "Helvetica Neue";color:rgb(69,69,69)"><span class="gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>rank:
            -2</p>
          <p class="gmail-m_-4220985776146207598gmail-p1" style="margin:0px;font:12px "Helvetica Neue";color:rgb(69,69,69)"><span class="gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>used:
            6058696</p>
          <p class="gmail-m_-4220985776146207598gmail-p1" style="margin:0px;font:12px "Helvetica Neue";color:rgb(69,69,69)"><span class="gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>free:
            824</p>
          <p class="gmail-m_-4220985776146207598gmail-p1" style="margin:0px;font:12px "Helvetica Neue";color:rgb(69,69,69)"><span class="gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>real_used:
            8387784</p>
          <p class="gmail-m_-4220985776146207598gmail-p1" style="margin:0px;font:12px "Helvetica Neue";color:rgb(69,69,69)"><span class="gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>total_size:
            8388608</p>
          <p class="gmail-m_-4220985776146207598gmail-p1">
          </p>
          <p class="gmail-m_-4220985776146207598gmail-p1"><span class="gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap">        </span>total_frags:
            30</p>
          <p class="gmail-m_-4220985776146207598gmail-p1"><br>
          </p>
          <p class="gmail-m_-4220985776146207598gmail-p1">This process ID is related
            with Evapi Dispatcher</p>
          <p class="gmail-m_-4220985776146207598gmail-p1"><br>
          </p>
          <p class="gmail-m_-4220985776146207598gmail-p1"><br>
          </p>
          <p class="gmail-m_-4220985776146207598gmail-p1" style="margin:0px;font:12px "Helvetica Neue";color:rgb(69,69,69)">kamcmd ps |
            grep 20565</p>
          <p class="gmail-m_-4220985776146207598gmail-p1" style="margin:0px;font:12px "Helvetica Neue";color:rgb(69,69,69)"><br>
          </p>
          <p class="gmail-m_-4220985776146207598gmail-p1">
          </p>
          <p class="gmail-m_-4220985776146207598gmail-p1">20565<span class="gmail-m_-4220985776146207598gmail-Apple-tab-span" style="white-space:pre-wrap">   </span>EvAPI
            Dispatcher</p>
          <div><br>
          </div>
          <div>After stop the sipp that was generating traffic to
            kamailio the evapi process still have all the memory
            consumed</div>
          <div><br>
          </div>
          <div>Can it be some memory leak in Evapi module?</div>
          <div><br>
          </div>
          <div>Best Regards</div>
          <div>José Seabra</div>
          -- <br>
          <div dir="ltr" class="gmail-m_-4220985776146207598gmail_signature">Cumprimentos
            <div>José Seabra</div>
          </div>
        </div>
      </div>
      <br>
      <fieldset class="gmail-m_-4220985776146207598mimeAttachmentHeader"></fieldset>
      <pre class="gmail-m_-4220985776146207598moz-quote-pre">_______________________________________________
Kamailio (SER) - Users Mailing List
<a class="gmail-m_-4220985776146207598moz-txt-link-abbreviated" href="mailto:sr-users@lists.kamailio.org" target="_blank">sr-users@lists.kamailio.org</a>
<a class="gmail-m_-4220985776146207598moz-txt-link-freetext" href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
    </blockquote>
    <pre class="gmail-m_-4220985776146207598moz-signature" cols="72">-- 
Daniel-Constantin Mierla -- <a class="gmail-m_-4220985776146207598moz-txt-link-abbreviated" href="http://www.asipto.com" target="_blank">www.asipto.com</a>
<a class="gmail-m_-4220985776146207598moz-txt-link-abbreviated" href="http://www.twitter.com/miconda" target="_blank">www.twitter.com/miconda</a> -- <a class="gmail-m_-4220985776146207598moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda" target="_blank">www.linkedin.com/in/miconda</a></pre>
  </div>

</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">Cumprimentos<div>José Seabra</div></div>