[SR-Users] Out of memory in UB 210: OOM killed process 12261 (kamailio) score 0 vm:1614768kB, rss:280200kB, swap:131408kB

[Daniel-Constantin Mierla]

Hello,

are you doing different external operations than on the other instances,
like mi/rpc commands.

>From the list of the modules you exposed, I think jansson has the higher
probability to work with system memory. Are you doing different json
operations in config that in the other instances of kamailio? Are you
using same version of libjansson everywhere?

Cheers,
Daniel

On 06/10/16 13:46, Jurijs Ivolga wrote:
> Hi Daniel,
>
> This modules what we are using:
>
> loadmodule "mi_fifo.so"
> loadmodule "kex.so"
> loadmodule "corex.so"
> loadmodule "tm.so"
> loadmodule "tmx.so"
> loadmodule "sl.so"
> loadmodule "rr.so"
> loadmodule "pv.so"
> loadmodule "maxfwd.so"
> loadmodule "textops.so"
> loadmodule "siputils.so"
> loadmodule "xlog.so"
> loadmodule "sanity.so"
> loadmodule "ctl.so"
> loadmodule "cfg_rpc.so"
> loadmodule "mi_rpc.so"
> loadmodule "dispatcher.so"
> loadmodule "utils.so"
> loadmodule "path.so"
> loadmodule "ipops.so"
> loadmodule "jansson.so"
> loadmodule "auth.so"
> loadmodule "nathelper.so"
> loadmodule "tls.so"
> loadmodule "htable.so"
> loadmodule "pike.so"
>
> We have several other Kamailio instances but they are not faced to
> internet and they do not have such memory issue. That other Kamailio
> instances have same modules,  except modules listed below. So if you
> think that issue is inside external library, probably we need to check
> first modules from list below.
>
> loadmodule "ipops.so"
> loadmodule "auth.so"
> loadmodule "nathelper.so"
> loadmodule "pike.so"
>
> But maybe this other Kamailio instances do not have this memory issue,
> just because they did not face to internet and did not have same load
> as instances with memory issue.
>
>  kamailio -v
> version: kamailio 4.4.3 (x86_64/linux) 5a2195
> flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS,
> USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM,
> SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY,
> USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER,
> USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
> MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
> id: 5a2195
> compiled on 08:30:51 Sep 15 2016 with gcc 4.4.7
>
>
> With kind regards,
>
>
> Jurijs
>
> On Thu, Oct 6, 2016 at 12:52 PM, Daniel-Constantin Mierla
> <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>
>     Hello,
>
>     it looks like a leak from the system memory, not from kamailio's
>     pkg or shm memory. This can be due to an improper use of an
>     external library (e.g., libxml2) by a kamailio module or because
>     of a problem in the library.
>
>     Can you list the modules used in your config (the loadmodule
>     lines)? I will try to guess from the list which one relying on
>     external libs with higher risk of leak issues.
>
>     Also, provide the version of kamailio you are using (kamailio -v).
>
>     Cheers,
>     Daniel
>
>
>     On 04/10/16 15:42, Jurijs Ivolga wrote:
>>     Hi,
>>
>>     Our Kamailio server is crashing once per week, with following error:
>>
>>     Oct  1 06:25:06 kamailio kernel: [26982632.803789] Out of memory
>>     in UB 210: OOM killed process 12261 (kamailio) score 0
>>     vm:1614768kB, rss:280200kB, swap:131408kB
>>
>>     Core dump was never created, probably it is because of my
>>     environment, but I will try to get it.
>>
>>     Server constantly eats memory, maybe some kind of memory leak?
>>
>>     Any help is highly appreciated!
>>
>>     Jurijs
>>
>>
>>     _______________________________________________
>>     SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
>>     sr-users at lists.sip-router.org <mailto:sr-users at lists.sip-router.org>
>>     http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>     <http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users>
>
>     -- 
>     Daniel-Constantin Mierla
>     http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> - http://www.linkedin.com/in/miconda
>     <http://www.linkedin.com/in/miconda>
>     Kamailio Advanced Training, Berlin, Oct 24-26, 2016 - http://www.asipto.com
>
>     _______________________________________________ SIP Express Router
>     (SER) and Kamailio (OpenSER) - sr-users mailing list
>     sr-users at lists.sip-router.org
>     <mailto:sr-users at lists.sip-router.org>
>     http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>     <http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users> 
>
-- 
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio Advanced Training, Berlin, Oct 24-26, 2016 - http://www.asipto.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20161006/cdb5bd36/attachment.html>