Hello,
Last Thursday our kamailio process crashed. There are a lot of memory related errors in there. See attachment for details.
Did the machine run out of memory? How do I prevent this from happening again? The machine has 4GB of memory.
Regards,
Grant
Forgot to mention the version:
version: kamailio 3.3.1 (i386/linux) 2d0f6f flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 2d0f6f compiled on 10:35:51 Oct 22 2012 with gcc 4.4.3
From: sr-users-bounces@lists.sip-router.org [mailto:sr-users-bounces@lists.sip-router.org] On Behalf Of Grant Bagdasarian Sent: Tuesday, February 19, 2013 2:36 PM To: sr-users@lists.sip-router.org Subject: [SR-Users] Errors leading to kamailio proc crash
Hello,
Last Thursday our kamailio process crashed. There are a lot of memory related errors in there. See attachment for details.
Did the machine run out of memory? How do I prevent this from happening again? The machine has 4GB of memory.
Regards,
Grant
Default shared memory is 32MB, that is quite low if you have a lot of traffic.
You can increase it with '-m' command line parameter.
Try to get a coredump and then send the backtrace. It will help to discover where is the issue, even when out of memory, there should be no crash. For that, use 'ulimit -c unlimited' before starting the sip server.
How long it was running? Can it be eventually reproduced?
Also, to be sure is not an issue already fixed, upgrade to latest 3.3.x, you don't have to change anything in database or config file.
Cheers, Daniel
On 2/19/13 2:38 PM, Grant Bagdasarian wrote:
Forgot to mention the version:
version: kamailio 3.3.1 (i386/linux) 2d0f6f
flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 2d0f6f
compiled on 10:35:51 Oct 22 2012 with gcc 4.4.3
*From:*sr-users-bounces@lists.sip-router.org [mailto:sr-users-bounces@lists.sip-router.org] *On Behalf Of *Grant Bagdasarian *Sent:* Tuesday, February 19, 2013 2:36 PM *To:* sr-users@lists.sip-router.org *Subject:* [SR-Users] Errors leading to kamailio proc crash
Hello,
Last Thursday our kamailio process crashed. There are a lot of memory related errors in there. See attachment for details.
Did the machine run out of memory? How do I prevent this from happening again?
The machine has 4GB of memory.
Regards,
Grant
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Can I increase the shared memory when kamailio is still running or configure it somewhere in a configuration file to use x amount of shared memory? The kamailio process is managed by heartbeat/pacemaker as a resource and I was advised not to manually restart a resource which is managed by heartbeat/pacemaker.
The virtual server Kamailio is running on is up for 133 days now. But the kamailio process was up for a couple of weeks, since it has been restarted a few times.
Where can I find the coredump?
From: sr-users-bounces@lists.sip-router.org [mailto:sr-users-bounces@lists.sip-router.org] On Behalf Of Daniel-Constantin Mierla Sent: Thursday, February 21, 2013 9:34 AM To: SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List Subject: Re: [SR-Users] Errors leading to kamailio proc crash
Default shared memory is 32MB, that is quite low if you have a lot of traffic.
You can increase it with '-m' command line parameter.
Try to get a coredump and then send the backtrace. It will help to discover where is the issue, even when out of memory, there should be no crash. For that, use 'ulimit -c unlimited' before starting the sip server.
How long it was running? Can it be eventually reproduced?
Also, to be sure is not an issue already fixed, upgrade to latest 3.3.x, you don't have to change anything in database or config file.
Cheers, Daniel On 2/19/13 2:38 PM, Grant Bagdasarian wrote: Forgot to mention the version:
version: kamailio 3.3.1 (i386/linux) 2d0f6f flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 2d0f6f compiled on 10:35:51 Oct 22 2012 with gcc 4.4.3
From: sr-users-bounces@lists.sip-router.orgmailto:sr-users-bounces@lists.sip-router.org [mailto:sr-users-bounces@lists.sip-router.org] On Behalf Of Grant Bagdasarian Sent: Tuesday, February 19, 2013 2:36 PM To: sr-users@lists.sip-router.orgmailto:sr-users@lists.sip-router.org Subject: [SR-Users] Errors leading to kamailio proc crash
Hello,
Last Thursday our kamailio process crashed. There are a lot of memory related errors in there. See attachment for details.
Did the machine run out of memory? How do I prevent this from happening again? The machine has 4GB of memory.
Regards,
Grant
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.orgmailto:sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio World Conference, April 16-17, 2013, Berlin
Hello,
On 2/21/13 9:59 AM, Grant Bagdasarian wrote:
Can I increase the shared memory when kamailio is still running or configure it somewhere in a configuration file to use x amount of shared memory? The kamailio process is managed by heartbeat/pacemaker as a resource and I was advised not to manually restart a resource which is managed by heartbeat/pacemaker.
usually kamailio is started via init.d script (even by monitoring tools such as heartbeat), there you can put the the parameter you want.
The virtual server Kamailio is running on is up for 133 days now. But the kamailio process was up for a couple of weeks, since it has been restarted a few times.
Where can I find the coredump?
You can set the path where to write core dump via -w command line parameter. If you don't set that, then it is either in / or in a folder configured at OS level (e.g., in macosx is /cores).
Cheers, Daniel
*From:*sr-users-bounces@lists.sip-router.org [mailto:sr-users-bounces@lists.sip-router.org] *On Behalf Of *Daniel-Constantin Mierla *Sent:* Thursday, February 21, 2013 9:34 AM *To:* SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List *Subject:* Re: [SR-Users] Errors leading to kamailio proc crash
Default shared memory is 32MB, that is quite low if you have a lot of traffic.
You can increase it with '-m' command line parameter.
Try to get a coredump and then send the backtrace. It will help to discover where is the issue, even when out of memory, there should be no crash. For that, use 'ulimit -c unlimited' before starting the sip server.
How long it was running? Can it be eventually reproduced?
Also, to be sure is not an issue already fixed, upgrade to latest 3.3.x, you don't have to change anything in database or config file.
Cheers, Daniel
On 2/19/13 2:38 PM, Grant Bagdasarian wrote:
Forgot to mention the version: version: kamailio 3.3.1 (i386/linux) 2d0f6f flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 2d0f6f compiled on 10:35:51 Oct 22 2012 with gcc 4.4.3 *From:*sr-users-bounces@lists.sip-router.org <mailto:sr-users-bounces@lists.sip-router.org> [mailto:sr-users-bounces@lists.sip-router.org] *On Behalf Of *Grant Bagdasarian *Sent:* Tuesday, February 19, 2013 2:36 PM *To:* sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> *Subject:* [SR-Users] Errors leading to kamailio proc crash Hello, Last Thursday our kamailio process crashed. There are a lot of memory related errors in there. See attachment for details. Did the machine run out of memory? How do I prevent this from happening again? The machine has 4GB of memory. Regards, Grant _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users-- Daniel-Constantin Mierla -http://www.asipto.com http://twitter.com/#!/miconda http://twitter.com/#%21/miconda -http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin -http://conference.kamailio.com -
-
Daniel-Constantin Mierla -
Grant Bagdasarian