THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.
The following task has a new comment added:
FS#236 - Kamailio crash when about 300,000 tls connections opened.
User who did this - Shaobin.Feng (saxon_leo)
----------
according to http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:signal9
================================================Memory Overcommit info
root@B2-Ubuntu-01:~# cat /proc/sys/vm/overcommit_memory
0
----------
More information can be found at the following URL:
http://sip-router.org/tracker/index.php?do=details&task_id=236#comment686
You are receiving this message because you have requested it from the Flyspray bugtracking system. If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.
THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.
The following task has a new comment added:
FS#236 - Kamailio crash when about 300,000 tls connections opened.
User who did this - Klaus Darilion (klaus3000)
----------
see also http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:signal9
----------
More information can be found at the following URL:
http://sip-router.org/tracker/index.php?do=details&task_id=236#comment684
You are receiving this message because you have requested it from the Flyspray bugtracking system. If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.
THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.
The following task has a new comment added:
FS#236 - Kamailio crash when about 300,000 tls connections opened.
User who did this - Daniel-Constantin Mierla (miconda)
----------
FYI, in v3.2.x, you can give -M parameter for specifying private memory, no need to change in config.h and recompile.
Try to reproduce and watch the top about used memory, if kamailio is increasing, it will be killed by OS. Look at the syslog messages, iirc, there should be a message about it.
----------
More information can be found at the following URL:
http://sip-router.org/tracker/index.php?do=details&task_id=236#comment683
You are receiving this message because you have requested it from the Flyspray bugtracking system. If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.
THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.
The following task has a new comment added:
FS#236 - Kamailio crash when about 300,000 tls connections opened.
User who did this - Shaobin.Feng (saxon_leo)
----------
============================================================================mem info
root@B2-Ubuntu-01:/usr/local/src/kamailio-3.2.3# cat /proc/meminfo
MemTotal: 8195876 kB
MemFree: 7881204 kB
Buffers: 18284 kB
Cached: 44848 kB
SwapCached: 25660 kB
Active: 46864 kB
Inactive: 64928 kB
Active(anon): 22488 kB
Inactive(anon): 26448 kB
Active(file): 24376 kB
Inactive(file): 38480 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 2928636 kB
SwapFree: 2384952 kB
Dirty: 32 kB
Writeback: 0 kB
AnonPages: 23608 kB
Mapped: 5860 kB
Shmem: 272 kB
Slab: 30632 kB
SReclaimable: 18280 kB
SUnreclaim: 12352 kB
KernelStack: 1704 kB
PageTables: 5292 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 7026572 kB
Committed_AS: 1715016 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 290428 kB
VmallocChunk: 34359445308 kB
HardwareCorrupted: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 45056 kB
DirectMap2M: 8343552 kB
=========================================================kamailio mem info
kamctlrc STARTOPTIONS="-m 6144"
config.h #define PKG_MEM_SIZE 32
=========================================================system info before crash
root@B2-Ubuntu-01: free -m
total used free shared buffers cached
Mem: 8003 7931 72 0 1 2939
-/+ buffers/cache: 4990 3013
Swap: 2859 2850 9
Well,didn't save top info.
----------
More information can be found at the following URL:
http://sip-router.org/tracker/index.php?do=details&task_id=236#comment682
You are receiving this message because you have requested it from the Flyspray bugtracking system. If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.
THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.
The following task has a new comment added:
FS#236 - Kamailio crash when about 300,000 tls connections opened.
User who did this - Daniel-Constantin Mierla (miconda)
----------
Signal 9 is sent by the kernel when the process is eating almost entire memory.
How much memory do you have on this server and how much is allocated to kamailio (private and shared).
Kamailio itself should not eat more than allocated, but other libraries may need own extra memory for internal stuff and can cause such a case. Can you grab also the info about used memory in the system (e.g., via top) just before the crash?
----------
More information can be found at the following URL:
http://sip-router.org/tracker/index.php?do=details&task_id=236#comment681
You are receiving this message because you have requested it from the Flyspray bugtracking system. If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.
THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.
A new Flyspray task has been opened. Details are below.
User who did this - Shaobin.Feng (saxon_leo)
Attached to Project - sip-router
Summary - Kamailio crash when about 300,000 tls connections opened.
Task Type - Bug Report
Category - Core
Status - Unconfirmed
Assigned To -
Operating System - Linux
Severity - Critical
Priority - Normal
Reported Version - 3.2
Due in Version - Undecided
Due Date - Undecided
Details - Kamailio crash when about 300,000 tls connections opened.
=========================================================version info
version: kamailio 3.2.3 (x86_64/linux) 59f87e
flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 32MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 59f87e
compiled on 10:51:38 Jun 13 2012 with gcc 4.5.2
openssl version 1.0.0e
==========================================================config info
relate config
enable_tls=yes
open_files_limit = 500000
tcp_children = 80
tcp_connection_lifetime=3600
tcp_max_connections = 500000
tcp_no_connect = yes
tcp_poll_method = epoll_et
modparam("tls", "ssl_release_buffers", 1)
modparam("tls", "ssl_freelist_max_len", 0)
modparam("tls", "ssl_max_send_fragment", 2048)
modparam("usrloc", "db_mode", 0)
============================================================operate info
Once a connection was created success,while send OPTIONS to keep the connection active before lifetime achieve.
==============================================================log info
426 Jun 14 13:58:40 B2-Ubuntu-01 /usr/local/sbin/kamailio[31516]: ALERT: <core> [main.c:751]: child process 31610 exited by a signal 9
427 Jun 14 13:58:40 B2-Ubuntu-01 /usr/local/sbin/kamailio[31516]: ALERT: <core> [main.c:754]: core was not generated
428 Jun 14 13:58:40 B2-Ubuntu-01 /usr/local/sbin/kamailio[31516]: INFO: <core> [main.c:766]: INFO: terminating due to SIGCHLD
429 Jun 14 13:58:40 B2-Ubuntu-01 /usr/local/sbin/kamailio[31600]: INFO: <core> [main.c:817]: INFO: signal 15 received
430 Jun 14 13:58:40 B2-Ubuntu-01 /usr/local/sbin/kamailio[31520]: INFO: <core> [main.c:817]: INFO: signal 15 received
....
522 Jun 14 13:59:39 B2-Ubuntu-01 /usr/local/sbin/kamailio[31516]: : <core> [main.c:660]: BUG: shutdown timeout triggered, dying...
==============================================================process info
sercmd core.ps
......
31610
tcp main process
===============================================================core file info
casue the size limit, attach the gdb screenshot
One or more files have been attached.
More information can be found at the following URL:
http://sip-router.org/tracker/index.php?do=details&task_id=236
You are receiving this message because you have requested it from the Flyspray bugtracking system. If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.
THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.
The following task has a new comment added:
FS#234 - kamailio crah [pass_fd.c:293]: ERROR: receive_fd: EOF on 46
User who did this - David Kovarik (kovik)
----------
Just check it to me to confirm my theory. it is probably wrong.
----------
More information can be found at the following URL:
http://sip-router.org/tracker/index.php?do=details&task_id=234#comment680
You are receiving this message because you have requested it from the Flyspray bugtracking system. If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.
THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.
The following task has a new comment added:
FS#234 - kamailio crah [pass_fd.c:293]: ERROR: receive_fd: EOF on 46
User who did this - Daniel-Constantin Mierla (miconda)
----------
Don't get it, that's just some check, does it prevents the crash or is just printing a message that got you to the conclusion with the protocols?
Btw, I did another commit, fetch again master branch. Although there seems to be an inconsistency in the number of items in the location table slot and the list of this items.
----------
More information can be found at the following URL:
http://sip-router.org/tracker/index.php?do=details&task_id=234#comment679
You are receiving this message because you have requested it from the Flyspray bugtracking system. If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.