[SR-Users] core dump if silo table > ~5000 records

Henning Westerholt hw at skalatan.de
Sat Sep 14 16:36:07 CEST 2019


Hello David,

strange. Do you have a timer in the msilo module configured that is executed every 20s?

About the core dump - this looks like the kamailio is already shutting down from some earlier error. The crash is not in the msilo code but in the tsilo code.

Can you print the value of urecord in frame 0?

Cheers,

Henning

Am 14.09.19 um 15:45 schrieb David Villasmil:
bt full:


root at sip:/home/carlos# gdb /usr/local/kamailio5/sbin/kamailio /usr/local/kamailio5/sbin/core
GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/local/kamailio5/sbin/kamailio...done.
[New LWP 22299]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/kamailio5/sbin/kamailio -f /usr/local/kamailio5/etc/kamailio/kamaili'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f32cd989140 in free_ts_urecord (urecord=0x7f328e347270) at ts_hash.c:54
54 urecord->transactions = urecord->transactions->next;
(gdb) bt
#0  0x00007f32cd989140 in free_ts_urecord (urecord=0x7f328e347270) at ts_hash.c:54
#1  0x00007f32cd989e9d in destroy_ts_table () at ts_hash.c:145
#2  0x00007f32cd9920d4 in destroy () at tsilo.c:193
#3  0x000000000051af99 in destroy_modules () at core/sr_module.c:811
#4  0x0000000000418199 in cleanup (show_status=1) at main.c:529
#5  0x0000000000419817 in shutdown_children (sig=15, show_status=1) at main.c:671
#6  0x000000000041c1de in handle_sigs () at main.c:763
#7  0x0000000000423ddd in main_loop () at main.c:1751
#8  0x0000000000429a61 in main (argc=13, argv=0x7fff1d3773f8) at main.c:2639



(gdb) bt full
#0  0x00007f32cd989140 in free_ts_urecord (urecord=0x7f328e347270) at ts_hash.c:54
        __FUNCTION__ = "free_ts_urecord"
        ptr = 0x72743b3536323831
#1  0x00007f32cd989e9d in destroy_ts_table () at ts_hash.c:145
        ts_u = 0x7f3200000023
        l_ts_u = 0x7f328e347270
        i = 7
        __FUNCTION__ = "destroy_ts_table"
#2  0x00007f32cd9920d4 in destroy () at tsilo.c:193
No locals.
#3  0x000000000051af99 in destroy_modules () at core/sr_module.c:811
        t = 0x7f32d68eb248
        foo = 0x7f32d68eaa18
        __FUNCTION__ = "destroy_modules"
#4  0x0000000000418199 in cleanup (show_status=1) at main.c:529
        memlog = 16711680
        __FUNCTION__ = "cleanup"
#5  0x0000000000419817 in shutdown_children (sig=15, show_status=1) at main.c:671
        __FUNCTION__ = "shutdown_children"
#6  0x000000000041c1de in handle_sigs () at main.c:763
        chld = 0
        chld_status = 139
        memlog = 4291896
        __FUNCTION__ = "handle_sigs"
#7  0x0000000000423ddd in main_loop () at main.c:1751
        i = 8
        pid = 22340
        si = 0x0
        si_desc = "udp receiver child=7 sock=169.239.137.3:5060<http://169.239.137.3:5060>\000\b\000\000P\335ό\001\000\000\000\300\244\244\214\062\177\000\000\220p7\035\377\177\000\000\372\071d\000\000\000\000\000PxA\000\000\000\000\000\350R\237\326\062\177", '\000' <repeats 14 times>, "\001\000\000\000\340p7\035\377\177\000\000I\273d\000\000\000\000"
        nrprocs = 8
        woneinit = 1
        __FUNCTION__ = "main_loop"
#8  0x0000000000429a61 in main (argc=13, argv=0x7fff1d3773f8) at main.c:2639
        cfg_stream = 0x202a010
        c = -1
        r = 0
        tmp = 0x7fff1d3788bf ""
        tmp_len = 0
        port = 1
        proto = 32562
        options = 0x72b100 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
        ret = -1
        seed = 1981610615
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 0
        n_lst = 0x7fff1d3772b0
        p = 0x7fff1d377468 "\361\210\067\035\377\177"
        st = {st_dev = 15, st_ino = 10098, st_nlink = 2, st_mode = 16832, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 80, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1568455518, tv_nsec = 802350062}, st_mtim = {
            tv_sec = 1568457084, tv_nsec = 898346157}, st_ctim = {tv_sec = 1568457084, tv_nsec = 898346157}, __glibc_reserved = {0, 0, 0}}
        __FUNCTION__ = "main"
(gdb)


Regards,

David Villasmil
email: david.villasmil.work at gmail.com<mailto:david.villasmil.work at gmail.com>
phone: +34669448337


On Sat, Sep 14, 2019 at 2:35 PM David Villasmil <david.villasmil.work at gmail.com<mailto:david.villasmil.work at gmail.com>> wrote:
Hello guys,

i've got a kamailo:

version: kamailio 5.0.0 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled on 19:50:58 Mar 28 2017 with gcc 4.9.2

which was core dumping every 20 seconds and we had no idea why.
We emptied out the silo table, which had:

mysql> select count(*) from silobackup;
+----------+
| count(*) |
+----------+
|     5240 |
+----------+

records.
Once we truncated it, kamailio started working again.

Is there a limit to how many records msilo can handle?

I will look into the data in the table, but in case there's a limit, i'd like to know.

core available on request.

Thanks

Regards,

David Villasmil
email: david.villasmil.work at gmail.com<mailto:david.villasmil.work at gmail.com>
phone: +34669448337



_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users at lists.kamailio.org<mailto:sr-users at lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users


--
Henning Westerholt - https://skalatan.de/blog/
Kamailio services - https://skalatan.de/services
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20190914/098e55b3/attachment.html>


More information about the sr-users mailing list