[SR-Users] Crash kamailio 4.3.1
Thibault Gueslin
thibault.gueslin at gmail.com
Thu Sep 10 14:07:05 CEST 2015
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/kamailio -f /etc/kamailio/kamailio.cfg -P
/var/run/kamailio/kamailio.'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 ts_append (msg=0x7f883bfd5490, ruri=0x7fff65fbfe30,
table=0x7f883bf7e390 "location") at ts_append.c:60
60 ts_append.c: No such file or directory.
(gdb) bt
#0 ts_append (msg=0x7f883bfd5490, ruri=0x7fff65fbfe30,
table=0x7f883bf7e390 "location") at ts_append.c:60
#1 0x00007f8837f638fc in w_ts_append (_msg=0x7f883bfd5490,
_table=0x7f883bf7e390 "location", _ruri=0x7f88339aa634 "\002") at
tsilo.c:225
#2 0x0000000000534db0 in do_action (h=0x7fff65fc1a40, a=0x7f883bf7d278,
msg=0x7f883bfd5490) at action.c:1059
#3 0x00000000005339e8 in run_actions (h=0x7f8837f65540, a=0x5,
msg=0x7f88339aa634) at action.c:1548
#4 0x00000000005352e6 in do_action (h=0x7fff65fc1a40, a=0x7f883bf7dc50,
msg=0x7f883bfd5490) at action.c:1048
#5 0x00000000005339e8 in run_actions (h=0x7f8837f65540, a=0x5,
msg=0x7f88339aa634) at action.c:1548
#6 0x00000000005352e6 in do_action (h=0x7fff65fc1a40, a=0x7f883bf7df48,
msg=0x7f883bfd5490) at action.c:1048
#7 0x00000000005339e8 in run_actions (h=0x7f8837f65540, a=0x5,
msg=0x7f88339aa634) at action.c:1548
#8 0x00000000005354a3 in do_action (h=0x7fff65fc1a40, a=0x7f883bfa8ab8,
msg=0x7f883bfd5490) at action.c:677
#9 0x00000000005339e8 in run_actions (h=0x7f8837f65540, a=0x5,
msg=0x7f88339aa634) at action.c:1548
#10 0x00000000005354a3 in do_action (h=0x7fff65fc1a40, a=0x7f883bfa8ab8,
msg=0x7f883bfd5490) at action.c:677
#11 0x00000000005339e8 in run_actions (h=0x7f8837f65540, a=0x5,
msg=0x7f88339aa634) at action.c:1548
#12 0x00000000005352e6 in do_action (h=0x7fff65fc1a40, a=0x7f883bf56068,
msg=0x7f883bfd5490) at action.c:1048
#13 0x00000000005339e8 in run_actions (h=0x7f8837f65540,
h at entry=0x7fff65fc1a40,
a=0x5, a at entry=0x7f883bf4f958, msg=0x7f88339aa634, msg at entry=0x7f883bfd5490)
at action.c:1548
#14 0x000000000053f885 in run_top_route (a=0x7f883bf4f958,
msg=0x7f883bfd5490, c=<optimized out>) at action.c:1634
#15 0x00000000005407e6 in receive_msg (buf=0x0, len=5,
rcv_info=0x7f88339b8638) at receive.c:196
#16 0x00000000005b806b in tcp_read_req (con=0x7f88339b8620,
bytes_read=0x7fff65fc1d50, read_flags=0x7fff65fc1d54) at tcp_read.c:1382
#17 0x00000000005bb4e1 in handle_io (fm=0xb, events=5, idx=865773108) at
tcp_read.c:1568
#18 0x00000000005c191a in io_wait_loop_epoll (h=<optimized out>,
t=<optimized out>, repeat=<optimized out>) at io_wait.h:1061
#19 tcp_receive_loop (unix_sock=938890560) at tcp_read.c:1733
#20 0x00000000004d3447 in tcp_init_children () at tcp_main.c:4787
#21 0x0000000000506863 in main_loop () at main.c:1658
#22 0x000000000041b944 in main (argc=0, argv=0x0) at main.c:2533
It seems to be on reception of REGISTER from client.
2015-09-10 12:47 GMT+02:00 Daniel-Constantin Mierla <miconda at gmail.com>:
> Hello,
>
> can you install the kamailio-dbg package to get access to debugging
> sysmbols? Once that package is installed, get the backtrace again and send
> it here.
>
> Cheers,
> Daniel
>
>
> On 10/09/15 10:55, Thibault Gueslin wrote:
>
> I have done again the trace and obtained 2 core dumps:
>
> Here is the second:
>
> Core was generated by `kamailio -f /etc/kamailio/kamailio.cfg -w /tmp'.
>
> Program terminated with signal SIGSEGV, Segmentation fault.
>
> #0 0x0000000000000264 in ?? ()
>
> (gdb) bt full
>
> #0 0x0000000000000264 in ?? ()
>
> No symbol table info available.
>
> #1 0x00007fac88d39ebb in free_faked_req () from
> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>
> No symbol table info available.
>
> #2 0x00007fac88d3a1e7 in run_failure_handlers () from
> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>
> No symbol table info available.
>
> #3 0x00007fac88d3bf59 in ?? () from
> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>
> No symbol table info available.
>
> #4 0x00007fac88d43105 in relay_reply () from
> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>
> No symbol table info available.
>
> #5 0x00007fac88d4764b in reply_received () from
> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>
> No symbol table info available.
>
> #6 0x00000000004fbfee in ?? ()
>
> No symbol table info available.
>
> #7 0x000000000054074c in receive_msg ()
>
> No symbol table info available.
>
> #8 0x00000000005b806b in tcp_read_req ()
>
> No symbol table info available.
>
> #9 0x00000000005bb704 in ?? ()
>
> No symbol table info available.
>
> #10 0x00000000005c191a in tcp_receive_loop ()
>
> No symbol table info available.
>
> #11 0x00000000004d3447 in tcp_init_children ()
>
> No symbol table info available.
>
> #12 0x0000000000506863 in main_loop ()
>
> No symbol table info available.
>
> #13 0x000000000041b944 in main ()
>
> No symbol table info available.
>
>
> The scenario is to call a remote which is started. After starting the
> regisration, the call is presented, then kill the app, then started again,
> call is presented, wait while ringing....
>
> After a few times, kamailio crashs.
>
> (sent to mailing list now)
>
> 2015-09-09 17:20 GMT+02:00 Daniel-Constantin Mierla < <miconda at gmail.com>
> miconda at gmail.com>:
>
>> This is during shut down clean up. Have you stopped kamailio manually? If
>> not, aave you got more than one core file? If not, enable one core file per
>> process.
>>
>> Cheers,
>> Daniel
>>
>>
>> On 09/09/15 17:03, Thibault Gueslin wrote:
>>
>> It seems related to tsilo module and tm module:
>>
>>
>> #0 0x00007f6a7c8978f5 in lock_entry () from
>> /usr/lib/x86_64-linux-gnu/kamailio/modules/tsilo.so
>>
>>
>> #0 0x00007f6a7c8978f5 in lock_entry () from
>> /usr/lib/x86_64-linux-gnu/kamailio/modules/tsilo.so
>>
>> (gdb) bt full
>>
>> #0 0x00007f6a7c8978f5 in lock_entry () from
>> /usr/lib/x86_64-linux-gnu/kamailio/modules/tsilo.so
>>
>> No symbol table info available.
>>
>> #1 0x00007f6a7c89d185 in ts_onreply () from
>> /usr/lib/x86_64-linux-gnu/kamailio/modules/tsilo.so
>>
>> No symbol table info available.
>>
>> #2 0x00007f6a7f688f67 in run_trans_callbacks_internal () from
>> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>>
>> No symbol table info available.
>>
>> #3 0x00007f6a7f68921e in run_trans_callbacks () from
>> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>>
>> No symbol table info available.
>>
>> #4 0x00007f6a7f6043e0 in free_cell () from
>> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>>
>> No symbol table info available.
>>
>> #5 0x00007f6a7f605c58 in free_hash_table () from
>> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>>
>> No symbol table info available.
>>
>> #6 0x00007f6a7f67f9a1 in tm_shutdown () from
>> /usr/lib/x86_64-linux-gnu/kamailio/modules/tm.so
>>
>> No symbol table info available.
>>
>> #7 0x0000000000509142 in destroy_modules ()
>>
>> No symbol table info available.
>>
>> #8 0x00000000004ff559 in cleanup ()
>>
>> No symbol table info available.
>>
>> #9 0x00000000005004e8 in ?? ()
>>
>> No symbol table info available.
>>
>> #10 0x00000000005023cb in handle_sigs ()
>>
>> No symbol table info available.
>>
>> #11 0x0000000000506a3e in main_loop ()
>>
>> No symbol table info available.
>>
>> #12 0x000000000041b944 in main ()
>>
>> No symbol table info available.
>>
>> version: kamailio 4.3.1 (x86_64/linux)
>>
>> flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS,
>> DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC,
>> F_MALLOC, DBG_F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
>> USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
>>
>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
>> MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
>>
>> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
>>
>> id: unknown
>>
>> compiled with gcc 4.9.2
>>
>>
>> I have seen some commits related to tm module after release 4.3.1
>>
>>
>> Should it work better in 4.3.2 ?
>>
>>
>> Regards
>>
>>
>> thibault
>>
>>
>>
>> 2015-09-09 14:26 GMT+02:00 Daniel-Constantin Mierla <miconda at gmail.com>:
>>
>>> Hello,
>>>
>>> do you get other error messages in syslog before the one "CRITICAL: ..."?
>>>
>>> It is hard to say it is from something already known or new. Right now
>>> there is an open issue for saving dialog variables in database that was not
>>> yet sorted out. Another one related to tm and memory manager, reported
>>> after 4.3.1, got a safety fix. Apart of these two, I don't remember any
>>> active crash report for the moment.
>>>
>>> If you get it very often, then it should be easy to catch and
>>> troubleshoot properly. Can you run it so it generates a core file?
>>>
>>> The easiest way is to run it as root, with 'ulimit -c unlimited'
>>> executed before starting kamailio.
>>>
>>> Can you share what operating system are you using and exact version of
>>> kamailio (output of kamailio -v)?
>>>
>>> Cheers,
>>> Daniel
>>>
>>>
>>> On 09/09/15 12:15, Thibault Gueslin wrote:
>>>
>>> After upgrading to kamailio 4.3.1 and modifying the config, I get this
>>> crash very often:
>>> ...
>>>
>>> CRITICAL: <core> [pass_fd.c:275]: receive_fd(): EOF on 21
>>>
>>> ALERT: <core> [main.c:728]: handle_sigs(): child process 9341 exited by
>>> a signal 11
>>>
>>> ALERT: <core> [main.c:731]: handle_sigs(): core was not generated
>>>
>>> INFO: <core> [main.c:743]: handle_sigs(): terminating due to SIGCHLD
>>>
>>> INFO: <core> [main.c:794]: sig_usr(): signal 15 received
>>>
>>> INFO: <core> [main.c:794]: sig_usr(): signal 15 received
>>>
>>> INFO: <core> [main.c:794]: sig_usr(): signal 15 received
>>>
>>> ...
>>>
>>>
>>> Is it known ? or may be related to my config ?
>>>
>>>
>>> Thank you
>>>
>>>
>>> Thibault
>>>
>>>
>>> _______________________________________________
>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users at lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>>
>>> --
>>> Daniel-Constantin Mierlahttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
>>> Book: SIP Routing With Kamailio - http://www.asipto.com
>>> Kamailio Advanced Training, Sep 28-30, 2015, in Berlin - http://asipto.com/u/kat
>>>
>>>
>>
>> --
>> Daniel-Constantin Mierlahttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
>> Book: SIP Routing With Kamailio - http://www.asipto.com
>> Kamailio Advanced Training, Sep 28-30, 2015, in Berlin - http://asipto.com/u/kat
>>
>>
>
> --
> Daniel-Constantin Mierlahttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
> Book: SIP Routing With Kamailio - http://www.asipto.com
> Kamailio Advanced Training, Sep 28-30, 2015, in Berlin - http://asipto.com/u/kat
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20150910/bd2a7d1d/attachment.html>
More information about the sr-users
mailing list