[sr-dev] SCSCF crashing during registration
Jason Penton
jason.penton at gmail.com
Fri Mar 14 07:46:27 CET 2014
Hey Daniel,
Can you build from source instead of using debian packages. This looks like
an inconsistent build that you are running. I suspect some modules have not
been built using the correct sources...
On Fri, Mar 14, 2014 at 12:22 AM, Daniel Ciprus
<daniel.ciprus at acision.com>wrote:
> For now iscscf crashed.
>
> mem_safety=1
>
> (gdb) bt full
> #0 0x00000031ba432925 in raise () from /lib64/libc.so.6
> No symbol table info available.
> #1 0x00000031ba434105 in abort () from /lib64/libc.so.6
> No symbol table info available.
> #2 0x0000000000546750 in qm_debug_frag (qm=0x7f1b2794e000,
> f=0x7f1b27a69a00) at mem/q_malloc.c:142
> __FUNCTION__ = "qm_debug_frag"
> #3 0x000000000054853a in qm_free (qm=0x7f1b2794e000, p=0x7f1b27a69a30,
> file=0x7f1b2ce7b0ea "tm: uac.c", func=0x7f1b2ce7baa0
> "free_local_ack_unsafe", line=600) at mem/q_malloc.c:464
> f = 0x7f1b27a69a00
> size = 695784701953
> next = 0x7f1b2ce60e08
> prev = 0x7f1b2ce60c6d
> __FUNCTION__ = "qm_free"
> #4 0x00007f1b2ce5c7de in free_local_ack_unsafe (lack=0x7f1b27a69a30) at
> uac.c:600
> __FUNCTION__ = "free_local_ack_unsafe"
> #5 0x00007f1b2cdf2a57 in free_cell (dead_cell=0x7f1b27a69730) at
> h_table.c:217
>
> b = 0x0
> i = 0
> rpl = 0x0
> tt = 0x0
> foo = 0x1
> cbs = 0x0
> cbs_tmp = 0x7c00000001
> __FUNCTION__ = "free_cell"
> #6 0x00007f1b2cdf32ee in free_hash_table () at h_table.c:441
> p_cell = 0x7f1b27a69730
> tmp_cell = 0x7f1b27bff350
>
> i = 580
> __FUNCTION__ = "free_hash_table"
> #7 0x00007f1b2ce06fca in tm_shutdown () at t_funcs.c:122
> __FUNCTION__ = "tm_shutdown"
> #8 0x00000000004f7c7a in destroy_modules () at sr_module.c:817
> t = 0x7f1b2d0d7df0
> foo = 0x0
> __FUNCTION__ = "destroy_modules"
> #9 0x0000000000466e63 in cleanup (show_status=1) at main.c:560
> memlog = 32767
> __FUNCTION__ = "cleanup"
> #10 0x0000000000467f65 in shutdown_children (sig=15, show_status=1) at
> main.c:702
> __FUNCTION__ = "shutdown_children"
> #11 0x0000000000469679 in handle_sigs () at main.c:793
> chld = 0
> chld_status = 139
> memlog = 0
> __FUNCTION__ = "handle_sigs"
> #12 0x000000000046db19 in main_loop () at main.c:1746
> i = 8
> pid = 13103
> si = 0x0
> si_desc = "udp receiver child=7 sock=10.67.97.63:5060\000-\033\177\000\000\001\000\000\000z\000\000\000\376}^\000\000\000\000\000\340o^\000\000\000\000\000\026\070\266M\000\000\000\000\300LA\000\000\000\000\000\260\236\033\346\377\177",
> '\000' <repeats 18 times>"\360,
> \234\033\346\377\177\000\000.\241K\000\000\000\000"
> nrprocs = 8
> __FUNCTION__ = "main_loop"
> #13 0x000000000047030b in main (argc=13, argv=0x7fffe61b9eb8) at
> main.c:2533
> cfg_stream = 0x2753010
> c = -1
> r = 0
> tmp = 0x7fffe61bae4b ""
> tmp_len = 0
> port = 0
> proto = 0
> options = 0x5e0340
> ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"
> ret = -1
> seed = 3395117202
> rfd = 4
> debug_save = 0
> debug_flag = 0
> dont_fork_cnt = 0
> n_lst = 0x31ba40fba0
> p = 0x5ca460 "H\211l$\330L\211d$\340H\215-\217\233*"
> __FUNCTION__ = "main"
> (gdb)
>
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13641]: ERROR: ims_usrloc_pcscf
> [udomain.c:400]: update_pcontact(): no more shm_mem
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13641]: ERROR:
> ims_registrar_pcscf [save.c:208]: update_contacts(): failed to update pcscf
> contact
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13641]: ERROR: ims_usrloc_pcscf
> [udomain.c:400]: update_pcontact(): no more shm_mem
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13641]: ERROR:
> ims_registrar_pcscf [save.c:208]: update_contacts(): failed to update pcscf
> contact
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13641]: ERROR: ims_usrloc_pcscf
> [udomain.c:400]: update_pcontact(): no more shm_mem
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13641]: ERROR:
> ims_registrar_pcscf [save.c:208]: update_contacts(): failed to update pcscf
> contact
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13657]: ERROR: <core>
> [tcp_main.c:4237]: handle_tcpconn_ev(): connect 10.67.64.29:1305 failed
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13657]: ERROR: <core>
> [tcp_main.c:4237]: handle_tcpconn_ev(): connect 10.67.64.29:1305 failed
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13657]: ERROR: <core>
> [tcp_main.c:4237]: handle_tcpconn_ev(): connect 10.67.64.29:1325 failed
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13657]: ERROR: <core>
> [tcp_main.c:4237]: handle_tcpconn_ev(): connect 10.67.64.29:1325 failed
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13638]: WARNING:
> ims_registrar_pcscf [service_routes.c:440]: assert_identity(): Contact not
> found based on Contact, trying IP/Port/Proto
> Mar 13 18:15:51 ricvmf-fusion01 kam-scscf[13512]: ERROR: <script>:
> SUBSCRIBE (sip:18047621111 at lab.ricuc.com (10.67.97.63:5090) to
> sip:18047621111 at lab.ricuc.com, d1dc7671-e6fb-0071-1042-edba53716ada)
> Mar 13 18:15:51 ricvmf-fusion01 kam-scscf[13512]: ERROR: tm [tm.c:1302]:
> w_t_reply(): ERROR: t_reply: cannot send a t_reply to a message for which
> no T-state has been established
> Mar 13 18:15:51 ricvmf-fusion01 kam-pcscf[13638]: WARNING:
> ims_registrar_pcscf [service_routes.c:440]: assert_identity(): Contact not
> found based on Contact, trying IP/Port/Proto
> Mar 13 18:15:51 ricvmf-fusion01 kam-scscf[13512]: ERROR: <script>:
> SUBSCRIBE (sip:18047621111 at lab.ricuc.com (10.67.97.63:5090) to
> sip:18047621111 at lab.ricuc.com, d1dc7671-e6fb-0071-1042-edba53716ada)
> Mar 13 18:15:51 ricvmf-fusion01 kam-scscf[13512]: ERROR: tm [tm.c:1302]:
> w_t_reply(): ERROR: t_reply: cannot send a t_reply to a message for which
> no T-state has been established
> Mar 13 18:15:52 ricvmf-fusion01 kernel: kamailio[13083]: segfault at
> 7f1b27c05af0 ip 00007f1b27c05af0 sp 00007fffe61b9bb8 error 15 in zero
> (deleted)[7f1b2794e000+2000000]
> Mar 13 18:15:52 ricvmf-fusion01 kernel: kamailio[13083]: segfault at
> 7f1b27c05af0 ip 00007f1b27c05af0 sp 00007fffe61b9bb8 error 15 in zero
> (deleted)[7f1b2794e000+2000000]
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13074]: ALERT: <core>
> [main.c:775]: handle_sigs(): child process 13083 exited by a signal 11
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13103]: : <core>
> [pass_fd.c:293]: receive_fd(): ERROR: receive_fd: EOF on 18
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13074]: ALERT: <core>
> [main.c:778]: handle_sigs(): core was generated
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13074]: INFO: <core>
> [main.c:790]: handle_sigs(): INFO: terminating due to SIGCHLD
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13103]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13102]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13101]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13100]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13099]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13098]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13097]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13095]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13096]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13091]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13094]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13088]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13093]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13074]: ALERT: <core>
> [main.c:775]: handle_sigs(): child process 13083 exited by a signal 11
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13103]: : <core>
> [pass_fd.c:293]: receive_fd(): ERROR: receive_fd: EOF on 18
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13074]: ALERT: <core>
> [main.c:778]: handle_sigs(): core was generated
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13074]: INFO: <core>
> [main.c:790]: handle_sigs(): INFO: terminating due to SIGCHLD
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13103]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13102]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13101]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13100]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13099]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13098]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13097]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13095]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13096]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13091]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13094]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13088]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13093]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13086]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13086]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13076]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13084]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13077]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13082]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13085]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13078]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13080]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13087]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13076]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13084]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13077]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13082]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13085]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13078]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13080]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13087]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13075]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13089]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13079]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13081]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13090]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13092]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13075]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13089]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13079]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13081]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13090]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13092]: INFO: <core>
> [main.c:841]: sig_usr(): INFO: signal 15 received
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13074]: INFO: cdp [mod.c:251]:
> cdp_exit(): CDiameterPeer child stopping ...
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13074]: INFO: cdp
> [diameter_peer.c:428]: diameter_peer_destroy(): destroy_diameter_peer():
> Terminating all children...
> Mar 13 18:15:52 ricvmf-fusion01 kam-icscf[13074]: INFO: cdp
> [diameter_peer.c:435]: diameter_peer_destroy(): destroy_diameter_peer():
> Waiting for child [13094] to terminate..
>
>
>
> On 03/13/2014 05:47 PM, Hugh Waite wrote:
>
> Dan,
> There are two cores because of a crash in one process followed by a crash
> when the other processes are trying to shutdown.
>
> What's interesting is that the bt doesn't show useful pointers. If you
> have installed from RPMs make sure the kamailio-debuginfo is from the same
> build as the other RPMs.
>
> Also, do the logs say anything? There should be a log entry from the
> kernel for the segfault/signal that says which module crashed (e.g.
> registrar.so) and possibly (hopefully) an error message just before that.
>
> Hugh
>
>
> On 13/03/2014 19:53, Daniel Ciprus wrote:
>
> Jason,
>
> I've tried multiple combinations for pattern but I'm getting only 2 core
> files ...
>
> Details:
>
> ~]# cat /proc/sys/kernel/core_pattern
> /tmp/core.%e.sig%s.%p
>
> ~]# lsb_release -a
> LSB Version:
> :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
> Distributor ID: RedHatEnterpriseServer
> Description: Red Hat Enterprise Linux Server release 6.5 (Santiago)
> Release: 6.5
> Codename: Santiago
>
>
>
> (gdb) bt
> #0 0x00000000005350b0 in ?? ()
> #1 0x000000000053542a in ?? ()
> #2 0x00000000005356c7 in timer_main ()
> #3 0x000000000046d572 in main_loop ()
> #4 0x000000000047030b in main ()
> (gdb) bt full
> #0 0x00000000005350b0 in ?? ()
> No symbol table info available.
> #1 0x000000000053542a in ?? ()
> No symbol table info available.
> #2 0x00000000005356c7 in timer_main ()
> No symbol table info available.
> #3 0x000000000046d572 in main_loop ()
> No symbol table info available.
> #4 0x000000000047030b in main ()
> No symbol table info available.
> (gdb)
>
> (gdb) bt
> #0 0x00000031ba432925 in raise () from /lib64/libc.so.6
> #1 0x00000031ba434105 in abort () from /lib64/libc.so.6
> #2 0x0000000000546750 in ?? ()
> #3 0x000000000054853a in qm_free ()
> #4 0x00007f23d98f87de in free_local_ack_unsafe (lack=0x7f23d3319d70) at
> uac.c:600
> #5 0x00007f23d988ea57 in free_cell (dead_cell=0x7f23d3319a70) at
> h_table.c:217
> #6 0x00007f23d988f2ee in free_hash_table () at h_table.c:441
> #7 0x00007f23d98a2fca in tm_shutdown () at t_funcs.c:122
> #8 0x00000000004f7c7a in destroy_modules ()
> #9 0x0000000000466e63 in cleanup ()
> #10 0x0000000000467f65 in ?? ()
> #11 0x0000000000469679 in handle_sigs ()
> #12 0x000000000046db19 in main_loop ()
> #13 0x000000000047030b in main ()
> (gdb) bt full
> #0 0x00000031ba432925 in raise () from /lib64/libc.so.6
> No symbol table info available.
> #1 0x00000031ba434105 in abort () from /lib64/libc.so.6
> No symbol table info available.
> #2 0x0000000000546750 in ?? ()
> No symbol table info available.
> #3 0x000000000054853a in qm_free ()
> No symbol table info available.
> #4 0x00007f23d98f87de in free_local_ack_unsafe (lack=0x7f23d3319d70) at
> uac.c:600
> __FUNCTION__ = "free_local_ack_unsafe"
> #5 0x00007f23d988ea57 in free_cell (dead_cell=0x7f23d3319a70) at
> h_table.c:217
> b = 0x0
> i = 0
> rpl = 0x0
> tt = 0x0
> foo = 0x2fd3221000
> cbs = 0x0
> cbs_tmp = 0x7f23d35386b8
> __FUNCTION__ = "free_cell"
> #6 0x00007f23d988f2ee in free_hash_table () at h_table.c:441
> p_cell = 0x7f23d3319a70
> tmp_cell = 0x7f23d353dca0
> i = 580
> __FUNCTION__ = "free_hash_table"
> #7 0x00007f23d98a2fca in tm_shutdown () at t_funcs.c:122
> __FUNCTION__ = "tm_shutdown"
> #8 0x00000000004f7c7a in destroy_modules ()
> No symbol table info available.
> #9 0x0000000000466e63 in cleanup ()
> No symbol table info available.
> #10 0x0000000000467f65 in ?? ()
> No symbol table info available.
> #11 0x0000000000469679 in handle_sigs ()
> No symbol table info available.
> #12 0x000000000046db19 in main_loop ()
> No symbol table info available.
> #13 0x000000000047030b in main ()
> No symbol table info available.
> (gdb)
>
>
>
> On 03/13/2014 02:58 PM, Jason Penton wrote:
>
> I don't think these cores indicate the real crash... I'd like to get some
> more detail on what actually happened? Daniel, can you re-create? Keep in
> mind that if your core dump config on your box is not configured to name
> your cores according to process id or timestamp one core will overwrite the
> other..... as a result you will never see the core that is the root cause.
>
> Which OS are you running?
>
> if Linux, I use the following in /etc/sysctl.conf:
>
> kernel.core_pattern=/tmp/core.%e.%p.%h.%t
>
>
> On Thu, Mar 13, 2014 at 8:45 PM, Carsten Bock <carsten at ng-voice.com>wrote:
>
>> It looks a little bit like a "double free".
>>
>> You could try to disable the call to "abort()" in case this happens:
>> mem_safety=1
>> See: http://www.kamailio.org/wiki/cookbooks/devel/core#mem_safety
>>
>> Kind regards,
>> Carsten
>>
>> 2014-03-13 19:44 GMT+01:00 Carsten Bock <carsten at ng-voice.com>:
>> > It looks a little bit like a "double free".
>> >
>> > You could try to disable the call to "abort()" in case this happens:
>> >
>> >
>> > 2014-03-13 17:22 GMT+01:00 Daniel Ciprus <daniel.ciprus at acision.com>:
>> >> There are no more core files on the filesystem :-(
>> >>
>> >> On 03/13/2014 12:18 PM, Jason Penton wrote:
>> >>
>> >> I'm afraid this is also not the correct core. Can you check the
>> timestamp on
>> >> the cores? Can you re-create the crash and send me the correct core?
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, Mar 13, 2014 at 5:36 PM, Daniel Ciprus <
>> daniel.ciprus at acision.com>
>> >> wrote:
>> >>>
>> >>> So I cleaned up my junkyard and I got 2 core files:
>> >>>
>> >>> (gdb) bt
>> >>> #0 0x00000000005350b0 in ?? ()
>> >>> #1 0x000000000053542a in ?? ()
>> >>> #2 0x00000000005356c7 in timer_main ()
>> >>> #3 0x000000000046d572 in main_loop ()
>> >>> #4 0x000000000047030b in main ()
>> >>> (gdb) bt full
>> >>> #0 0x00000000005350b0 in ?? ()
>> >>>
>> >>> No symbol table info available.
>> >>> #1 0x000000000053542a in ?? ()
>> >>>
>> >>> No symbol table info available.
>> >>> #2 0x00000000005356c7 in timer_main ()
>> >>>
>> >>> No symbol table info available.
>> >>> #3 0x000000000046d572 in main_loop ()
>> >>>
>> >>> No symbol table info available.
>> >>> #4 0x000000000047030b in main ()
>> >>>
>> >>> No symbol table info available.
>> >>> (gdb)
>> >>>
>> >>>
>> >>> (gdb) bt full
>> >>> #0 0x00000031ba432925 in raise () from /lib64/libc.so.6
>> >>> No symbol table info available.
>> >>> #1 0x00000031ba434105 in abort () from /lib64/libc.so.6
>> >>> No symbol table info available.
>> >>> #2 0x0000000000546750 in ?? ()
>> >>> No symbol table info available.
>> >>> #3 0x000000000054853a in qm_free ()
>> >>> No symbol table info available.
>> >>> #4 0x00007f5bf7d5a7de in free_local_ack_unsafe (lack=0x7f5bf1894528)
>> at
>> >>> uac.c:600
>> >>> __FUNCTION__ = "free_local_ack_unsafe"
>> >>> #5 0x00007f5bf7cf0a57 in free_cell (dead_cell=0x7f5bf1894228) at
>> >>> h_table.c:217
>> >>>
>> >>> b = 0x0
>> >>> i = 0
>> >>> rpl = 0x0
>> >>> tt = 0x0
>> >>> foo = 0x2ff1683000
>> >>> cbs = 0x0
>> >>> cbs_tmp = 0x7f5bf198e508
>> >>> __FUNCTION__ = "free_cell"
>> >>> #6 0x00007f5bf7cf12ee in free_hash_table () at h_table.c:441
>> >>> p_cell = 0x7f5bf1894228
>> >>> tmp_cell = 0x7f5bf1894228
>> >>> i = 3533
>> >>> __FUNCTION__ = "free_hash_table"
>> >>> #7 0x00007f5bf7d04fca in tm_shutdown () at t_funcs.c:122
>> >>>
>> >>> __FUNCTION__ = "tm_shutdown"
>> >>> #8 0x00000000004f7c7a in destroy_modules ()
>> >>> No symbol table info available.
>> >>> #9 0x0000000000466e63 in cleanup ()
>> >>> No symbol table info available.
>> >>> #10 0x0000000000467f65 in ?? ()
>> >>> No symbol table info available.
>> >>> #11 0x0000000000469679 in handle_sigs ()
>> >>> No symbol table info available.
>> >>> #12 0x000000000046db19 in main_loop ()
>> >>> No symbol table info available.
>> >>> #13 0x000000000047030b in main ()
>> >>> No symbol table info available.
>> >>> (gdb)
>> >>>
>> >>>
>> >>> On 03/13/2014 11:18 AM, Jason Penton wrote:
>> >>>
>> >>> Hi Daniel,
>> >>>
>> >>> this is the wrong core file. This is the one created on shutdown of
>> >>> kamailio. Can you do a bt on the other core file that you probably
>> have...
>> >>>
>> >>> Cheers
>> >>> Jason
>> >>>
>> >>>
>> >>> On Thu, Mar 13, 2014 at 5:05 PM, Daniel Ciprus <
>> daniel.ciprus at acision.com>
>> >>> wrote:
>> >>>>
>> >>>> Folks,
>> >>>>
>> >>>> This is happening during the registration on SCSCF.
>> >>>>
>> >>>> Server:: kamailio (4.2.0-dev2 (x86_64/linux))
>> >>>> Build:: mi_core.c compiled on 10:01:09 Mar 13 2014 with gcc 4.4.6
>> >>>> Flags:: STATS: Off, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS,
>> >>>> DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC,
>> >>>> DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
>> >>>> USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
>> >>>> GIT:: unknown
>> >>>> Now:: Thu Mar 13 11:04:47 2014
>> >>>> Up since:: Thu Mar 13 10:58:12 2014
>> >>>> Up time:: 395 [sec]
>> >>>>
>> >>>> (gdb) bt
>> >>>> #0 0x00000031ba432925 in raise () from /lib64/libc.so.6
>> >>>> #1 0x00000031ba434105 in abort () from /lib64/libc.so.6
>> >>>> #2 0x0000000000546750 in ?? ()
>> >>>> #3 0x000000000054853a in qm_free ()
>> >>>> #4 0x00007fb4def5b7de in free_local_ack_unsafe
>> (lack=0x7fb4d8b31728) at
>> >>>> uac.c:600
>> >>>> #5 0x00007fb4deef1a57 in free_cell (dead_cell=0x7fb4d8b31428) at
>> >>>> h_table.c:217
>> >>>> #6 0x00007fb4deef22ee in free_hash_table () at h_table.c:441
>> >>>> #7 0x00007fb4def05fca in tm_shutdown () at t_funcs.c:122
>> >>>> #8 0x00000000004f7c7a in destroy_modules ()
>> >>>> #9 0x0000000000466e63 in cleanup ()
>> >>>> #10 0x0000000000467f65 in ?? ()
>> >>>> #11 0x0000000000469679 in handle_sigs ()
>> >>>> #12 0x000000000046db19 in main_loop ()
>> >>>> #13 0x000000000047030b in main ()
>> >>>> (gdb) bt full
>> >>>> #0 0x00000031ba432925 in raise () from /lib64/libc.so.6
>> >>>> No symbol table info available.
>> >>>> #1 0x00000031ba434105 in abort () from /lib64/libc.so.6
>> >>>> No symbol table info available.
>> >>>> #2 0x0000000000546750 in ?? ()
>> >>>> No symbol table info available.
>> >>>> #3 0x000000000054853a in qm_free ()
>> >>>> No symbol table info available.
>> >>>> #4 0x00007fb4def5b7de in free_local_ack_unsafe
>> (lack=0x7fb4d8b31728) at
>> >>>> uac.c:600
>> >>>> __FUNCTION__ = "free_local_ack_unsafe"
>> >>>> #5 0x00007fb4deef1a57 in free_cell (dead_cell=0x7fb4d8b31428) at
>> >>>> h_table.c:217
>> >>>> b = 0x0
>> >>>> i = 0
>> >>>> rpl = 0x0
>> >>>> tt = 0x0
>> >>>> foo = 0x2fd8a8b000
>> >>>> cbs = 0x0
>> >>>> cbs_tmp = 0x7fb4d8d9c9e0
>> >>>> __FUNCTION__ = "free_cell"
>> >>>> #6 0x00007fb4deef22ee in free_hash_table () at h_table.c:441
>> >>>> p_cell = 0x7fb4d8b31428
>> >>>> tmp_cell = 0x7fb4d8b31428
>> >>>> i = 11517
>> >>>> __FUNCTION__ = "free_hash_table"
>> >>>> #7 0x00007fb4def05fca in tm_shutdown () at t_funcs.c:122
>> >>>> __FUNCTION__ = "tm_shutdown"
>> >>>> #8 0x00000000004f7c7a in destroy_modules ()
>> >>>> No symbol table info available.
>> >>>> #9 0x0000000000466e63 in cleanup ()
>> >>>> No symbol table info available.
>> >>>> #10 0x0000000000467f65 in ?? ()
>> >>>> No symbol table info available.
>> >>>> #11 0x0000000000469679 in handle_sigs ()
>> >>>> No symbol table info available.
>> >>>> #12 0x000000000046db19 in main_loop ()
>> >>>> No symbol table info available.
>> >>>> #13 0x000000000047030b in main ()
>> >>>> No symbol table info available.
>> >>>> (gdb)
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Daniel Ciprus
>> >>>> Integration engineer
>> >>>> http://www.acision.com
>> >>>>
>> >>>> 9954 Mayland Dr
>> >>>> Suite 3100
>> >>>> Richmond, VA 23233
>> >>>> USA
>> >>>> T: +1 804 762 5601
>> >>>> E: daniel.ciprus at acision.com
>> >>>>
>> >>>> ________________________________
>> >>>> This e-mail and any attachment is for authorised use by the intended
>> >>>> recipient(s) only. It may contain proprietary material, confidential
>> >>>> information and/or be subject to legal privilege. It should not be
>> copied,
>> >>>> disclosed to, retained or used by, any other party. If you are not an
>> >>>> intended recipient then please promptly delete this e-mail and any
>> >>>> attachment and all copies and inform the sender. Thank you for
>> >>>> understanding.
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> sr-dev mailing list
>> >>>> sr-dev at lists.sip-router.org
>> >>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> Daniel Ciprus
>> >>> Integration engineer
>> >>> http://www.acision.com
>> >>>
>> >>> 9954 Mayland Dr
>> >>> Suite 3100
>> >>> Richmond, VA 23233
>> >>> USA
>> >>> T: +1 804 762 5601
>> >>> E: daniel.ciprus at acision.com
>> >>>
>> >>> ________________________________
>> >>> This e-mail and any attachment is for authorised use by the intended
>> >>> recipient(s) only. It may contain proprietary material, confidential
>> >>> information and/or be subject to legal privilege. It should not be
>> copied,
>> >>> disclosed to, retained or used by, any other party. If you are not an
>> >>> intended recipient then please promptly delete this e-mail and any
>> >>> attachment and all copies and inform the sender. Thank you for
>> >>> understanding.
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> sr-dev mailing list
>> >>> sr-dev at lists.sip-router.org
>> >>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
>> >>>
>> >>
>> >>
>> >> --
>> >> Daniel Ciprus
>> >> Integration engineer
>> >> http://www.acision.com
>> >>
>> >> 9954 Mayland Dr
>> >> Suite 3100
>> >> Richmond, VA 23233
>> >> USA
>> >> T: +1 804 762 5601
>> >> E: daniel.ciprus at acision.com
>> >>
>> >> ________________________________
>> >> This e-mail and any attachment is for authorised use by the intended
>> >> recipient(s) only. It may contain proprietary material, confidential
>> >> information and/or be subject to legal privilege. It should not be
>> copied,
>> >> disclosed to, retained or used by, any other party. If you are not an
>> >> intended recipient then please promptly delete this e-mail and any
>> >> attachment and all copies and inform the sender. Thank you for
>> >> understanding.
>> >>
>> >>
>> >> _______________________________________________
>> >> sr-dev mailing list
>> >> sr-dev at lists.sip-router.org
>> >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
>> >>
>> >
>> >
>> >
>> > --
>> > Carsten Bock
>> > CEO (Geschäftsführer)
>> >
>> > ng-voice GmbH
>> > Schomburgstr. 80
>> > D-22767 Hamburg / Germany
>> >
>> > http://www.ng-voice.com
>> > mailto:carsten at ng-voice.com
>> >
>> > Office +49 40 34927219
>> > Fax +49 40 34927220
>> >
>> > Sitz der Gesellschaft: Hamburg
>> > Registergericht: Amtsgericht Hamburg, HRB 120189
>> > Geschäftsführer: Carsten Bock
>> > Ust-ID: DE279344284
>> >
>> > Hier finden Sie unsere handelsrechtlichen Pflichtangaben:
>> > http://www.ng-voice.com/imprint/
>>
>>
>>
>> --
>> Carsten Bock
>> CEO (Geschäftsführer)
>>
>> ng-voice GmbH
>> Schomburgstr. 80
>> D-22767 Hamburg / Germany
>>
>> http://www.ng-voice.com
>> mailto:carsten at ng-voice.com
>>
>> Office +49 40 34927219
>> Fax +49 40 34927220
>>
>> Sitz der Gesellschaft: Hamburg
>> Registergericht: Amtsgericht Hamburg, HRB 120189
>> Geschäftsführer: Carsten Bock
>> Ust-ID: DE279344284
>>
>> Hier finden Sie unsere handelsrechtlichen Pflichtangaben:
>> http://www.ng-voice.com/imprint/
>>
>
>
> --
> *Daniel Ciprus*
> Integration engineer
> http://www.acision.com
>
> 9954 Mayland Dr
> Suite 3100
> Richmond, VA 23233
> USA
> T: +1 804 762 5601
> E: daniel.ciprus at acision.com
>
> ------------------------------
> This e-mail and any attachment is for authorised use by the intended
> recipient(s) only. It may contain proprietary material, confidential
> information and/or be subject to legal privilege. It should not be copied,
> disclosed to, retained or used by, any other party. If you are not an
> intended recipient then please promptly delete this e-mail and any
> attachment and all copies and inform the sender. Thank you for
> understanding.
>
>
>
> _______________________________________________
> sr-dev mailing listsr-dev at lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
>
>
>
> --
> Hugh Waite
> Principal Design Engineer
> Crocodile RCS Ltd.
>
>
> --
> *Daniel Ciprus*
> Integration engineer
> http://www.acision.com
>
> 9954 Mayland Dr
> Suite 3100
> Richmond, VA 23233
> USA
> T: +1 804 762 5601
> E: daniel.ciprus at acision.com
>
> ------------------------------
> This e-mail and any attachment is for authorised use by the intended
> recipient(s) only. It may contain proprietary material, confidential
> information and/or be subject to legal privilege. It should not be copied,
> disclosed to, retained or used by, any other party. If you are not an
> intended recipient then please promptly delete this e-mail and any
> attachment and all copies and inform the sender. Thank you for
> understanding.
>
>
> _______________________________________________
> sr-dev mailing list
> sr-dev at lists.sip-router.org
> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-dev/attachments/20140314/33febea8/attachment-0001.html>
More information about the sr-dev
mailing list