Re: [SR-Users] Multiple crashes of Kamailio 4.2.1

Hello,

there are newer releases in branch 4.2, was there any reason to stop at 4.2.3?

Back to the backtrace, this one is from shutdown, happening during the cleanup, not showing the reason of the crash at runtime, unless there was a manual shut down triggered at that moment. Do you have another core file?

Cheers, Daniel

On 10/09/15 12:36, Igor Potjevlesch wrote:

Hello,

We got some others crashes even after updated to 4.2.3.

From the last one, I got the following into "bt full":

Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid -m 256 -M 64'.

Program terminated with signal 11, Segmentation fault.

#0 0x0000000000617612 in fm_insert_free (qm=0x7f7d4206d000, frag=0x7f7d422ea368) at mem/f_malloc.c:245

245 if (*f) (*f)->prv_free = &(frag->u.nxt_free);

Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.5-7.el6_0.x86_64 db4-4.7.25-18.el6_4.x86_64 elfutils-libelf-0.152-1.el6.x86_64 glibc-2.12-1.132.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.6.x86_64 libacl-2.2.49-6.el6.x86_64 libattr-2.4.44-7.el6.x86_64 libcap-2.16-5.5.el6.x86_64 libcom_err-1.41.12-18.el6.x86_64 libgcc-4.4.7-11.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 lm_sensors-libs-3.1.1-17.el6.x86_64 lua-5.1.4-4.1.el6.x86_64 mysql-libs-5.1.73-3.el6_5.x86_64 net-snmp-libs-5.5-50.el6_6.1.x86_64 nspr-4.10.0-1.el6.x86_64 nss-3.15.1-15.el6.x86_64 nss-softokn-freebl-3.14.3-9.el6.x86_64 nss-util-3.15.1-3.el6.x86_64 openssl-1.0.1e-30.el6_6.4.x86_64 pcre-7.8-6.el6.x86_64 perl-libs-5.10.1-136.el6.x86_64 popt-1.13-7.el6.x86_64 rpm-libs-4.8.0-37.el6.x86_64 tcp_wrappers-libs-7.6-57.el6.x86_64 xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 zlib-1.2.3-29.el6.x86_64

(gdb) bt full

#0 0x0000000000617612 in fm_insert_free (qm=0x7f7d4206d000, frag=0x7f7d422ea368) at mem/f_malloc.c:245

    f = 0x7f7d4206d188

    hash = 4

#1 0x000000000061a2fe in fm_free (qm=0x7f7d4206d000, p=0x7f7d422ea3a0, file=0x7f7d53408ff0 "dialog: dlg_timer.c", func=0x7f7d53409d00 "destroy_dlg_timer", line=95)

at mem/f_malloc.c:614

    f = 0x7f7d422ea368

    __FUNCTION__ = "fm_free"

#2 0x00007f7d533e441f in destroy_dlg_timer () at dlg_timer.c:95

    __FUNCTION__ = "destroy_dlg_timer"

#3 0x00007f7d53393268 in mod_destroy () at dialog.c:784

No locals.

#4 0x0000000000590d79 in destroy_modules () at sr_module.c:811

    t = 0x7f7d59e9d670

    foo = 0x7f7d59e9d440

    __FUNCTION__ = "destroy_modules"

#5 0x000000000049bb43 in cleanup (show_status=1) at main.c:569

    memlog = 0

   __FUNCTION__ = "cleanup"

#6 0x000000000049d10b in shutdown_children (sig=15, show_status=1) at main.c:711

    __FUNCTION__ = "shutdown_children"

#7 0x000000000049f6e1 in handle_sigs () at main.c:802

    chld = 0

    chld_status = 139

   memlog = 1107776752

    __FUNCTION__ = "handle_sigs"

#8 0x00000000004a6fbf in main_loop () at main.c:1757

    i = 8

    pid = 4147

    si = 0x0

    si_desc = "udp receiver child=7

sock=91.213.145.60:5060\000\000\000\000\016\b\000\000\377\177\000\000\260T\aB}\177\000\000\000\000\000\020\004\000\000\000\260T\aB}\177\000\000\060SA\000\000\000\000\000\260:\240\343\001\000\000\000@8\240\343\377\177\000\000\032dN\000\000\000\000\000h\261\346Yz\000\000\000\276}p\000\000\000\000"

    nrprocs = 8

    __FUNCTION__ = "main_loop"

#9 0x00000000004ab8bf in main (argc=7, argv=0x7fffe3a03ab8) at main.c:2578

    cfg_stream = 0x1a04010

    c = -1

    r = 0

    tmp = 0x7fffe3a04f70 ""

    tmp_len = 0

    port = 0

    proto = 32767

    options = 0x6fcc00

":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"

    ret = -1

    seed = 3320376299

    rfd = 4

    debug_save = 0

    debug_flag = 0

    dont_fork_cnt = 0

    n_lst = 0xc2

    p = 0x7fffe3a0398e ""

    __FUNCTION__ = "main"

Regards,

Igor.

*De :*Igor Potjevlesch [mailto:igor.potjevlesch@gmail.com] *Envoyé :* lundi 20 juillet 2015 16:22 *À :* sr-users@lists.sip-router.org *Objet :* Multiple crashes of Kamailio 4.2.1

Hello,

3 crashes occurred today (we never seen crashes on this server before) during a period of 20 minutes.

I had a look to the coredumps. Here is a quick overview of the first entries of "bt full":

core.27671 : #0 0x00007f5577048d65 in run_trans_callbacks_internal (cb_lst=0x7f5560abbd50, type=2, trans=0x7f5560abbce0, params=0x7fff3712acb0) at t_hooks.c:286

core.27656 : #0 0x00007f557700a048 in free_cell (dead_cell=0x7f5560abbce0) at h_table.c:175

core.27685 : #0 0x00007f5577048d65 in run_trans_callbacks_internal (cb_lst=0x7f5560abbd50, type=2, trans=0x7f5560abbce0, params=0x7fff3712acb0) at t_hooks.c:286

core.27689 : #0 0x00007f5577048d65 in run_trans_callbacks_internal (cb_lst=0x7f5560abbd50, type=2, trans=0x7f5560abbce0, params=0x7fff3712acb0) at t_hooks.c:286

core.27691 : #0 0x00007f5577048d65 in run_trans_callbacks_internal (cb_lst=0x7f5560abbd50, type=2, trans=0x7f5560abbce0, params=0x7fff3712acb0) at t_hooks.c:286

core.31977 : #0 0x00007f038e66fdee in free_cell (dead_cell=0x7f037811ebe0) at h_table.c:157

core.31994 : #0 0x00007f038e6ab65a in t_forward_nonack (t=0x7f037811ebe0, p_msg=0x7f038fdca438, proxy=0x0, proto=0) at t_fwd.c:1762

core.3700 : #0 0x000000000061697a in fm_insert_free (qm=0x7f19861fa000, frag=0x7f19864d53e8) at mem/f_malloc.c:245

core.3727 : #0 0x00000000006167a4 in fm_extract_free (qm=0x7f19861fa000, frag=0x7f19864f4cc8) at mem/f_malloc.c:208

I don't know how to proceed now. Can you help me to highlight the issue here? Thank you.

Regards,

Igor.

---

SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users