[sr-dev] [kamailio/kamailio] Core dumps possibly related to #2616 or http_async_query (#2632)

Thu Feb 18 12:21:21 CET 2021

I've have similar crashes on syslog(). But I don't use HTTP(S).

### Troubleshooting

#### Debugging Data

```
(lldb) target create "kamailio" --core "/var/coredump/986/kamailio.4087.core"
Core file '/var/coredump/986/kamailio.4087.core' (x86_64) was loaded.
(lldb) bt all
* thread #1, name = 'kamailio', stop reason = signal SIGSEGV
  * frame #0: 0x00000008009fa56d libc.so.7`__je_tcache_bin_flush_small [inlined] extent_arena_get(extent=0x0000000000000000) at extent_inlines.h:39:43
    frame #1: 0x00000008009fa56d libc.so.7`__je_tcache_bin_flush_small(tsd=<unavailable>, tcache=<unavailable>, tbin=0x0000000800d1a2a8, binind=<unavailable>, rem=33) at jemalloc_tcache.c:123
    frame #2: 0x00000008009fa17c libc.so.7`__je_tcache_event_hard(tsd=<unavailable>, tcache=0x0000000800d1a250) at jemalloc_tcache.c:54:4
    frame #3: 0x0000000800a37934 libc.so.7`__malloc [inlined] arena_malloc(tsdn=0x0000000800d1a090, arena=0x0000000000000000, size=<unavailable>, ind=<unavailable>, zero=false, tcache=<unavailable>, slow_path=false) at arena_inlines_b.h:0
    frame #4: 0x0000000800a378a7 libc.so.7`__malloc [inlined] iallocztm(tsdn=0x0000000800d1a090, size=<unavailable>, ind=<unavailable>, zero=false, tcache=<unavailable>, is_internal=false, arena=0x0000000000000000, slow_path=false) at jemalloc_internal_inlines_c.h:53
    frame #5: 0x0000000800a378a7 libc.so.7`__malloc [inlined] imalloc_no_sample(sopts=<unavailable>, dopts=<unavailable>, tsd=<unavailable>, size=<unavailable>, usize=<unavailable>, ind=<unavailable>) at jemalloc_jemalloc.c:1713
    frame #6: 0x0000000800a378a7 libc.so.7`__malloc [inlined] imalloc_body(sopts=<unavailable>, dopts=<unavailable>, tsd=<unavailable>) at jemalloc_jemalloc.c:1909
    frame #7: 0x0000000800a378a7 libc.so.7`__malloc [inlined] imalloc(sopts=<unavailable>, dopts=<unavailable>) at jemalloc_jemalloc.c:2009
    frame #8: 0x0000000800a3768b libc.so.7`__malloc(size=<unavailable>) at jemalloc_jemalloc.c:2042
    frame #9: 0x0000000800acaf34 libc.so.7`__smakebuf(fp=0x0000000800d06010) at makebuf.c:73:11
    frame #10: 0x0000000800acae39 libc.so.7`__swsetup(fp=0x0000000800d06010) at wsetup.c:82:3
    frame #11: 0x0000000800a67b53 libc.so.7`__vfprintf(fp=0x0000000800d06010, locale=0x0000000800ade698, fmt0="", ap=0x00007fffffff8cf0) at vfprintf.c:462:6
    frame #12: 0x0000000800a678c5 libc.so.7`vfprintf_l(fp=0x0000000800d06010, locale=0x0000000800ade698, fmt0="", ap=0x00007fffffff8cf0) at vfprintf.c:285:9
    frame #13: 0x0000000800a6f113 libc.so.7`fprintf(fp=<unavailable>, fmt=<unavailable>) at fprintf.c:57:8
    frame #14: 0x0000000800aa8f01 libc.so.7`vsyslog [inlined] vsyslog1(pri=135, fmt="", ap=0x00007fffffff9c50) at syslog.c:173:8
    frame #15: 0x0000000800aa8e4c libc.so.7`vsyslog(pri=<unavailable>, fmt="", ap=0x00007fffffff9c50) at syslog.c:363
    frame #16: 0x0000000800aa8dcd libc.so.7`syslog(pri=<unavailable>, fmt=<unavailable>) at syslog.c:129:2
    frame #17: 0x00000000006655dc kamailio`handle_tcp_child(tcp_c=0x00000008014dcf08, fd_i=-1) at tcp_main.c:3559:2
    frame #18: 0x000000000065b7e9 kamailio`handle_io(fm=0x00000008014e6080, ev=1, idx=-1) at tcp_main.c:4523:8
    frame #19: 0x0000000000644502 kamailio`io_wait_loop_kqueue(h=0x00000000008dbf50, t=5, repeat=0) at io_wait.h:1187:9
    frame #20: 0x000000000063cb62 kamailio`tcp_main_loop at tcp_main.c:4824:5
    frame #21: 0x00000000002e973a kamailio`main_loop at main.c:1779:5
    frame #22: 0x00000000002f7650 kamailio`main(argc=7, argv=0x00007fffffffec70) at main.c:2856:6
    frame #23: 0x00000000002d1af0 kamailio`_start(ap=<unavailable>, cleanup=<unavailable>) at crt1.c:76:7
```

#### Log Messages

```
03:20:21.349512 4087 - - DEBUG: <core> [core/tcp_main.c:4130]: send2child(): selected tcp worker idx:1 proc:26 pid:4080 for activity on [tcp:91.217.xx.yy:5060], 0x802e70718
03:20:27.364691 4087 - - DEBUG: <core> [core/tcp_main.c:3560]: handle_tcp_child(): reader response= 802e70718, 1 from 1 
03:20:27.364727 4087 - - DEBUG: <core> [core/io_wait.h:375]: io_watch_add(): DBG: io_watch_add(0x8dbf50, 53, 2, 0x802e70718), fd_no=43
03:20:27.364746 4087 - - DEBUG: <core> [core/tcp_main.c:3687]: handle_tcp_child(): CONN_RELEASE  0x802e70718 refcnt= 1
03:20:29.740878 4062 - - DEBUG: {1 1001 SUBSCRIBE ceccb6ad-add35f9c at 10.10.10.251} <core> [core/socket_info.c:646]: grep_sock_info(): checking if host==us: 13==13 && [78.37.aa.bb] == [91.217.xx.yy]
03:20:51.496077 4087 - - DEBUG: <core> [core/io_wait.h:600]: io_watch_del(): DBG: io_watch_del (0x8dbf50, 53, -1, 0x0) fd_no=44 called
03:20:51.496127 4087 - - DEBUG: <core> [core/tcp_main.c:4457]: handle_tcpconn_ev(): sending to child, events 1
03:20:51.496147 4087 - - DEBUG: <core> [core/tcp_main.c:4130]: send2child(): selected tcp worker idx:2 proc:27 pid:4081 for activity on [tcp:91.217.xx.yy:5060], 0x802e70718
03:20:58.190139 4054 - - ALERT: <core> [main.c:777]: handle_sigs(): child process 4087 exited by a signal 11
```

### Additional Information

  * **Kamailio Version** - output of `kamailio -v`

```
kamailio 5.4.3 (x86_64/freebsd) e19ae3
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, select, kqueue.
id: e19ae3 
compiled on 14:45:07 Feb 11 2021 with cc FreeBSD clang version 10.0.1 (git at github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
```

* **Operating System**:

```
FreeBSD 12.2-STABLE r369029
OpenSSL 1.1.1i-freebsd  8 Dec 2020
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2632#issuecomment-781274045
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-dev/attachments/20210218/8a6a955e/attachment.htm>