[sr-dev] [kamailio/kamailio] q_malloc crash on Kamailio 5.2.3 with http_async_query function (#2091)
Michal Škuta
notifications at github.com
Thu Oct 3 17:32:39 CEST 2019
### Description
We added http_async_client module to our configuration. We want to send HTTP POST request after the call is ended (in event_route[dialog_end]).
We used the function **http_async_query** and right after calling the function Kamailio child exited by a signal 6 (more in the log down).
### Troubleshooting
#### Reproduction
In our system we just added http_async_module and its function http_async_query. It is located in own route HTTP_TEST_CALL_END called from event_route[dialog:end]
Somewhat like this:
```
route[HTTP_TEST_CALL_END] {
t_newtran();
$http_req(body) = "TEST";
http_async_query("http://192.168.6.23:2080/cdr_http", "HTTP_TEST_REPLY");
}
```
<!--
If the issue can be reproduced, describe how it can be done.
-->
#### Debugging Data
<!--
If you got a core dump, use gdb to extract troubleshooting data - full backtrace,
local variables and the list of the code at the issue location.
gdb /path/to/kamailio /path/to/corefile
bt full
info locals
list
If you are familiar with gdb, feel free to attach more of what you consider to
be relevant.
-->
```
Core was generated by `/opt/ipgate/kamailio/sbin/kamailio -f /opt/ipgate/kamailio/etc/kamailio/kamaili'.
Program terminated with signal 6, Aborted.
#0 0xb7791424 in __kernel_vsyscall ()
(gdb) bt full
#0 0xb7791424 in __kernel_vsyscall ()
No symbol table info available.
#1 0xb7633661 in raise () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
No symbol table info available.
#2 0xb7636a92 in abort () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
No symbol table info available.
#3 0x0829fed6 in qm_debug_check_frag (qm=0x97607008, f=0x97a7ff3c,
file=0x966f22d6 "http_async_client: hm_hash.c", line=82, efile=0x83f5093 "core/mem/q_malloc.c",
eline=384) at core/mem/q_malloc.c:151
__FUNCTION__ = "qm_debug_check_frag"
#4 0x082a16aa in qm_malloc (qmp=0x97607008, size=8, file=0x966f22d6 "http_async_client: hm_hash.c",
func=0x966f2bfb "build_hash_key", line=82, mname=0x966f22c4 "http_async_client")
at core/mem/q_malloc.c:384
qm = 0x97607008
f = 0x97a7ff3c
hash = 4
list_cntr = 1
__FUNCTION__ = "qm_malloc"
#5 0x966d5323 in build_hash_key (p=0x567d9620) at hm_hash.c:82
hash_str = 0x966f2be9
pointer_str = 0x97a7ff1c "0x567d9620"
len = 10
hash = 360
__FUNCTION__ = "build_hash_key"
#6 0x966d5d94 in build_http_m_cell (p=0x567d9620) at hm_hash.c:116
cell = 0x567cab3c
len = 360
__FUNCTION__ = "build_http_m_cell"
#7 0x966eafac in new_request (query=0xbfe65718, query_params=0xbfe65720,
cb=0x966c9125 <async_http_cb>, param=0x567be804) at http_multi.c:442
__FUNCTION__ = "new_request"
easy = 0x567d9620
rc = -1217109748
cell = 0x0
#8 0x966ce473 in notification_socket_cb (fd=16, event=2, arg=0x5679c988) at async_http.c:390
worker = 0x5679c988
received = 4
i = 0
len = -1075423144
aq = 0x567be804
query_params = {timeout = 500, tls_verify_host = 1, tls_verify_peer = 1, headers = 0x0,
method = 0, tls_client_cert = 0x0, tls_client_key = 0x0, tls_ca_path = 0x0, body = {
s = 0x567cb650 "TEST\300\300\300\300\355\357ͫ", len = 4}, authmethod = 3,
username = 0x0, password = 0x0}
query = {s = 0x567bf75c "http://192.168.6.23:2080/cdr_httpt_", len = 33}
__FUNCTION__ = "notification_socket_cb"
#9 0x9661d522 in event_base_loop () from /usr/lib/i386-linux-gnu/libevent-2.0.so.5
No symbol table info available.
#10 0x9661e683 in event_base_dispatch () from /usr/lib/i386-linux-gnu/libevent-2.0.so.5
No symbol table info available.
#11 0x966c8a4c in async_http_run_worker (worker=0x5679c988) at async_http.c:92
No locals.
#12 0x966dabcd in child_init (rank=0) at http_async_client_mod.c:352
---Type <return> to continue, or q <return> to quit---
pid = 0
i = 0
__FUNCTION__ = "child_init"
#13 0x08200daa in init_mod_child (m=0x976a286c, rank=0) at core/sr_module.c:846
__FUNCTION__ = "init_mod_child"
#14 0x082011ad in init_child (rank=0) at core/sr_module.c:874
ret = 23
#15 0x0806b409 in main_loop () at main.c:1737
i = 3
pid = 5318
si = 0x0
si_desc = "udp receiver child=2 sock=192.168.5.67:5060\000]\330g\267\360\243v\267X\037\000\000\300\243v\267\364\217v\267\300\243v\267\b\376R\t`\\\346\277]\330g\267\251\000\000\000\b\376R\t\364\217v\267\025\000\000\000\000]\346\277\367\273m\267\060+Z\t0+Z\t\\\v\232\227\000@\000\000@\200v\267\000\000\000"
nrprocs = 3
woneinit = 1
__FUNCTION__ = "main_loop"
#16 0x08071eb4 in main (argc=13, argv=0xbfe660b4) at main.c:2696
cfg_stream = 0x93f3008
c = -1
r = 0
tmp = 0xbfe6678c ""
tmp_len = -1216949440
port = 2210
proto = 1
options = 0x8376690 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
ret = -1
seed = 2661251522
rfd = 4
debug_save = 0
debug_flag = 0
dont_fork_cnt = 0
n_lst = 0xbfe65fe0
p = 0x805e4a0 "[\201\303p\374?"
st = {st_dev = 14, __pad1 = 0, st_ino = 5047, st_mode = 16877, st_nlink = 2, st_uid = 109,
st_gid = 111, st_rdev = 0, __pad2 = 0, st_size = 60, st_blksize = 4096, st_blocks = 0,
st_atim = {tv_sec = 1561470746, tv_nsec = 145357408}, st_mtim = {tv_sec = 1570111971,
tv_nsec = 819339961}, st_ctim = {tv_sec = 1570111972, tv_nsec = 31337425}, __unused4 = 0,
__unused5 = 0}
__FUNCTION__ = "main"
(gdb) info locals
No symbol table info available.
(gdb) list
1876 int proto;
1877 char *options;
1878 int ret;
1879 unsigned int seed;
1880 int rfd;
1881 int debug_save, debug_flag;
1882 int dont_fork_cnt;
1883 struct name_lst* n_lst;
1884 char *p;
1885 struct stat st = {0};
```
#### Log Messages
<!--
Check the syslog file and if there are relevant log messages printed by Kamailio, add them next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->
```
Oct 3 16:13:37 sip_server /opt/ipgate/kamailio/sbin/kamailio[5331]: CRITICAL: <core> [core/mem/q_malloc.c:149]: qm_debug_check_frag(): BUG: qm: prev. fragm. tail overwritten(c0003032, abcdefed)[0x97a7ff3c:0x97a7ff58]! Memory allocator was called from http_async_client: hm_hash.c:82. Fragment marked by http_async_client: hm_hash.c:71. Exec from core/mem/q_malloc.c:384.
Oct 3 16:13:37 sip_server /opt/ipgate/kamailio/sbin/kamailio[5315]: INFO: <script>: XLOG: 93ffd680-1f94-4f60-b3b0-0fd599a46677 [onsend_route] Request BYE sent to sip:FreeSWITCH at 192.168.5.94:5060;transport=udp;gw=balancer1
Oct 3 16:13:42 sip_server call-control[5004]: Starting factory <callcontrol.rating.RatingEngineFactory instance at 0xb6e8114c>
Oct 3 16:13:42 sip_server call-control[5004]: <twisted.internet.tcp.Connector instance at 0xb6e8116c> will retry in 14 seconds
Oct 3 16:13:42 sip_server call-control[5004]: Stopping factory <callcontrol.rating.RatingEngineFactory instance at 0xb6e8114c>
Oct 3 16:13:44 sip_server /opt/ipgate/kamailio/sbin/kamailio[5312]: ALERT: <core> [main.c:756]: handle_sigs(): child process 5331 exited by a signal 6
Oct 3 16:13:44 sip_server /opt/ipgate/kamailio/sbin/kamailio[5312]: ALERT: <core> [main.c:759]: handle_sigs(): core was generated
Oct 3 16:13:44 sip_server /opt/ipgate/kamailio/sbin/kamailio[5312]: INFO: <core> [main.c:781]: handle_sigs(): terminating due to SIGCHLD
Oct 3 16:13:44 sip_server /opt/ipgate/kamailio/sbin/kamailio[5321]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
...
```
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
```
version: kamailio 5.2.3 (i386/linux) dcce68
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144 MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: dcce68
compiled on 17:08:25 Jun 20 2019 with gcc 4.7.2
```
* **Operating System**:
<!--
Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...;
Kernel details (output of `uname -a`)
-->
```
Debian GNU/Linux 7.1 (wheezy)
Linux munda 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1+deb7u1 i686 GNU/Linux
```
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2091
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-dev/attachments/20191003/2d11b1f5/attachment.html>
More information about the sr-dev
mailing list