[sr-dev] [kamailio/kamailio] Crash on qm_debug_check_frag() (#2503)

9 Oct 2020

### Description

I am experimenting with fuzzing on Kamailio SIP. A malformed INVITE (with a long tag)
crashes the server, raised by qm_debug_check_frag().

### Troubleshooting

The error message:

```
qm_debug_check_frag(): BUG: qm: fragm. 0x7ffff03642e8 (address 0x7ffff0364320) end
overwritten (9191919191919191, 9191919191919191)! Memory allocator was called from tm:
t_reply.c:2410. Fragment marked by tm: t_msgbuilder.c:327. Exec from
core/mem/q_malloc.c:511.
```

Output from GDB:
```
(gdb) watch *0x7ffff0364320
Hardware watchpoint 1: *0x7ffff0364320
(gdb) run -f kamailio-basic.cfg -L src/modules -Y runtime_dir/ -n 1 -D -E
Starting program: /home/rnatella/workdir-sip/kamailio/src/kamailio -f kamailio-basic.cfg
-L src/modules -Y runtime_dir/ -n 1 -D -E
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
 0(29450) INFO: <core> [core/sctp_core.c:75]: sctp_core_check_support(): SCTP API
not enabled - if you want to use it, load sctp module
Listening on
             udp: 127.0.0.1 [127.0.0.1]:5060
Aliases:

WARNING: no fork mode
 0(29450) INFO: rr [./../outbound/api.h:52]: ob_load_api(): unable to import bind_ob -
maybe module is not loaded
 0(29450) INFO: rr [rr_mod.c:185]: mod_init(): outbound module not available
 0(29450) INFO: <core> [main.c:2841]: main(): processes (at least): 4 - shm size:
67108864 - pkg size: 8388608
 0(29450) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer():
SO_RCVBUF is initially 212992
 0(29450) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer():
SO_RCVBUF is finally 425984
 0(29450) ERROR: {1 2 INVITE 1-670(a)127.0.0.1} <core> [core/parser/parse_rr.c:78]:
do_parse_rr_body(): Failed parsing name-addr
(<sip:127.0"0tttttttttttttttttttttttttttK-670-1-7)
 0(29450) ERROR: {1 2 INVITE 1-670(a)127.0.0.1} <core> [core/parser/parse_rr.c:140]:
do_parse_rr_body(): Failed parsing rr header body
[<sip:127.0"0tttttttttttttttttttttttttttK-670-1-7]
 0(29450) ERROR: {1 2 INVITE 1-670(a)127.0.0.1} rr [loose.c:468]: find_rem_target(): failed
to parse last Route HF
 0(29450) ERROR: {1 2 INVITE 1-670(a)127.0.0.1} rr [loose.c:700]: after_strict(): searching
for last Route URI failed

Hardware watchpoint 1: *0x7ffff0364320

Old value = <unreadable>
New value = 4932352
__memmove_sse2_unaligned_erms () at
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:316
316     ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or
directory.
(gdb) bt
#0  __memmove_sse2_unaligned_erms () at
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:316
#1  0x00007ffff602e4ff in build_local_reparse (Trans=<optimized out>,
branch=<optimized out>, len=<optimized out>, method=<optimized out>,
    method_len=<optimized out>, to=<optimized out>, reason=<optimized
out>) at t_msgbuilder.c:336
#2  0x00007ffff607eaff in build_ack (rpl=<optimized out>, trans=<optimized
out>, branch=0, ret_len=<optimized out>) at t_reply.c:360
#3  reply_received (p_msg=<optimized out>) at t_reply.c:2398
#4  0x00000000005a0af3 in do_forward_reply (msg=0x7ffff68eed28, mode=<optimized
out>) at core/forward.c:757
#5  0x0000000000688cae in receive_msg (buf=<optimized out>, len=<optimized
out>, rcv_info=<optimized out>) at core/receive.c:509
#6  0x00000000004a6b39 in udp_rcv_loop () at core/udp_server.c:543
#7  0x000000000042c938 in main_loop () at main.c:1480
#8  0x000000000043c574 in main (argc=<optimized out>, argv=<optimized out>) at
main.c:2863
(gdb) frame \
Quit
(gdb) frame
No symbol "frame" in current context.
(gdb) frame frame 1
No symbol "frame" in current context.
(gdb) frame 1
#1  0x00007ffff602e4ff in build_local_reparse (Trans=<optimized out>,
branch=<optimized out>, len=<optimized out>, method=<optimized out>,
    method_len=<optimized out>, to=<optimized out>, reason=<optimized
out>) at t_msgbuilder.c:336
336             append_str(d, method, method_len);
(gdb) print d
$1 = 0x7ffff0364320 ""
(gdb) c
Continuing.

Hardware watchpoint 1: *0x7ffff0364320

Old value = 4932352
New value = 4932417
__memmove_sse2_unaligned_erms () at
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:317
317     ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or
directory.
(gdb) bt
#0  __memmove_sse2_unaligned_erms () at
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:317
#1  0x00007ffff602e4ff in build_local_reparse (Trans=<optimized out>,
branch=<optimized out>, len=<optimized out>, method=<optimized out>,
    method_len=<optimized out>, to=<optimized out>, reason=<optimized
out>) at t_msgbuilder.c:336
#2  0x00007ffff607eaff in build_ack (rpl=<optimized out>, trans=<optimized
out>, branch=0, ret_len=<optimized out>) at t_reply.c:360
#3  reply_received (p_msg=<optimized out>) at t_reply.c:2398
#4  0x00000000005a0af3 in do_forward_reply (msg=0x7ffff68eed28, mode=<optimized
out>) at core/forward.c:757
#5  0x0000000000688cae in receive_msg (buf=<optimized out>, len=<optimized
out>, rcv_info=<optimized out>) at core/receive.c:509
#6  0x00000000004a6b39 in udp_rcv_loop () at core/udp_server.c:543
#7  0x000000000042c938 in main_loop () at main.c:1480
#8  0x000000000043c574 in main (argc=<optimized out>, argv=<optimized out>) at
main.c:2863
(gdb) c
Continuing.

Hardware watchpoint 1: *0x7ffff0364320

Old value = 4932417
New value = 541803329
0x00007ffff602e504 in build_local_reparse (Trans=<optimized out>,
branch=<optimized out>, len=<optimized out>, method=<optimized out>,
    method_len=<optimized out>, to=<optimized out>, reason=<optimized
out>) at t_msgbuilder.c:337
337             *d = ' ';
(gdb) bt
#0  0x00007ffff602e504 in build_local_reparse (Trans=<optimized out>,
branch=<optimized out>, len=<optimized out>, method=<optimized out>,
    method_len=<optimized out>, to=<optimized out>, reason=<optimized
out>) at t_msgbuilder.c:337
#1  0x00007ffff607eaff in build_ack (rpl=<optimized out>, trans=<optimized
out>, branch=0, ret_len=<optimized out>) at t_reply.c:360
#2  reply_received (p_msg=<optimized out>) at t_reply.c:2398
#3  0x00000000005a0af3 in do_forward_reply (msg=0x7ffff68eed28, mode=<optimized
out>) at core/forward.c:757
#4  0x0000000000688cae in receive_msg (buf=<optimized out>, len=<optimized
out>, rcv_info=<optimized out>) at core/receive.c:509
#5  0x00000000004a6b39 in udp_rcv_loop () at core/udp_server.c:543
#6  0x000000000042c938 in main_loop () at main.c:1480
#7  0x000000000043c574 in main (argc=<optimized out>, argv=<optimized out>) at
main.c:2863
(gdb) c
Continuing.
 0(29450) CRITICAL: {2 2 INVITE 1-670(a)127.0.0.1} <core> [core/mem/q_malloc.c:138]:
qm_debug_check_frag(): BUG: qm: fragm. 0x7ffff03642e8 (address 0x7ffff0364320) end
overwritten (9191919191919191, 9191919191919191)! Memory allocator was called from tm:
t_reply.c:2410. Fragment marked by tm: t_msgbuilder.c:327. Exec from
core/mem/q_malloc.c:511.

Program received signal SIGSEGV, Segmentation fault.
0x000000000082f45f in qm_status (qmp=<optimized out>) at core/mem/q_malloc.c:902
902                                     f!=&(qm->free_hash[h].head);
f=f->u.nxt_free, i++, j++){
```

#### Reproduction

I am running the server with a basic configuration (attached kamailio-basic.cfg), using
the command:

```
./src/kamailio -f kamailio-basic.cfg -L src/modules -Y runtime_dir/ -n 1 -D -E
```

[kamailio-basic.cfg.txt](https://github.com/kamailio/kamailio/files/5354439/…

On the same machine, I am sending the malformed message (also attached):

```
cat sip-crash.txt | nc -4u -w1 localhost 5060
```

[sip-crash.txt](https://github.com/kamailio/kamailio/files/5354468/sip-crash…

You can find more information about my fuzzing setup at:
[](https://github.com/rnatella/aflnet-kamailio-sip)

#### Debugging Data

<!--
If you got a core dump, use gdb to extract troubleshooting data - full backtrace,
local variables and the list of the code at the issue location.

  gdb /path/to/kamailio /path/to/corefile
  bt full
  info locals
  list

If you are familiar with gdb, feel free to attach more of what you consider to
be relevant.
-->

See previous section

#### Log Messages

<!--
Check the syslog file and if there are relevant log messages printed by Kamailio, add them
next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->

See previous section

#### SIP Traffic

<!--
If the issue is exposed by processing specific SIP messages, grab them with ngrep or save
in a pcap file, then add them next, or attach to issue, or provide a link to download them
(e.g., to a pastebin site).
-->

See previos section

### Possible Solutions

<!--
If you found a solution or workaround for the issue, describe it. Ideally, provide a pull
request with a fix.
-->

### Additional Information

  * **Kamailio Version** - output of `kamailio -v`

```
version: kamailio 5.5.0-dev2 (x86_64/linux) 6049a1-dirty
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST,
DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY,
DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER,
USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535,
DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 6049a1 -dirty
compiled on 10:12:13 Oct  9 2020 with /home/rnatella/aflnet/afl-clang-fast 6.0
```

* **Operating System**:

<!--
Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04,
CentOS 7.1, ...), MacOS, xBSD, Solaris, ...;
Kernel details (output of `uname -a`)
-->

```
Ubuntu 18.04.2 LTS

Linux dockertest1 4.15.0-109-generic #110-Ubuntu SMP Tue Jun 23 02:39:32 UTC 2020 x86_64
x86_64 x86_64 GNU/Linux
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2503

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

[sr-dev] [kamailio/kamailio] Crash on qm_debug_check_frag() (#2503)