[sr-dev] Crash in free_sip_msg -> reset_ruid

[Hugh Waite]

On 02/10/2013 19:18, Daniel-Constantin Mierla wrote:
> Hello,
>
> can you give bt full as well as kamailio -v output? Any log error 
> messages?
>
> Also, it would be good to recompile with MEMDBG=1 and watch for errors 
> in the logs to see if there is a buffer overflow.
>
> Cheers,
> Daniel
>
> On 10/2/13 7:19 PM, Hugh Waite wrote:
>> Hi,
>> We've had some more crashes on the current master build.
>> (gdb) bt
>> #0  qm_insert_free (qm=0x7fc1e1b9e010, p=<value optimized out>) at 
>> mem/q_malloc.c:181
>> #1  qm_free (qm=0x7fc1e1b9e010, p=<value optimized out>) at 
>> mem/q_malloc.c:527
>> #2  0x000000000055027f in reset_ruid (msg=0x7fc1e1c35360) at 
>> parser/msg_parser.c:911
>> #3  free_sip_msg (msg=0x7fc1e1c35360) at parser/msg_parser.c:730
>> #4  0x00000000004a4012 in receive_msg (buf=<value optimized out>, 
>> len=<value optimized out>, rcv_info=<value optimized out>) at 
>> receive.c:297
>> #5  0x000000000052a251 in tcp_read_req (con=0x7fc1ca4c6e00, 
>> bytes_read=0x7fff041b327c, read_flags=0x7fff041b3274) at tcp_read.c:1387
>> #6  0x000000000052c53b in handle_io (fm=<value optimized out>, 
>> events=1, idx=-1) at tcp_read.c:1617
>> #7  0x000000000052eb69 in io_wait_loop_epoll (unix_sock=<value 
>> optimized out>) at io_wait.h:1092
>> #8  tcp_receive_loop (unix_sock=<value optimized out>) at 
>> tcp_read.c:1728
>> #9  0x00000000004fc0eb in tcp_init_children () at tcp_main.c:4959
>> #10 0x000000000046c3d5 in main_loop () at main.c:1702
>> #11 0x000000000046dec9 in main (argc=<value optimized out>, 
>> argv=<value optimized out>) at main.c:2533
>>
>> (gdb) frame 2
>> #2  0x000000000055027f in reset_ruid (msg=0x7fc1e1c35360) at 
>> parser/msg_parser.c:911
>> 911                     pkg_free(msg->ruid.s);
>> (gdb) p msg->ruid
>> $7 = {s = 0x845d20 "", len = 20}
>>
>> Might this be related to the changes made on Sept 19th to the 
>> free_sip_msg functions?
>>
>> Regards,
>> Hugh
>>
>
Extra output below.
Nothing was printed in the logs (WARNING or ERROR level) before the 
crash. It seemed to be quite reproduceable when there was traffic being 
sent to registered websocket clients, but there is no-one online now. 
We'll have multiple people logged on tomorrow morning.

Regards,
Hugh

kamailio -v
version: kamailio 4.1.0-dev9 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, 
DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, 
USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, 
USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, 
MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled on 13:35:36 Oct  2 2013 with gcc 4.4.7

(gdb) bt full
#0  qm_insert_free (qm=0x7fc1e1b9e010, p=<value optimized out>) at 
mem/q_malloc.c:181
         f = 0x845d10
         prev = 0x65332d3231653163
         hash = 0
#1  qm_free (qm=0x7fc1e1b9e010, p=<value optimized out>) at 
mem/q_malloc.c:527
         f = 0x845d10
         size = <value optimized out>
         next = <value optimized out>
         prev = <value optimized out>
         __FUNCTION__ = "qm_free"
#2  0x000000000055027f in reset_ruid (msg=0x7fc1e1c35360) at 
parser/msg_parser.c:911
No locals.
#3  free_sip_msg (msg=0x7fc1e1c35360) at parser/msg_parser.c:730
No locals.
#4  0x00000000004a4012 in receive_msg (buf=<value optimized out>, 
len=<value optimized out>, rcv_info=<value optimized out>) at receive.c:297
         msg = 0x7fc1e1c35360
         ctx = {rec_lev = 0, run_flags = 0, last_retcode = 1, jmp_env = 
{{__jmpbuf = {1048575, -3596212518023615478, 140470693039152,
                 140470299422208, 140470299422208, 140733262279292, 
140733262279292, 140733262279284}, __mask_was_saved = -900960744,
               __saved_mask = {__val = {5406222, 532575944923, 
541165879417, 4294967297, 1042, 140470299422952, 5409666, 65535,
                   140733262279284, 140470300510264, 140470697171281, 
1042, 140470300510984, 18446744072809678880, 16, 17179869210}}}}}
         ret = <value optimized out>
         inb = {
           s = 0x23d7cc0 "OPTIONS sip:gavin.llewellyn at crocodiletalk.com 
SIP/2.0\r\nVia: SIP/2.0/TCP 
edge00-int.crocodilertc.net:5080;branch=z9hG4bKab92.bb8249afcf13f20080f25121e49865b8.0\r\nVia: 
SIP/2.0/WSS qvis2mie4gas.invalid;rp"..., len = 1028}
         __FUNCTION__ = "receive_msg"
#5  0x000000000052a251 in tcp_read_req (con=0x7fc1ca4c6e00, 
bytes_read=0x7fff041b327c, read_flags=0x7fff041b3274) at tcp_read.c:1387
         bytes = <value optimized out>
         total_bytes = 1028
         resp = 1
         size = <value optimized out>
         req = 0x7fc1ca4c6e80
         dst = {send_sock = 0x14, to = {s = {sa_family = 1,
               sa_data = "\000\000\001\000\000\000\001 
\000\000x\313\306", <incomplete sequence \341>}, sin = {sin_family = 1, 
sin_port = 0,
               sin_addr = {s_addr = 1}, sin_zero = "\001 
\000\000x\313\306", <incomplete sequence \341>}, sin6 = {sin6_family = 1,
               sin6_port = 0, sin6_flowinfo = 1, sin6_addr = {__in6_u = {
                   __u6_addr8 = "\001 
\000\000x\313\306\341\301\177\000\000\000\000\000", __u6_addr16 = {8193, 
0, 52088, 57798, 32705, 0, 0,
                     0}, __u6_addr32 = {8193, 3787901816, 32705, 0}}}, 
sin6_scope_id = 68891240}}, id = 32767, proto = 8 '\b', send_flags = {
             f = 0 '\000', blst_imask = 0 '\000'}}
         c = 13 '\r'
         ret = <value optimized out>
         __FUNCTION__ = "tcp_read_req"
#6  0x000000000052c53b in handle_io (fm=<value optimized out>, events=1, 
idx=-1) at tcp_read.c:1617
         ret = <value optimized out>
         n = <value optimized out>
         read_flags = 1
         con = 0x7fc1ca4c6e00
         s = <value optimized out>
         resp = <value optimized out>
         t = <value optimized out>
         __FUNCTION__ = "handle_io"
#7  0x000000000052eb69 in io_wait_loop_epoll (unix_sock=<value optimized 
out>) at io_wait.h:1092


-- 
Hugh Waite
Principal Design Engineer
Crocodile RCS Ltd.