[sr-dev] segfault with t_suspend() / t_continue()

Daniel-Constantin Mierla miconda at gmail.com
Sat Apr 6 21:40:22 CEST 2013 

Hello,

you can send the log to me, no problem.

However, I realized I was asking details for the wrong pointer, send the 
output of gdb for following commands:

frame 1
p *(struct qm_frag_end*)(((char*)f) - sizeof(struct qm_frag_end))
p *f

Cheers,
Daniel

On 4/5/13 3:50 PM, Dragos Oancea wrote:
> Hi,
>
> (gdb) p *(struct qm_frag_end*)(((char*)qm) - sizeof(struct qm_frag_end))
> $1 = {size = 0, prev_free = 0x0}
>
> I also have more information (I found meanwhile the kamailio debug log ):
> -  i am using async_sleep() somewhere in the script
> -  it indeed crashes when it receives a reply from a voicemail server 
> (the call should go to voicemail if the second user - callee -  does 
> not REGISTER )
>
>
>
>  0(21290) DEBUG: tm [t_lookup.c:1666]: DEBUG: t_lookup_ident: 
> transaction not found
>  0(21290) ERROR: tm [t_suspend.c:128]: ERROR: t_continue: transaction 
> not found
>  0(21290) ERROR: tmx [tmx_mod.c:567]: resuming the processing of 
> transaction [44781:1223434987] failed
>
> [...]
>
>  0(21290) ERROR: <script>: route[REGISTER]:transaction 
> 15704:1312607975 should continue 0(21290) ERROR: *** cfgtrace: 
> c=[suspend.cfg] l=668 a=27 n=t_continue
>  0(21290) DEBUG: tm [t_lookup.c:1657]: DEBUG: t_lookup_ident: 
> transaction found
>  0(21290) ERROR: *** cfgtrace: c=[suspend.cfg] l=678 a=63 n=assign
>
>
> Daniel, let me know if I may send the log to you in private, it 
> contains alot of private information.
>
> Regards,
> Dragos
>
> ------------------------------------------------------------------------
> *From:* Daniel-Constantin Mierla <miconda at gmail.com>
> *To:* Dragos Oancea <droancea at yahoo.com>; Kamailio (SER) - Development 
> Mailing List <sr-dev at lists.sip-router.org>
> *Sent:* Friday, April 5, 2013 2:28 PM
> *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
>
> Hello,
>
> On 4/4/13 9:26 PM, Dragos Oancea wrote:
>> Hi Daniel,  hi Guys,
>>
>> I did not stress test it, I only had one registered device (which 
>> sent the INVITE) , and another one was supposed to register after 
>> some event triggered by kamailio via a http query (using the utils 
>> module) - that's
>> the reason I parked the transaction with "t_suspend()" .
>> When the REGISTER for the second device comes I send the INVITE to 
>> the it .( lookup() , t_relay() ) . This is a scenario that works for 
>> me. I could say I tested 10 times and 1 time crashed.
>> I plan to stress test these days, let me know if I should get some 
>> information for you while doing it.
> can you spot if there are any error messages in the syslog just before 
> the crash? It can be a buffer overflow done on the previous packet.
>
> If you are doing some tests, maybe is better to re-compile with 
> 'MEMDBG=1', to have details about what functions allocated the chunks.
>
> For now, try to get the following from gdb:
>
> p *(struct qm_frag_end*)(((char*)qm) - sizeof(struct qm_frag_end))
>
> Cheers,
> Daniel
>
>>
>>
>> Regards,
>> Dragos
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Daniel-Constantin Mierla <miconda at gmail.com> 
>> <mailto:miconda at gmail.com>
>> *To:* Dragos Oancea <droancea at yahoo.com> <mailto:droancea at yahoo.com>; 
>> Kamailio (SER) - Development Mailing List 
>> <sr-dev at lists.sip-router.org> <mailto:sr-dev at lists.sip-router.org>
>> *Sent:* Thursday, April 4, 2013 8:53 PM
>> *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
>>
>> Hello,
>>
>> at first sight looks like a missing safety check, but I may be wrong 
>> after a long day. I need to look a bit deeper at it.
>>
>> Did you stress test it or was just casual trying? You said it 
>> happened one, couldn't be reproduced, right?
>>
>> Cheers,
>> Daniel
>>
>> On 4/4/13 8:28 PM, Dragos Oancea wrote:
>>> Hi Daniel,
>>>
>>> Here is the output of `bt full`:
>>> http://pastebin.com/tnrWML3L
>>> Let me know if I can be of any assistance further.
>>>
>>> Regards,
>>> Dragos
>>>
>>> ------------------------------------------------------------------------
>>> *From:* Daniel-Constantin Mierla <miconda at gmail.com> 
>>> <mailto:miconda at gmail.com>
>>> *To:* Dragos Oancea <droancea at yahoo.com> 
>>> <mailto:droancea at yahoo.com>; Kamailio (SER) - Development Mailing 
>>> List <sr-dev at lists.sip-router.org> <mailto:sr-dev at lists.sip-router.org>
>>> *Sent:* Thursday, April 4, 2013 7:28 PM
>>> *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
>>>
>>> Hello,
>>>
>>> could be related, but the issue seems to be while processing a SIP 
>>> reply, not with suspend/continue of a request. Can you send the 
>>> output of 'bt full'?
>>>
>>> Cheers,
>>> Daniel
>>>
>>> On 4/4/13 6:56 PM, Dragos Oancea wrote:
>>>> Hello,
>>>>
>>>>
>>>> I got the following coredump while doing some testing with 
>>>> t_suspend() and t_continue() on the first INVITE:
>>>>
>>>> (gdb) bt
>>>> #0  qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269
>>>> #1  qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386
>>>> #2  0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, 
>>>> ok=<value optimized out>) at t_reply.c:372
>>>> #3  0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, 
>>>> p_msg=<value optimized out>, branch=2, msg_status=200, 
>>>> cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854
>>>> #4  0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at 
>>>> t_reply.c:2370
>>>> #5  0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, 
>>>> mode=<value optimized out>) at forward.c:799
>>>> #6  0x000000000049a5ba in receive_msg (buf=<value optimized out>, 
>>>> len=961, rcv_info=0x7fff63fdfc40) at receive.c:270
>>>> #7  0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544
>>>> #8  0x0000000000464d3a in main_loop () at main.c:1638
>>>> #9  0x0000000000467a6c in main (argc=<value optimized out>, 
>>>> argv=<value optimized out>) at main.c:2566
>>>>
>>>>
>>>>
>>>> I suspend the transaction in one route and then I continue it in 
>>>> another route, similar to below:
>>>>
>>>> route[INVITE]
>>>> # Suspend and put to memcache:
>>>> t_suspend();
>>>>  xlog("proceesing suspended in transaction 
>>>> [$T(id_index):$T(id_label)]\n");
>>>>  $var(transaction_index)=$T(id_index);
>>>>  $var(transaction_label)=$T(id_label);
>>>>  $mct(wait_$tU)= "" + $var(transaction_index) + ":" + 
>>>> $var(transaction_label);
>>>>  $mctex(wait_$tU) = 100;
>>>> }
>>>>
>>>> route[INVITE_resume] {
>>>>
>>>> # get the transaction index and lable from memcache
>>>>  $avp(check) = $mct(wait_$tU) ; # I should have the transaction 
>>>> index and the transaction label here
>>>>  $var(my_transaction_index) =  $(avp(check){s.select,0,:});
>>>>  $var(my_transaction_label) =  $(avp(check){s.select,1,:});
>>>>  $var(a)= $(var(my_transaction_index){s.int});
>>>>  $var(b) = $(var(my_transaction_label){s.int});
>>>>
>>>>  t_continue( "$var(a)",  "$var(b)" , "my_relay_route");
>>>> }
>>>>
>>>> route[INVITE_resume] executes only on an asynchronous event (a 
>>>> certain SIP message that might come or not) .
>>>>
>>>> I use kamailio 4.0.0/Centos Linux. I suspect it happened when I 
>>>> sent CANCEL, but I could not reproduce. I still have the core file.
>>>>
>>>> Regards,
>>>> Dragos
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> sr-dev mailing list
>>>> sr-dev at lists.sip-router.org  <mailto:sr-dev at lists.sip-router.org>
>>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
>>>
>>> -- 
>>> Daniel-Constantin Mierla -http://www.asipto.com  <http://www.asipto.com/>
>>> http://twitter.com/#!/miconda  <http://twitter.com/#%21/miconda>  -http://www.linkedin.com/in/miconda
>>> Kamailio World Conference, April 16-17, 2013, Berlin
>>>   -http://conference.kamailio.com  <http://conference.kamailio.com/>  -
>>>
>>>
>>
>> -- 
>> Daniel-Constantin Mierla -http://www.asipto.com  <http://www.asipto.com/>
>> http://twitter.com/#!/miconda  <http://twitter.com/#%21/miconda>  -http://www.linkedin.com/in/miconda
>> Kamailio World Conference, April 16-17, 2013, Berlin
>>   -http://conference.kamailio.com  <http://conference.kamailio.com/>  -
>>
>>
>
> -- 
> Daniel-Constantin Mierla -http://www.asipto.com  <http://www.asipto.com/>
> http://twitter.com/#!/miconda  <http://twitter.com/#%21/miconda>  -http://www.linkedin.com/in/miconda
> Kamailio World Conference, April 16-17, 2013, Berlin
>   -http://conference.kamailio.com  <http://conference.kamailio.com/>  -
>
>

-- 
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio World Conference, April 16-17, 2013, Berlin
  - http://conference.kamailio.com -

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-dev/attachments/20130406/1a6ad49e/attachment-0001.htm>

More information about the sr-dev mailing list