Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route"); }
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
Hello,
could be related, but the issue seems to be while processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route"); }
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Hi Daniel,
Here is the output of `bt full`: http://pastebin.com/tnrWML3L
Let me know if I can be of any assistance further.
Regards, Dragos
________________________________ From: Daniel-Constantin Mierla miconda@gmail.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org Sent: Thursday, April 4, 2013 7:28 PM Subject: Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
could be related, but the issue seems to be while processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route"); }
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Hey Guys,
yes we can confirm that we also got crashes when trying to use suspend/resume in reply processing.
Cheers Jason
On Thu, Apr 4, 2013 at 8:28 PM, Dragos Oancea droancea@yahoo.com wrote:
Hi Daniel,
Here is the output of `bt full`: http://pastebin.com/tnrWML3L Let me know if I can be of any assistance further.
Regards, Dragos
*From:* Daniel-Constantin Mierla miconda@gmail.com *To:* Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org *Sent:* Thursday, April 4, 2013 7:28 PM *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
could be related, but the issue seems to be while processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label);
$mct(wait_$tU)= "" + $var(transaction_index) + ":" +$var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route");}
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing listsr-dev@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
-- Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
t_suspend()/t_continue() are not designed for sip replies in 4.0.x. You have some patches to extend that in devel version.
Cheers, Daniel
On 4/4/13 8:40 PM, Jason Penton wrote:
Hey Guys,
yes we can confirm that we also got crashes when trying to use suspend/resume in reply processing.
Cheers Jason
On Thu, Apr 4, 2013 at 8:28 PM, Dragos Oancea <droancea@yahoo.com mailto:droancea@yahoo.com> wrote:
Hi Daniel, Here is the output of `bt full`: http://pastebin.com/tnrWML3L Let me know if I can be of any assistance further. Regards, Dragos ------------------------------------------------------------------------ *From:* Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> *To:* Dragos Oancea <droancea@yahoo.com <mailto:droancea@yahoo.com>>; Kamailio (SER) - Development Mailing List <sr-dev@lists.sip-router.org <mailto:sr-dev@lists.sip-router.org>> *Sent:* Thursday, April 4, 2013 7:28 PM *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue() Hello, could be related, but the issue seems to be while processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'? Cheers, Daniel On 4/4/13 6:56 PM, Dragos Oancea wrote:Hello, I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE: (gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566 I suspend the transaction in one route and then I continue it in another route, similar to below: route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; } route[INVITE_resume] { # get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int <http://s.int>}); $var(b) = $(var(my_transaction_label){s.int <http://s.int>}); t_continue( "$var(a)", "$var(b)" , "my_relay_route"); } route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) . I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file. Regards, Dragos _______________________________________________ sr-dev mailing list sr-dev@lists.sip-router.org <mailto:sr-dev@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev-- Daniel-Constantin Mierla -http://www.asipto.com http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> -http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin -http://conference.kamailio.com - _______________________________________________ sr-dev mailing list sr-dev@lists.sip-router.org <mailto:sr-dev@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Hello,
at first sight looks like a missing safety check, but I may be wrong after a long day. I need to look a bit deeper at it.
Did you stress test it or was just casual trying? You said it happened one, couldn't be reproduced, right?
Cheers, Daniel
On 4/4/13 8:28 PM, Dragos Oancea wrote:
Hi Daniel,
Here is the output of `bt full`: http://pastebin.com/tnrWML3L Let me know if I can be of any assistance further.
Regards, Dragos
*From:* Daniel-Constantin Mierla miconda@gmail.com *To:* Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org *Sent:* Thursday, April 4, 2013 7:28 PM *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
could be related, but the issue seems to be while processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route"); }
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing list sr-dev@lists.sip-router.org mailto:sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
-- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin
Hi Daniel, hi Guys,
I did not stress test it, I only had one registered device (which sent the INVITE) , and another one was supposed to register after some event triggered by kamailio via a http query (using the utils module) - that's the reason I parked the transaction with "t_suspend()" . When the REGISTER for the second device comes I send the INVITE to the it .( lookup() , t_relay() ) . This is a scenario that works for me. I could say I tested 10 times and 1 time crashed. I plan to stress test these days, let me know if I should get some information for you while doing it.
Regards, Dragos
________________________________ From: Daniel-Constantin Mierla miconda@gmail.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org Sent: Thursday, April 4, 2013 8:53 PM Subject: Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
at first sight looks like a missing safety check, but I may be wrong after a long day. I need to look a bit deeper at it.
Did you stress test it or was just casual trying? You said it happened one, couldn't be reproduced, right?
Cheers, Daniel
On 4/4/13 8:28 PM, Dragos Oancea wrote:
Hi Daniel,
Here is the output of `bt full`: http://pastebin.com/tnrWML3L
Let me know if I can be of any assistance further.
Regards, Dragos
From: Daniel-Constantin Mierla miconda@gmail.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org Sent: Thursday, April 4, 2013 7:28 PM Subject: Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
could be related, but the issue seems to be while
processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route"); }
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
--
Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin - http://conference.kamailio.com -
Hey Dragos,
okay I just looked at your config and I see you are not using suspend/continue in a reply block so you shouldn't have crash.
Cheers jason
On Thu, Apr 4, 2013 at 9:26 PM, Dragos Oancea droancea@yahoo.com wrote:
Hi Daniel, hi Guys,
I did not stress test it, I only had one registered device (which sent the INVITE) , and another one was supposed to register after some event triggered by kamailio via a http query (using the utils module) - that's the reason I parked the transaction with "t_suspend()" . When the REGISTER for the second device comes I send the INVITE to the it .( lookup() , t_relay() ) . This is a scenario that works for me. I could say I tested 10 times and 1 time crashed. I plan to stress test these days, let me know if I should get some information for you while doing it.
Regards, Dragos
*From:* Daniel-Constantin Mierla miconda@gmail.com *To:* Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org *Sent:* Thursday, April 4, 2013 8:53 PM
*Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
at first sight looks like a missing safety check, but I may be wrong after a long day. I need to look a bit deeper at it.
Did you stress test it or was just casual trying? You said it happened one, couldn't be reproduced, right?
Cheers, Daniel
On 4/4/13 8:28 PM, Dragos Oancea wrote:
Hi Daniel,
Here is the output of `bt full`: http://pastebin.com/tnrWML3L Let me know if I can be of any assistance further.
Regards, Dragos
*From:* Daniel-Constantin Mierla miconda@gmail.com miconda@gmail.com *To:* Dragos Oancea droancea@yahoo.com droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.orgsr-dev@lists.sip-router.org *Sent:* Thursday, April 4, 2013 7:28 PM *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
could be related, but the issue seems to be while processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label);
$mct(wait_$tU)= "" + $var(transaction_index) + ":" +$var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route");}
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing listsr-dev@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
-- Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin
-- Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Hi Jason ,
It could be that it crashed in a CANCEL / 200 OK race condition. I clearly remember sending CANCEL for that call. Maybe the reply that shows in "bt full" is the 200 OK reply that Kamailio generates for CANCEL ? It might also be possible that it crashed after a succesfull t_suspend() , but I am not sure the other side received the INVITE and answered.
Dragos
________________________________ From: Jason Penton jason.penton@gmail.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org Cc: "miconda@gmail.com" miconda@gmail.com Sent: Thursday, April 4, 2013 9:33 PM Subject: Re: [sr-dev] segfault with t_suspend() / t_continue()
Hey Dragos,
okay I just looked at your config and I see you are not using suspend/continue in a reply block so you shouldn't have crash.
Cheers jason
On Thu, Apr 4, 2013 at 9:26 PM, Dragos Oancea droancea@yahoo.com wrote:
Hi Daniel, hi Guys,
I did not stress test it, I only had one registered device (which sent the INVITE) , and another one was supposed to register after some event triggered by kamailio via a http query (using the utils module) - that's the reason I parked the transaction with "t_suspend()" . When the REGISTER for the second device comes I send the INVITE to the it .( lookup() , t_relay() ) . This is a scenario that works for me. I could say I tested 10 times and 1 time crashed. I plan to stress test these days, let me know if I should get some information for you while doing it.
Regards, Dragos
From: Daniel-Constantin Mierla miconda@gmail.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org Sent: Thursday, April 4, 2013 8:53 PM
Subject: Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
at first sight looks like a missing safety check, but I may be wrong
after a long day. I need to look a bit deeper at it.
Did you stress test it or was just casual trying? You said it
happened one, couldn't be reproduced, right?
Cheers, Daniel
On 4/4/13 8:28 PM, Dragos Oancea wrote:
Hi Daniel,
Here is the output of `bt full`: http://pastebin.com/tnrWML3L
Let me know if I can be of any assistance further.
Regards, Dragos
From: Daniel-Constantin Mierla miconda@gmail.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org Sent: Thursday, April 4, 2013 7:28 PM Subject: Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
could be related, but the issue seems to be while
processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route"); }
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
--
Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin - http://conference.kamailio.com -
--
Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin - http://conference.kamailio.com -
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Hello,
On 4/4/13 9:26 PM, Dragos Oancea wrote:
Hi Daniel, hi Guys,
I did not stress test it, I only had one registered device (which sent the INVITE) , and another one was supposed to register after some event triggered by kamailio via a http query (using the utils module)
- that's
the reason I parked the transaction with "t_suspend()" . When the REGISTER for the second device comes I send the INVITE to the it .( lookup() , t_relay() ) . This is a scenario that works for me. I could say I tested 10 times and 1 time crashed. I plan to stress test these days, let me know if I should get some information for you while doing it.
can you spot if there are any error messages in the syslog just before the crash? It can be a buffer overflow done on the previous packet.
If you are doing some tests, maybe is better to re-compile with 'MEMDBG=1', to have details about what functions allocated the chunks.
For now, try to get the following from gdb:
p *(struct qm_frag_end*)(((char*)qm) - sizeof(struct qm_frag_end))
Cheers, Daniel
Regards, Dragos
*From:* Daniel-Constantin Mierla miconda@gmail.com *To:* Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org *Sent:* Thursday, April 4, 2013 8:53 PM *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
at first sight looks like a missing safety check, but I may be wrong after a long day. I need to look a bit deeper at it.
Did you stress test it or was just casual trying? You said it happened one, couldn't be reproduced, right?
Cheers, Daniel
On 4/4/13 8:28 PM, Dragos Oancea wrote:
Hi Daniel,
Here is the output of `bt full`: http://pastebin.com/tnrWML3L Let me know if I can be of any assistance further.
Regards, Dragos
*From:* Daniel-Constantin Mierla miconda@gmail.com mailto:miconda@gmail.com *To:* Dragos Oancea droancea@yahoo.com mailto:droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org mailto:sr-dev@lists.sip-router.org *Sent:* Thursday, April 4, 2013 7:28 PM *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
could be related, but the issue seems to be while processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route"); }
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing list sr-dev@lists.sip-router.org mailto:sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
-- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin
-- Daniel-Constantin Mierla -http://www.asipto.com http://www.asipto.com/ http://twitter.com/#!/miconda http://twitter.com/#%21/miconda -http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin -http://conference.kamailio.com http://conference.kamailio.com/ -
Hi,
(gdb) p *(struct qm_frag_end*)(((char*)qm) - sizeof(struct qm_frag_end)) $1 = {size = 0, prev_free = 0x0}
I also have more information (I found meanwhile the kamailio debug log ): - i am using async_sleep() somewhere in the script - it indeed crashes when it receives a reply from a voicemail server (the call should go to voicemail if the second user - callee - does not REGISTER )
0(21290) DEBUG: tm [t_lookup.c:1666]: DEBUG: t_lookup_ident: transaction not found 0(21290) ERROR: tm [t_suspend.c:128]: ERROR: t_continue: transaction not found 0(21290) ERROR: tmx [tmx_mod.c:567]: resuming the processing of transaction [44781:1223434987] failed
[...]
0(21290) ERROR: <script>: route[REGISTER]:transaction 15704:1312607975 should continue 0(21290) ERROR: *** cfgtrace: c=[suspend.cfg] l=668 a=27 n=t_continue 0(21290) DEBUG: tm [t_lookup.c:1657]: DEBUG: t_lookup_ident: transaction found 0(21290) ERROR: *** cfgtrace: c=[suspend.cfg] l=678 a=63 n=assign
Daniel, let me know if I may send the log to you in private, it contains alot of private information.
Regards, Dragos
________________________________ From: Daniel-Constantin Mierla miconda@gmail.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org Sent: Friday, April 5, 2013 2:28 PM Subject: Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
On 4/4/13 9:26 PM, Dragos Oancea wrote:
Hi Daniel, hi Guys,
I did not stress test it, I only had one registered device (which sent the INVITE) , and another one was supposed to register after some event triggered by kamailio via a http query (using the utils module) - that's the reason I parked the transaction with "t_suspend()" . When the REGISTER for the second device comes I send the INVITE to the it .( lookup() , t_relay() ) . This is a scenario that works for me. I could say I tested 10 times and 1 time crashed. I plan to stress test these days, let me know if I should get some information for you while doing it.
can you spot if there are any error messages in the syslog just before the crash? It can be a buffer overflow done on the previous packet.
If you are doing some tests, maybe is better to re-compile with 'MEMDBG=1', to have details about what functions allocated the chunks.
For now, try to get the following from gdb:
p *(struct qm_frag_end*)(((char*)qm) - sizeof(struct qm_frag_end))
Cheers, Daniel
Regards, Dragos
From: Daniel-Constantin Mierla miconda@gmail.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org Sent: Thursday, April 4, 2013 8:53 PM Subject: Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
at first sight looks like a missing safety check, but I
may be wrong after a long day. I need to look a bit deeper at it.
Did you stress test it or was just casual trying? You
said it happened one, couldn't be reproduced, right?
Cheers, Daniel
On 4/4/13 8:28 PM, Dragos Oancea wrote:
Hi Daniel,
Here is the output of `bt full`: http://pastebin.com/tnrWML3L
Let me know if I can be of any assistance further.
Regards, Dragos
From: Daniel-Constantin Mierla miconda@gmail.com To: Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org Sent: Thursday, April 4, 2013 7:28 PM Subject: Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
could be related, but the issue seems to be
while processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route"); }
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
--
Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin - http://conference.kamailio.com -
--
Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin - http://conference.kamailio.com -
Hello,
you can send the log to me, no problem.
However, I realized I was asking details for the wrong pointer, send the output of gdb for following commands:
frame 1 p *(struct qm_frag_end*)(((char*)f) - sizeof(struct qm_frag_end)) p *f
Cheers, Daniel
On 4/5/13 3:50 PM, Dragos Oancea wrote:
Hi,
(gdb) p *(struct qm_frag_end*)(((char*)qm) - sizeof(struct qm_frag_end)) $1 = {size = 0, prev_free = 0x0}
I also have more information (I found meanwhile the kamailio debug log ):
- i am using async_sleep() somewhere in the script
- it indeed crashes when it receives a reply from a voicemail server
(the call should go to voicemail if the second user - callee - does not REGISTER )
0(21290) DEBUG: tm [t_lookup.c:1666]: DEBUG: t_lookup_ident: transaction not found 0(21290) ERROR: tm [t_suspend.c:128]: ERROR: t_continue: transaction not found 0(21290) ERROR: tmx [tmx_mod.c:567]: resuming the processing of transaction [44781:1223434987] failed
[...]
0(21290) ERROR: <script>: route[REGISTER]:transaction 15704:1312607975 should continue 0(21290) ERROR: *** cfgtrace: c=[suspend.cfg] l=668 a=27 n=t_continue 0(21290) DEBUG: tm [t_lookup.c:1657]: DEBUG: t_lookup_ident: transaction found 0(21290) ERROR: *** cfgtrace: c=[suspend.cfg] l=678 a=63 n=assign
Daniel, let me know if I may send the log to you in private, it contains alot of private information.
Regards, Dragos
*From:* Daniel-Constantin Mierla miconda@gmail.com *To:* Dragos Oancea droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org *Sent:* Friday, April 5, 2013 2:28 PM *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
On 4/4/13 9:26 PM, Dragos Oancea wrote:
Hi Daniel, hi Guys,
I did not stress test it, I only had one registered device (which sent the INVITE) , and another one was supposed to register after some event triggered by kamailio via a http query (using the utils module) - that's the reason I parked the transaction with "t_suspend()" . When the REGISTER for the second device comes I send the INVITE to the it .( lookup() , t_relay() ) . This is a scenario that works for me. I could say I tested 10 times and 1 time crashed. I plan to stress test these days, let me know if I should get some information for you while doing it.
can you spot if there are any error messages in the syslog just before the crash? It can be a buffer overflow done on the previous packet.
If you are doing some tests, maybe is better to re-compile with 'MEMDBG=1', to have details about what functions allocated the chunks.
For now, try to get the following from gdb:
p *(struct qm_frag_end*)(((char*)qm) - sizeof(struct qm_frag_end))
Cheers, Daniel
Regards, Dragos
*From:* Daniel-Constantin Mierla miconda@gmail.com mailto:miconda@gmail.com *To:* Dragos Oancea droancea@yahoo.com mailto:droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org mailto:sr-dev@lists.sip-router.org *Sent:* Thursday, April 4, 2013 8:53 PM *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
at first sight looks like a missing safety check, but I may be wrong after a long day. I need to look a bit deeper at it.
Did you stress test it or was just casual trying? You said it happened one, couldn't be reproduced, right?
Cheers, Daniel
On 4/4/13 8:28 PM, Dragos Oancea wrote:
Hi Daniel,
Here is the output of `bt full`: http://pastebin.com/tnrWML3L Let me know if I can be of any assistance further.
Regards, Dragos
*From:* Daniel-Constantin Mierla miconda@gmail.com mailto:miconda@gmail.com *To:* Dragos Oancea droancea@yahoo.com mailto:droancea@yahoo.com; Kamailio (SER) - Development Mailing List sr-dev@lists.sip-router.org mailto:sr-dev@lists.sip-router.org *Sent:* Thursday, April 4, 2013 7:28 PM *Subject:* Re: [sr-dev] segfault with t_suspend() / t_continue()
Hello,
could be related, but the issue seems to be while processing a SIP reply, not with suspend/continue of a request. Can you send the output of 'bt full'?
Cheers, Daniel
On 4/4/13 6:56 PM, Dragos Oancea wrote:
Hello,
I got the following coredump while doing some testing with t_suspend() and t_continue() on the first INVITE:
(gdb) bt #0 qm_detach_free (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:269 #1 qm_malloc (qm=0x7fe2dae65000, size=16) at mem/q_malloc.c:386 #2 0x00007fe2e5e6d541 in update_totag_set (t=0x7fe2db0f42d0, ok=<value optimized out>) at t_reply.c:372 #3 0x00007fe2e5e72387 in relay_reply (t=<value optimized out>, p_msg=<value optimized out>, branch=2, msg_status=200, cancel_data=0x7fff63fdf9b0, do_put_on_wait=1) at t_reply.c:1854 #4 0x00007fe2e5e733e6 in reply_received (p_msg=0x7fe2e6a55ba0) at t_reply.c:2370 #5 0x00000000004547f5 in do_forward_reply (msg=0x7fe2e6a55ba0, mode=<value optimized out>) at forward.c:799 #6 0x000000000049a5ba in receive_msg (buf=<value optimized out>, len=961, rcv_info=0x7fff63fdfc40) at receive.c:270 #7 0x00000000005281a6 in udp_rcv_loop () at udp_server.c:544 #8 0x0000000000464d3a in main_loop () at main.c:1638 #9 0x0000000000467a6c in main (argc=<value optimized out>, argv=<value optimized out>) at main.c:2566
I suspend the transaction in one route and then I continue it in another route, similar to below:
route[INVITE] # Suspend and put to memcache: t_suspend(); xlog("proceesing suspended in transaction [$T(id_index):$T(id_label)]\n"); $var(transaction_index)=$T(id_index); $var(transaction_label)=$T(id_label); $mct(wait_$tU)= "" + $var(transaction_index) + ":" + $var(transaction_label); $mctex(wait_$tU) = 100; }
route[INVITE_resume] {
# get the transaction index and lable from memcache $avp(check) = $mct(wait_$tU) ; # I should have the transaction index and the transaction label here $var(my_transaction_index) = $(avp(check){s.select,0,:}); $var(my_transaction_label) = $(avp(check){s.select,1,:}); $var(a)= $(var(my_transaction_index){s.int}); $var(b) = $(var(my_transaction_label){s.int});
t_continue( "$var(a)", "$var(b)" , "my_relay_route"); }
route[INVITE_resume] executes only on an asynchronous event (a certain SIP message that might come or not) .
I use kamailio 4.0.0/Centos Linux. I suspect it happened when I sent CANCEL, but I could not reproduce. I still have the core file.
Regards, Dragos
sr-dev mailing list sr-dev@lists.sip-router.org mailto:sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
-- Daniel-Constantin Mierla -http://www.asipto.com http://www.asipto.com/ http://twitter.com/#!/miconda http://twitter.com/#%21/miconda -http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin -http://conference.kamailio.com http://conference.kamailio.com/ -
-- Daniel-Constantin Mierla -http://www.asipto.com http://www.asipto.com/ http://twitter.com/#!/miconda http://twitter.com/#%21/miconda -http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin -http://conference.kamailio.com http://conference.kamailio.com/ -
-- Daniel-Constantin Mierla -http://www.asipto.com http://www.asipto.com/ http://twitter.com/#!/miconda http://twitter.com/#%21/miconda -http://www.linkedin.com/in/miconda Kamailio World Conference, April 16-17, 2013, Berlin -http://conference.kamailio.com http://conference.kamailio.com/ -
-
Daniel-Constantin Mierla -
Dragos Oancea -
Jason Penton