<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">Hi,<span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span> </span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">I have been testing one Kamailio v5.1.6 instance with one rtpengine
instance, using sipp playing media files at 40 cps (-r 40) with up to 1600
concurrent calls. During the load tests if rtpengine is pkill'ed/restarted a few times Kamailio would crash. It is quite repeatable and every time the backtrace
from gdb points to the same place as shown below.<br></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>
However the same tests on Kamailio v5.0.7 with the same cfg files and the same rtpengine instance did not cause any crash. <br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><br></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">Here’s what I got from gdb backtrace for v5.1.6 using a dbg build: 2 core dump files:</p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><br></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">1. <span> UDP receiver processes</span>
<span>14483</span>
<br><span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>{{{</span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>[New LWP 14483]<br>Core was generated by `/usr/sbin/kamailio -P /var/run/kamailio/kamailio.pid -f /etc/kamailio/kamailio.'.<br>Program terminated with signal SIGSEGV, Segmentation fault.<br>#0 0x00007fadfa824d8e in t_should_relay_response (Trans=0x7fadf4207730, new_code=200, branch=0, should_store=0x7ffd5038fce4, should_relay=0x7ffd5038fce0, cancel_data=0x7ffd5038fed0, <br> reply=0x7fadfb545210) at t_reply.c:1282<br>1282 t_reply.c: No such file or directory.<br>(gdb) bt<br>#0 0x00007fadfa824d8e in t_should_relay_response (Trans=0x7fadf4207730, new_code=200, branch=0, should_store=0x7ffd5038fce4, should_relay=0x7ffd5038fce0, cancel_data=0x7ffd5038fed0, <br> reply=0x7fadfb545210) at t_reply.c:1282<br>#1 0x00007fadfa829577 in relay_reply (t=0x7fadf4207730, p_msg=0x7fadfb545210, branch=0, msg_status=200, cancel_data=0x7ffd5038fed0, do_put_on_wait=1) at t_reply.c:1786<br>#2 0x00007fadfa82f54c in reply_received (p_msg=0x7fadfb545210) at t_reply.c:2537<br>#3 0x000000000054624b in do_forward_reply (msg=0x7fadfb545210, mode=0) at core/forward.c:747<br>#4 0x0000000000547e4c in forward_reply (msg=0x7fadfb545210) at core/forward.c:852<br>#5 0x000000000058e186 in receive_msg (<br> buf=0xa595a0 <buf> "SIP/2.0 200 OK\r\nVia: SIP/2.0/UDP 192.168.70.102;branch=z9hG4bKa042.afac8eb973f1dfad7a549af0ab1a8ccc.0, SIP/2.0/UDP 192.168.60.80:5060;branch=z9hG4bK-3750-978-0\r\nFrom: sipp <<a href="mailto:sip%3ACustomer68@192.168.60.8">sip:Customer68@192.168.60.8</a>"..., len=888, rcv_info=0x7ffd50390480) at core/receive.c:364<br>#6 0x00000000004af6b1 in udp_rcv_loop () at core/udp_server.c:554<br>#7 0x00000000004246ac in main_loop () at main.c:1619<br>#8 0x000000000042bd5c in main (argc=13, argv=0x7ffd50390b38) at main.c:2638<br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>}}}</span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><br></span></p></div><div dir="ltr"><br></div><div>2. Main process 14468<br></div><div>{{{</div><div>[New LWP 14468]<br>Core was generated by `/usr/sbin/kamailio -P /var/run/kamailio/kamailio.pid -f /etc/kamailio/kamailio.'.<br>Program terminated with signal SIGABRT, Aborted.<br>#0 0x00007fadfbc77428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54<br>54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.<br>(gdb) bt<br>#0 0x00007fadfbc77428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54<br>#1 0x00007fadfbc7902a in __GI_abort () at abort.c:89<br>#2 0x000000000041a029 in sig_alarm_abort (signo=14) at main.c:646<br>#3 <signal handler called><br>#4 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:37<br>#5 0x00007fadf354e67d in futex_get (lock=0x7fadf3e94e50) at ../../core/parser/../mem/../futexlock.h:121<br>#6 0x00007fadf3561113 in mod_destroy () at rtpengine.c:1810<br>#7 0x000000000055132b in destroy_modules () at core/sr_module.c:832<br>#8 0x0000000000418c9f in cleanup (show_status=1) at main.c:521<br>#9 0x000000000041a313 in shutdown_children (sig=15, show_status=1) at main.c:663<br>#10 0x000000000041cfa5 in handle_sigs () at main.c:768<br>#11 0x0000000000425fb5 in main_loop () at main.c:1752<br>#12 0x000000000042bd5c in main (argc=13, argv=0x7ffd50390b38) at main.c:2638<br>}}}<br></div><div><br></div><div dir="ltr"><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>The parameters for rtpengine:</span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>{{{<br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>loadmodule "rtpengine.so"<br>modparam("rtpengine", "db_url", "text:///usr/share/kamailio/dbtext/kamailio")<br>modparam("rtpengine", "hash_table_size", 4)<br>modparam("rtpengine", "setid_default", 1)<br>modparam("rtpengine", "rtpengine_disable_tout", 20)<br>modparam("rtpengine", "rtpengine_retr", 1)<br>modparam("rtpengine", "setid_avp", "$avp(setid)")<br>modparam("rtpengine", "rtp_inst_pvar", "$avp(rtpInstance)")<br>modparam("rtpengine", "rtpengine_tout_ms", 1000)<br>modparam("rtpengine", "read_sdp_pv", "$var(sdpToRtpengine)")<br>modparam("rtpengine", "write_sdp_pv", "$var(sdpFromRtpengine)")<br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>}}}<br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>I'm using a simplified kamailio.cfg from installation, and here are calls to rtpengine:<br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>{{{</span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>... <br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>route[INVITE]<br>{<br> $var(sdpToRtpengine) = $rb;<br> $var(ret) = rtpengine_manage("direction=dirty direction=clean ICE=remove");<br> xlog("L_INFO", "$ci INVITE: rtpengine chosen: $avp(rtpInstance)");<br> remove_body();<br> replace_body(".*", $var(sdpFromRtpengine));<br> t_on_reply("RESPONSE");<br> <br> route(RELAY);<br>}<br><br>onreply_route[RESPONSE]<br>{<br> $var(sdpToRtpengine) = $rb;<br> $var(ret) = rtpengine_manage("direction=clean direction=dirty ICE=remove");<br> remove_body();<br> replace_body(".*", $var(sdpFromRtpengine));<br> xlog("L_INFO", "$ci RESPONSE: $rm - $rs $rr, cseq=$cs, by [$hdr(Server)], from $si:$sp");<br>}<br>...</span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span>}}}<br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span><br></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">When rtpengine is down for a
couple of seconds, there were a lot of SIP retransmissions and timeouts. Doing
a netstat and I can see Kamailio’s receive buffer is quite filled up.<span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span> </span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">Please let me know if more information is needed.<span></span> Thank you!<br></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif"><span> </span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">Cheers,<span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:"Calibri",sans-serif">Yufei<span></span></p>
</div></div></div></div></div>