<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello,</p>
<p>I got the backtrace, I will analyse it and see what I find.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div class="moz-cite-prefix">On 19.06.20 14:09, David VILLAUME
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:PR2P264MB0607368BBAD7675592313AA5FD980@PR2P264MB0607.FRAP264.PROD.OUTLOOK.COM">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"Préformaté HTML Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.PrformatHTMLCar
{mso-style-name:"Préformaté HTML Car";
mso-style-priority:99;
mso-style-link:"Préformaté HTML";
font-family:Consolas;
mso-fareast-language:EN-US;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-US">I just restarted the
service I wait for another occurrence to perform it.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="mso-fareast-language:FR"
lang="EN-US">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:FR"
lang="EN-US">David<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="mso-fareast-language:FR">From:</span></b><span
style="mso-fareast-language:FR"> Daniel-Constantin
Mierla <a class="moz-txt-link-rfc2396E" href="mailto:miconda@gmail.com"><miconda@gmail.com></a>
<br>
<b>Sent:</b> Friday, June 19, 2020 2:07 PM<br>
<b>To:</b> David VILLAUME
<a class="moz-txt-link-rfc2396E" href="mailto:david.villaume@sewan.fr"><david.villaume@sewan.fr></a>; Kamailio (SER) - Users
Mailing List <a class="moz-txt-link-rfc2396E" href="mailto:sr-users@lists.kamailio.org"><sr-users@lists.kamailio.org></a><br>
<b>Subject:</b> Re: [SR-Users] RPC Socket stops
answering after a few time.<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p>Hello,<o:p></o:p></p>
<p>get the output of:<o:p></o:p></p>
<p>kamctl trap<o:p></o:p></p>
<p>You need gdb installed. The output is written a file, with
the path to file printed by the command. Sent that file so we
can investigate and see what each process is doing.<o:p></o:p></p>
<p>Cheers.<br>
Daniel<o:p></o:p></p>
<div>
<p class="MsoNormal">On 19.06.20 13:53, David VILLAUME wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span lang="EN-US">Hello,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">After a new check ,
can’t get answer, on the node for : “kamctl rpc
dlg.stats_active”</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">And I see in logs that
about for dmq replicated calls (dialog [dialog.c:2400]:
rpc_dlg_stats_active(): not active - state: 5) , disappear
after the previous message appears.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">After trying a “kamctl
rpc dlg.stats_active”, I cannot even have answer to
‘kamctl rpc core.uptime’ anymore.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Have you any idea to
go ahead with troubleshooting ?
</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="mso-fareast-language:FR"
lang="EN-US">Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:FR"
lang="EN-US">David</span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="mso-fareast-language:FR">From:</span></b><span
style="mso-fareast-language:FR"> David VILLAUME
<br>
<b>Sent:</b> Thursday, June 18, 2020 3:31 PM<br>
<b>To:</b> <a href="mailto:miconda@gmail.com"
moz-do-not-send="true">miconda@gmail.com</a>;
Kamailio (SER) - Users Mailing List
<a href="mailto:sr-users@lists.kamailio.org"
moz-do-not-send="true"><sr-users@lists.kamailio.org></a><br>
<b>Subject:</b> RE: [SR-Users] RPC Socket stops
answering after a few time.</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Hello Daniel,<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Thanks for your quick
answer.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">In fact I cannot even
enter in kamcmd interactive mode after this happens.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Seems that “ kamctl
rpc core.uptime “ can still do the job.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">I’m using
kamailio_exporter with the following methods :
tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">I haven’t process that
requests longest data on the host, so I’m not sure if it
can be related.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="mso-fareast-language:FR">Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:FR">David</span><o:p></o:p></p>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="mso-fareast-language:FR">From:</span></b><span
style="mso-fareast-language:FR"> Daniel-Constantin
Mierla <<a href="mailto:miconda@gmail.com"
moz-do-not-send="true">miconda@gmail.com</a>>
<br>
<b>Sent:</b> Thursday, June 18, 2020 3:16 PM<br>
<b>To:</b> Kamailio (SER) - Users Mailing List <<a
href="mailto:sr-users@lists.kamailio.org"
moz-do-not-send="true">sr-users@lists.kamailio.org</a>>;
David VILLAUME <<a
href="mailto:david.villaume@sewan.fr"
moz-do-not-send="true">david.villaume@sewan.fr</a>><br>
<b>Subject:</b> Re: [SR-Users] RPC Socket stops
answering after a few time.</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p>Hello,<o:p></o:p></p>
<p>what is the command you run? Maybe the result is too big
and you need to change the params for ctl module.<o:p></o:p></p>
<p>Or is it for all commands, including those printing small
data, such case core.uptime?<o:p></o:p></p>
<p>You can also try the 'kamctl rpc COMMAND', which uses
jsonrpcs, the error you pasted is from ctl module, used by
kamcmd.<o:p></o:p></p>
<p>Cheers,<br>
Daniel<o:p></o:p></p>
<div>
<p class="MsoNormal">On 18.06.20 15:00, David VILLAUME
wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">I’m having problem
with RPC socket that stops answering after
approximatively one or two hours.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Version is 5.2.7</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">My principal host
with exact same version and configuration has no
problem, only the backup node (Keepalived + DMQ)
encounters the problem.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">I could catch this
log about the problem, but didn’t find any reason for
this :</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Jun 18 10:53:33
sbc-kam-partner-b kamailio[83170]: ERROR: ctl
[binrpc_run.c:600]: rpc_send(): ERROR: binrprc:
rpc_send: send failed</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><u><span lang="EN-US">My configuration
about rpc : </span></u><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">loadmodule
"jsonrpcs.so"</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">modparam("ctl",
"binrpc", "unix:/var/run/kamailio/kamailio_ctl")</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">modparam("ctl",
"binrpc_max_body_size", 1024)</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Do you have any idea
to troubleshoot this ?</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">David</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="mso-fareast-language:FR"> </span><o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>Kamailio (SER) - Users Mailing List<o:p></o:p></pre>
<pre><a href="mailto:sr-users@lists.kamailio.org" moz-do-not-send="true">sr-users@lists.kamailio.org</a><o:p></o:p></pre>
<pre><a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a><o:p></o:p></pre>
</blockquote>
<pre>-- <o:p></o:p></pre>
<pre>Daniel-Constantin Mierla -- <a href="http://www.asipto.com" moz-do-not-send="true">www.asipto.com</a><o:p></o:p></pre>
<pre><a href="http://www.twitter.com/miconda" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" moz-do-not-send="true">www.linkedin.com/in/miconda</a><o:p></o:p></pre>
<pre>Funding: <a href="https://www.paypal.me/dcmierla" moz-do-not-send="true">https://www.paypal.me/dcmierla</a><o:p></o:p></pre>
</blockquote>
<pre>-- <o:p></o:p></pre>
<pre>Daniel-Constantin Mierla -- <a href="http://www.asipto.com" moz-do-not-send="true">www.asipto.com</a><o:p></o:p></pre>
<pre><a href="http://www.twitter.com/miconda" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" moz-do-not-send="true">www.linkedin.com/in/miconda</a><o:p></o:p></pre>
<pre>Funding: <a href="https://www.paypal.me/dcmierla" moz-do-not-send="true">https://www.paypal.me/dcmierla</a><o:p></o:p></pre>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla -- <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/miconda">www.twitter.com/miconda</a> -- <a class="moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda">www.linkedin.com/in/miconda</a>
Funding: <a class="moz-txt-link-freetext" href="https://www.paypal.me/dcmierla">https://www.paypal.me/dcmierla</a></pre>
</body>
</html>