Hello,
I’m having problem with RPC socket that stops answering after approximatively one or two hours.
Version is 5.2.7
My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem.
I could catch this log about the problem, but didn’t find any reason for this :
Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed
My configuration about rpc :
loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024)
Do you have any idea to troubleshoot this ?
Regards, David
Hello,
what is the command you run? Maybe the result is too big and you need to change the params for ctl module.
Or is it for all commands, including those printing small data, such case core.uptime?
You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd.
Cheers, Daniel
On 18.06.20 15:00, David VILLAUME wrote:
Hello,
I’m having problem with RPC socket that stops answering after approximatively one or two hours.
Version is 5.2.7
My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem.
I could catch this log about the problem, but didn’t find any reason for this :
Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed
_My configuration about rpc : _
loadmodule "jsonrpcs.so"
modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl")
modparam("ctl", "binrpc_max_body_size", 1024)
Do you have any idea to troubleshoot this ?
Regards,
David
Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hello Daniel,
Thanks for your quick answer.
In fact I cannot even enter in kamcmd interactive mode after this happens.
Seems that “ kamctl rpc core.uptime “ can still do the job.
I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active.
I haven’t process that requests longest data on the host, so I’m not sure if it can be related.
Regards, David
From: Daniel-Constantin Mierla miconda@gmail.com Sent: Thursday, June 18, 2020 3:16 PM To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org; David VILLAUME david.villaume@sewan.fr Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
what is the command you run? Maybe the result is too big and you need to change the params for ctl module.
Or is it for all commands, including those printing small data, such case core.uptime?
You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd.
Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello,
I’m having problem with RPC socket that stops answering after approximatively one or two hours.
Version is 5.2.7
My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem.
I could catch this log about the problem, but didn’t find any reason for this :
Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed
My configuration about rpc :
loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024)
Do you have any idea to troubleshoot this ?
Regards, David
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
Hello,
After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active”
And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears.
After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore.
Have you any idea to go ahead with troubleshooting ?
Regards, David
From: David VILLAUME Sent: Thursday, June 18, 2020 3:31 PM To: miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hello Daniel,
Thanks for your quick answer.
In fact I cannot even enter in kamcmd interactive mode after this happens.
Seems that “ kamctl rpc core.uptime “ can still do the job.
I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active.
I haven’t process that requests longest data on the host, so I’m not sure if it can be related.
Regards, David
From: Daniel-Constantin Mierla <miconda@gmail.commailto:miconda@gmail.com> Sent: Thursday, June 18, 2020 3:16 PM To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org>; David VILLAUME <david.villaume@sewan.frmailto:david.villaume@sewan.fr> Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
what is the command you run? Maybe the result is too big and you need to change the params for ctl module.
Or is it for all commands, including those printing small data, such case core.uptime?
You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd.
Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello,
I’m having problem with RPC socket that stops answering after approximatively one or two hours.
Version is 5.2.7
My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem.
I could catch this log about the problem, but didn’t find any reason for this :
Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed
My configuration about rpc :
loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024)
Do you have any idea to troubleshoot this ?
Regards, David
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
Hello,
get the output of:
kamctl trap
You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing.
Cheers. Daniel
On 19.06.20 13:53, David VILLAUME wrote:
Hello,
After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active”
And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears.
After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore.
Have you any idea to go ahead with troubleshooting ?
Regards,
David
*From:*David VILLAUME *Sent:* Thursday, June 18, 2020 3:31 PM *To:* miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* RE: [SR-Users] RPC Socket stops answering after a few time.
Hello Daniel,
Thanks for your quick answer.
In fact I cannot even enter in kamcmd interactive mode after this happens.
Seems that “ kamctl rpc core.uptime “ can still do the job.
I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active.
I haven’t process that requests longest data on the host, so I’m not sure if it can be related.
Regards,
David
*From:*Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> *Sent:* Thursday, June 18, 2020 3:16 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org mailto:sr-users@lists.kamailio.org>; David VILLAUME <david.villaume@sewan.fr mailto:david.villaume@sewan.fr> *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
what is the command you run? Maybe the result is too big and you need to change the params for ctl module.
Or is it for all commands, including those printing small data, such case core.uptime?
You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd.
Cheers, Daniel
On 18.06.20 15:00, David VILLAUME wrote:
Hello, I’m having problem with RPC socket that stops answering after approximatively one or two hours. Version is 5.2.7 My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem. I could catch this log about the problem, but didn’t find any reason for this : Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed _My configuration about rpc : _ loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024) Do you have any idea to troubleshoot this ? Regards, David _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- www.asipto.com http://www.asipto.com www.twitter.com/miconda http://www.twitter.com/miconda -- www.linkedin.com/in/miconda http://www.linkedin.com/in/miconda Funding: https://www.paypal.me/dcmierla
Hello,
I just restarted the service I wait for another occurrence to perform it.
Regards, David
From: Daniel-Constantin Mierla miconda@gmail.com Sent: Friday, June 19, 2020 2:07 PM To: David VILLAUME david.villaume@sewan.fr; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
get the output of:
kamctl trap
You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing.
Cheers. Daniel On 19.06.20 13:53, David VILLAUME wrote: Hello,
After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active”
And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears.
After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore.
Have you any idea to go ahead with troubleshooting ?
Regards, David
From: David VILLAUME Sent: Thursday, June 18, 2020 3:31 PM To: miconda@gmail.commailto:miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hello Daniel,
Thanks for your quick answer.
In fact I cannot even enter in kamcmd interactive mode after this happens.
Seems that “ kamctl rpc core.uptime “ can still do the job.
I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active.
I haven’t process that requests longest data on the host, so I’m not sure if it can be related.
Regards, David
From: Daniel-Constantin Mierla <miconda@gmail.commailto:miconda@gmail.com> Sent: Thursday, June 18, 2020 3:16 PM To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org>; David VILLAUME <david.villaume@sewan.frmailto:david.villaume@sewan.fr> Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
what is the command you run? Maybe the result is too big and you need to change the params for ctl module.
Or is it for all commands, including those printing small data, such case core.uptime?
You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd.
Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello,
I’m having problem with RPC socket that stops answering after approximatively one or two hours.
Version is 5.2.7
My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem.
I could catch this log about the problem, but didn’t find any reason for this :
Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed
My configuration about rpc :
loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024)
Do you have any idea to troubleshoot this ?
Regards, David
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
Hello,
I got the backtrace, I will analyse it and see what I find.
Cheers, Daniel
On 19.06.20 14:09, David VILLAUME wrote:
Hello,
I just restarted the service I wait for another occurrence to perform it.
Regards,
David
*From:*Daniel-Constantin Mierla miconda@gmail.com *Sent:* Friday, June 19, 2020 2:07 PM *To:* David VILLAUME david.villaume@sewan.fr; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
get the output of:
kamctl trap
You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing.
Cheers. Daniel
On 19.06.20 13:53, David VILLAUME wrote:
Hello, After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active” And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears. After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore. Have you any idea to go ahead with troubleshooting ? Regards, David *From:*David VILLAUME *Sent:* Thursday, June 18, 2020 3:31 PM *To:* miconda@gmail.com <mailto:miconda@gmail.com>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> <mailto:sr-users@lists.kamailio.org> *Subject:* RE: [SR-Users] RPC Socket stops answering after a few time. Hello Daniel, Thanks for your quick answer. In fact I cannot even enter in kamcmd interactive mode after this happens. Seems that “ kamctl rpc core.uptime “ can still do the job. I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active. I haven’t process that requests longest data on the host, so I’m not sure if it can be related. Regards, David *From:*Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> *Sent:* Thursday, June 18, 2020 3:16 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org>>; David VILLAUME <david.villaume@sewan.fr <mailto:david.villaume@sewan.fr>> *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time. Hello, what is the command you run? Maybe the result is too big and you need to change the params for ctl module. Or is it for all commands, including those printing small data, such case core.uptime? You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd. Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello, I’m having problem with RPC socket that stops answering after approximatively one or two hours. Version is 5.2.7 My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem. I could catch this log about the problem, but didn’t find any reason for this : Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed _My configuration about rpc : _ loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024) Do you have any idea to troubleshoot this ? Regards, David _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Funding: https://www.paypal.me/dcmierla
-- Daniel-Constantin Mierla -- www.asipto.com http://www.asipto.com www.twitter.com/miconda http://www.twitter.com/miconda -- www.linkedin.com/in/miconda http://www.linkedin.com/in/miconda Funding: https://www.paypal.me/dcmierla
Hello,
can you try with the patch from the commit:
- https://github.com/kamailio/kamailio/commit/f4036c535033a333cb551b5af7070313...
If all works ok, then I will backport to stable branches.
Cheers, Daniel
On 22.06.20 17:53, Daniel-Constantin Mierla wrote:
Hello,
I got the backtrace, I will analyse it and see what I find.
Cheers, Daniel
On 19.06.20 14:09, David VILLAUME wrote:
Hello,
I just restarted the service I wait for another occurrence to perform it.
Regards,
David
*From:*Daniel-Constantin Mierla miconda@gmail.com *Sent:* Friday, June 19, 2020 2:07 PM *To:* David VILLAUME david.villaume@sewan.fr; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
get the output of:
kamctl trap
You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing.
Cheers. Daniel
On 19.06.20 13:53, David VILLAUME wrote:
Hello, After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active” And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears. After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore. Have you any idea to go ahead with troubleshooting ? Regards, David *From:*David VILLAUME *Sent:* Thursday, June 18, 2020 3:31 PM *To:* miconda@gmail.com <mailto:miconda@gmail.com>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> <mailto:sr-users@lists.kamailio.org> *Subject:* RE: [SR-Users] RPC Socket stops answering after a few time. Hello Daniel, Thanks for your quick answer. In fact I cannot even enter in kamcmd interactive mode after this happens. Seems that “ kamctl rpc core.uptime “ can still do the job. I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active. I haven’t process that requests longest data on the host, so I’m not sure if it can be related. Regards, David *From:*Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> *Sent:* Thursday, June 18, 2020 3:16 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org>>; David VILLAUME <david.villaume@sewan.fr <mailto:david.villaume@sewan.fr>> *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time. Hello, what is the command you run? Maybe the result is too big and you need to change the params for ctl module. Or is it for all commands, including those printing small data, such case core.uptime? You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd. Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello, I’m having problem with RPC socket that stops answering after approximatively one or two hours. Version is 5.2.7 My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem. I could catch this log about the problem, but didn’t find any reason for this : Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed _My configuration about rpc : _ loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024) Do you have any idea to troubleshoot this ? Regards, David _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Funding: https://www.paypal.me/dcmierla
-- Daniel-Constantin Mierla -- www.asipto.com http://www.asipto.com www.twitter.com/miconda http://www.twitter.com/miconda -- www.linkedin.com/in/miconda http://www.linkedin.com/in/miconda Funding: https://www.paypal.me/dcmierla
-- Daniel-Constantin Mierla -- www.asipto.com www.twitter.com/miconda -- www.linkedin.com/in/miconda Funding: https://www.paypal.me/dcmierla
Hi,
Thanks a lot, the patch is in test, I‘ll be able to confirm you tomorrow.
Regards, David
From: Daniel-Constantin Mierla miconda@gmail.com Sent: Tuesday, June 23, 2020 8:15 AM To: David VILLAUME david.villaume@sewan.fr; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
can you try with the patch from the commit:
- https://github.com/kamailio/kamailio/commit/f4036c535033a333cb551b5af7070313...
If all works ok, then I will backport to stable branches.
Cheers, Daniel On 22.06.20 17:53, Daniel-Constantin Mierla wrote:
Hello,
I got the backtrace, I will analyse it and see what I find.
Cheers, Daniel On 19.06.20 14:09, David VILLAUME wrote: Hello,
I just restarted the service I wait for another occurrence to perform it.
Regards, David
From: Daniel-Constantin Mierla miconda@gmail.commailto:miconda@gmail.com Sent: Friday, June 19, 2020 2:07 PM To: David VILLAUME david.villaume@sewan.frmailto:david.villaume@sewan.fr; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
get the output of:
kamctl trap
You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing.
Cheers. Daniel On 19.06.20 13:53, David VILLAUME wrote: Hello,
After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active”
And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears.
After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore.
Have you any idea to go ahead with troubleshooting ?
Regards, David
From: David VILLAUME Sent: Thursday, June 18, 2020 3:31 PM To: miconda@gmail.commailto:miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hello Daniel,
Thanks for your quick answer.
In fact I cannot even enter in kamcmd interactive mode after this happens.
Seems that “ kamctl rpc core.uptime “ can still do the job.
I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active.
I haven’t process that requests longest data on the host, so I’m not sure if it can be related.
Regards, David
From: Daniel-Constantin Mierla <miconda@gmail.commailto:miconda@gmail.com> Sent: Thursday, June 18, 2020 3:16 PM To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org>; David VILLAUME <david.villaume@sewan.frmailto:david.villaume@sewan.fr> Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
what is the command you run? Maybe the result is too big and you need to change the params for ctl module.
Or is it for all commands, including those printing small data, such case core.uptime?
You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd.
Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello,
I’m having problem with RPC socket that stops answering after approximatively one or two hours.
Version is 5.2.7
My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem.
I could catch this log about the problem, but didn’t find any reason for this :
Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed
My configuration about rpc :
loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024)
Do you have any idea to troubleshoot this ?
Regards, David
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
Hi Daniel,
The patched worked correctly.
Big thanks.
Regards, David
From: David VILLAUME Sent: Sunday, June 28, 2020 12:16 PM To: miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hi,
Thanks a lot, the patch is in test, I‘ll be able to confirm you tomorrow.
Regards, David
From: Daniel-Constantin Mierla <miconda@gmail.commailto:miconda@gmail.com> Sent: Tuesday, June 23, 2020 8:15 AM To: David VILLAUME <david.villaume@sewan.frmailto:david.villaume@sewan.fr>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org> Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
can you try with the patch from the commit:
- https://github.com/kamailio/kamailio/commit/f4036c535033a333cb551b5af7070313...
If all works ok, then I will backport to stable branches.
Cheers, Daniel On 22.06.20 17:53, Daniel-Constantin Mierla wrote:
Hello,
I got the backtrace, I will analyse it and see what I find.
Cheers, Daniel On 19.06.20 14:09, David VILLAUME wrote: Hello,
I just restarted the service I wait for another occurrence to perform it.
Regards, David
From: Daniel-Constantin Mierla miconda@gmail.commailto:miconda@gmail.com Sent: Friday, June 19, 2020 2:07 PM To: David VILLAUME david.villaume@sewan.frmailto:david.villaume@sewan.fr; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
get the output of:
kamctl trap
You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing.
Cheers. Daniel On 19.06.20 13:53, David VILLAUME wrote: Hello,
After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active”
And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears.
After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore.
Have you any idea to go ahead with troubleshooting ?
Regards, David
From: David VILLAUME Sent: Thursday, June 18, 2020 3:31 PM To: miconda@gmail.commailto:miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hello Daniel,
Thanks for your quick answer.
In fact I cannot even enter in kamcmd interactive mode after this happens.
Seems that “ kamctl rpc core.uptime “ can still do the job.
I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active.
I haven’t process that requests longest data on the host, so I’m not sure if it can be related.
Regards, David
From: Daniel-Constantin Mierla <miconda@gmail.commailto:miconda@gmail.com> Sent: Thursday, June 18, 2020 3:16 PM To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org>; David VILLAUME <david.villaume@sewan.frmailto:david.villaume@sewan.fr> Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
what is the command you run? Maybe the result is too big and you need to change the params for ctl module.
Or is it for all commands, including those printing small data, such case core.uptime?
You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd.
Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello,
I’m having problem with RPC socket that stops answering after approximatively one or two hours.
Version is 5.2.7
My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem.
I could catch this log about the problem, but didn’t find any reason for this :
Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed
My configuration about rpc :
loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024)
Do you have any idea to troubleshoot this ?
Regards, David
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
Hello Daniel,
Do you have any idea about when the pâcth could be backported in stable ?
Regards, David
From: David VILLAUME Sent: Monday, June 29, 2020 11:21 PM To: miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hi Daniel,
The patched worked correctly.
Big thanks.
Regards, David
From: David VILLAUME Sent: Sunday, June 28, 2020 12:16 PM To: miconda@gmail.commailto:miconda@gmail.com; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org> Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hi,
Thanks a lot, the patch is in test, I‘ll be able to confirm you tomorrow.
Regards, David
From: Daniel-Constantin Mierla <miconda@gmail.commailto:miconda@gmail.com> Sent: Tuesday, June 23, 2020 8:15 AM To: David VILLAUME <david.villaume@sewan.frmailto:david.villaume@sewan.fr>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org> Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
can you try with the patch from the commit:
- https://github.com/kamailio/kamailio/commit/f4036c535033a333cb551b5af7070313...
If all works ok, then I will backport to stable branches.
Cheers, Daniel On 22.06.20 17:53, Daniel-Constantin Mierla wrote:
Hello,
I got the backtrace, I will analyse it and see what I find.
Cheers, Daniel On 19.06.20 14:09, David VILLAUME wrote: Hello,
I just restarted the service I wait for another occurrence to perform it.
Regards, David
From: Daniel-Constantin Mierla miconda@gmail.commailto:miconda@gmail.com Sent: Friday, June 19, 2020 2:07 PM To: David VILLAUME david.villaume@sewan.frmailto:david.villaume@sewan.fr; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
get the output of:
kamctl trap
You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing.
Cheers. Daniel On 19.06.20 13:53, David VILLAUME wrote: Hello,
After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active”
And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears.
After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore.
Have you any idea to go ahead with troubleshooting ?
Regards, David
From: David VILLAUME Sent: Thursday, June 18, 2020 3:31 PM To: miconda@gmail.commailto:miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hello Daniel,
Thanks for your quick answer.
In fact I cannot even enter in kamcmd interactive mode after this happens.
Seems that “ kamctl rpc core.uptime “ can still do the job.
I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active.
I haven’t process that requests longest data on the host, so I’m not sure if it can be related.
Regards, David
From: Daniel-Constantin Mierla <miconda@gmail.commailto:miconda@gmail.com> Sent: Thursday, June 18, 2020 3:16 PM To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org>; David VILLAUME <david.villaume@sewan.frmailto:david.villaume@sewan.fr> Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
what is the command you run? Maybe the result is too big and you need to change the params for ctl module.
Or is it for all commands, including those printing small data, such case core.uptime?
You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd.
Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello,
I’m having problem with RPC socket that stops answering after approximatively one or two hours.
Version is 5.2.7
My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem.
I could catch this log about the problem, but didn’t find any reason for this :
Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed
My configuration about rpc :
loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024)
Do you have any idea to troubleshoot this ?
Regards, David
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
Hello,
it is already backported in branch 5.3. Are you still running 5.2.x?
Cheers, Daniel
On 15.07.20 13:07, David VILLAUME wrote:
Hello Daniel,
Do you have any idea about when the pâcth could be backported in stable ?
Regards,
David
*From:*David VILLAUME *Sent:* Monday, June 29, 2020 11:21 PM *To:* miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* RE: [SR-Users] RPC Socket stops answering after a few time.
Hi Daniel,
The patched worked correctly.
Big thanks.
Regards,
David
*From:*David VILLAUME *Sent:* Sunday, June 28, 2020 12:16 PM *To:* miconda@gmail.com mailto:miconda@gmail.com; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org mailto:sr-users@lists.kamailio.org> *Subject:* RE: [SR-Users] RPC Socket stops answering after a few time.
Hi,
Thanks a lot, the patch is in test, I‘ll be able to confirm you tomorrow.
Regards,
David
*From:*Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> *Sent:* Tuesday, June 23, 2020 8:15 AM *To:* David VILLAUME <david.villaume@sewan.fr mailto:david.villaume@sewan.fr>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org mailto:sr-users@lists.kamailio.org> *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
can you try with the patch from the commit:
- https://github.com/kamailio/kamailio/commit/f4036c535033a333cb551b5af7070313...
If all works ok, then I will backport to stable branches.
Cheers, Daniel
On 22.06.20 17:53, Daniel-Constantin Mierla wrote:
Hello, I got the backtrace, I will analyse it and see what I find. Cheers, Daniel On 19.06.20 14:09, David VILLAUME wrote: Hello, I just restarted the service I wait for another occurrence to perform it. Regards, David *From:*Daniel-Constantin Mierla <miconda@gmail.com> <mailto:miconda@gmail.com> *Sent:* Friday, June 19, 2020 2:07 PM *To:* David VILLAUME <david.villaume@sewan.fr> <mailto:david.villaume@sewan.fr>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> <mailto:sr-users@lists.kamailio.org> *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time. Hello, get the output of: kamctl trap You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing. Cheers. Daniel On 19.06.20 13:53, David VILLAUME wrote: Hello, After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active” And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears. After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore. Have you any idea to go ahead with troubleshooting ? Regards, David *From:*David VILLAUME *Sent:* Thursday, June 18, 2020 3:31 PM *To:* miconda@gmail.com <mailto:miconda@gmail.com>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> <mailto:sr-users@lists.kamailio.org> *Subject:* RE: [SR-Users] RPC Socket stops answering after a few time. Hello Daniel, Thanks for your quick answer. In fact I cannot even enter in kamcmd interactive mode after this happens. Seems that “ kamctl rpc core.uptime “ can still do the job. I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active. I haven’t process that requests longest data on the host, so I’m not sure if it can be related. Regards, David *From:*Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> *Sent:* Thursday, June 18, 2020 3:16 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org>>; David VILLAUME <david.villaume@sewan.fr <mailto:david.villaume@sewan.fr>> *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time. Hello, what is the command you run? Maybe the result is too big and you need to change the params for ctl module. Or is it for all commands, including those printing small data, such case core.uptime? You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd. Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello, I’m having problem with RPC socket that stops answering after approximatively one or two hours. Version is 5.2.7 My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem. I could catch this log about the problem, but didn’t find any reason for this : Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed _My configuration about rpc : _ loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024) Do you have any idea to troubleshoot this ? Regards, David _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Funding: https://www.paypal.me/dcmierla -- Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Funding: https://www.paypal.me/dcmierla -- Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Funding: https://www.paypal.me/dcmierla
-- Daniel-Constantin Mierla -- www.asipto.com http://www.asipto.com www.twitter.com/miconda http://www.twitter.com/miconda -- www.linkedin.com/in/miconda http://www.linkedin.com/in/miconda Funding: https://www.paypal.me/dcmierla
Hello,
Was still running 5.2 on this server, I moved to 5.3.
Thanks for everything.
Regards, David
From: Daniel-Constantin Mierla miconda@gmail.com Sent: Wednesday, July 15, 2020 7:44 PM To: David VILLAUME david.villaume@sewan.fr; 'Kamailio (SER) - Users Mailing List' sr-users@lists.kamailio.org Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
it is already backported in branch 5.3. Are you still running 5.2.x?
Cheers, Daniel On 15.07.20 13:07, David VILLAUME wrote: Hello Daniel,
Do you have any idea about when the pâcth could be backported in stable ?
Regards, David
From: David VILLAUME Sent: Monday, June 29, 2020 11:21 PM To: miconda@gmail.commailto:miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hi Daniel,
The patched worked correctly.
Big thanks.
Regards, David
From: David VILLAUME Sent: Sunday, June 28, 2020 12:16 PM To: miconda@gmail.commailto:miconda@gmail.com; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org> Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hi,
Thanks a lot, the patch is in test, I‘ll be able to confirm you tomorrow.
Regards, David
From: Daniel-Constantin Mierla <miconda@gmail.commailto:miconda@gmail.com> Sent: Tuesday, June 23, 2020 8:15 AM To: David VILLAUME <david.villaume@sewan.frmailto:david.villaume@sewan.fr>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org> Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
can you try with the patch from the commit:
- https://github.com/kamailio/kamailio/commit/f4036c535033a333cb551b5af7070313...
If all works ok, then I will backport to stable branches.
Cheers, Daniel On 22.06.20 17:53, Daniel-Constantin Mierla wrote:
Hello,
I got the backtrace, I will analyse it and see what I find.
Cheers, Daniel On 19.06.20 14:09, David VILLAUME wrote: Hello,
I just restarted the service I wait for another occurrence to perform it.
Regards, David
From: Daniel-Constantin Mierla miconda@gmail.commailto:miconda@gmail.com Sent: Friday, June 19, 2020 2:07 PM To: David VILLAUME david.villaume@sewan.frmailto:david.villaume@sewan.fr; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
get the output of:
kamctl trap
You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing.
Cheers. Daniel On 19.06.20 13:53, David VILLAUME wrote: Hello,
After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active”
And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears.
After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore.
Have you any idea to go ahead with troubleshooting ?
Regards, David
From: David VILLAUME Sent: Thursday, June 18, 2020 3:31 PM To: miconda@gmail.commailto:miconda@gmail.com; Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Subject: RE: [SR-Users] RPC Socket stops answering after a few time.
Hello Daniel,
Thanks for your quick answer.
In fact I cannot even enter in kamcmd interactive mode after this happens.
Seems that “ kamctl rpc core.uptime “ can still do the job.
I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active.
I haven’t process that requests longest data on the host, so I’m not sure if it can be related.
Regards, David
From: Daniel-Constantin Mierla <miconda@gmail.commailto:miconda@gmail.com> Sent: Thursday, June 18, 2020 3:16 PM To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org>; David VILLAUME <david.villaume@sewan.frmailto:david.villaume@sewan.fr> Subject: Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
what is the command you run? Maybe the result is too big and you need to change the params for ctl module.
Or is it for all commands, including those printing small data, such case core.uptime?
You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd.
Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello,
I’m having problem with RPC socket that stops answering after approximatively one or two hours.
Version is 5.2.7
My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem.
I could catch this log about the problem, but didn’t find any reason for this :
Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed
My configuration about rpc :
loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024)
Do you have any idea to troubleshoot this ?
Regards, David
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
--
Daniel-Constantin Mierla -- www.asipto.comhttp://www.asipto.com
www.twitter.com/micondahttp://www.twitter.com/miconda -- www.linkedin.com/in/micondahttp://www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
Hello,
ok, I plan to backports my commits to 5.2 these days, probably a release in that series will follow soon as well.
Cheers, Daniel
On 16.07.20 07:50, David VILLAUME wrote:
Hello,
Was still running 5.2 on this server, I moved to 5.3.
Thanks for everything.
Regards,
David
*From:*Daniel-Constantin Mierla miconda@gmail.com *Sent:* Wednesday, July 15, 2020 7:44 PM *To:* David VILLAUME david.villaume@sewan.fr; 'Kamailio (SER) - Users Mailing List' sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time.
Hello,
it is already backported in branch 5.3. Are you still running 5.2.x?
Cheers, Daniel
On 15.07.20 13:07, David VILLAUME wrote:
Hello Daniel, Do you have any idea about when the pâcth could be backported in stable ? Regards, David *From:*David VILLAUME *Sent:* Monday, June 29, 2020 11:21 PM *To:* miconda@gmail.com <mailto:miconda@gmail.com>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> <mailto:sr-users@lists.kamailio.org> *Subject:* RE: [SR-Users] RPC Socket stops answering after a few time. Hi Daniel, The patched worked correctly. Big thanks. Regards, David *From:*David VILLAUME *Sent:* Sunday, June 28, 2020 12:16 PM *To:* miconda@gmail.com <mailto:miconda@gmail.com>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org>> *Subject:* RE: [SR-Users] RPC Socket stops answering after a few time. Hi, Thanks a lot, the patch is in test, I‘ll be able to confirm you tomorrow. Regards, David *From:*Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> *Sent:* Tuesday, June 23, 2020 8:15 AM *To:* David VILLAUME <david.villaume@sewan.fr <mailto:david.villaume@sewan.fr>>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org>> *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time. Hello, can you try with the patch from the commit: - https://github.com/kamailio/kamailio/commit/f4036c535033a333cb551b5af70703135305e7b3 If all works ok, then I will backport to stable branches. Cheers, Daniel On 22.06.20 17:53, Daniel-Constantin Mierla wrote: Hello, I got the backtrace, I will analyse it and see what I find. Cheers, Daniel On 19.06.20 14:09, David VILLAUME wrote: Hello, I just restarted the service I wait for another occurrence to perform it. Regards, David *From:*Daniel-Constantin Mierla <miconda@gmail.com> <mailto:miconda@gmail.com> *Sent:* Friday, June 19, 2020 2:07 PM *To:* David VILLAUME <david.villaume@sewan.fr> <mailto:david.villaume@sewan.fr>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> <mailto:sr-users@lists.kamailio.org> *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time. Hello, get the output of: kamctl trap You need gdb installed. The output is written a file, with the path to file printed by the command. Sent that file so we can investigate and see what each process is doing. Cheers. Daniel On 19.06.20 13:53, David VILLAUME wrote: Hello, After a new check , can’t get answer, on the node for : “kamctl rpc dlg.stats_active” And I see in logs that about for dmq replicated calls (dialog [dialog.c:2400]: rpc_dlg_stats_active(): not active - state: 5) , disappear after the previous message appears. After trying a “kamctl rpc dlg.stats_active”, I cannot even have answer to ‘kamctl rpc core.uptime’ anymore. Have you any idea to go ahead with troubleshooting ? Regards, David *From:*David VILLAUME *Sent:* Thursday, June 18, 2020 3:31 PM *To:* miconda@gmail.com <mailto:miconda@gmail.com>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> <mailto:sr-users@lists.kamailio.org> *Subject:* RE: [SR-Users] RPC Socket stops answering after a few time. Hello Daniel, Thanks for your quick answer. In fact I cannot even enter in kamcmd interactive mode after this happens. Seems that “ kamctl rpc core.uptime “ can still do the job. I’m using kamailio_exporter with the following methods : tm.stats,sl.stats,core.shmmem,core.uptime,dispatcher.list,dlg.stats_active. I haven’t process that requests longest data on the host, so I’m not sure if it can be related. Regards, David *From:*Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> *Sent:* Thursday, June 18, 2020 3:16 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org>>; David VILLAUME <david.villaume@sewan.fr <mailto:david.villaume@sewan.fr>> *Subject:* Re: [SR-Users] RPC Socket stops answering after a few time. Hello, what is the command you run? Maybe the result is too big and you need to change the params for ctl module. Or is it for all commands, including those printing small data, such case core.uptime? You can also try the 'kamctl rpc COMMAND', which uses jsonrpcs, the error you pasted is from ctl module, used by kamcmd. Cheers, Daniel On 18.06.20 15:00, David VILLAUME wrote: Hello, I’m having problem with RPC socket that stops answering after approximatively one or two hours. Version is 5.2.7 My principal host with exact same version and configuration has no problem, only the backup node (Keepalived + DMQ) encounters the problem. I could catch this log about the problem, but didn’t find any reason for this : Jun 18 10:53:33 sbc-kam-partner-b kamailio[83170]: ERROR: ctl [binrpc_run.c:600]: rpc_send(): ERROR: binrprc: rpc_send: send failed _My configuration about rpc : _ loadmodule "jsonrpcs.so" modparam("ctl", "binrpc", "unix:/var/run/kamailio/kamailio_ctl") modparam("ctl", "binrpc_max_body_size", 1024) Do you have any idea to troubleshoot this ? Regards, David _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Funding: https://www.paypal.me/dcmierla -- Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Funding: https://www.paypal.me/dcmierla -- Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Funding: https://www.paypal.me/dcmierla -- Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Funding: https://www.paypal.me/dcmierla
-- Daniel-Constantin Mierla -- www.asipto.com http://www.asipto.com www.twitter.com/miconda http://www.twitter.com/miconda -- www.linkedin.com/in/miconda http://www.linkedin.com/in/miconda Funding: https://www.paypal.me/dcmierla