Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions. Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards, Angelo
Hi,
Any ideas on this topic? Anyone with same behavior?
Regards, Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper sippro97@gmail.com wrote:
Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions. Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards, Angelo
Hello,
Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..?
Cheers,
Henning
-- Henning Westerholt – https://skalatan.de/blog/ Kamailio services – https://gilawa.comhttps://gilawa.com/
From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of Angelo Sipper Sent: Monday, January 31, 2022 12:32 PM To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Any ideas on this topic? Anyone with same behavior?
Regards, Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper <sippro97@gmail.commailto:sippro97@gmail.com> wrote: Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions. Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards, Angelo
Hello,
Do use domain name as destination for homer?
On Mon, Jan 31, 2022, 23:12 Henning Westerholt hw@gilawa.com wrote:
Hello,
Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Monday, January 31, 2022 12:32 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Any ideas on this topic?
Anyone with same behavior?
Regards,
Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper sippro97@gmail.com wrote:
Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions.
Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards,
Angelo
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Hi,
Memory on the system is 8GB and on Kamailio we have SHM 4096 and PKG 32. Each of these VM handles only a few hundred of concurrent calls and each VM has a few hundred of TLS user registrations. Kamailio uses IP to send the traffic to Homer and not domain name.
Any ideas how to solve this issue?
Kind Regards, Angelo
On Tue, Feb 1, 2022, 8:43 PM Nick F nik.oloakt@gmail.com wrote:
Hello,
Do use domain name as destination for homer?
On Mon, Jan 31, 2022, 23:12 Henning Westerholt hw@gilawa.com wrote:
Hello,
Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Monday, January 31, 2022 12:32 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Any ideas on this topic?
Anyone with same behavior?
Regards,
Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper sippro97@gmail.com wrote:
Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions.
Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards,
Angelo
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Hello,
any feedback regarding my previous question about log files, errors?
Cheers,
Henning
-- Henning Westerholt – https://skalatan.de/blog/ Kamailio services – https://gilawa.comhttps://gilawa.com/
From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of Angelo Sipper Sent: Wednesday, February 2, 2022 5:45 PM To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Memory on the system is 8GB and on Kamailio we have SHM 4096 and PKG 32. Each of these VM handles only a few hundred of concurrent calls and each VM has a few hundred of TLS user registrations. Kamailio uses IP to send the traffic to Homer and not domain name.
Any ideas how to solve this issue?
Kind Regards, Angelo
On Tue, Feb 1, 2022, 8:43 PM Nick F <nik.oloakt@gmail.commailto:nik.oloakt@gmail.com> wrote: Hello,
Do use domain name as destination for homer?
On Mon, Jan 31, 2022, 23:12 Henning Westerholt <hw@gilawa.commailto:hw@gilawa.com> wrote: Hello,
Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..?
Cheers,
Henning
-- Henning Westerholt – https://skalatan.de/blog/ Kamailio services – https://gilawa.comhttps://gilawa.com/
From: sr-users <sr-users-bounces@lists.kamailio.orgmailto:sr-users-bounces@lists.kamailio.org> On Behalf Of Angelo Sipper Sent: Monday, January 31, 2022 12:32 PM To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org> Subject: Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Any ideas on this topic? Anyone with same behavior?
Regards, Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper <sippro97@gmail.commailto:sippro97@gmail.com> wrote: Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions. Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards, Angelo __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions * sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions * sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi,
try to define an extra sendsocket.
And add an listen directive for that sendsocket. Sorry for the formating wrote this on my smartphone.
Works for me with an similar setup and 5.3
# capture node or webhomer target #!ifdef WITH_HOMER #!substdef "!MY_HOMER_CAPTURE!sip:MY_HOMER_IP_ADDR:MY_HOMER_PORT!g" #!substdef "!MY_LOCAL_SENDSOCK!sip:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!substdef "!MY_LOCAL_SENDLISTEN!udp:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!endif
#!ifdef WITH_HOMER #Siptrace modparam("siptrace", "force_send_sock", "MY_LOCAL_SENDSOCK") # local interface from where to send the duplicated traffic modparam("siptrace", "duplicate_uri", "MY_HOMER_CAPTURE") modparam("siptrace", "hep_mode_on", 1) # enable/disable Homer encapsulate mode modparam("siptrace", "hep_version", 3) # HEPv3 the timestamp and capture agent ID will be included in the HEP header modparam("siptrace", "hep_capture_id", MY_HOMER_CAPTURE_ID) # capture agent id Limitation: 32-bit for HEPv3. modparam("siptrace", "trace_to_database", 0) # trace to database modparam("siptrace", "trace_on", 1) # enable/disable trace modparam("siptrace", "trace_mode", 1) # default 0, if 1 then you dont need call siptrace flag or sip_trace() #!endif
# sendsocket for homer #!ifdef WITH_HOMER socket_workers=2 listen=MY_LOCAL_SENDLISTEN #!endif
Henning Westerholt hw@gilawa.com schrieb am Mi., 2. Feb. 2022, 18:44:
Hello,
any feedback regarding my previous question about log files, errors?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Wednesday, February 2, 2022 5:45 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Memory on the system is 8GB and on Kamailio we have SHM 4096 and PKG 32. Each of these VM handles only a few hundred of concurrent calls and each VM has a few hundred of TLS user registrations.
Kamailio uses IP to send the traffic to Homer and not domain name.
Any ideas how to solve this issue?
Kind Regards,
Angelo
On Tue, Feb 1, 2022, 8:43 PM Nick F nik.oloakt@gmail.com wrote:
Hello,
Do use domain name as destination for homer?
On Mon, Jan 31, 2022, 23:12 Henning Westerholt hw@gilawa.com wrote:
Hello,
Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Monday, January 31, 2022 12:32 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Any ideas on this topic?
Anyone with same behavior?
Regards,
Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper sippro97@gmail.com wrote:
Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions.
Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards,
Angelo
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Hi,
I have the exact same configuration with two workers and same syntax. It works fine until I shut down homer. After that Kamailio stop precessing any traffic and all clients try to register as they are disconnected.
Kind Regards, Angelo
On Wed, Feb 2, 2022, 8:45 PM Karsten Horsmann khorsmann@gmail.com wrote:
Hi,
try to define an extra sendsocket.
And add an listen directive for that sendsocket. Sorry for the formating wrote this on my smartphone.
Works for me with an similar setup and 5.3
# capture node or webhomer target #!ifdef WITH_HOMER #!substdef "!MY_HOMER_CAPTURE!sip:MY_HOMER_IP_ADDR:MY_HOMER_PORT!g" #!substdef "!MY_LOCAL_SENDSOCK!sip:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!substdef "!MY_LOCAL_SENDLISTEN!udp:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!endif
#!ifdef WITH_HOMER #Siptrace modparam("siptrace", "force_send_sock", "MY_LOCAL_SENDSOCK") # local interface from where to send the duplicated traffic modparam("siptrace", "duplicate_uri", "MY_HOMER_CAPTURE") modparam("siptrace", "hep_mode_on", 1) # enable/disable Homer encapsulate mode modparam("siptrace", "hep_version", 3) # HEPv3 the timestamp and capture agent ID will be included in the HEP header modparam("siptrace", "hep_capture_id", MY_HOMER_CAPTURE_ID) # capture agent id Limitation: 32-bit for HEPv3. modparam("siptrace", "trace_to_database", 0) # trace to database modparam("siptrace", "trace_on", 1) # enable/disable trace modparam("siptrace", "trace_mode", 1) # default 0, if 1 then you dont need call siptrace flag or sip_trace() #!endif
# sendsocket for homer #!ifdef WITH_HOMER socket_workers=2 listen=MY_LOCAL_SENDLISTEN #!endif
Henning Westerholt hw@gilawa.com schrieb am Mi., 2. Feb. 2022, 18:44:
Hello,
any feedback regarding my previous question about log files, errors?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Wednesday, February 2, 2022 5:45 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Memory on the system is 8GB and on Kamailio we have SHM 4096 and PKG 32. Each of these VM handles only a few hundred of concurrent calls and each VM has a few hundred of TLS user registrations.
Kamailio uses IP to send the traffic to Homer and not domain name.
Any ideas how to solve this issue?
Kind Regards,
Angelo
On Tue, Feb 1, 2022, 8:43 PM Nick F nik.oloakt@gmail.com wrote:
Hello,
Do use domain name as destination for homer?
On Mon, Jan 31, 2022, 23:12 Henning Westerholt hw@gilawa.com wrote:
Hello,
Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Monday, January 31, 2022 12:32 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Any ideas on this topic?
Anyone with same behavior?
Regards,
Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper sippro97@gmail.com wrote:
Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions.
Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards,
Angelo
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
I've encountered a very similar problem in the past.The siptrace module uses UDP and does not expect a response, so it can be confusing that kamailio's behavior changes depending on whether the homer server is online.
The problem I've seen in the past is that when kamailio and a statsd server are in the same IP subnet, and the statds server goes offline, an ARP request is made whenever traffic is to be sent, and this ARP request is blocking until it times out (a few seconds). It blocks all workers because they all want to send a UDP packet and all have to wait.
If this is what's going on, then moving the siptrace server to a different subnet should resolve the problem.
James
On Wed, 2 Feb 2022 at 21:43, Angelo Sipper sippro97@gmail.com wrote:
Hi,
I have the exact same configuration with two workers and same syntax. It works fine until I shut down homer. After that Kamailio stop precessing any traffic and all clients try to register as they are disconnected.
Kind Regards, Angelo
On Wed, Feb 2, 2022, 8:45 PM Karsten Horsmann khorsmann@gmail.com wrote:
Hi,
try to define an extra sendsocket.
And add an listen directive for that sendsocket. Sorry for the formating wrote this on my smartphone.
Works for me with an similar setup and 5.3
# capture node or webhomer target #!ifdef WITH_HOMER #!substdef "!MY_HOMER_CAPTURE!sip:MY_HOMER_IP_ADDR:MY_HOMER_PORT!g" #!substdef "!MY_LOCAL_SENDSOCK!sip:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!substdef "!MY_LOCAL_SENDLISTEN!udp:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!endif
#!ifdef WITH_HOMER #Siptrace modparam("siptrace", "force_send_sock", "MY_LOCAL_SENDSOCK") # local interface from where to send the duplicated traffic modparam("siptrace", "duplicate_uri", "MY_HOMER_CAPTURE") modparam("siptrace", "hep_mode_on", 1) # enable/disable Homer encapsulate mode modparam("siptrace", "hep_version", 3) # HEPv3 the timestamp and capture agent ID will be included in the HEP header modparam("siptrace", "hep_capture_id", MY_HOMER_CAPTURE_ID) # capture agent id Limitation: 32-bit for HEPv3. modparam("siptrace", "trace_to_database", 0) # trace to database modparam("siptrace", "trace_on", 1) # enable/disable trace modparam("siptrace", "trace_mode", 1) # default 0, if 1 then you dont need call siptrace flag or sip_trace() #!endif
# sendsocket for homer #!ifdef WITH_HOMER socket_workers=2 listen=MY_LOCAL_SENDLISTEN #!endif
Henning Westerholt hw@gilawa.com schrieb am Mi., 2. Feb. 2022, 18:44:
Hello,
any feedback regarding my previous question about log files, errors?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Wednesday, February 2, 2022 5:45 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Memory on the system is 8GB and on Kamailio we have SHM 4096 and PKG 32. Each of these VM handles only a few hundred of concurrent calls and each VM has a few hundred of TLS user registrations.
Kamailio uses IP to send the traffic to Homer and not domain name.
Any ideas how to solve this issue?
Kind Regards,
Angelo
On Tue, Feb 1, 2022, 8:43 PM Nick F nik.oloakt@gmail.com wrote:
Hello,
Do use domain name as destination for homer?
On Mon, Jan 31, 2022, 23:12 Henning Westerholt hw@gilawa.com wrote:
Hello,
Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Monday, January 31, 2022 12:32 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Any ideas on this topic?
Anyone with same behavior?
Regards,
Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper sippro97@gmail.com wrote:
Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions.
Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards,
Angelo
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Hi James an Angelo,
most of the time my homer or hepic servers are in different subnets. So that arp think sounds good to me.
@Angelo Take a look at the HEP Fidelity Proxy, it's capable to be in the middle between your kamailio and the hep-server and can store the HEP packets if your hep-server is unavailable. After the hep-server (homer 7, hepic, whatever) is available again it sends the stored data automatically to the hep-server.
https://github.com/ivlovric/HFP/
James Browne james@frideo.com schrieb am Do., 3. Feb. 2022, 12:26:
I've encountered a very similar problem in the past.The siptrace module uses UDP and does not expect a response, so it can be confusing that kamailio's behavior changes depending on whether the homer server is online.
The problem I've seen in the past is that when kamailio and a statsd server are in the same IP subnet, and the statds server goes offline, an ARP request is made whenever traffic is to be sent, and this ARP request is blocking until it times out (a few seconds). It blocks all workers because they all want to send a UDP packet and all have to wait.
If this is what's going on, then moving the siptrace server to a different subnet should resolve the problem.
James
On Wed, 2 Feb 2022 at 21:43, Angelo Sipper sippro97@gmail.com wrote:
Hi,
I have the exact same configuration with two workers and same syntax. It works fine until I shut down homer. After that Kamailio stop precessing any traffic and all clients try to register as they are disconnected.
Kind Regards, Angelo
On Wed, Feb 2, 2022, 8:45 PM Karsten Horsmann khorsmann@gmail.com wrote:
Hi,
try to define an extra sendsocket.
And add an listen directive for that sendsocket. Sorry for the formating wrote this on my smartphone.
Works for me with an similar setup and 5.3
# capture node or webhomer target #!ifdef WITH_HOMER #!substdef "!MY_HOMER_CAPTURE!sip:MY_HOMER_IP_ADDR:MY_HOMER_PORT!g" #!substdef "!MY_LOCAL_SENDSOCK!sip:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!substdef "!MY_LOCAL_SENDLISTEN!udp:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!endif
#!ifdef WITH_HOMER #Siptrace modparam("siptrace", "force_send_sock", "MY_LOCAL_SENDSOCK") # local interface from where to send the duplicated traffic modparam("siptrace", "duplicate_uri", "MY_HOMER_CAPTURE") modparam("siptrace", "hep_mode_on", 1) # enable/disable Homer encapsulate mode modparam("siptrace", "hep_version", 3) # HEPv3 the timestamp and capture agent ID will be included in the HEP header modparam("siptrace", "hep_capture_id", MY_HOMER_CAPTURE_ID) # capture agent id Limitation: 32-bit for HEPv3. modparam("siptrace", "trace_to_database", 0) # trace to database modparam("siptrace", "trace_on", 1) # enable/disable trace modparam("siptrace", "trace_mode", 1) # default 0, if 1 then you dont need call siptrace flag or sip_trace() #!endif
# sendsocket for homer #!ifdef WITH_HOMER socket_workers=2 listen=MY_LOCAL_SENDLISTEN #!endif
Henning Westerholt hw@gilawa.com schrieb am Mi., 2. Feb. 2022, 18:44:
Hello,
any feedback regarding my previous question about log files, errors?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Wednesday, February 2, 2022 5:45 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Memory on the system is 8GB and on Kamailio we have SHM 4096 and PKG 32. Each of these VM handles only a few hundred of concurrent calls and each VM has a few hundred of TLS user registrations.
Kamailio uses IP to send the traffic to Homer and not domain name.
Any ideas how to solve this issue?
Kind Regards,
Angelo
On Tue, Feb 1, 2022, 8:43 PM Nick F nik.oloakt@gmail.com wrote:
Hello,
Do use domain name as destination for homer?
On Mon, Jan 31, 2022, 23:12 Henning Westerholt hw@gilawa.com wrote:
Hello,
Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Monday, January 31, 2022 12:32 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Any ideas on this topic?
Anyone with same behavior?
Regards,
Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper sippro97@gmail.com wrote:
Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions.
Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards,
Angelo
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Hello,
when Kamailio is not processing sip messages, so it looks like stuck, you can grab the backtraces with gdb to see what each kamailio process is executing at that moment. You need gdb installed and then run:
kamctl trap
It writes to a file in /tmp, the file name is printed by the command.
Cheers, Daniel
On 03.02.22 12:22, James Browne wrote:
I've encountered a very similar problem in the past.The siptrace module uses UDP and does not expect a response, so it can be confusing that kamailio's behavior changes depending on whether the homer server is online.
The problem I've seen in the past is that when kamailio and a statsd server are in the same IP subnet, and the statds server goes offline, an ARP request is made whenever traffic is to be sent, and this ARP request is blocking until it times out (a few seconds). It blocks all workers because they all want to send a UDP packet and all have to wait.
If this is what's going on, then moving the siptrace server to a different subnet should resolve the problem.
James
On Wed, 2 Feb 2022 at 21:43, Angelo Sipper sippro97@gmail.com wrote:
Hi, I have the exact same configuration with two workers and same syntax. It works fine until I shut down homer. After that Kamailio stop precessing any traffic and all clients try to register as they are disconnected. Kind Regards, Angelo On Wed, Feb 2, 2022, 8:45 PM Karsten Horsmann <khorsmann@gmail.com> wrote: Hi, try to define an extra sendsocket. And add an listen directive for that sendsocket. Sorry for the formating wrote this on my smartphone. Works for me with an similar setup and 5.3 # capture node or webhomer target #!ifdef WITH_HOMER #!substdef "!MY_HOMER_CAPTURE!sip:MY_HOMER_IP_ADDR:MY_HOMER_PORT!g" #!substdef "!MY_LOCAL_SENDSOCK!sip:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!substdef "!MY_LOCAL_SENDLISTEN!udp:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!endif #!ifdef WITH_HOMER #Siptrace modparam("siptrace", "force_send_sock", "MY_LOCAL_SENDSOCK") # local interface from where to send the duplicated traffic modparam("siptrace", "duplicate_uri", "MY_HOMER_CAPTURE") modparam("siptrace", "hep_mode_on", 1) # enable/disable Homer encapsulate mode modparam("siptrace", "hep_version", 3) # HEPv3 the timestamp and capture agent ID will be included in the HEP header modparam("siptrace", "hep_capture_id", MY_HOMER_CAPTURE_ID) # capture agent id Limitation: 32-bit for HEPv3. modparam("siptrace", "trace_to_database", 0) # trace to database modparam("siptrace", "trace_on", 1) # enable/disable trace modparam("siptrace", "trace_mode", 1) # default 0, if 1 then you dont need call siptrace flag or sip_trace() #!endif # sendsocket for homer #!ifdef WITH_HOMER socket_workers=2 listen=MY_LOCAL_SENDLISTEN #!endif Henning Westerholt <hw@gilawa.com> schrieb am Mi., 2. Feb. 2022, 18:44: Hello, any feedback regarding my previous question about log files, errors? Cheers, Henning -- Henning Westerholt – https://skalatan.de/blog/ <https://skalatan.de/blog/> Kamailio services – https://gilawa.com <https://gilawa.com/> *From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *Angelo Sipper *Sent:* Wednesday, February 2, 2022 5:45 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3 Hi, Memory on the system is 8GB and on Kamailio we have SHM 4096 and PKG 32. Each of these VM handles only a few hundred of concurrent calls and each VM has a few hundred of TLS user registrations. Kamailio uses IP to send the traffic to Homer and not domain name. Any ideas how to solve this issue? Kind Regards, Angelo On Tue, Feb 1, 2022, 8:43 PM Nick F <nik.oloakt@gmail.com> wrote: Hello, Do use domain name as destination for homer? On Mon, Jan 31, 2022, 23:12 Henning Westerholt <hw@gilawa.com> wrote: Hello, Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..? Cheers, Henning -- Henning Westerholt – https://skalatan.de/blog/ <https://skalatan.de/blog/> Kamailio services – https://gilawa.com <https://gilawa.com/> *From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *Angelo Sipper *Sent:* Monday, January 31, 2022 12:32 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3 Hi, Any ideas on this topic? Anyone with same behavior? Regards, Angelo On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper <sippro97@gmail.com> wrote: Hi to all, We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions. Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio? Thank you! Kind Regards, Angelo __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions * sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions * sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions * sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions * sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions * sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Hi,
Thank you all for the suggestions. It makes sense the ARP to be the issue as we have homer on the same network. So we will relocate it to a another subnet and we will also consider adding the proxy in-between but again not on the same subnet. We will also install the gdb to have some backtraces when needed.
I will update you with results when ready.
Kind regards, Angelo
On Thu, Feb 3, 2022, 6:57 PM Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
when Kamailio is not processing sip messages, so it looks like stuck, you can grab the backtraces with gdb to see what each kamailio process is executing at that moment. You need gdb installed and then run:
kamctl trap
It writes to a file in /tmp, the file name is printed by the command.
Cheers, Daniel On 03.02.22 12:22, James Browne wrote:
I've encountered a very similar problem in the past.The siptrace module uses UDP and does not expect a response, so it can be confusing that kamailio's behavior changes depending on whether the homer server is online.
The problem I've seen in the past is that when kamailio and a statsd server are in the same IP subnet, and the statds server goes offline, an ARP request is made whenever traffic is to be sent, and this ARP request is blocking until it times out (a few seconds). It blocks all workers because they all want to send a UDP packet and all have to wait.
If this is what's going on, then moving the siptrace server to a different subnet should resolve the problem.
James
On Wed, 2 Feb 2022 at 21:43, Angelo Sipper sippro97@gmail.com wrote:
Hi,
I have the exact same configuration with two workers and same syntax. It works fine until I shut down homer. After that Kamailio stop precessing any traffic and all clients try to register as they are disconnected.
Kind Regards, Angelo
On Wed, Feb 2, 2022, 8:45 PM Karsten Horsmann khorsmann@gmail.com wrote:
Hi,
try to define an extra sendsocket.
And add an listen directive for that sendsocket. Sorry for the formating wrote this on my smartphone.
Works for me with an similar setup and 5.3
# capture node or webhomer target #!ifdef WITH_HOMER #!substdef "!MY_HOMER_CAPTURE!sip:MY_HOMER_IP_ADDR:MY_HOMER_PORT!g" #!substdef "!MY_LOCAL_SENDSOCK!sip:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!substdef "!MY_LOCAL_SENDLISTEN!udp:MY_PRV_IP:MY_LOCAL_SEND_PORT!g" #!endif
#!ifdef WITH_HOMER #Siptrace modparam("siptrace", "force_send_sock", "MY_LOCAL_SENDSOCK") # local interface from where to send the duplicated traffic modparam("siptrace", "duplicate_uri", "MY_HOMER_CAPTURE") modparam("siptrace", "hep_mode_on", 1) # enable/disable Homer encapsulate mode modparam("siptrace", "hep_version", 3) # HEPv3 the timestamp and capture agent ID will be included in the HEP header modparam("siptrace", "hep_capture_id", MY_HOMER_CAPTURE_ID) # capture agent id Limitation: 32-bit for HEPv3. modparam("siptrace", "trace_to_database", 0) # trace to database modparam("siptrace", "trace_on", 1) # enable/disable trace modparam("siptrace", "trace_mode", 1) # default 0, if 1 then you dont need call siptrace flag or sip_trace() #!endif
# sendsocket for homer #!ifdef WITH_HOMER socket_workers=2 listen=MY_LOCAL_SENDLISTEN #!endif
Henning Westerholt hw@gilawa.com schrieb am Mi., 2. Feb. 2022, 18:44:
Hello,
any feedback regarding my previous question about log files, errors?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Wednesday, February 2, 2022 5:45 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Memory on the system is 8GB and on Kamailio we have SHM 4096 and PKG 32. Each of these VM handles only a few hundred of concurrent calls and each VM has a few hundred of TLS user registrations.
Kamailio uses IP to send the traffic to Homer and not domain name.
Any ideas how to solve this issue?
Kind Regards,
Angelo
On Tue, Feb 1, 2022, 8:43 PM Nick F nik.oloakt@gmail.com wrote:
Hello,
Do use domain name as destination for homer?
On Mon, Jan 31, 2022, 23:12 Henning Westerholt hw@gilawa.com wrote:
Hello,
Interesting observation. Do you see anything related to this error period in the log files? Maybe out of memory errors etc..?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users sr-users-bounces@lists.kamailio.org *On Behalf Of *Angelo Sipper *Sent:* Monday, January 31, 2022 12:32 PM *To:* Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org *Subject:* Re: [SR-Users] Homer7 and Kamailio 5.3
Hi,
Any ideas on this topic?
Anyone with same behavior?
Regards,
Angelo
On Thu, Jan 27, 2022, 9:18 PM Angelo Sipper sippro97@gmail.com wrote:
Hi to all,
We have a Kamailio 5.3 that sends all traffic to a homer7 using siptrace module with parameters force_send_sock and hep_mode_on,1 and hep_version,3 and trace_mode,1. Everything works fine getting all traffic to Homer7 but, in case Homer7 is not reachable, Kamailio then no longer accepts traffic or registrations and it is like it crashes. When we revive Homer7 and is it again reachable accepting traffic, Kamailio then works again fine without any restart or actions.
Any ideas why this may be happening? Is there anything we need to add to the config to avoid this behavior as having Homer7 now is a single point of failure for Kamailio?
Thank you!
Kind Regards,
Angelo
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Daniel-Constantin Mierla -- www.asipto.comwww.twitter.com/miconda -- www.linkedin.com/in/miconda Kamailio Advanced Training - Online Feb 21-24, 2022 (America Timezone)
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-
Angelo Sipper -
Daniel-Constantin Mierla -
Henning Westerholt -
James Browne -
Karsten Horsmann -
Nick F