I have a situation where I am seeing intermittently Kamailio having delays when relaying the BYE received from an external party
This is only affecting the BYE
We have 6 Kamailios stood up
3 linked to the SIP Providers (SBC) 3 for Registration (REG) Language KEMI | Python
version: kamailio 5.5.1 (x86_64/linux) 278772-dirty flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 278772 -dirty compiled with gcc 9.3.0
Simplified Flow PSTN >>>> Kamailio SBC >>>> FREESWITCH >>>> Kamailio REG >>>> SIP DEVICE
SIP DEVICE >>>> Kamailio REG >>>> FREESWITCH >>>> Kamailio SBC >>>> PSTN
This happens on all of them intermittently when receiving the BYE from either a SIP trunk or a SIP Device Publicly, it can take upto 1s to pass this onto the next hope (Freeswitch) on the local network. All BYEs passed between the local network relay in fractions of a second. All BYEs passed out of the KAMAILIO SBC or KAMAILIO REG to their respective public enpoints relay in fractions of a second but BYEs received from public on advertised address take upto 1s for Kamailio to process and relay.
i.e
PSTN >>> BYE >>>> KAMAILIO SBC (kamailio will sometimes take upto 1 second to relay this on to other kit on the internal solution. Freeswitch etc)
SIP DEVICE >>> BYE >>>> KAMAILIO REG (kamailio will sometimes take upto 1 second to relay this on to other kit on the internal solution. Freeswitch etc)
I am basing these timings on SNgrep and PCAP traces from homer.
Does anyone have any hints and tips on how I can debug this issue. I have enabled debug logging and replicated the issue but it doesn't show anything from what I can see as to why there is delays.
Thanks in advance
lewis
The `benchmark` module allows you to create points can be used to generate comparative timestamps for desired parts of your code. I'm not sure if using it on a production server adds much overhead or not, though. Look to set points before and after things like database calls that are blocking operation. As a possible reason, if you're doing accounting and the db is blocked you may have latency in processing the BYE.
Kaufman Sr. VoIP Engineer - BCM One
From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of Lewis Hutchinson Sent: Thursday, August 25, 2022 8:37 AM To: sr-users@lists.kamailio.org Subject: [SR-Users] Intermittent delays relaying BYE (up to 1s)
I have a situation where I am seeing intermittently Kamailio having delays when relaying the BYE received from an external party
This is only affecting the BYE
We have 6 Kamailios stood up
3 linked to the SIP Providers (SBC) 3 for Registration (REG) Language KEMI | Python
version: kamailio 5.5.1 (x86_64/linux) 278772-dirty flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 278772 -dirty compiled with gcc 9.3.0
Simplified Flow PSTN >>>> Kamailio SBC >>>> FREESWITCH >>>> Kamailio REG >>>> SIP DEVICE
SIP DEVICE >>>> Kamailio REG >>>> FREESWITCH >>>> Kamailio SBC >>>> PSTN
This happens on all of them intermittently when receiving the BYE from either a SIP trunk or a SIP Device Publicly, it can take upto 1s to pass this onto the next hope (Freeswitch) on the local network. All BYEs passed between the local network relay in fractions of a second. All BYEs passed out of the KAMAILIO SBC or KAMAILIO REG to their respective public enpoints relay in fractions of a second but BYEs received from public on advertised address take upto 1s for Kamailio to process and relay.
i.e
PSTN >>> BYE >>>> KAMAILIO SBC (kamailio will sometimes take upto 1 second to relay this on to other kit on the internal solution. Freeswitch etc) SIP DEVICE >>> BYE >>>> KAMAILIO REG (kamailio will sometimes take upto 1 second to relay this on to other kit on the internal solution. Freeswitch etc)
I am basing these timings on SNgrep and PCAP traces from homer.
Does anyone have any hints and tips on how I can debug this issue. I have enabled debug logging and replicated the issue but it doesn't show anything from what I can see as to why there is delays.
Thanks in advance
lewis
Hi,
While you have not said much about how your BYEs are processed, I would speculate that the operation in your config script does not consist purely of relaying these on the network, but also some sort of blocking database operation that is delaying execution of the config script.
— Alex
On Aug 25, 2022, at 9:36 AM, Lewis Hutchinson lewis.hutchinson@missionlabs.co.uk wrote:
I have a situation where I am seeing intermittently Kamailio having delays when relaying the BYE received from an external party
This is only affecting the BYE
We have 6 Kamailios stood up
3 linked to the SIP Providers (SBC) 3 for Registration (REG) Language KEMI | Python
version: kamailio 5.5.1 (x86_64/linux) 278772-dirty flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 278772 -dirty compiled with gcc 9.3.0
Simplified Flow PSTN >>>> Kamailio SBC >>>> FREESWITCH >>>> Kamailio REG >>>> SIP DEVICE
SIP DEVICE >>>> Kamailio REG >>>> FREESWITCH >>>> Kamailio SBC >>>> PSTN
This happens on all of them intermittently when receiving the BYE from either a SIP trunk or a SIP Device Publicly, it can take upto 1s to pass this onto the next hope (Freeswitch) on the local network. All BYEs passed between the local network relay in fractions of a second. All BYEs passed out of the KAMAILIO SBC or KAMAILIO REG to their respective public enpoints relay in fractions of a second but BYEs received from public on advertised address take upto 1s for Kamailio to process and relay.
i.e
PSTN >>> BYE >>>> KAMAILIO SBC (kamailio will sometimes take upto 1 second to relay this on to other kit on the internal solution. Freeswitch etc)
SIP DEVICE >>> BYE >>>> KAMAILIO REG (kamailio will sometimes take upto 1 second to relay this on to other kit on the internal solution. Freeswitch etc)
I am basing these timings on SNgrep and PCAP traces from homer.
Does anyone have any hints and tips on how I can debug this issue. I have enabled debug logging and replicated the issue but it doesn’t show anything from what I can see as to why there is delays.
Thanks in advance
lewis
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-
Alex Balashov -
Kaufman -
Lewis Hutchinson