The `benchmark` module allows you to create points can be used to generate comparative timestamps for desired parts of your code.  I’m not sure if using it on a production server adds much overhead or not, though.  Look to set points before and after things like database calls that are blocking operation.  As a possible reason, if you’re doing accounting and the db is blocked you may have latency in processing the BYE.

Kaufman

Sr. VoIP Engineer - BCM One

From: sr-users <sr-users-bounces@lists.kamailio.org> On Behalf Of Lewis Hutchinson
Sent: Thursday, August 25, 2022 8:37 AM
To: sr-users@lists.kamailio.org
Subject: [SR-Users] Intermittent delays relaying BYE (up to 1s)

I have a situation where I am seeing intermittently Kamailio having delays when relaying the BYE received from an external party

This is only affecting the BYE

We have 6 Kamailios stood up

3 linked to the SIP Providers (SBC)
3 for Registration (REG)
Language KEMI | Python

version: kamailio 5.5.1 (x86_64/linux) 278772-dirty

flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED

ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB

poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.

id: 278772 -dirty

compiled with gcc 9.3.0

Simplified Flow
PSTN >>>> Kamailio SBC >>>> FREESWITCH >>>> Kamailio REG >>>> SIP DEVICE

SIP DEVICE >>>> Kamailio REG >>>> FREESWITCH >>>> Kamailio SBC >>>> PSTN

This happens on all of them intermittently when receiving the BYE from either a SIP trunk or a SIP Device Publicly, it can take upto 1s to pass this onto the next hope (Freeswitch) on the local network. All BYEs passed between the local network relay  in fractions of a second. All BYEs passed out of the KAMAILIO SBC or KAMAILIO REG to their respective public enpoints relay  in fractions of a second but BYEs received from public on advertised address take upto 1s for Kamailio to process and relay.

i.e

PSTN >>> BYE >>>> KAMAILIO SBC (kamailio will sometimes take upto 1 second to relay this on to other kit on the internal solution. Freeswitch etc)

SIP DEVICE  >>> BYE >>>> KAMAILIO REG (kamailio will sometimes take upto 1 second to relay this on to other kit on the internal solution. Freeswitch etc)

I am basing these timings on SNgrep and PCAP traces from homer.

Does anyone have any hints and tips on how I can debug this issue. I have enabled debug logging and replicated the issue but it doesn’t show anything from what I can see as to why there is delays.  

Thanks in advance

lewis