Hey all,

looking for a little help.

I've been tracking an issue, where a number of SIP Messages ( typically INVITE ) are sent, but Kamailio only see one of them ( the last one). I have verified that the messages are definitely received on the host, as verified with TCPDUMP, but the INVITES never hit the request route block. ( the very first thing is to log the message )

What i've managed to find is, that if I increase worker children, the problem goes away.

So I've got a problem, I think I've found the solution, but what I'm struggling with is how to monitor better, so I can be alerted to this in the future.

I started by looking at "kamctl stats | grep drop"

"core:drop_replies = 14",
"core:drop_requests = 10577"

however drop_requests seems to include explicit drops in my config.

which I do for many reasons, but mainly bad UA or blocked IP's, so this dosnt seem to be what im after.

I then went digging in the code ( often the best way to find things ).

in receive.c receive_msg() we find some conditions, where we drop the packet ( before calling request route ).

right now im looking around line 251 at

LM_DBG("dropping the received message\n");

goto error00;

Following error00 I see it calls STATS_RX_DROPS which increments stats->received_drops.

This is where it gets fun, I cant find where received_drops is ever returned in the stats output.

I can see where -SIGUSR1 will trigger this to be dumped, if Kamailio is compiled with that option ( dosnt seem the debian packages are ). however, what im seeking clarification on is, does this get reported in "kamctl stats"

Am I just going crazy or is something not quite right here?

TLDNR; how do I monitor for Kamailio not having enough worker threads to process incoming messages.

FYI I'm seeing about 50Mbps of incoming SIP on this box, and I do perform a decent amount of work on some message types.

Sincerely

Jay