New subject: lost sip messages

31 Jul 2019

Hey all,
looking for a little help.

I've been tracking an issue, where a number of SIP Messages ( typically
INVITE ) are sent, but Kamailio only see one of them ( the last one).  I
have verified that the messages are definitely received on the host, as
verified with TCPDUMP, but the INVITES never hit the request route block. (
the very first thing is to log the message )

What i've managed to find is, that if I increase worker children, the
problem goes away.

So I've got a problem, I think I've found the solution, but what I'm
struggling with is how to monitor better, so I can be alerted to this in
the future.

I started by looking at "kamctl stats | grep drop"
    "core:drop_replies = 14",
    "core:drop_requests = 10577"

however drop_requests seems to include explicit drops in my config.
which I do for many reasons, but mainly bad UA or blocked IP's, so this
dosnt seem to be what im after.

I then went digging in the code ( often the best way to find things ).
in receive.c receive_msg() we find some conditions, where we drop the
packet ( before calling request route ).

right now im looking around line 251 at
LM_DBG("dropping the received message\n");
goto error00;

Following error00 I see it calls STATS_RX_DROPS which increments
stats->received_drops.

This is where it gets fun, I cant find where received_drops is ever
returned in the stats output.

I can see where -SIGUSR1 will trigger this to be dumped, if Kamailio is
compiled with that option ( dosnt seem the debian packages are ).  however,
what im seeking clarification on is, does this get reported in "kamctl
stats"

Am I just going crazy or is something not quite right here?

TLDNR;  how do I monitor for Kamailio not having enough worker threads to
process incoming messages.

FYI I'm seeing about 50Mbps of incoming SIP on this box, and I do perform a
decent amount of work on some message types.

-- 
Sincerely

Jay

(no subject)