Description

Since we upgraded from v5.0.2 to v5.1.4 we are seeing spikes in our monitoring platform regarding current active and early dialogs.

Every 10s we gather metrics using Kamailio's HTTP server:

    # Statistics endpoint
    if ($hu =~ "^/statistics") {
        jsonrpc_exec('{"jsonrpc": "2.0","method": "stats.get_statistics","params": ["all"],"id": 1}');
        xhttp_reply("200", "OK", "application/json", "$jsonrpl(body)");
        exit;
    }

stats.get_statistics has lots of metrics, the problematic ones we have found are:

    "dialog.active_dialogs":  "84",
    "dialog.early_dialogs": "16",

The issue is that randomly, instead of reporting the real active/early dialogs, it will return 9223372036854776000.

Example:

...
    "dialog.active_dialogs":  "77",
    "dialog.early_dialogs": "9223372036854776000",
...

...
    "dialog.active_dialogs":  "9223372036854776000",
    "dialog.early_dialogs": "20",
...

...
    "dialog.active_dialogs":  "83",
    "dialog.early_dialogs": "18",
...

Troubleshooting

I added logging for kamailio to print the value for active-dialogs just to make sure it wasn't a problem further down in the pipeline, and I could see that same number, so it's definitely coming from Kamailio. Also, on version v5.0.2 installed from deb repo this did not happen.

Reproduction

I cannot reproduce on purpose, I see it happen several times a day though.

Debugging

Please let me know what I can provide to troubleshoot this, I don't know where to start, I'm concerned of activating debug logs in prod and waiting for this to happen might overload the system, any alternatives?

Additional Information

# kamailio -v
version: kamailio 5.1.4 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled with gcc 6.3.0

OS: Debian stretch 9.4

Kernel: Linux kamailio 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.