[kamailio/kamailio] KEMI: Add htable count/match API equivalent to $shtcn() (to avoid pv_cache growth with dynamic PV strings) (Issue #4426) - sr-dev

8 Oct 2025


      wedhornsipgate created an issue (kamailio/kamailio#4426)
**Description:**
We're missing a KEMI function in the htable module that provides the same functionality as the PV function `$shtcn(table=>pattern)` to count the number of keys in an htable that match a given pattern (prefix/suffix/regex). The goal is to let KEMI scripts perform pattern counts without constructing dynamic PV expressions, which currently causes unbounded pv_cache growth in PKG memory.
**Use case / motivation:**
We implemented rolling-window rate limiting by writing short-lived, per-call entries into an htable and then counting keys that match a number-derived pattern. Using $shtcn() works well for the logic, but in KEMI (JS/TypeScript) we must call it via pv.gete("$shtcn(...)"). Because the PV string varies per INVITE (different pattern/key each time), these dynamic PV expressions accumulate in the PV cache, growing PKG/private memory over time.
We’d like a native KEMI function to perform the same count/match so we can avoid the PV layer entirely.
**Environment**
Kamailio: 5.8.6 on Debian 12
Scripting: KEMI (JavaScript/TypeScript)
Symptom: steady PKG/private memory increase over ~24h under load when using pv.gete("$shtcn(...)"). Disabling the $shtcn() call stops the growth.
Related log line: the one discussed in [#3440](https://github.com/kamailio/kamailio/issues/3440) (“pv cache … pkg memory may get filled with pv declarations”).
Current line in our script:
```if (ksr.pv.gete(`$shtcn(${htable_count}=>%~${number})`) > threshold) { ... }```
**Current workaround**
Restarting the Kamailio every day or disabling rate limit on instances with KEMI.
**Idea for API**
Minimal, PV-parity:
```
// KEMI export
int shtcn_match(str table, str pattern, int mode /* 0=exact, 1=prefix, 2=suffix, 3=regex */);
```
JS example:
`const n = KSR.htable.shtcn_match("from_number_count", number, 3); // regex`
More explicit helpers (avoids integer modes):
```
KSR.htable.shtcn_exact(table, key)      -> int
KSR.htable.shtcn_prefix(table, prefix)  -> int
KSR.htable.shtcn_suffix(table, suffix)  -> int
KSR.htable.shtcn_regex(table, pattern)  -> int
```
**Behavioral notes / expectations**
Should mirror $shtcn() semantics where reasonable (esp. regex behavior), but returning an integer directly to KEMI.
Must not allocate PV expressions or interact with pv_cache.
**Why this belongs in htable KEMI**
$shtcn() is already offered at the PV level; parity in KEMI keeps feature consistency across scripting engines.
It enables memory-safe implementations of rolling-window rate limits and similar patterns (without touching PV).
**Alternatives considered**
Replace pattern counting with per-bucket counters (sht_inc), which leads to other problems like increment/decrement handling and locking.
Use ratelimit/pike modules—useful for IP/method limits, but our use case is PAI/From/To based and currently hinges on pattern counts.
**Acceptance criteria**
A KEMI function exists to count matching keys in an htable without PV parsing.
Verified that repeated calls with dynamic patterns do not grow PV cache / PKG memory.
Basic tests for exact/prefix/suffix/regex modes (and skip_expired if implemented).
Thank you!
This would let KEMI users keep $shtcn()-style logic with the same performance characteristics, but without the PV-cache side effects.
-- 
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/4426
You are receiving this because you are subscribed to this thread.

Message ID: kamailio/kamailio/issues/4426@github.com