[sr-dev] SHM memory usage Kamailio 1.4.4

[Robin Vleij]

Hi guys,

Since 1.3.0 (now running 1.4.4) I'm seeing a very slow uptake of SHM
memory on our low traffic setup (less than 5 cps per machine). I'm
looking for some basis to go further on in my research to the cause. :)

I compiled Kamailio 1.4.4-notls with #define SHM_MEM_SIZE 4*32 in
config.h in production. For my testing setup I'm running on the standard
32 there.

After about 3 weeks uptime I start top, sort on memory size and find the
kamailio processes (I'm running with 16 children) to all have about 40mb
in the SHR column. My understanding is that this should also go down,
but it only goes up, slowly. More CPS (for example a benchmark using
sipp) makes this go up faster, but it never seems to go down this
figure. I think this is wrong, but I could be wrong myself. :)

On a seperate machine with no traffic I compiled the memory debugging
according to the "memory troubleshooting" page on the wiki. LOTS of info
in the logs. Also ran with valgrind, didn't find anything interesting
(but I'm no dev myself really).

My plan now is to take away our acc module (compiled with radius
support) and see if it's maybe that module that's causing this. My test
on this traffic-less machine is as follows: start, run 20cps for a while
(we do no registers, just routing and auth) and note the SHR data from
top. Then according to my understanding this figure should drop down
after a period of 20 minutes with no traffic. Is this a right assumption?

On the test setup the top data looks like this after about 10 calls:

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
15975 root      18   0 69240 1536  568 S  0.0  0.6   0:00.00 kamailio
15972 root      18   0 69240 1568  588 S  0.0  0.6   0:00.00 kamailio
15970 root      18   0 69240 1568  588 S  0.0  0.6   0:00.00 kamailio
15969 root      19   0 69244 1488  568 S  0.0  0.6   0:00.00 kamailio
15968 root      15   0 69240 1860  936 S  0.0  0.7   0:00.00 kamailio
15967 root      15   0 71436 3376 2268 S  0.0  1.3   0:00.00 kamailio
15966 root      15   0 71436 3396 2288 S  0.0  1.3   0:00.00 kamailio
15965 root      23   0 69240 5552 4640 S  0.0  2.1   0:00.02 kamailio

As far as I know, it never goes down, the SHR entries. When running with
very little SHM i config.h, the process goes out of shm memory and
complains, as expected.

Are my assumptions about all of this correct?

-- 
Robin Vleij
robin at swip.net