15 Oct
2013
15 Oct
'13
5:12 a.m.
Hi Daniel,
Thanks for the reply again. Looking at the email history, I'm not sure how
we decided it was definitely a pkg memory problem. What we see is the
output of "ps aux" as follows:
root@sip0-test:~# ps aux | egrep -i "kamailio|mem"
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 6794 0.0 0.0 22480 1868 ? Ss Oct02 0:12
/opt/testuser/current/sbin/testuser_safe_kamailio
testuser 6822 0.0 0.2 556528 4580 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6824 0.3 8.7 825552 180244 ? S Oct02 56:40
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6825 0.3 8.7 825536 180776 ? S Oct02 56:20
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6826 0.3 8.7 825912 180296 ? S Oct02 55:54
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6827 0.3 8.7 825744 180580 ? S Oct02 56:19
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6828 0.3 8.7 825536 180092 ? S Oct02 56:25
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6829 0.3 8.7 825536 180632 ? S Oct02 56:21
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6830 0.3 8.7 825472 180968 ? S Oct02 56:37
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6831 0.3 8.7 825276 180272 ? S Oct02 56:41
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6832 0.0 0.0 556528 1324 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6833 0.0 0.0 556528 1324 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6834 0.0 0.0 556528 1324 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6835 0.0 0.0 556528 1324 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6836 0.0 0.0 556528 1324 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6837 0.0 0.0 556528 1324 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6838 0.0 0.0 556528 1324 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6839 0.0 0.0 556528 1324 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6840 0.0 0.0 556528 1776 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6841 0.0 0.0 556528 1780 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6842 0.0 0.0 556528 1780 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6843 0.0 0.0 556528 1328 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6844 0.0 0.0 556528 1780 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6845 0.0 0.0 556528 1328 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6846 0.0 0.0 556528 1328 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6847 0.0 0.0 556528 1328 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6848 0.0 0.0 556528 1676 ? S Oct02 0:02
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6849 0.0 0.1 556528 3568 ? S Oct02 5:28
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6850 0.0 0.0 556612 1600 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6851 0.0 0.0 556532 1188 ? S Oct02 0:00
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
testuser 6852 0.0 0.0 556528 1360 ? S Oct02 0:02
/sbin/kamailio -m 512 -P /var/run/testuser/kamailio.pid
You'll see for example that process 6824 is using 8.7% of memory, which is
much more than it was using 5 days ago. Yet if I run the same sercmd again
I get (exactly!) the same numbers:
root@sip0-test:~# sercmd pkg.stats pid 6824
{
entry: 1
pid: 6824
rank: 1
used: 209836
free: 3704080
real_used: 490224
}
Any ideas?
On 12 October 2013 00:23, Daniel-Constantin Mierla <miconda(a)gmail.com>wrote;wrote:
Hi David,
On 10/10/13 11:36 PM, David Cunningham wrote:
--
David Cunningham, Voisonics
http://voisonics.com/
USA: +1 213 221 1092
UK: +44 (0) 20 3298 1642
Australia: +61 (0) 2 8063 9019
Hi Daniel,
Thanks for the reply. Perhaps what we're seeing is normal, and the memory
use is meant to increase as time progresses. Would you expect to see an
ongoing memory use increase, or when should it stop increasing?
private memory (pkg) should stay rather constant. It increases when
there is a
sip message processed, but once is sent out, it should come back
around the average.
There are couple of functions that can fill the private memory and keep it
up, such as doing an sql_query() that returns a big data and the result is
not freed (sql_result_free()). It is not actually a leak as the next
sql_query() will free previous result, but in case you have such query for
some corner case that is not executed frequently, then the memory can stay
filled in. Another example will be storing very large value in a $var(...)
(e.g., $var(x) ).
This is private memory, per process, which is meant for temporary
operations. Shared memory (shm) can increase over the time, being the place
where the dynamic data required at runtime is stored (e.g., location
records, hash tables, transactions) - so as you get more traffic or more
phones using kamailio, more shm is used. But your problem was reported for
pkg.
Anyhow, keep an eye on the pkg.stats and if you see constant increase
which is substantial, then get a mem log dump.
Cheers,
Daniel
--
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda -
http://www.linkedin.com/in/**miconda<http://www.linkedin.com/in/miconda&…
Kamailio Advanced Trainings - Berlin, Nov 25-28; Miami, Nov 18-20, 2013
- more details about Kamailio trainings at http://www.asipto.com -