Hello,
We use Kamailio as a stateless load balancer. It runs on a VM, with 512 MB RAM, and it seems to have some kind of memory leak. The kamailio processes consume all available memory, until Kamailio crashes and restart. This occurs about once a day. Since it's stateless, and the restart is very fast, it's not a big deal, but I'd like to fix it. The strange thing is that I updated kamailio from 1.5.3 to 3.0.3 and the problem still ocurrs. I followed the instructions on http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory in order to determine the cause, and stopped the traffic on the proxy. The allocated memory were not freed, after 1 hour the amount of free memory is still the same. Anybody has an ideia about what's happening, or how can I debug it?
Thank you!
Santiago Soares
Hello
Can you give us more information on the modules you are loading/ your specific config ?
Marius ________________________________________ From: sr-users-bounces@lists.sip-router.org [sr-users-bounces@lists.sip-router.org] On Behalf Of Santiago Soares [santiagosoares@gmail.com] Sent: Monday, September 20, 2010 9:21 PM To: SR-Users; users@lists.kamailio.org Subject: [SR-Users] Memory leak
Hello,
We use Kamailio as a stateless load balancer. It runs on a VM, with 512 MB RAM, and it seems to have some kind of memory leak. The kamailio processes consume all available memory, until Kamailio crashes and restart. This occurs about once a day. Since it's stateless, and the restart is very fast, it's not a big deal, but I'd like to fix it. The strange thing is that I updated kamailio from 1.5.3 to 3.0.3 and the problem still ocurrs. I followed the instructions on http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory in order to determine the cause, and stopped the traffic on the proxy. The allocated memory were not freed, after 1 hour the amount of free memory is still the same. Anybody has an ideia about what's happening, or how can I debug it?
Thank you!
Santiago Soares
On 09/21/2010 09:18 AM, Marius Zbihlei wrote:
Hello
Can you give us more information on the modules you are loading/ your specific config ?
Marius
Also, is it shared memory or private memory? If you have a dump of the memory distribution, can you spot anything out of the ordinary. As a tip look for functions in the ending part of the dump that repeat for multiple times. The lower offset is usually alloc'ed during module initialization so any big leak will happen at higher offsets (I don't know if this is the rule but this is what I observed).
Have you tried increasing the memory pool?
Cheers Marius
From: sr-users-bounces@lists.sip-router.org [sr-users-bounces@lists.sip-router.org] On Behalf Of Santiago Soares [santiagosoares@gmail.com] Sent: Monday, September 20, 2010 9:21 PM To: SR-Users; users@lists.kamailio.org Subject: [SR-Users] Memory leak
Hello,
We use Kamailio as a stateless load balancer. It runs on a VM, with 512 MB RAM, and it seems to have some kind of memory leak. The kamailio processes consume all available memory, until Kamailio crashes and restart. This occurs about once a day. Since it's stateless, and the restart is very fast, it's not a big deal, but I'd like to fix it. The strange thing is that I updated kamailio from 1.5.3 to 3.0.3 and the problem still ocurrs. I followed the instructions on http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory in order to determine the cause, and stopped the traffic on the proxy. The allocated memory were not freed, after 1 hour the amount of free memory is still the same. Anybody has an ideia about what's happening, or how can I debug it?
Thank you!
Santiago Soares
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
For a shorter version of memory status you can set memory_summary=2 or 3:
http://www.kamailio.org/dokuwiki/doku.php/core-cookbook:3.0.x#mem_summary
This will show quickly if there is a leak. Cheers, Daniel
On 9/21/10 9:57 AM, marius zbihlei wrote:
On 09/21/2010 09:18 AM, Marius Zbihlei wrote:
Hello
Can you give us more information on the modules you are loading/ your specific config ?
Marius
Also, is it shared memory or private memory? If you have a dump of the memory distribution, can you spot anything out of the ordinary. As a tip look for functions in the ending part of the dump that repeat for multiple times. The lower offset is usually alloc'ed during module initialization so any big leak will happen at higher offsets (I don't know if this is the rule but this is what I observed).
Have you tried increasing the memory pool?
Cheers Marius
From: sr-users-bounces@lists.sip-router.org [sr-users-bounces@lists.sip-router.org] On Behalf Of Santiago Soares [santiagosoares@gmail.com] Sent: Monday, September 20, 2010 9:21 PM To: SR-Users; users@lists.kamailio.org Subject: [SR-Users] Memory leak
Hello,
We use Kamailio as a stateless load balancer. It runs on a VM, with 512 MB RAM, and it seems to have some kind of memory leak. The kamailio processes consume all available memory, until Kamailio crashes and restart. This occurs about once a day. Since it's stateless, and the restart is very fast, it's not a big deal, but I'd like to fix it. The strange thing is that I updated kamailio from 1.5.3 to 3.0.3 and the problem still ocurrs. I followed the instructions on http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory in order to determine the cause, and stopped the traffic on the proxy. The allocated memory were not freed, after 1 hour the amount of free memory is still the same. Anybody has an ideia about what's happening, or how can I debug it?
Thank you!
Santiago Soares
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Thanks for the answers. Here are the config file and the memory status right now. I didn't change the memory pool. I don't know how to interpret the memory log. Can you help me?
Config file
# ----------- global configuration parameters ------------------------ # vim:bg=dark
check_via=no # (cmd. line: -v) dns=no # (cmd. line: -r) rev_dns=no # (cmd. line: -R) dns_cache_init=no use_dns_cache=no disable_tcp=yes server_header="Server: SIP ROUTER" user_agent_header="User-Agent: SIP ROUTER" sip_warning=0 debug=2 memlog=1
port=5060
# ------------------ module loading ----------------------------------
mpath="/usr/local/lib/kamailio/modules/:/usr/local/lib/kamailio/modules_k/"
loadmodule "maxfwd.so" loadmodule "exec.so" loadmodule "tm.so" loadmodule "sl.so" loadmodule "dispatcher.so" loadmodule "textops.so" loadmodule "xlog.so" loadmodule "mi_fifo.so" loadmodule "regex.so" loadmodule "pv.so"
# ----------- setting module-specific parameters ---------------------
modparam("exec", "setvars", 0) modparam("dispatcher", "list_file", "/usr/local/etc/kamailio/servers.list") modparam("mi_fifo", "fifo_name", "/tmp/kamailio_fifo") modparam("dispatcher", "ds_ping_interval", 5) modparam("dispatcher", "ds_probing_mode", 1) modparam("dispatcher", "ds_probing_threshhold", 3) modparam("dispatcher", "flags", 2) modparam("tm", "fr_timer", 3)
# ------------------ request routing logic ---------------------------
route {
# initial sanity checks: # messages with max_forwards==0 if (!mf_process_maxfwd_header("10")) { sl_send_reply("483","Too Many Hops"); exit; };
# excessively long requests if (msg:len >= 2048 ) { sl_send_reply("513", "Message too big"); exit; };
# excessively long display name if (!pcre_match("$fn","^.{0,62}$$")) { sl_send_reply("403", "Forbidden - Display Name Max Size Exceeded"); exit; };
#Register if (method=="REGISTER") { if (from_uri=~".*(<?sip:[a-z][a-z0-9_.]{3,11}(\$[0-9]{1,3})?@voiprovider\.com(:[0-9]{4})?(;[^>]+)?>?) ?(;tag=[a-zA-Z0-9]{4,}.*)?$" ||
from_uri=~".*(<?sip:\$voipmonitor@voiprovider\.com(:[0-9]{4})?(;[^>]+)?>?) ?(;tag=[a-zA-Z0-9]{4,}.*)?$") { route(4); exit; }; sl_send_reply("403","Forbidden - Wrong domain or Username format"); exit; };
#Deny OPTIONS if (method=="OPTIONS") { sl_send_reply("404", "Not found"); exit; };
#Deny PING if (method=="PING") { sl_send_reply("404", "Not found"); exit; };
# INVITE if (method=="INVITE") {
if (from_uri=~".*(<?sip:[a-z][a-z0-9_.]{3,11}(\$[0-9]{1,3})?@voiprovider\.com(:[0-9]{4})?(;[^>]+)?>?) ?(;tag=[a-zA-Z0-9]{4,}.*)?$" || from_uri=~"@equip[0-9][0-9]?.voiprovider.com($|;|:|>)" || from_uri=~"@10.0.0.1($|;|:|>)" || from_uri=~"@10.0.0.2($|;|:|>)" || from_uri=~"@10.0.0.3($|;|:|>)") {
# Skype if (to_uri=~"#skype@") { route(2); exit; };
# Skype if (to_uri=~"%23skype@") { route(2); exit; };
# default route(1); exit;
};
sl_send_reply("403","Forbidden - Wrong domain or Username format"); exit;
};
# Default route(1); exit; } #Default (softswitches) route[1] { ds_select_dst("1", "0"); forward(); exit; } #Skype route[2] { ds_select_dst("2", "0"); forward(); exit; }
#Register route[4] { ds_select_dst("4", "0"); forward(); exit; }
Memory log:
Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: <core> [main.c:691]: Memory status (pkg): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: heap size= 4194304 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 126576, used+overhead=169192, free=4025112 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: dumping free list: Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 4 fragments no.: 1, unused: 0 bucket size: 32 - 32 (first 32) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3989736) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: TOTAL: 37 free fragments = 4025112 free bytes Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: ----------------------------- Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: <core> [main.c:703]: Memory status (shm): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: fm_status (0x2b982b03b000): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: heap size= 33554432 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 3071760, used+overhead=3109608, free=30444824 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: max used (+overhead)= 3235640 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: dumping free list: Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 3 fragments no.: 2, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 5 fragments no.: 8, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 8 fragments no.: 19, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 27 fragments no.: 1, unused: 0 bucket size: 216 - 216 (first 216) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 37 fragments no.: 1, unused: 0 bucket size: 296 - 296 (first 296) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 38 fragments no.: 22, unused: 0 bucket size: 304 - 304 (first 304) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 39 fragments no.: 2, unused: 0 bucket size: 312 - 312 (first 312) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 40 fragments no.: 1, unused: 0 bucket size: 320 - 320 (first 320) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 583 fragments no.: 25, unused: 0 bucket size: 4664 - 4664 (first 4664) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2059 fragments no.: 1, unused: 0 bucket size: 16777216 - 33554432 (first 30318496) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: TOTAL: 82 free fragments = 30444824 free bytes Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: -----------------------------
Santiago Soares
On Tue, Sep 21, 2010 at 5:29 AM, Daniel-Constantin Mierla <miconda@gmail.com
wrote:
For a shorter version of memory status you can set memory_summary=2 or 3:
http://www.kamailio.org/dokuwiki/doku.php/core-cookbook:3.0.x#mem_summary
This will show quickly if there is a leak. Cheers, Daniel
On 9/21/10 9:57 AM, marius zbihlei wrote:
On 09/21/2010 09:18 AM, Marius Zbihlei wrote:
Hello
Can you give us more information on the modules you are loading/ your specific config ?
Marius
Also, is it shared memory or private memory? If you have a dump of the memory distribution, can you spot anything out of the ordinary. As a tip look for functions in the ending part of the dump that repeat for multiple times. The lower offset is usually alloc'ed during module initialization so any big leak will happen at higher offsets (I don't know if this is the rule but this is what I observed).
Have you tried increasing the memory pool?
Cheers Marius
From: sr-users-bounces@lists.sip-router.org [ sr-users-bounces@lists.sip-router.org] On Behalf Of Santiago Soares [ santiagosoares@gmail.com] Sent: Monday, September 20, 2010 9:21 PM To: SR-Users; users@lists.kamailio.org Subject: [SR-Users] Memory leak
Hello,
We use Kamailio as a stateless load balancer. It runs on a VM, with 512 MB RAM, and it seems to have some kind of memory leak. The kamailio processes consume all available memory, until Kamailio crashes and restart. This occurs about once a day. Since it's stateless, and the restart is very fast, it's not a big deal, but I'd like to fix it. The strange thing is that I updated kamailio from 1.5.3 to 3.0.3 and the problem still ocurrs. I followed the instructions on http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory in order to determine the cause, and stopped the traffic on the proxy. The allocated memory were not freed, after 1 hour the amount of free memory is still the same. Anybody has an ideia about what's happening, or how can I debug it?
Thank you!
Santiago Soares
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla http://www.asipto.com
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hello,
is it compile with memory debug on? The status shows it is still fast malloc.
Cheers, Daniel
On 9/21/10 4:41 PM, Santiago Soares wrote:
Thanks for the answers. Here are the config file and the memory status right now. I didn't change the memory pool. I don't know how to interpret the memory log. Can you help me?
Config file
# ----------- global configuration parameters ------------------------ # vim:bg=dark
check_via=no # (cmd. line: -v) dns=no # (cmd. line: -r) rev_dns=no # (cmd. line: -R) dns_cache_init=no use_dns_cache=no disable_tcp=yes server_header="Server: SIP ROUTER" user_agent_header="User-Agent: SIP ROUTER" sip_warning=0 debug=2 memlog=1
port=5060
# ------------------ module loading ----------------------------------
mpath="/usr/local/lib/kamailio/modules/:/usr/local/lib/kamailio/modules_k/"
loadmodule "maxfwd.so" loadmodule "exec.so" loadmodule "tm.so" loadmodule "sl.so" loadmodule "dispatcher.so" loadmodule "textops.so" loadmodule "xlog.so" loadmodule "mi_fifo.so" loadmodule "regex.so" loadmodule "pv.so"
# ----------- setting module-specific parameters ---------------------
modparam("exec", "setvars", 0) modparam("dispatcher", "list_file", "/usr/local/etc/kamailio/servers.list") modparam("mi_fifo", "fifo_name", "/tmp/kamailio_fifo") modparam("dispatcher", "ds_ping_interval", 5) modparam("dispatcher", "ds_probing_mode", 1) modparam("dispatcher", "ds_probing_threshhold", 3) modparam("dispatcher", "flags", 2) modparam("tm", "fr_timer", 3)
# ------------------ request routing logic ---------------------------
route {
# initial sanity checks: # messages with max_forwards==0 if (!mf_process_maxfwd_header("10")) { sl_send_reply("483","Too Many Hops"); exit; }; # excessively long requests if (msg:len >= 2048 ) { sl_send_reply("513", "Message too big"); exit; }; # excessively long display name if (!pcre_match("$fn","^.{0,62}$$")) { sl_send_reply("403", "Forbidden - Display Name MaxSize Exceeded"); exit; };
#Register if (method=="REGISTER") { if(from_uri=~".*(<?sip:[a-z][a-z0-9_.]{3,11}(\$[0-9]{1,3})?@voiprovider\.com(:[0-9]{4})?(;[^>]+)?>?) ?(;tag=[a-zA-Z0-9]{4,}.*)?$" ||
from_uri=~".*(<?sip:\$voipmonitor@voiprovider\.com(:[0-9]{4})?(;[^>]+)?>?) ?(;tag=[a-zA-Z0-9]{4,}.*)?$") { route(4); exit; }; sl_send_reply("403","Forbidden - Wrong domain or Username format"); exit; };
#Deny OPTIONS if (method=="OPTIONS") { sl_send_reply("404", "Not found"); exit; }; #Deny PING if (method=="PING") { sl_send_reply("404", "Not found"); exit; }; # INVITE if (method=="INVITE") { if(from_uri=~".*(<?sip:[a-z][a-z0-9_.]{3,11}(\$[0-9]{1,3})?@voiprovider\.com(:[0-9]{4})?(;[^>]+)?>?) ?(;tag=[a-zA-Z0-9]{4,}.*)?$" ||
from_uri=~"@equip[0-9][0-9]?.voiprovider.com($|;|:|>)" || from_uri=~"@10.0.0.1 http://10.0.0.1($|;|:|>)" || from_uri=~"@10.0.0.2 http://10.0.0.2($|;|:|>)" || from_uri=~"@10.0.0.3 http://10.0.0.3($|;|:|>)") {
# Skype if (to_uri=~"\#skype@") { route(2); exit; }; # Skype if (to_uri=~"%23skype@") { route(2); exit; }; # default route(1); exit; }; sl_send_reply("403","Forbidden - Wrong domain orUsername format"); exit;
}; # Default route(1); exit;} #Default (softswitches) route[1] { ds_select_dst("1", "0"); forward(); exit; } #Skype route[2] { ds_select_dst("2", "0"); forward(); exit; }
#Register route[4] { ds_select_dst("4", "0"); forward(); exit; }
Memory log:
Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: <core> [main.c:691]: Memory status (pkg): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: heap size= 4194304 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 126576, used+overhead=169192, free=4025112 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: dumping free list: Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 4 fragments no.: 1, unused: 0 bucket size: 32 - 32 (first 32) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3989736) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: TOTAL: 37 free fragments = 4025112 free bytes Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: ----------------------------- Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: <core> [main.c:703]: Memory status (shm): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: fm_status (0x2b982b03b000): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: heap size= 33554432 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 3071760, used+overhead=3109608, free=30444824 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: max used (+overhead)= 3235640 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: dumping free list: Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 3 fragments no.: 2, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 5 fragments no.: 8, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 8 fragments no.: 19, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 27 fragments no.: 1, unused: 0 bucket size: 216 - 216 (first 216) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 37 fragments no.: 1, unused: 0 bucket size: 296 - 296 (first 296) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 38 fragments no.: 22, unused: 0 bucket size: 304 - 304 (first 304) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 39 fragments no.: 2, unused: 0 bucket size: 312 - 312 (first 312) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 40 fragments no.: 1, unused: 0 bucket size: 320 - 320 (first 320) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 583 fragments no.: 25, unused: 0 bucket size: 4664 - 4664 (first 4664) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2059 fragments no.: 1, unused: 0 bucket size: 16777216 - 33554432 (first 30318496) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: TOTAL: 82 free fragments = 30444824 free bytes Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: -----------------------------
Santiago Soares
On Tue, Sep 21, 2010 at 5:29 AM, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:
For a shorter version of memory status you can set memory_summary=2 or 3: http://www.kamailio.org/dokuwiki/doku.php/core-cookbook:3.0.x#mem_summary This will show quickly if there is a leak. Cheers, Daniel On 9/21/10 9:57 AM, marius zbihlei wrote: On 09/21/2010 09:18 AM, Marius Zbihlei wrote: Hello Can you give us more information on the modules you are loading/ your specific config ? Marius Also, is it shared memory or private memory? If you have a dump of the memory distribution, can you spot anything out of the ordinary. As a tip look for functions in the ending part of the dump that repeat for multiple times. The lower offset is usually alloc'ed during module initialization so any big leak will happen at higher offsets (I don't know if this is the rule but this is what I observed). Have you tried increasing the memory pool? Cheers Marius ________________________________________ From: sr-users-bounces@lists.sip-router.org <mailto:sr-users-bounces@lists.sip-router.org> [sr-users-bounces@lists.sip-router.org <mailto:sr-users-bounces@lists.sip-router.org>] On Behalf Of Santiago Soares [santiagosoares@gmail.com <mailto:santiagosoares@gmail.com>] Sent: Monday, September 20, 2010 9:21 PM To: SR-Users; users@lists.kamailio.org <mailto:users@lists.kamailio.org> Subject: [SR-Users] Memory leak Hello, We use Kamailio as a stateless load balancer. It runs on a VM, with 512 MB RAM, and it seems to have some kind of memory leak. The kamailio processes consume all available memory, until Kamailio crashes and restart. This occurs about once a day. Since it's stateless, and the restart is very fast, it's not a big deal, but I'd like to fix it. The strange thing is that I updated kamailio from 1.5.3 to 3.0.3 and the problem still ocurrs. I followed the instructions on http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory in order to determine the cause, and stopped the traffic on the proxy. The allocated memory were not freed, after 1 hour the amount of free memory is still the same. Anybody has an ideia about what's happening, or how can I debug it? Thank you! Santiago Soares _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla http://www.asipto.com _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
On 09/21/2010 05:41 PM, Santiago Soares wrote:
Thanks for the answers. Here are the config file and the memory status right now. I didn't change the memory pool. I don't know how to interpret the memory log. Can you help me?
Hello
Yes sure. But you still haven't answer my question. Is it private memory or shared memory ? Have you any log entries that are useful for debugging?
I think pkg memory is all right
Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 126576, used+overhead=169192, free=4025112
SHM mem looks as well pretty good:
Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 3071760, used+overhead=3109608, free=30444824
I think you need to compile Kamailio with MEMDBG=1 (Makefile.defs - I think Daniel already suggested that)
Please send the SIGUSR1 to the main process when the mem leak condition appears.
Marius
Memory log:
Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: <core> [main.c:691]: Memory status (pkg): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: heap size= 4194304 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 126576, used+overhead=169192, free=4025112 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: dumping free list: Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 4 fragments no.: 1, unused: 0 bucket size: 32 - 32 (first 32) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3989736) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: TOTAL: 37 free fragments = 4025112 free bytes Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: ----------------------------- Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: <core> [main.c:703]: Memory status (shm): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: fm_status (0x2b982b03b000): Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: heap size= 33554432 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 3071760, used+overhead=3109608, free=30444824 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: max used (+overhead)= 3235640 Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: dumping free list: Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 3 fragments no.: 2, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 5 fragments no.: 8, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 8 fragments no.: 19, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 27 fragments no.: 1, unused: 0 bucket size: 216 - 216 (first 216) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 37 fragments no.: 1, unused: 0 bucket size: 296 - 296 (first 296) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 38 fragments no.: 22, unused: 0 bucket size: 304 - 304 (first 304) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 39 fragments no.: 2, unused: 0 bucket size: 312 - 312 (first 312) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 40 fragments no.: 1, unused: 0 bucket size: 320 - 320 (first 320) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 583 fragments no.: 25, unused: 0 bucket size: 4664 - 4664 (first 4664) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2059 fragments no.: 1, unused: 0 bucket size: 16777216 - 33554432 (first 30318496) Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: TOTAL: 82 free fragments = 30444824 free bytes Sep 21 11:31:02 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: -----------------------------
Santiago Soares
Am 20.09.2010 21:21, schrieb Santiago Soares:
Hello,
We use Kamailio as a stateless load balancer. It runs on a VM, with 512 MB RAM, and it seems to have some kind of memory leak. The kamailio processes consume all available memory, until Kamailio crashes and restart.
What do you mean with "crash"? Usually if Kamailio runs out of memory it logs error messages but it does not crash.
How do you verify the memory consumption? With top or other tools? If the memory consumption reported by the OS is increasing then there is a problem with an external library.
Kamailio internally allocates at startup a fixed size of memory which is never increased. If system memory consumption is increasing then it is by an external library which does not use Kamailio's memory allocator.
So what exactly is happening? Is Kamilio just running out of pre-allocated memory or is system-memory increasing? How does it crash in detail?
regards Klaus
This occurs about once a day. Since it's stateless, and the restart is very fast, it's not a big deal, but I'd like to fix it. The strange thing is that I updated kamailio from 1.5.3 to 3.0.3 and the problem still ocurrs. I followed the instructions on http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory in order to determine the cause, and stopped the traffic on the proxy. The allocated memory were not freed, after 1 hour the amount of free memory is still the same. Anybody has an ideia about what's happening, or how can I debug it?
Thank you!
Santiago Soares
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Thank's for the answers. Based on your explanation, I think that the problem is an external library. The kamailio process is taking all the system memory. When the system runs out of memory, it kills kamailio. But how can I find out what is the library? Do I still have to recompile it with memory debug on? This is what I have on /var/log/messages when kamailio crashes:
Sep 21 22:15:21 siprouter kernel: kamailio invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Sep 21 22:15:21 siprouter kernel: Sep 21 22:15:21 siprouter kernel: Call Trace: Sep 21 22:15:21 siprouter kernel: [<ffffffff800c7161>] out_of_memory+0x8e/0x2f3 Sep 21 22:15:21 siprouter kernel: [<ffffffff8000f540>] __alloc_pages+0x27f/0x308 Sep 21 22:15:21 siprouter kernel: [<ffffffff80012f11>] __do_page_cache_readahead+0x96/0x179 Sep 21 22:15:21 siprouter kernel: [<ffffffff800138c9>] filemap_nopage+0x14c/0x360 Sep 21 22:15:21 siprouter kernel: [<ffffffff8000895e>] __handle_mm_fault+0x1fb/0x1039 Sep 21 22:15:21 siprouter kernel: [<ffffffff80066b55>] do_page_fault+0x4cb/0x874 Sep 21 22:15:21 siprouter kernel: [<ffffffff8001ad22>] remove_vma+0x5d/0x64 Sep 21 22:15:21 siprouter kernel: [<ffffffff8005dde9>] error_exit+0x0/0x84 Sep 21 22:15:21 siprouter kernel: Sep 21 22:15:21 siprouter kernel: Mem-info: Sep 21 22:15:22 siprouter kernel: Node 0 DMA per-cpu: Sep 21 22:15:22 siprouter kernel: cpu 0 hot: high 0, batch 1 used:0 Sep 21 22:15:22 siprouter kernel: cpu 0 cold: high 0, batch 1 used:0 Sep 21 22:15:22 siprouter kernel: cpu 1 hot: high 0, batch 1 used:0 Sep 21 22:15:22 siprouter kernel: cpu 1 cold: high 0, batch 1 used:0 Sep 21 22:15:22 siprouter kernel: cpu 2 hot: high 0, batch 1 used:0 Sep 21 22:15:22 siprouter kernel: cpu 2 cold: high 0, batch 1 used:0 Sep 21 22:15:22 siprouter kernel: cpu 3 hot: high 0, batch 1 used:0 Sep 21 22:15:22 siprouter kernel: cpu 3 cold: high 0, batch 1 used:0 Sep 21 22:15:22 siprouter kernel: Node 0 DMA32 per-cpu: Sep 21 22:15:22 siprouter kernel: cpu 0 hot: high 186, batch 31 used:106 Sep 21 22:15:22 siprouter kernel: cpu 0 cold: high 62, batch 15 used:46 Sep 21 22:15:22 siprouter kernel: cpu 1 hot: high 186, batch 31 used:58 Sep 21 22:15:22 siprouter kernel: cpu 1 cold: high 62, batch 15 used:47 Sep 21 22:15:22 siprouter kernel: cpu 2 hot: high 186, batch 31 used:48 Sep 21 22:15:22 siprouter kernel: cpu 2 cold: high 62, batch 15 used:15 Sep 21 22:15:22 siprouter kernel: cpu 3 hot: high 186, batch 31 used:170 Sep 21 22:15:22 siprouter kernel: cpu 3 cold: high 62, batch 15 used:36 Sep 21 22:15:22 siprouter kernel: Node 0 Normal per-cpu: empty Sep 21 22:15:22 siprouter kernel: Node 0 HighMem per-cpu: empty Sep 21 22:15:22 siprouter kernel: Free pages: 4788kB (0kB HighMem) Sep 21 22:15:22 siprouter kernel: Active:117857 inactive:673 dirty:0 writeback:0 unstable:0 free:1197 slab:3158 mapped-file:956 mapped-anon:116864 pagetables:1417 Sep 21 22:15:22 siprouter kernel: Node 0 DMA free:2012kB min:56kB low:68kB high:84kB active:1724kB inactive:0kB present:10520kB pages_scanned:30577 all_unreclaimable? yes Sep 21 22:15:22 siprouter kernel: lowmem_reserve[]: 0 489 489 489 Sep 21 22:15:22 siprouter kernel: Node 0 DMA32 free:2776kB min:2800kB low:3500kB high:4200kB active:469832kB inactive:2564kB present:500896kB pages_scanned:2204224 all_unreclaimable? yes Sep 21 22:15:22 siprouter kernel: lowmem_reserve[]: 0 0 0 0 Sep 21 22:15:22 siprouter kernel: Node 0 Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Sep 21 22:15:22 siprouter kernel: lowmem_reserve[]: 0 0 0 0 Sep 21 22:15:22 siprouter kernel: Node 0 HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Sep 21 22:15:22 siprouter kernel: lowmem_reserve[]: 0 0 0 0 Sep 21 22:15:22 siprouter kernel: Node 0 DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2012kB Sep 21 22:15:22 siprouter kernel: Node 0 DMA32: 28*4kB 1*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2776kB Sep 21 22:15:22 siprouter kernel: Node 0 Normal: empty Sep 21 22:15:22 siprouter kernel: Node 0 HighMem: empty Sep 21 22:15:22 siprouter kernel: 1796 pagecache pages Sep 21 22:15:22 siprouter kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0 Sep 21 22:15:22 siprouter kernel: Free swap = 0kB Sep 21 22:15:22 siprouter kernel: Total swap = 0kB Sep 21 22:15:22 siprouter kernel: Free swap: 0kB Sep 21 22:15:22 siprouter kernel: 131072 pages of RAM Sep 21 22:15:22 siprouter kernel: 3531 reserved pages Sep 21 22:15:22 siprouter kernel: 5863 pages shared Sep 21 22:15:22 siprouter kernel: 0 pages swap cached Sep 21 22:15:22 siprouter kernel: Out of memory: Killed process 11091, UID 89, (pickup). Sep 21 22:15:35 siprouter kernel: kamailio invoked oom-killer: gfp_mask=0x280d2, order=0, oomkilladj=0 Sep 21 22:15:35 siprouter kernel: Sep 21 22:15:35 siprouter kernel: Call Trace: Sep 21 22:15:35 siprouter kernel: [<ffffffff800c7161>] out_of_memory+0x8e/0x2f3 Sep 21 22:15:35 siprouter kernel: [<ffffffff8002e23f>] __wake_up+0x38/0x4f Sep 21 22:15:35 siprouter kernel: [<ffffffff800a09d8>] autoremove_wake_function+0x0/0x2e Sep 21 22:15:35 siprouter kernel: [<ffffffff8000f540>] __alloc_pages+0x27f/0x308 Sep 21 22:15:35 siprouter kernel: [<ffffffff80008e9f>] __handle_mm_fault+0x73c/0x1039 Sep 21 22:15:35 siprouter kernel: [<ffffffff80062ff8>] thread_return+0x62/0xfe Sep 21 22:15:35 siprouter kernel: [<ffffffff80066b55>] do_page_fault+0x4cb/0x874 Sep 21 22:15:35 siprouter kernel: [<ffffffff8006383f>] schedule_timeout+0x1e/0xad Sep 21 22:15:35 siprouter kernel: [<ffffffff8005dde9>] error_exit+0x0/0x84 Sep 21 22:15:35 siprouter kernel: [<ffffffff8000d1b7>] file_read_actor+0x2b/0x159 Sep 21 22:15:35 siprouter kernel: [<ffffffff80014068>] page_cache_readahead+0xd6/0x1af Sep 21 22:15:35 siprouter kernel: [<ffffffff8000c317>] do_generic_mapping_read+0x183/0x359 Sep 21 22:15:35 siprouter kernel: [<ffffffff8000d18c>] file_read_actor+0x0/0x159 Sep 21 22:15:35 siprouter kernel: [<ffffffff8000c639>] __generic_file_aio_read+0x14c/0x198 Sep 21 22:15:35 siprouter kernel: [<ffffffff80016e31>] generic_file_aio_read+0x34/0x39 Sep 21 22:15:35 siprouter kernel: [<ffffffff8000ceb5>] do_sync_read+0xc7/0x104 Sep 21 22:15:35 siprouter kernel: [<ffffffff8002a6ef>] __vma_link+0x42/0x4b Sep 21 22:15:35 siprouter kernel: [<ffffffff800a09d8>] autoremove_wake_function+0x0/0x2e Sep 21 22:15:35 siprouter kernel: [<ffffffff8000e1b8>] do_mmap_pgoff+0x615/0x780 Sep 21 22:15:35 siprouter kernel: [<ffffffff8000b729>] vfs_read+0xcb/0x171 Sep 21 22:15:35 siprouter kernel: [<ffffffff80011c3b>] sys_read+0x45/0x6e Sep 21 22:15:35 siprouter kernel: [<ffffffff8005d116>] system_call+0x7e/0x83 Sep 21 22:15:35 siprouter kernel: Sep 21 22:15:35 siprouter kernel: Mem-info: Sep 21 22:15:35 siprouter kernel: Node 0 DMA per-cpu: Sep 21 22:15:35 siprouter kernel: cpu 0 hot: high 0, batch 1 used:0 Sep 21 22:15:35 siprouter kernel: cpu 0 cold: high 0, batch 1 used:0 Sep 21 22:15:35 siprouter kernel: cpu 1 hot: high 0, batch 1 used:0 Sep 21 22:15:35 siprouter kernel: cpu 1 cold: high 0, batch 1 used:0 Sep 21 22:15:35 siprouter kernel: cpu 2 hot: high 0, batch 1 used:0 Sep 21 22:15:35 siprouter kernel: cpu 2 cold: high 0, batch 1 used:0 Sep 21 22:15:35 siprouter kernel: cpu 3 hot: high 0, batch 1 used:0 Sep 21 22:15:35 siprouter kernel: cpu 3 cold: high 0, batch 1 used:0 Sep 21 22:15:35 siprouter kernel: Node 0 DMA32 per-cpu: Sep 21 22:15:35 siprouter kernel: cpu 0 hot: high 186, batch 31 used:115 Sep 21 22:15:35 siprouter kernel: cpu 0 cold: high 62, batch 15 used:12 Sep 21 22:15:35 siprouter kernel: cpu 1 hot: high 186, batch 31 used:96 Sep 21 22:15:35 siprouter kernel: cpu 1 cold: high 62, batch 15 used:16 Sep 21 22:15:35 siprouter kernel: cpu 2 hot: high 186, batch 31 used:71 Sep 21 22:15:35 siprouter kernel: cpu 2 cold: high 62, batch 15 used:17 Sep 21 22:15:35 siprouter kernel: cpu 3 hot: high 186, batch 31 used:120 Sep 21 22:15:35 siprouter kernel: cpu 3 cold: high 62, batch 15 used:38 Sep 21 22:15:35 siprouter kernel: Node 0 Normal per-cpu: empty Sep 21 22:15:35 siprouter kernel: Node 0 HighMem per-cpu: empty Sep 21 22:15:35 siprouter kernel: Free pages: 4800kB (0kB HighMem) Sep 21 22:15:35 siprouter kernel: Active:117577 inactive:1347 dirty:0 writeback:0 unstable:0 free:1200 slab:3058 mapped-file:1126 mapped-anon:116771 pagetables:1374 Sep 21 22:15:35 siprouter kernel: Node 0 DMA free:2012kB min:56kB low:68kB high:84kB active:1728kB inactive:0kB present:10520kB pages_scanned:13345 all_unreclaimable? yes Sep 21 22:15:35 siprouter kernel: lowmem_reserve[]: 0 489 489 489 Sep 21 22:15:35 siprouter kernel: Node 0 DMA32 free:2788kB min:2800kB low:3500kB high:4200kB active:469212kB inactive:5012kB present:500896kB pages_scanned:1029046 all_unreclaimable? yes Sep 21 22:15:35 siprouter kernel: lowmem_reserve[]: 0 0 0 0 Sep 21 22:15:35 siprouter kernel: Node 0 Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Sep 21 22:15:35 siprouter kernel: lowmem_reserve[]: 0 0 0 0 Sep 21 22:15:35 siprouter kernel: Node 0 HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Sep 21 22:15:35 siprouter kernel: lowmem_reserve[]: 0 0 0 0 Sep 21 22:15:35 siprouter kernel: Node 0 DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2012kB Sep 21 22:15:35 siprouter kernel: Node 0 DMA32: 23*4kB 5*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2788kB Sep 21 22:15:35 siprouter kernel: Node 0 Normal: empty Sep 21 22:15:35 siprouter kernel: Node 0 HighMem: empty Sep 21 22:15:35 siprouter kernel: 2213 pagecache pages Sep 21 22:15:35 siprouter kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0 Sep 21 22:15:35 siprouter kernel: Free swap = 0kB Sep 21 22:15:35 siprouter kernel: Total swap = 0kB Sep 21 22:15:35 siprouter kernel: Free swap: 0kB Sep 21 22:15:35 siprouter kernel: 131072 pages of RAM Sep 21 22:15:35 siprouter kernel: 3531 reserved pages Sep 21 22:15:35 siprouter kernel: 7119 pages shared Sep 21 22:15:35 siprouter kernel: 0 pages swap cached Sep 21 22:15:35 siprouter kernel: Out of memory: Killed process 767, UID 0, (kamailio). Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[766]: ALERT: <core> [main.c:722]: child process 767 exited by a signal 9 Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[766]: ALERT: <core> [main.c:725]: core was not generated Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[766]: INFO: <core> [main.c:737]: INFO: terminating due to SIGCHLD Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[784]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[784]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: used= 134800, used+overhead=178120, free=4016184 Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: dumping free list: Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[777]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[780]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[785]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[778]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[777]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[773]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[773]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[776]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[776]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[769]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[781]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[785]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[782]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[772]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[778]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[770]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[783]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[774]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[768]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[771]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[780]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 6 fragments no.: 3, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[779]: INFO: <core> [main.c:788]: INFO: signal 15 received Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[769]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[781]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[782]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[772]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[770]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[783]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: used= 134800, used+overhead=178216, free=4016088 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 7 fragments no.: 2, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: <core> [main.c:799]: Memory status (pkg): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: used= 134800, used+overhead=177432, free=4016872 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: used= 138264, used+overhead=181616, free=4012688 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 8 fragments no.: 20, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: used= 144816, used+overhead=187480, free=4006824 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: used= 134800, used+overhead=178120, free=4016184 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: used= 134768, used+overhead=177416, free=4016888 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:36 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: used= 134800, used+overhead=178104, free=4016200 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 14 fragments no.: 1, unused: 0 bucket size: 112 - 112 (first 112) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: used= 134800, used+overhead=177432, free=4016872 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: used= 138640, used+overhead=181912, free=4012392 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: used= 138176, used+overhead=181528, free=4012776 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: used= 134800, used+overhead=177432, free=4016872 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: used= 134800, used+overhead=177432, free=4016872 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: used= 134800, used+overhead=177432, free=4016872 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: used= 134800, used+overhead=177432, free=4016872 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: used= 134800, used+overhead=177432, free=4016872 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 15 fragments no.: 1, unused: 0 bucket size: 120 - 120 (first 120) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: used= 138408, used+overhead=181776, free=4012528 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 31 fragments no.: 1, unused: 0 bucket size: 248 - 248 (first 248) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 2 fragments no.: 1, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 55 fragments no.: 1, unused: 0 bucket size: 440 - 440 (first 440) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 2 fragments no.: 1, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 6 fragments no.: 2, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: used= 134768, used+overhead=177400, free=4016904 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 57 fragments no.: 1, unused: 0 bucket size: 456 - 456 (first 456) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 2 fragments no.: 1, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 7 fragments no.: 4, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3981528) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 66 fragments no.: 1, unused: 0 bucket size: 528 - 528 (first 528) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 2 fragments no.: 1, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: hash = 4 fragments no.: 1, unused: 0 bucket size: 32 - 32 (first 32) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 8 fragments no.: 19, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 14 fragments no.: 2, unused: 0 bucket size: 112 - 112 (first 112) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: TOTAL: 36 free fragments = 4016872 free bytes Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[773]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 6 fragments no.: 1, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 100 fragments no.: 1, unused: 0 bucket size: 800 - 800 (first 800) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 6 fragments no.: 1, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: hash = 4 fragments no.: 1, unused: 0 bucket size: 32 - 32 (first 32) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 6 fragments no.: 2, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 17 fragments no.: 1, unused: 0 bucket size: 136 - 136 (first 136) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 7 fragments no.: 2, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 114 fragments no.: 1, unused: 0 bucket size: 912 - 912 (first 912) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 6 fragments no.: 3, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 7 fragments no.: 2, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 6 fragments no.: 1, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: dumping free list: Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 7 fragments no.: 3, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: hash = 9 fragments no.: 1, unused: 0 bucket size: 72 - 72 (first 72) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 29 fragments no.: 1, unused: 0 bucket size: 232 - 232 (first 232) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 8 fragments no.: 12, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 216 fragments no.: 17, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 6 fragments no.: 1, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 7 fragments no.: 2, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 8 fragments no.: 11, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: hash = 36 fragments no.: 1, unused: 0 bucket size: 288 - 288 (first 288) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 7 fragments no.: 3, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 8 fragments no.: 19, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: hash = 43 fragments no.: 1, unused: 0 bucket size: 344 - 344 (first 344) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 47 fragments no.: 1, unused: 0 bucket size: 376 - 376 (first 376) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 12 fragments no.: 1, unused: 0 bucket size: 96 - 96 (first 96) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 251 fragments no.: 1, unused: 0 bucket size: 2008 - 2008 (first 2008) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 7 fragments no.: 2, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: hash = 1 fragments no.: 1, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: hash = 2 fragments no.: 2, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3981528) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 16 fragments no.: 1, unused: 0 bucket size: 128 - 128 (first 128) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: hash = 216 fragments no.: 19, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 8 fragments no.: 13, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3981528) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 13 fragments no.: 1, unused: 0 bucket size: 104 - 104 (first 104) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3981528) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 50 fragments no.: 1, unused: 0 bucket size: 400 - 400 (first 400) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3981528) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 15 fragments no.: 1, unused: 0 bucket size: 120 - 120 (first 120) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 261 fragments no.: 1, unused: 0 bucket size: 2088 - 2088 (first 2088) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 8 fragments no.: 9, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: hash = 3 fragments no.: 8, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 8 fragments no.: 21, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: TOTAL: 36 free fragments = 4016872 free bytes Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 44 fragments no.: 1, unused: 0 bucket size: 352 - 352 (first 352) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3973320) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 14 fragments no.: 1, unused: 0 bucket size: 112 - 112 (first 112) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: TOTAL: 36 free fragments = 4016872 free bytes Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 14 fragments no.: 1, unused: 0 bucket size: 112 - 112 (first 112) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: TOTAL: 36 free fragments = 4016872 free bytes Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3981528) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 52 fragments no.: 1, unused: 0 bucket size: 416 - 416 (first 416) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: TOTAL: 36 free fragments = 4016872 free bytes Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 47 fragments no.: 1, unused: 0 bucket size: 376 - 376 (first 376) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 279 fragments no.: 1, unused: 0 bucket size: 2232 - 2232 (first 2232) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 12 fragments no.: 1, unused: 0 bucket size: 96 - 96 (first 96) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 13 fragments no.: 1, unused: 0 bucket size: 104 - 104 (first 104) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[769]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 90 fragments no.: 1, unused: 0 bucket size: 720 - 720 (first 720) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: TOTAL: 36 free fragments = 4006824 free bytes Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 15 fragments no.: 2, unused: 0 bucket size: 120 - 120 (first 120) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[772]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 15 fragments no.: 1, unused: 0 bucket size: 120 - 120 (first 120) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: hash = 4 fragments no.: 1, unused: 0 bucket size: 32 - 32 (first 32) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[770]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: TOTAL: 38 free fragments = 4016888 free bytes Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3981528) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[783]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[768]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 50 fragments no.: 1, unused: 0 bucket size: 400 - 400 (first 400) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 286 fragments no.: 1, unused: 0 bucket size: 2288 - 2288 (first 2288) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 15 fragments no.: 3, unused: 0 bucket size: 120 - 120 (first 120) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 14 fragments no.: 1, unused: 0 bucket size: 112 - 112 (first 112) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 124 fragments no.: 1, unused: 0 bucket size: 992 - 992 (first 992) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[785]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 25 fragments no.: 1, unused: 0 bucket size: 200 - 200 (first 200) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 16 fragments no.: 1, unused: 0 bucket size: 128 - 128 (first 128) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: TOTAL: 36 free fragments = 4016872 free bytes Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 54 fragments no.: 2, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 52 fragments no.: 1, unused: 0 bucket size: 416 - 416 (first 416) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 333 fragments no.: 1, unused: 0 bucket size: 2664 - 2664 (first 2664) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 46 fragments no.: 1, unused: 0 bucket size: 368 - 368 (first 368) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 15 fragments no.: 1, unused: 0 bucket size: 120 - 120 (first 120) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 216 fragments no.: 16, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 45 fragments no.: 1, unused: 0 bucket size: 360 - 360 (first 360) Sep 21 22:15:37 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: hash = 5 fragments no.: 3, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 29 fragments no.: 1, unused: 0 bucket size: 232 - 232 (first 232) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[774]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 55 fragments no.: 1, unused: 0 bucket size: 440 - 440 (first 440) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3981528) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 53 fragments no.: 1, unused: 0 bucket size: 424 - 424 (first 424) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 363 fragments no.: 1, unused: 0 bucket size: 2904 - 2904 (first 2904) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 94 fragments no.: 1, unused: 0 bucket size: 752 - 752 (first 752) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 19 fragments no.: 1, unused: 0 bucket size: 152 - 152 (first 152) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 227 fragments no.: 1, unused: 0 bucket size: 1816 - 1816 (first 1816) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 48 fragments no.: 1, unused: 0 bucket size: 384 - 384 (first 384) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 46 fragments no.: 1, unused: 0 bucket size: 368 - 368 (first 368) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 60 fragments no.: 1, unused: 0 bucket size: 480 - 480 (first 480) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: TOTAL: 36 free fragments = 4016872 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 80 fragments no.: 1, unused: 0 bucket size: 640 - 640 (first 640) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 372 fragments no.: 1, unused: 0 bucket size: 2976 - 2976 (first 2976) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 120 fragments no.: 1, unused: 0 bucket size: 960 - 960 (first 960) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 29 fragments no.: 1, unused: 0 bucket size: 232 - 232 (first 232) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 237 fragments no.: 1, unused: 0 bucket size: 1896 - 1896 (first 1896) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 51 fragments no.: 1, unused: 0 bucket size: 408 - 408 (first 408) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 47 fragments no.: 1, unused: 0 bucket size: 376 - 376 (first 376) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 80 fragments no.: 1, unused: 0 bucket size: 640 - 640 (first 640) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[771]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 95 fragments no.: 1, unused: 0 bucket size: 760 - 760 (first 760) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 373 fragments no.: 1, unused: 0 bucket size: 2984 - 2984 (first 2984) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 216 fragments no.: 16, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 45 fragments no.: 1, unused: 0 bucket size: 360 - 360 (first 360) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 247 fragments no.: 1, unused: 0 bucket size: 1976 - 1976 (first 1976) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 92 fragments no.: 1, unused: 0 bucket size: 736 - 736 (first 736) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 48 fragments no.: 1, unused: 0 bucket size: 384 - 384 (first 384) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 134 fragments no.: 1, unused: 0 bucket size: 1072 - 1072 (first 1072) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 119 fragments no.: 1, unused: 0 bucket size: 952 - 952 (first 952) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3981528) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 404 fragments no.: 1, unused: 0 bucket size: 3232 - 3232 (first 3232) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 220 fragments no.: 1, unused: 0 bucket size: 1760 - 1760 (first 1760) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 55 fragments no.: 2, unused: 0 bucket size: 440 - 440 (first 440) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 259 fragments no.: 1, unused: 0 bucket size: 2072 - 2072 (first 2072) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 96 fragments no.: 1, unused: 0 bucket size: 768 - 768 (first 768) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 55 fragments no.: 2, unused: 0 bucket size: 440 - 440 (first 440) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 216 fragments no.: 16, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 216 fragments no.: 15, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 422 fragments no.: 1, unused: 0 bucket size: 3376 - 3376 (first 3376) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 230 fragments no.: 1, unused: 0 bucket size: 1840 - 1840 (first 1840) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 81 fragments no.: 1, unused: 0 bucket size: 648 - 648 (first 648) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 260 fragments no.: 1, unused: 0 bucket size: 2080 - 2080 (first 2080) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 118 fragments no.: 1, unused: 0 bucket size: 944 - 944 (first 944) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 56 fragments no.: 1, unused: 0 bucket size: 448 - 448 (first 448) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: TOTAL: 37 free fragments = 4016904 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 220 fragments no.: 1, unused: 0 bucket size: 1760 - 1760 (first 1760) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 222 fragments no.: 1, unused: 0 bucket size: 1776 - 1776 (first 1776) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 482 fragments no.: 1, unused: 0 bucket size: 3856 - 3856 (first 3856) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 237 fragments no.: 1, unused: 0 bucket size: 1896 - 1896 (first 1896) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 112 fragments no.: 1, unused: 0 bucket size: 896 - 896 (first 896) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 263 fragments no.: 1, unused: 0 bucket size: 2104 - 2104 (first 2104) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 216 fragments no.: 15, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 59 fragments no.: 1, unused: 0 bucket size: 472 - 472 (first 472) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[784]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 236 fragments no.: 1, unused: 0 bucket size: 1888 - 1888 (first 1888) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 226 fragments no.: 1, unused: 0 bucket size: 1808 - 1808 (first 1808) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 546 fragments no.: 1, unused: 0 bucket size: 4368 - 4368 (first 4368) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 260 fragments no.: 1, unused: 0 bucket size: 2080 - 2080 (first 2080) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 216 fragments no.: 17, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 324 fragments no.: 1, unused: 0 bucket size: 2592 - 2592 (first 2592) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 217 fragments no.: 1, unused: 0 bucket size: 1736 - 1736 (first 1736) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 80 fragments no.: 1, unused: 0 bucket size: 640 - 640 (first 640) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 241 fragments no.: 1, unused: 0 bucket size: 1928 - 1928 (first 1928) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 258 fragments no.: 1, unused: 0 bucket size: 2064 - 2064 (first 2064) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 553 fragments no.: 1, unused: 0 bucket size: 4424 - 4424 (first 4424) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 288 fragments no.: 1, unused: 0 bucket size: 2304 - 2304 (first 2304) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 220 fragments no.: 1, unused: 0 bucket size: 1760 - 1760 (first 1760) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 372 fragments no.: 1, unused: 0 bucket size: 2976 - 2976 (first 2976) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 258 fragments no.: 1, unused: 0 bucket size: 2064 - 2064 (first 2064) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 134 fragments no.: 1, unused: 0 bucket size: 1072 - 1072 (first 1072) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 253 fragments no.: 1, unused: 0 bucket size: 2024 - 2024 (first 2024) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 259 fragments no.: 1, unused: 0 bucket size: 2072 - 2072 (first 2072) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 574 fragments no.: 1, unused: 0 bucket size: 4592 - 4592 (first 4592) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 298 fragments no.: 1, unused: 0 bucket size: 2384 - 2384 (first 2384) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 280 fragments no.: 1, unused: 0 bucket size: 2240 - 2240 (first 2240) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 373 fragments no.: 1, unused: 0 bucket size: 2984 - 2984 (first 2984) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 261 fragments no.: 1, unused: 0 bucket size: 2088 - 2088 (first 2088) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 216 fragments no.: 16, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 260 fragments no.: 1, unused: 0 bucket size: 2080 - 2080 (first 2080) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 260 fragments no.: 1, unused: 0 bucket size: 2080 - 2080 (first 2080) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3937312) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 343 fragments no.: 1, unused: 0 bucket size: 2744 - 2744 (first 2744) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 318 fragments no.: 1, unused: 0 bucket size: 2544 - 2544 (first 2544) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 552 fragments no.: 1, unused: 0 bucket size: 4416 - 4416 (first 4416) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 297 fragments no.: 1, unused: 0 bucket size: 2376 - 2376 (first 2376) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 299 fragments no.: 1, unused: 0 bucket size: 2392 - 2392 (first 2392) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 261 fragments no.: 1, unused: 0 bucket size: 2088 - 2088 (first 2088) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 297 fragments no.: 1, unused: 0 bucket size: 2376 - 2376 (first 2376) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: TOTAL: 79 free fragments = 4016184 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 344 fragments no.: 1, unused: 0 bucket size: 2752 - 2752 (first 2752) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 343 fragments no.: 1, unused: 0 bucket size: 2744 - 2744 (first 2744) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 574 fragments no.: 1, unused: 0 bucket size: 4592 - 4592 (first 4592) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 318 fragments no.: 1, unused: 0 bucket size: 2544 - 2544 (first 2544) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 342 fragments no.: 1, unused: 0 bucket size: 2736 - 2736 (first 2736) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 266 fragments no.: 1, unused: 0 bucket size: 2128 - 2128 (first 2128) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 327 fragments no.: 1, unused: 0 bucket size: 2616 - 2616 (first 2616) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[775]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 352 fragments no.: 1, unused: 0 bucket size: 2816 - 2816 (first 2816) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 351 fragments no.: 1, unused: 0 bucket size: 2808 - 2808 (first 2808) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3951848) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: TOTAL: 59 free fragments = 4012392 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 344 fragments no.: 1, unused: 0 bucket size: 2752 - 2752 (first 2752) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 334 fragments no.: 1, unused: 0 bucket size: 2672 - 2672 (first 2672) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 352 fragments no.: 1, unused: 0 bucket size: 2816 - 2816 (first 2816) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 373 fragments no.: 1, unused: 0 bucket size: 2984 - 2984 (first 2984) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 372 fragments no.: 1, unused: 0 bucket size: 2976 - 2976 (first 2976) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[781]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 352 fragments no.: 1, unused: 0 bucket size: 2816 - 2816 (first 2816) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 372 fragments no.: 1, unused: 0 bucket size: 2976 - 2976 (first 2976) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 351 fragments no.: 1, unused: 0 bucket size: 2808 - 2808 (first 2808) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 372 fragments no.: 1, unused: 0 bucket size: 2976 - 2976 (first 2976) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 445 fragments no.: 1, unused: 0 bucket size: 3560 - 3560 (first 3560) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 373 fragments no.: 1, unused: 0 bucket size: 2984 - 2984 (first 2984) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 373 fragments no.: 1, unused: 0 bucket size: 2984 - 2984 (first 2984) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 394 fragments no.: 1, unused: 0 bucket size: 3152 - 3152 (first 3152) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 363 fragments no.: 1, unused: 0 bucket size: 2904 - 2904 (first 2904) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 373 fragments no.: 1, unused: 0 bucket size: 2984 - 2984 (first 2984) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 552 fragments no.: 1, unused: 0 bucket size: 4416 - 4416 (first 4416) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 441 fragments no.: 1, unused: 0 bucket size: 3528 - 3528 (first 3528) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 464 fragments no.: 1, unused: 0 bucket size: 3712 - 3712 (first 3712) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 413 fragments no.: 1, unused: 0 bucket size: 3304 - 3304 (first 3304) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 373 fragments no.: 1, unused: 0 bucket size: 2984 - 2984 (first 2984) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 431 fragments no.: 1, unused: 0 bucket size: 3448 - 3448 (first 3448) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 562 fragments no.: 1, unused: 0 bucket size: 4496 - 4496 (first 4496) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 454 fragments no.: 1, unused: 0 bucket size: 3632 - 3632 (first 3632) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 473 fragments no.: 1, unused: 0 bucket size: 3784 - 3784 (first 3784) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 491 fragments no.: 1, unused: 0 bucket size: 3928 - 3928 (first 3928) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 375 fragments no.: 1, unused: 0 bucket size: 3000 - 3000 (first 3000) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 546 fragments no.: 1, unused: 0 bucket size: 4368 - 4368 (first 4368) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 565 fragments no.: 1, unused: 0 bucket size: 4520 - 4520 (first 4520) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 562 fragments no.: 1, unused: 0 bucket size: 4496 - 4496 (first 4496) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 556 fragments no.: 1, unused: 0 bucket size: 4448 - 4448 (first 4448) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 547 fragments no.: 1, unused: 0 bucket size: 4376 - 4376 (first 4376) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 376 fragments no.: 1, unused: 0 bucket size: 3008 - 3008 (first 3008) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 574 fragments no.: 1, unused: 0 bucket size: 4592 - 4592 (first 4592) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3949296) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 565 fragments no.: 1, unused: 0 bucket size: 4520 - 4520 (first 4520) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 562 fragments no.: 1, unused: 0 bucket size: 4496 - 4496 (first 4496) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 565 fragments no.: 1, unused: 0 bucket size: 4520 - 4520 (first 4520) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 385 fragments no.: 1, unused: 0 bucket size: 3080 - 3080 (first 3080) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3945344) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: TOTAL: 78 free fragments = 4016200 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 566 fragments no.: 1, unused: 0 bucket size: 4528 - 4528 (first 4528) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 565 fragments no.: 1, unused: 0 bucket size: 4520 - 4520 (first 4520) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 571 fragments no.: 1, unused: 0 bucket size: 4568 - 4568 (first 4568) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 565 fragments no.: 1, unused: 0 bucket size: 4520 - 4520 (first 4520) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: TOTAL: 66 free fragments = 4012688 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[780]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3934032) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 566 fragments no.: 1, unused: 0 bucket size: 4528 - 4528 (first 4528) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3937872) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3946664) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[776]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: TOTAL: 85 free fragments = 4016088 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[777]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 566 fragments no.: 1, unused: 0 bucket size: 4528 - 4528 (first 4528) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: TOTAL: 79 free fragments = 4016184 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: TOTAL: 65 free fragments = 4012528 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3939224) Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[778]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[779]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: TOTAL: 69 free fragments = 4012776 free bytes Sep 21 22:15:38 siprouter /usr/local/sbin/kamailio[782]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: <core> [main.c:546]: Memory status (pkg): Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: fm_status (0x88d5c0): Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: heap size= 4194304 Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 96632, used+overhead=139248, free=4055056 Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: max used (+overhead)= 204008 Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: dumping free list: Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 1 fragments no.: 2, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2 fragments no.: 6, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 3 fragments no.: 12, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 4 fragments no.: 1, unused: 0 bucket size: 32 - 32 (first 32) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 5 fragments no.: 15, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 6 fragments no.: 1, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 7 fragments no.: 3, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 8 fragments no.: 1, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 9 fragments no.: 3, unused: 0 bucket size: 72 - 72 (first 72) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 11 fragments no.: 106, unused: 0 bucket size: 88 - 88 (first 88) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 16 fragments no.: 8, unused: 0 bucket size: 128 - 128 (first 128) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 17 fragments no.: 6, unused: 0 bucket size: 136 - 136 (first 136) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 18 fragments no.: 7, unused: 0 bucket size: 144 - 144 (first 144) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 19 fragments no.: 11, unused: 0 bucket size: 152 - 152 (first 152) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 20 fragments no.: 9, unused: 0 bucket size: 160 - 160 (first 160) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 21 fragments no.: 1, unused: 0 bucket size: 168 - 168 (first 168) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 22 fragments no.: 1, unused: 0 bucket size: 176 - 176 (first 176) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 25 fragments no.: 2, unused: 0 bucket size: 200 - 200 (first 200) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 31 fragments no.: 1, unused: 0 bucket size: 248 - 248 (first 248) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 32 fragments no.: 1, unused: 0 bucket size: 256 - 256 (first 256) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 34 fragments no.: 1, unused: 0 bucket size: 272 - 272 (first 272) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 41 fragments no.: 1, unused: 0 bucket size: 328 - 328 (first 328) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 54 fragments no.: 1, unused: 0 bucket size: 432 - 432 (first 432) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 57 fragments no.: 1, unused: 0 bucket size: 456 - 456 (first 456) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 64 fragments no.: 2, unused: 0 bucket size: 512 - 512 (first 512) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 72 fragments no.: 1, unused: 0 bucket size: 576 - 576 (first 576) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 105 fragments no.: 1, unused: 0 bucket size: 840 - 840 (first 840) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 170 fragments no.: 1, unused: 0 bucket size: 1360 - 1360 (first 1360) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 216 fragments no.: 20, unused: 0 bucket size: 1728 - 1728 (first 1728) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 926 fragments no.: 1, unused: 0 bucket size: 7408 - 7408 (first 7408) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 3989736) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: TOTAL: 228 free fragments = 4055056 free bytes Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: ----------------------------- Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: <core> [main.c:560]: Memory status (shm): Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: fm_status (0x2b982b03b000): Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: heap size= 33554432 Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: used= 2968, used+overhead=40816, free=33513616 Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: max used (+overhead)= 3235640 Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: dumping free list: Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 1 fragments no.: 14, unused: 0 bucket size: 8 - 8 (first 8) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2 fragments no.: 17, unused: 0 bucket size: 16 - 16 (first 16) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 3 fragments no.: 15, unused: 0 bucket size: 24 - 24 (first 24) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 5 fragments no.: 33, unused: 0 bucket size: 40 - 40 (first 40) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 6 fragments no.: 1, unused: 0 bucket size: 48 - 48 (first 48) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 7 fragments no.: 1, unused: 0 bucket size: 56 - 56 (first 56) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 8 fragments no.: 23, unused: 0 bucket size: 64 - 64 (first 64) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 27 fragments no.: 1, unused: 0 bucket size: 216 - 216 (first 216) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 32 fragments no.: 1, unused: 0 bucket size: 256 - 256 (first 256) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 37 fragments no.: 1, unused: 0 bucket size: 296 - 296 (first 296) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 38 fragments no.: 44, unused: 0 bucket size: 304 - 304 (first 304) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 39 fragments no.: 4, unused: 0 bucket size: 312 - 312 (first 312) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 40 fragments no.: 2, unused: 0 bucket size: 320 - 320 (first 320) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 53 fragments no.: 1, unused: 0 bucket size: 424 - 424 (first 424) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 144 fragments no.: 1, unused: 0 bucket size: 1152 - 1152 (first 1152) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 350 fragments no.: 1, unused: 0 bucket size: 2800 - 2800 (first 2800) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 583 fragments no.: 50, unused: 0 bucket size: 4664 - 4664 (first 4664) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 640 fragments no.: 1, unused: 0 bucket size: 5120 - 5120 (first 5120) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2048 fragments no.: 2, unused: 0 bucket size: 16384 - 16384 (first 16384) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2053 fragments no.: 1, unused: 0 bucket size: 262144 - 524288 (first 278544) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2056 fragments no.: 1, unused: 0 bucket size: 2097152 - 4194304 (first 2621440) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: hash = 2059 fragments no.: 1, unused: 0 bucket size: 16777216 - 33554432 (first 30318496) Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: TOTAL: 216 free fragments = 33513616 free bytes Sep 21 22:15:39 siprouter /usr/local/sbin/kamailio[766]: NOTICE: fm_status: -----------------------------
Santiago Soares
On Wed, Sep 22, 2010 at 4:36 AM, Klaus Darilion < klaus.mailinglists@pernau.at> wrote:
Am 20.09.2010 21:21, schrieb Santiago Soares:
Hello,
We use Kamailio as a stateless load balancer. It runs on a VM, with 512 MB RAM, and it seems to have some kind of memory leak. The kamailio processes consume all available memory, until Kamailio crashes and restart.
What do you mean with "crash"? Usually if Kamailio runs out of memory it logs error messages but it does not crash.
How do you verify the memory consumption? With top or other tools? If the memory consumption reported by the OS is increasing then there is a problem with an external library.
Kamailio internally allocates at startup a fixed size of memory which is never increased. If system memory consumption is increasing then it is by an external library which does not use Kamailio's memory allocator.
So what exactly is happening? Is Kamilio just running out of pre-allocated memory or is system-memory increasing? How does it crash in detail?
regards Klaus
This occurs about once a day.
Since it's stateless, and the restart is very fast, it's not a big deal, but I'd like to fix it. The strange thing is that I updated kamailio from 1.5.3 to 3.0.3 and the problem still ocurrs. I followed the instructions on http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory in order to determine the cause, and stopped the traffic on the proxy. The allocated memory were not freed, after 1 hour the amount of free memory is still the same. Anybody has an ideia about what's happening, or how can I debug it?
Thank you!
Santiago Soares
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Am 22.09.2010 15:31, schrieb Santiago Soares:
Thank's for the answers. Based on your explanation, I think that the problem is an external library. The kamailio process is taking all the system memory. When the system runs out of memory, it kills kamailio. But how can I find out what is the library?
This means that either the library is buggy, or the library is used wrongly (e.g. not releasing buffers ...). Let's take a look at your modules:
loadmodule "maxfwd.so" loadmodule "exec.so" loadmodule "tm.so" loadmodule "sl.so" loadmodule "dispatcher.so" loadmodule "textops.so" loadmodule "xlog.so" loadmodule "mi_fifo.so" loadmodule "pv.so"
I think they do not use any external libraries
loadmodule "regex.so"
This one uses libpcre. Maybe there is a problem with the regular expressions?
regards Klaus
On 09/22/2010 06:08 PM, Klaus Darilion wrote:
Am 22.09.2010 15:31, schrieb Santiago Soares:
Thank's for the answers. Based on your explanation, I think that the problem is an external library. The kamailio process is taking all the system memory. When the system runs out of memory, it kills kamailio. But how can I find out what is the library?
This means that either the library is buggy, or the library is used wrongly (e.g. not releasing buffers ...). Let's take a look at your modules:
loadmodule "maxfwd.so" loadmodule "exec.so" loadmodule "tm.so" loadmodule "sl.so" loadmodule "dispatcher.so" loadmodule "textops.so" loadmodule "xlog.so" loadmodule "mi_fifo.so" loadmodule "pv.so"
I think they do not use any external libraries
loadmodule "regex.so"
This one uses libpcre. Maybe there is a problem with the regular expressions?
Hello I have some suggestions
# excessively long display name if (!pcre_match("$fn","^.{0,62}$$")) { sl_send_reply("403", "Forbidden - Display Name Max Size Exceeded"); exit; };
this will be much better handled by a transformation
if ( $(fn{s.len}) > 62 ) {...}
Make this change and see if you still have problems with OOM Killer
Marius
regards Klaus
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
On 09/22/2010 06:21 PM, marius zbihlei wrote:
On 09/22/2010 06:08 PM, Klaus Darilion wrote:
Am 22.09.2010 15:31, schrieb Santiago Soares:
Thank's for the answers. Based on your explanation, I think that the problem is an external library. The kamailio process is taking all the system memory. When the system runs out of memory, it kills kamailio. But how can I find out what is the library?
This means that either the library is buggy, or the library is used wrongly (e.g. not releasing buffers ...). Let's take a look at your modules:
loadmodule "maxfwd.so" loadmodule "exec.so" loadmodule "tm.so" loadmodule "sl.so" loadmodule "dispatcher.so" loadmodule "textops.so" loadmodule "xlog.so" loadmodule "mi_fifo.so" loadmodule "pv.so"
I think they do not use any external libraries
loadmodule "regex.so"
This one uses libpcre. Maybe there is a problem with the regular expressions?
Hello again,
There seems to be a memory leak in w_pcre_match . I have fixed it in commit 61f4d7f. Please compile with this patch and test (applied to git kamailio_3.0 branch)
Marius
Hello I have some suggestions
# excessively long display name if (!pcre_match("$fn","^.{0,62}$$")) { sl_send_reply("403", "Forbidden - Display Name Max SizeExceeded"); exit; };
this will be much better handled by a transformation
if ( $(fn{s.len})> 62 ) {...}
Make this change and see if you still have problems with OOM Killer
Marius
regards Klaus
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
2010/9/22 marius zbihlei marius.zbihlei@1and1.ro:
this will be much better handled by a transformation
if ( $(fn{s.len}) > 62 ) {...}
Yes, using pcre_math is good when we need to perform a regular expression against a PV (which is not possible using "=~"). For example I use it to check if the RURI username matches a prefix extracted from a database:
if ! pcre_match("$rU", "^$var(client_prefix_in)") {
This works because the regular expression is compiled in runtime.
This would not be possible with "=~":
if ! $rU =~ "^$var(client_prefix_in)") {
It fails because the regular expression is compiled when parsing the config script.
On 9/22/10 10:25 PM, Iñaki Baz Castillo wrote:
2010/9/22 marius zbihleimarius.zbihlei@1and1.ro:
this will be much better handled by a transformation
if ( $(fn{s.len})> 62 ) {...}
Yes, using pcre_math is good when we need to perform a regular expression against a PV (which is not possible using "=~"). For example I use it to check if the RURI username matches a prefix extracted from a database:
if ! pcre_match("$rU", "^$var(client_prefix_in)") {
This works because the regular expression is compiled in runtime.
This would not be possible with "=~":
if ! $rU =~ "^$var(client_prefix_in)") {
It fails because the regular expression is compiled when parsing the config script.
have you tried like next?
$var(re) = "^" + $var(client_prefix_in);
if(!$rU =~ $var(re)) {
Cheers, Daniel
2010/9/22 Daniel-Constantin Mierla miconda@gmail.com:
Yes, using pcre_math is good when we need to perform a regular expression against a PV (which is not possible using "=~"). For example I use it to check if the RURI username matches a prefix extracted from a database:
if ! pcre_match("$rU", "^$var(client_prefix_in)") {
This works because the regular expression is compiled in runtime.
This would not be possible with "=~":
if ! $rU =~ "^$var(client_prefix_in)") {
It fails because the regular expression is compiled when parsing the config script.
have you tried like next?
$var(re) = "^" + $var(client_prefix_in);
if(!$rU =~ $var(re)) {
Yes, I tryed that (hope I'm not wrong) and it also didn't do the job. Perhaps it should work?
On 9/22/10 10:38 PM, Iñaki Baz Castillo wrote:
2010/9/22 Daniel-Constantin Mierlamiconda@gmail.com:
Yes, using pcre_math is good when we need to perform a regular expression against a PV (which is not possible using "=~"). For example I use it to check if the RURI username matches a prefix extracted from a database:
if ! pcre_match("$rU", "^$var(client_prefix_in)") {
This works because the regular expression is compiled in runtime.
This would not be possible with "=~":
if ! $rU =~ "^$var(client_prefix_in)") {
It fails because the regular expression is compiled when parsing the config script.
have you tried like next?
$var(re) = "^" + $var(client_prefix_in);
if(!$rU =~ $var(re)) {
Yes, I tryed that (hope I'm not wrong) and it also didn't do the job. Perhaps it should work?
I expect so, but I haven't tried with 3.x -- iirc, I added it in openser some time ago (or maybe I only wanted to :-) ?!?!) -- i have to test it.
Thanks, Daniel
2010/9/22 Daniel-Constantin Mierla miconda@gmail.com:
if(!$rU =~ $var(re)) {
Yes, I tryed that (hope I'm not wrong) and it also didn't do the job. Perhaps it should work?
I expect so, but I haven't tried with 3.x -- iirc, I added it in openser some time ago (or maybe I only wanted to :-) ?!?!) -- i have to test it.
I'll try it in a few hours and let you know, but I think I already tryed it (note: in kamailio 1.5.4).
2010/9/22 Iñaki Baz Castillo ibc@aliax.net:
2010/9/22 Daniel-Constantin Mierla miconda@gmail.com:
if(!$rU =~ $var(re)) {
Yes, I tryed that (hope I'm not wrong) and it also didn't do the job. Perhaps it should work?
I expect so, but I haven't tried with 3.x -- iirc, I added it in openser some time ago (or maybe I only wanted to :-) ?!?!) -- i have to test it.
I'll try it in a few hours and let you know, but I think I already tryed it (note: in kamailio 1.5.4).
It *does* work. Not sure what I did in my previous tests but I didn't get it working, so you are right :)
On 9/22/10 11:33 PM, Iñaki Baz Castillo wrote:
2010/9/22 Iñaki Baz Castilloibc@aliax.net:
2010/9/22 Daniel-Constantin Mierlamiconda@gmail.com:
if(!$rU =~ $var(re)) {
Yes, I tryed that (hope I'm not wrong) and it also didn't do the job. Perhaps it should work?
I expect so, but I haven't tried with 3.x -- iirc, I added it in openser some time ago (or maybe I only wanted to :-) ?!?!) -- i have to test it.
I'll try it in a few hours and let you know, but I think I already tryed it (note: in kamailio 1.5.4).
It *does* work. Not sure what I did in my previous tests but I didn't get it working, so you are right :)
which versions have you tried this time?
Cheers, Daniel
2010/9/22 Daniel-Constantin Mierla miconda@gmail.com:
It *does* work. Not sure what I did in my previous tests but I didn't get it working, so you are right :)
which versions have you tried this time?
K 1.5.4.
On Sep 22, 2010 at 22:25, Iñaki Baz Castillo ibc@aliax.net wrote:
2010/9/22 marius zbihlei marius.zbihlei@1and1.ro:
this will be much better handled by a transformation
if ( $(fn{s.len}) > 62 ) {...}
Yes, using pcre_math is good when we need to perform a regular expression against a PV (which is not possible using "=~"). For example I use it to check if the RURI username matches a prefix extracted from a database:
if ! pcre_match("$rU", "^$var(client_prefix_in)") {
This works because the regular expression is compiled in runtime.
This would not be possible with "=~":
if ! $rU =~ "^$var(client_prefix_in)") {
It fails because the regular expression is compiled when parsing the config script.
With sr/ser/kamailio 3.x it should work. E.g.: $foo="test2"; if ($foo =~ "^tes"+"t["+1+"2]$") ...
works.
Andrei
On 9/23/10 9:46 AM, Andrei Pelinescu-Onciul wrote:
On Sep 22, 2010 at 22:25, Iñaki Baz Castilloibc@aliax.net wrote:
2010/9/22 marius zbihleimarius.zbihlei@1and1.ro:
this will be much better handled by a transformation
if ( $(fn{s.len})> 62 ) {...}
Yes, using pcre_math is good when we need to perform a regular expression against a PV (which is not possible using "=~"). For example I use it to check if the RURI username matches a prefix extracted from a database:
if ! pcre_match("$rU", "^$var(client_prefix_in)") {
This works because the regular expression is compiled in runtime.
This would not be possible with "=~":
if ! $rU =~ "^$var(client_prefix_in)") {
It fails because the regular expression is compiled when parsing the config script.
With sr/ser/kamailio 3.x it should work. E.g.: $foo="test2"; if ($foo =~ "^tes"+"t["+1+"2]$") ...
but can it be a variable in the right side? E.g.,
$bar="^tes"+"t["+1+"2]$"; if($foo =~ $bar) {
Cheers, Daniel
On Sep 23, 2010 at 09:56, Daniel-Constantin Mierla miconda@gmail.com wrote:
On 9/23/10 9:46 AM, Andrei Pelinescu-Onciul wrote:
On Sep 22, 2010 at 22:25, Iñaki Baz Castilloibc@aliax.net wrote:
2010/9/22 marius zbihleimarius.zbihlei@1and1.ro:
this will be much better handled by a transformation
if ( $(fn{s.len})> 62 ) {...}
Yes, using pcre_math is good when we need to perform a regular expression against a PV (which is not possible using "=~"). For example I use it to check if the RURI username matches a prefix extracted from a database:
if ! pcre_match("$rU", "^$var(client_prefix_in)") {
This works because the regular expression is compiled in runtime.
This would not be possible with "=~":
if ! $rU =~ "^$var(client_prefix_in)") {
It fails because the regular expression is compiled when parsing the config script.
With sr/ser/kamailio 3.x it should work. E.g.: $foo="test2"; if ($foo =~ "^tes"+"t["+1+"2]$") ...
but can it be a variable in the right side? E.g.,
ups, wrong example :-)
$bar="^tes"+"t["+1+"2]$"; if($foo =~ $bar) {
Yes, it will work.
Andrei
It worked! It was the pcre_match that was consuming all the memory available. Kamailio has been running for more than 24 hours, and memory comsumption is not increasing. Thank you very much!
Santiago Soares Fone: (41) 8488-0537
On Wed, Sep 22, 2010 at 12:21 PM, marius zbihlei marius.zbihlei@1and1.rowrote:
On 09/22/2010 06:08 PM, Klaus Darilion wrote:
Am 22.09.2010 15:31, schrieb Santiago Soares:
Thank's for the answers. Based on your explanation, I think that the problem is an external library. The kamailio process is taking all the system memory. When the system runs out of memory, it kills kamailio. But how can I find out what is the library?
This means that either the library is buggy, or the library is used wrongly (e.g. not releasing buffers ...). Let's take a look at your modules:
loadmodule "maxfwd.so" loadmodule "exec.so" loadmodule "tm.so" loadmodule "sl.so" loadmodule "dispatcher.so" loadmodule "textops.so" loadmodule "xlog.so" loadmodule "mi_fifo.so" loadmodule "pv.so"
I think they do not use any external libraries
loadmodule "regex.so"
This one uses libpcre. Maybe there is a problem with the regular expressions?
Hello I have some suggestions
# excessively long display name if (!pcre_match("$fn","^.{0,62}$$")) { sl_send_reply("403", "Forbidden - Display Name Max Size Exceeded"); exit; };
this will be much better handled by a transformation
if ( $(fn{s.len}) > 62 ) {...}
Make this change and see if you still have problems with OOM Killer
Marius
regards
Klaus
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Just a comment.
As a 'rookie' SER user I am impressed with your knowledge and problem solving skills.
Great work gentlemen!
Jeff
On 2010-09-23, at 2:01 PM, Santiago Soares wrote:
It worked! It was the pcre_match that was consuming all the memory available. Kamailio has been running for more than 24 hours, and memory comsumption is not increasing. Thank you very much!
Santiago Soares Fone: (41) 8488-0537
On Wed, Sep 22, 2010 at 12:21 PM, marius zbihlei marius.zbihlei@1and1.ro wrote: On 09/22/2010 06:08 PM, Klaus Darilion wrote:
Am 22.09.2010 15:31, schrieb Santiago Soares:
Thank's for the answers. Based on your explanation, I think that the problem is an external library. The kamailio process is taking all the system memory. When the system runs out of memory, it kills kamailio. But how can I find out what is the library?
This means that either the library is buggy, or the library is used wrongly (e.g. not releasing buffers ...). Let's take a look at your modules:
loadmodule "maxfwd.so" loadmodule "exec.so" loadmodule "tm.so" loadmodule "sl.so" loadmodule "dispatcher.so" loadmodule "textops.so" loadmodule "xlog.so" loadmodule "mi_fifo.so" loadmodule "pv.so"
I think they do not use any external libraries
loadmodule "regex.so"
This one uses libpcre. Maybe there is a problem with the regular expressions?
Hello I have some suggestions
# excessively long display name if (!pcre_match("$fn","^.{0,62}$$")) { sl_send_reply("403", "Forbidden - Display Name Max Size Exceeded"); exit; };
this will be much better handled by a transformation
if ( $(fn{s.len}) > 62 ) {...}
Make this change and see if you still have problems with OOM Killer
Marius
regards Klaus
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-
Andrei Pelinescu-Onciul -
Daniel-Constantin Mierla -
Iñaki Baz Castillo -
Jeff Russell -
Klaus Darilion -
Marius Zbihlei -
Santiago Soares