Hi.
My kamailio v4.4.5 installation ( -m 512 -M 64 ) stop sip processing with repeatable error:
Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:415]:
fm_search_defrag(): fm_search_defrag(0x7f39917fa010, 952); Free fragment not found! Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:498]: fm_malloc(): fm_malloc(0x7f39917fa010, 952) called from core: rvalue.c: rval_new_empty(224), module: core; Free fragment not foun
while it was enough free shared memory according to
kamcmd> core.shmmem m
{ total: 512 free: 472 used: 27 real_used: 39 max_used: 51m fragments: 37035 }
whether it was about package memory or smth else? Thanks!
Hello,
for how long it was running? Was it any stress test or just normal traffic?
The issue is related to pkg memory, to get the stats of it per each process, run:
kamcmd pkg.stats
Cheers, Daniel
On 15/02/2017 21:04, Dmitri Savolainen wrote:
Hi. My kamailio v4.4.5 installation ( -m 512 -M 64 ) stop sip processing with repeatable error:
Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:415]: fm_search_defrag(): fm_search_defrag(0x7f39917fa010, 952); Free fragment not found! Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:498]: fm_malloc(): fm_malloc(0x7f39917fa010, 952) called from core: rvalue.c: rval_new_empty(224), module: core; Free fragment not founwhile it was enough free shared memory according to
kamcmd> core.shmmem m { total: 512 free: 472 used: 27 real_used: 39 max_used: 51m fragments: 37035 }whether it was aboutpackage memory or smth else? Thanks!
Savolainen Dmitri
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
No any tests. It was running two weeks and crushed in out of peak period. i have never seen this before in the months of uptime. Will start monitoring pkg memory from this moment too.
2017-02-16 9:58 GMT+03:00 Daniel-Constantin Mierla miconda@gmail.com:
Hello,
for how long it was running? Was it any stress test or just normal traffic?
The issue is related to pkg memory, to get the stats of it per each process, run:
kamcmd pkg.stats
Cheers, Daniel
On 15/02/2017 21:04, Dmitri Savolainen wrote:
Hi. My kamailio v4.4.5 installation ( -m 512 -M 64 ) stop sip processing with repeatable error:
Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:415]: fm_search_defrag(): fm_search_defrag(0x7f39917fa010, 952); Free fragment not found! Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:498]: fm_malloc(): fm_malloc(0x7f39917fa010, 952) called from core: rvalue.c: rval_new_empty(224), module: core; Free fragment not foun
while it was enough free shared memory according to
kamcmd> core.shmmem m { total: 512 free: 472 used:
27 real_used: 39 max_used: 51m fragments: 37035 }
whether it was about package memory or smth else? Thanks!
Savolainen Dmitri
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierlawww.twitter.com/miconda -- www.linkedin.com/in/miconda Kamailio Advanced Training - Mar 6-8 (Europe) and Mar 20-22 (USA) - www.asipto.com Kamailio World Conference - May 8-10, 2017 - www.kamailioworld.com
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
If it is already running for some time, run:
kamcmd corex.pkg_summary idx 1
and then get the related qm/fm status/summary log messages from syslog and send them here to check if there is any potential pkg leak in the sip worker one.
Cheers, Daniel
On 16/02/2017 08:27, Dmitri Savolainen wrote:
No any tests. It was running two weeks and crushed in out of peak period. i have never seen this before in the months of uptime. Will start monitoring pkg memory from this moment too.
2017-02-16 9:58 GMT+03:00 Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com>:
Hello, for how long it was running? Was it any stress test or just normal traffic? The issue is related to pkg memory, to get the stats of it per each process, run: kamcmd pkg.stats Cheers, Daniel On 15/02/2017 21:04, Dmitri Savolainen wrote:Hi. My kamailio v4.4.5 installation ( -m 512 -M 64 ) stop sip processing with repeatable error: Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:415]: fm_search_defrag(): fm_search_defrag(0x7f39917fa010, 952); Free fragment not found! Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:498]: fm_malloc(): fm_malloc(0x7f39917fa010, 952) called from core: rvalue.c: rval_new_empty(224), module: core; Free fragment not foun while it was enough free shared memory according to kamcmd> core.shmmem m { total: 512 free: 472 used: 27 real_used: 39 max_used: 51m fragments: 37035 } whether it was aboutpackage memory or smth else? Thanks! -- Savolainen Dmitri _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users <http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users>-- Daniel-Constantin Mierla www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Kamailio Advanced Training - Mar 6-8 (Europe) and Mar 20-22 (USA) - www.asipto.com <http://www.asipto.com> Kamailio World Conference - May 8-10, 2017 - www.kamailioworld.com <http://www.kamailioworld.com> _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users <http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users>-- Savolainen Dmitri
create github issue for this https://github.com/kamailio/kamailio/issues/1001
2017-02-16 10:34 GMT+03:00 Daniel-Constantin Mierla miconda@gmail.com:
If it is already running for some time, run:
kamcmd corex.pkg_summary idx 1
and then get the related qm/fm status/summary log messages from syslog and send them here to check if there is any potential pkg leak in the sip worker one.
Cheers, Daniel
On 16/02/2017 08:27, Dmitri Savolainen wrote:
No any tests. It was running two weeks and crushed in out of peak period. i have never seen this before in the months of uptime. Will start monitoring pkg memory from this moment too.
2017-02-16 9:58 GMT+03:00 Daniel-Constantin Mierla miconda@gmail.com:
Hello,
for how long it was running? Was it any stress test or just normal traffic?
The issue is related to pkg memory, to get the stats of it per each process, run:
kamcmd pkg.stats
Cheers, Daniel
On 15/02/2017 21:04, Dmitri Savolainen wrote:
Hi. My kamailio v4.4.5 installation ( -m 512 -M 64 ) stop sip processing with repeatable error:
Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:415]: fm_search_defrag(): fm_search_defrag(0x7f39917fa010, 952); Free fragment not found! Feb 15 21:45:05 kamailio[14173] ERROR: <core> [mem/f_malloc.c:498]: fm_malloc(): fm_malloc(0x7f39917fa010, 952) called from core: rvalue.c: rval_new_empty(224), module: core; Free fragment not foun
while it was enough free shared memory according to
kamcmd> core.shmmem m { total: 512 free: 472 used: 27 real_used: 39 max_used: 51m fragments: 37035 }
whether it was about package memory or smth else? Thanks!
Savolainen Dmitri
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierlawww.twitter.com/miconda -- www.linkedin.com/in/miconda Kamailio Advanced Training - Mar 6-8 (Europe) and Mar 20-22 (USA) - www.asipto.com Kamailio World Conference - May 8-10, 2017 - www.kamailioworld.com
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cg i-bin/mailman/listinfo/sr-users
-- Savolainen Dmitri
-- Daniel-Constantin Mierlawww.twitter.com/miconda -- www.linkedin.com/in/miconda Kamailio Advanced Training - Mar 6-8 (Europe) and Mar 20-22 (USA) - www.asipto.com Kamailio World Conference - May 8-10, 2017 - www.kamailioworld.com
-
Daniel-Constantin Mierla -
Dmitri Savolainen