17 Aug
2011
17 Aug
'11
11:27 a.m.
Hi,
we have a script running on our Kamailio servers which dumps all online
users with kamctl ul show, parses it and then sends it to database. (For
explanation, we started doing it this way because we noticed at least with
an older (open)ser release that SER would completely stop working for a
couple of seconds while writing registered users to database every 5
minutes. When dumping it, we don't see any disruption.)
With Kamailio 1.5 this works, with Kamailio 3.1 it works in the beginning,
but when you pass the 2000 online user mark, it just stops working.
I've started the Kamailio completely empty, then let users register against
it. I did a "kamctl ul show" every 5 seconds, this is what the results were:
Domain:: location table=512 records=82 max_slot=2
Domain:: location table=512 records=412 max_slot=4
Domain:: location table=512 records=640 max_slot=5
Domain:: location table=512 records=973 max_slot=7
Domain:: location table=512 records=1209 max_slot=8
Domain:: location table=512 records=1454 max_slot=11
Domain:: location table=512 records=1667 max_slot=12
Domain:: location table=512 records=1907 max_slot=12
500 command 'ul_dump' failed
500 command 'ul_dump' failed
500 command 'ul_dump' failed
500 command 'ul_dump' failed
500 command 'ul_dump' failed
500 command 'ul_dump' failed
Is there a hard limit for dumping users through the fifo? Why? Or should we
do it differently?
Best Regards,
Sebastian
--
Dipl.-Inf.
Sebastian Damm - VoIP-Engineer - damm(a)sipgate.de
sipgate GmbH - Gladbacher Str. 74 - 40219 Düsseldorf
HRB Düsseldorf 39841 - Geschäftsführer: Thilo Salmon, Tim Mois
Steuernummer: 106/5724/7147, Umsatzsteuer-ID: DE219349391
www.sipgate.de - www.sipgate.at - www.sipgate.co.uk
17 Aug
17 Aug
11:33 a.m.
On Wednesday 17 August 2011, Sebastian Damm wrote:
we have a script running on our Kamailio servers which
dumps all online
users with kamctl ul show, parses it and then sends it to database. (For
explanation, we started doing it this way because we noticed at least with
an older (open)ser release that SER would completely stop working for a
couple of seconds while writing registered users to database every 5
minutes. When dumping it, we don't see any disruption.)
Hi Sebastian,
what about fixing this bug instead of using this workaround? This sounds
really inefficient.
With Kamailio 1.5 this works, with Kamailio 3.1 it
works in the beginning,
but when you pass the 2000 online user mark, it just stops working.
I've started the Kamailio completely empty, then let users register against
it. I did a "kamctl ul show" every 5 seconds, this is what the results
were:
Domain:: location table=512 records=82 max_slot=2
[..]
500 command 'ul_dump' failed
Is there a hard limit for dumping users through the fifo? Why? Or should we
do it differently?
I don't think there is a hard limit. Do you see any errors (e.g. memory
allocation issues) in the server logs? Another idea, if you're somehow stuck
to the old/ buggy version of the server, what about using DB_ONLY mode?
Best regards,
Henning
1:41 p.m.
Hi Henning,
On Wed, Aug 17, 2011 at 11:33 AM, Henning Westerholt <
henning.westerholt(a)1und1.de> wrote:
On Wednesday 17 August 2011, Sebastian Damm wrote:
Actually, piping the output through awk and generating a file for LOAD DATA
INFILE" is not too bad. Additionally, we need to have that data in more than
one DB, which prevents us from using one of the built-in db modes. We use an
own DB mode which reads only on startup and then never touches the DB again.
we have a script running on our Kamailio servers
which dumps all online
users with kamctl ul show, parses it and then sends it to database. (For
explanation, we started doing it this way because we noticed at least
with
an older (open)ser release that SER would
completely stop working for a
couple of seconds while writing registered users to database every 5
minutes. When dumping it, we don't see any disruption.)
what about fixing this bug instead of using this workaround? This sounds
really inefficient.
I don't think there is a hard limit. Do you see
any errors (e.g. memory
allocation issues) in the server logs? Another idea, if you're somehow
stuck
to the old/ buggy version of the server, what about using DB_ONLY mode?
I don't see any errors in the log. And actually, that's the only bug left
preventing our migration from 1.5 to 3.1. And I'd say, DB_ONLY is not an
option, should be a lot less performant than if Kamailio keeps the state in
memory (although I haven't benchmarked it).
Other ideas?
Best regards,
Sebastian
--
Dipl.-Inf.
Sebastian Damm - VoIP-Engineer - damm(a)sipgate.de
sipgate GmbH - Gladbacher Str. 74 - 40219 Düsseldorf
HRB Düsseldorf 39841 - Geschäftsführer: Thilo Salmon, Tim Mois
Steuernummer: 106/5724/7147, Umsatzsteuer-ID: DE219349391
www.sipgate.de - www.sipgate.at - www.sipgate.co.uk
2:08 p.m.
Hello,
On 8/17/11 1:41 PM, Sebastian Damm wrote:
Hi Henning,
On Wed, Aug 17, 2011 at 11:33 AM, Henning Westerholt
<henning.westerholt(a)1und1.de <mailto:henning.westerholt@1und1.de>> wrote:
On Wednesday 17 August 2011, Sebastian Damm wrote:
not sure what was the default in latest 1.x, maybe MI was using the
system memory manager. Since location records can need quite some memory
if there are lot of entries, it may be better to use the system memory
directly in MI to build the response to the commands.
In 3.1, the option is to edit lib/kmi/Makefile and turn on the define of
MI_SYSTEM_MALLOC
Then recompile/reinstall and try again. Let us know the results.
As alternative, in 3.1, you can do RPC command with sercmd through ctl
module in order to dump location records. There you can control the size
of the buffers used to build the output:
*
http://kamailio.org/docs/modules/stable/modules/ctl.html#binrpc_max_body_si…
Cheers,
Daniel
--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Oct 10-13, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda
we have a script running on our Kamailio servers
which dumps all
online
users with kamctl ul show, parses it and then
sends it to
database. (For
explanation, we started doing it this way because
we noticed at
least with
an older (open)ser release that SER would
completely stop
working for a
couple of seconds while writing registered users
to database every 5
minutes. When dumping it, we don't see any disruption.)
what about fixing this bug instead of using this workaround? This
sounds
really inefficient.
Actually, piping the output through awk and generating a file for LOAD
DATA INFILE" is not too bad. Additionally, we need to have that data
in more than one DB, which prevents us from using one of the built-in
db modes. We use an own DB mode which reads only on startup and then
never touches the DB again.
I don't think there is a hard limit. Do you see any errors (e.g.
memory
allocation issues) in the server logs? Another idea, if you're
somehow stuck
to the old/ buggy version of the server, what about using DB_ONLY
mode?
I don't see any errors in the log. And actually, that's the only bug
left preventing our migration from 1.5 to 3.1. And I'd say, DB_ONLY is
not an option, should be a lot less performant than if Kamailio keeps
the state in memory (although I haven't benchmarked it).
Other ideas? 
25 Aug
25 Aug
2:21 p.m.
MI_SYSTEM_MALLOC did the trick, dumping >100k AOR worked that way.
Should this be mentioned somewhere (like documentation) or is a
permanent fix needed?
Cheers, Marcus
On Wed, Aug 17, 2011 at 2:08 PM, Daniel-Constantin Mierla
<miconda(a)gmail.com> wrote:
Hello,
On 8/17/11 1:41 PM, Sebastian Damm wrote:
Hi Henning,
On Wed, Aug 17, 2011 at 11:33 AM, Henning Westerholt
<henning.westerholt(a)1und1.de> wrote:
--
Dipl.-Inf. (FH)
Marcus Hunger - hunger(a)sipgate.de
Telefon: +49 (0)211-63 55 55-61
Telefax: +49 (0)211-63 55 55-22
sipgate GmbH - Gladbacher Str. 74 - 40219 Düsseldorf
HRB Düsseldorf 39841 - Geschäftsführer: Thilo Salmon, Tim Mois
Steuernummer: 106 / 5724 / 7147, Umsatzsteuer-ID: DE219349391
www.sipgate.de - www.sipgate.at - www.sipgate.co.uk
On Wednesday 17 August 2011, Sebastian Damm wrote:
Actually, piping the output through awk and generating a file for LOAD DATA
INFILE" is not too bad. Additionally, we need to have that data in more than
one DB, which prevents us from using one of the built-in db modes. We use an
own DB mode which reads only on startup and then never touches the DB again.
we have a script running on our Kamailio servers
which dumps all online
users with kamctl ul show, parses it and then sends it to database. (For
explanation, we started doing it this way because we noticed at least
with
an older (open)ser release that SER would completely stop working for a
couple of seconds while writing registered users to database every 5
minutes. When dumping it, we don't see any disruption.)
what about fixing this bug instead of using this workaround? This sounds
really inefficient.
I don't think there is a hard limit. Do you see any errors (e.g. memory
allocation issues) in the server logs? Another idea, if you're somehow
stuck
to the old/ buggy version of the server, what about using DB_ONLY mode?
I don't see any errors in the log. And actually, that's the only bug left
preventing our migration from 1.5 to 3.1. And I'd say, DB_ONLY is not an
option, should be a lot less performant than if Kamailio keeps the state in
memory (although I haven't benchmarked it).
Other ideas?
not sure what was the default in latest 1.x, maybe MI was using the system
memory manager. Since location records can need quite some memory if there
are lot of entries, it may be better to use the system memory directly in MI
to build the response to the commands.
In 3.1, the option is to edit lib/kmi/Makefile and turn on the define of
MI_SYSTEM_MALLOC
Then recompile/reinstall and try again. Let us know the results.
As alternative, in 3.1, you can do RPC command with sercmd through ctl
module in order to dump location records. There you can control the size of
the buffers used to build the output:
*
http://kamailio.org/docs/modules/stable/modules/ctl.html#binrpc_max_body_si…
Cheers,
Daniel
--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Oct 10-13, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users(a)lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
7:11 p.m.
Hello,
On 8/25/11 2:21 PM, Marcus Hunger wrote:
MI_SYSTEM_MALLOC did the trick, dumping>100k AOR
worked that way.
Should this be mentioned somewhere (like documentation) or is a
permanent fix needed?
maybe we could make this default. Documentation about it
should be added
in several places, like the mi modules and eventually to the MI commands
known to dump large about of data, like dumping usrloc, dialplan rules,
lcr records ... If you like to add a bit to documentations, you are most
welcomed, I can grant you git write access.
Cheers,
Daniel
Cheers, Marcus
On Wed, Aug 17, 2011 at 2:08 PM, Daniel-Constantin Mierla
<miconda(a)gmail.com> wrote:
--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Oct 10-13, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda
Hello,
On 8/17/11 1:41 PM, Sebastian Damm wrote:
Hi Henning,
On Wed, Aug 17, 2011 at 11:33 AM, Henning Westerholt
<henning.westerholt(a)1und1.de> wrote:
On Wednesday 17 August 2011, Sebastian Damm
wrote:
Actually, piping the output through awk and generating a file
for LOAD DATA
INFILE" is not too bad. Additionally, we need to have that data in more than
one DB, which prevents us from using one of the built-in db modes. We use an
own DB mode which reads only on startup and then never touches the DB again.
we have a script running on our Kamailio servers
which dumps all online
users with kamctl ul show, parses it and then sends it to database. (For
explanation, we started doing it this way because we noticed at least
with
an older (open)ser release that SER would completely stop working for a
couple of seconds while writing registered users to database every 5
minutes. When dumping it, we don't see any disruption.)
what about fixing this
bug instead of using this workaround? This sounds
really inefficient. I don't think there is a hard limit. Do you
see any errors (e.g. memory
allocation issues) in the server logs? Another idea, if you're somehow
stuck
to the old/ buggy version of the server, what about using DB_ONLY mode?
I
don't see any errors in the log. And actually, that's the only bug left
preventing our migration from 1.5 to 3.1. And I'd say, DB_ONLY is not an
option, should be a lot less performant than if Kamailio keeps the state in
memory (although I haven't benchmarked it).
Other ideas?
not sure what was the default in latest 1.x, maybe MI was using the system
memory manager. Since location records can need quite some memory if there
are lot of entries, it may be better to use the system memory directly in MI
to build the response to the commands.
In 3.1, the option is to edit lib/kmi/Makefile and turn on the define of
MI_SYSTEM_MALLOC
Then recompile/reinstall and try again. Let us know the results.
As alternative, in 3.1, you can do RPC command with sercmd through ctl
module in order to dump location records. There you can control the size of
the buffers used to build the output:
*
http://kamailio.org/docs/modules/stable/modules/ctl.html#binrpc_max_body_si…
Cheers,
Daniel
--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Oct 10-13, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users(a)lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
4633
days inactive
4641
days old
5 comments
4 participants
participants (4)
-
Daniel-Constantin Mierla -
Henning Westerholt -
Marcus Hunger -
Sebastian Damm