6 Oct
2020
6 Oct
'20
1:28 a.m.
Hello,
I'm trying to periodically get all registered contacts from the Kamailio
memory in the json format.
JSONRPCS module have 3 different types of transport to get this data. I
just wonder what will be the safest transport for the Kamailio to get all
this data? Because all this data it's not so important, I want to gather it
without affecting the main process or with the minimal effect on the
Kamailio SIP Processing module.
1. The HTTP transport - very useful, working async, but because it depends
on the xhttp module - which works synchronously I'm afraid that for big
data with contacts this type of transport can affect the kamailio
performance.
2. FIFO - this transport also is very fast, working locally - but it's
synchronous, I'm not sure how it will affect Kamailio SIP Processing when
the local database will have few millions of users registered.
3. DGRAM - also seems to be very fast, working locally or remotely through
the UDP protocol, but again it's synchronous and I'm not sure that this
transport is returning any output for the request, I tried - but didn't get
back anything, while for changing something it worked fine. Maybe it was a
misconfiguration in my config file.
What would be your recommendation for this case?
Thank you.
7 Oct
7 Oct
11:27 p.m.
Hello,
Your requirement to frequently dump millions of user location entries on a loaded
production system sounds a bit odd to me. Maybe you can elaborate a bit on your
requirements. If you have the requirement to access this data frequently, maybe getting it
from the database directly is the better idea. If you want to get it from kamailio, choose
a module that provides the option to configure the number of children (e.g. like
dgram_workers in jsonrpcs) and set it to a sufficient number.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com<https://gilawa.com/>
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of Ilie Soltanici
Sent: Tuesday, October 6, 2020 12:29 AM
To: Kamailio (SER) - Users Mailing List <sr-users(a)lists.kamailio.org>
Subject: [SR-Users] jsonrpcs - safest way to get data
Hello,
I'm trying to periodically get all registered contacts from the Kamailio memory in the
json format.
JSONRPCS module have 3 different types of transport to get this data. I just wonder what
will be the safest transport for the Kamailio to get all this data? Because all this data
it's not so important, I want to gather it without affecting the main process or with
the minimal effect on the Kamailio SIP Processing module.
1. The HTTP transport - very useful, working async, but because it depends on the xhttp
module - which works synchronously I'm afraid that for big data with contacts this
type of transport can affect the kamailio performance.
2. FIFO - this transport also is very fast, working locally - but it's synchronous,
I'm not sure how it will affect Kamailio SIP Processing when the local database will
have few millions of users registered.
3. DGRAM - also seems to be very fast, working locally or remotely through the UDP
protocol, but again it's synchronous and I'm not sure that this transport is
returning any output for the request, I tried - but didn't get back anything, while
for changing something it worked fine. Maybe it was a misconfiguration in my config
file.
What would be your recommendation for this case?
Thank you.
8 Oct
8 Oct
2:10 p.m.
Hello,
OK, so the reason to dump all contacts periodically from the Kamailio
memory is that I have a task to monitor the network latency for every
extension, which in this case we can do with the KA function from the
usrloc module.
This information it's not in the DB, we can get this information either
from the kamailio memory, or by parsing the log file. I was thinking that
the first option can be better because for the second one Kamailio will
have to do a lot of IO operation to the disk, which can affect somehow the
performance, even if the local storage it's based on SSD's.
I was trying to use dgram for this case - but I'm not getting any output
from the kamailio by sending any request.
For ex the following command:
echo '{"id": "1", "jsonrpc": "2.0",
"method": "core.psx" }' > /dev/udp/
127.0.0.1/8090
gave the following output to the log file:
Oct 8 11:20:47 devsrv devsrv[11468]: DEBUG: jsonrpcs
[jsonrpcs_sock.c:608]: jsonrpc_dgram_server():
received {"jsonrpc": "2.0",
"method": "core.ps", "id": 1}
Oct 8 11:20:47 devsrv devsrv[11468]: DEBUG: jsonrpcs
[jsonrpcs_sock.c:620]: jsonrpc_dgram_server(): buf is {"jsonrpc":
"2.0",
"method": "core.ps", "id": 1}#012 and we have received 48
bytes
Oct 8 11:20:47 devsrv devsrv[11468]: DEBUG: jsonrpcs
[jsonrpcs_sock.c:629]: jsonrpc_dgram_server(): command executed - result:
[200] [0x18b5430]
[{#012#011"jsonrpc":#011"2.0",#012#011"result":#011[11432,
"main process -
attendant", 11434, "udp receiver child=0 sock=127.0.0.1:5060", 11435,
"udp receiver child=1 sock=127.0.0.1:5060", 11436, "udp receiver child=0
sock=172.16.50.231:5060", 11437, "udp receiver child=1 sock=
172.16.50.231:5060", 11438, "udp receiver child=2
sock=172.16.50.231:5060",
11439, "udp receiver child=3 sock=172.16.50.231:5060", 11440, "udp
receiver child=4 sock=172.16.50.231:5060", 11441, "udp receiver child=5
sock=172.16.50.231:5060", 11442, "udp receiver child=6 sock=
172.16.50.231:5060", 11443, "udp receiver child=7
sock=172.16.50.231:5060",
11444, "udp receiver child=0 sock=192.168.0.10:5060", 11445, "udp
receiver child=1sock=192.168.0.10:5060", 11446, "udp receiver child=2
sock=192.168.0.10:5060", 11447, "udp receiver child=3 sock=
192.168.0.10:5060", 11448, "udp receiver child=4 sock=192.168.0.10:5060",
11449, "udp receiver child=5 sock=192.168.0.10:5060", 11450, "udp
receiver child=6 sock=192.168.0.10:5060", 11451, "udp receiver child=7
sock=192.168.0.10:5060", 11452, "udp receiver child=0 sock=
192.168.0.10:5090", 11453, "udp receiver child=1 sock=192.168.0.10:5090",
11454, "udp receiver child=2 sock=192.168.0.10:5090", 11455, "udp
receiver child=3 sock=192.168.0.10:5090", 11456, "udp receiver child=4
sock=192.168.0.10:5090", 11457, "udp receiver child=5 sock=
192.168.0.10:5090", 11458, "udp receiver child=6 sock=192.168.0.10:5090",
11459, "udp receiver child=7 sock=192.168.0.10:5090", 11460, "slow
timer", 11461, "timer", 11462, "secondary timer", 11463,
"Async Task
Worker", 11464, "Async Task Worker", 11465, "Async Task Worker",
11466,
"Async Task Worker", 11467, "JSONRPCS FIFO", 11468, "JSONRPCS
DATAGRAM",
11469, "USRLOC Timer", 11470, "USRLOC Timer", 11471, "USRLOC
Timer", 11472,
"USRLOC Timer", 11473, "Dialog Clean Timer", 11474, "ctl
handler", 11475,
"TIMER UAC REG", 11476, "SNMP AgentX", 11478, "Http Async
Worker", 11479,
"tcp receiver (generic) child=0", 11480, "tcp receiver (generic)
child=1",
11481, "tcp receiver (generic) child=2", 11482, "tcp receiver (generic)
child=3", 11483, "tcp receiver (generic) child=4", 11484, "tcp
receiver
(generic) child=5", 11485, "tcp receiver (generic) child=6", 11486,
"tcp
receiver (generic) child=7", 11487, "tcp main
process"],#012#011"id":#0111#012}]
I expected to receive this output back to the udp client in this case.
Also, the following request it's not getting anything even in the log file:
echo '{"id": "1", "jsonrpc": "2.0",
"method": "ul.dump" }' > /dev/udp/
127.0.0.1/8090
If there is a way to have all this data directly in the database - that
would be perfect (I already created a feature request for that, maybe in
the next versions that will be implemented).
Regards,
On Wed, 7 Oct 2020 at 21:27, Henning Westerholt <hw(a)skalatan.de> wrote:
Hello,
Your requirement to frequently dump millions of user location entries on a
loaded production system sounds a bit odd to me. Maybe you can elaborate a
bit on your requirements. If you have the requirement to access this data
frequently, maybe getting it from the database directly is the better idea.
If you want to get it from kamailio, choose a module that provides the
option to configure the number of children (e.g. like dgram_workers in
jsonrpcs) and set it to a sufficient number.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces(a)lists.kamailio.org> *On Behalf Of *Ilie
Soltanici
*Sent:* Tuesday, October 6, 2020 12:29 AM
*To:* Kamailio (SER) - Users Mailing List <sr-users(a)lists.kamailio.org>
*Subject:* [SR-Users] jsonrpcs - safest way to get data
Hello,
I'm trying to periodically get all registered contacts from the Kamailio
memory in the json format.
JSONRPCS module have 3 different types of transport to get this data. I
just wonder what will be the safest transport for the Kamailio to get all
this data? Because all this data it's not so important, I want to gather it
without affecting the main process or with the minimal effect on the
Kamailio SIP Processing module.
1. The HTTP transport - very useful, working async, but because it depends
on the xhttp module - which works synchronously I'm afraid that for big
data with contacts this type of transport can affect the kamailio
performance.
2. FIFO - this transport also is very fast, working locally - but it's
synchronous, I'm not sure how it will affect Kamailio SIP Processing when
the local database will have few millions of users registered.
3. DGRAM - also seems to be very fast, working locally or remotely through
the UDP protocol, but again it's synchronous and I'm not sure that this
transport is returning any output for the request, I tried - but didn't get
back anything, while for changing something it worked fine. Maybe it was a
misconfiguration in my config file.
What would be your recommendation for this case?
Thank you.
9 Oct
9 Oct
2:46 p.m.
Hello,
do you need to fetch for all users at the same time? If not, you can
fetch for one user at a time, like it is done by:
kamctl ul show user@domain
Regarding storing to database, of course it is open source and pretty
much anything can be implemented, however if you talk about millions of
active users, probably the database system will become overloaded by
doing updates to all records during the rather-short keepalive interval:
say you have 10mil active users and keepalive every 20-30secs (for
keeping nat pinhole open), then there are quire some updates performed
to db.
If someone wants to add db persistency for keepalive attributes, it is
fine for me, just make it based on a modparam.
Cheers,
Daniel
On 08.10.20 13:10, Ilie Soltanici wrote:
Hello,
OK, so the reason to dump all contacts periodically from the Kamailio
memory is that I have a task to monitor the network latency for every
extension, which in this case we can do with the KA function from the
usrloc module.
This information it's not in the DB, we can get this information
either from the kamailio memory, or by parsing the log file. I was
thinking that the first option can be better because for the second
one Kamailio will have to do a lot of IO operation to the disk, which
can affect somehow the performance, even if the local storage it's
based on SSD's.
I was trying to use dgram for this case - but I'm not getting any
output from the kamailio by sending any request.
For ex the following command:
echo '{"id": "1", "jsonrpc": "2.0",
"method": "core.psx" }' >
/dev/udp/127.0.0.1/8090 <http://127.0.0.1/8090>
gave the following output to the log file:
Oct 8 11:20:47 devsrv devsrv[11468]: DEBUG: jsonrpcs
[jsonrpcs_sock.c:608]: jsonrpc_dgram_server(): received
{"jsonrpc": "2.0", "method": "core.ps
<http://core.ps>", "id": 1}
Oct 8 11:20:47 devsrv devsrv[11468]: DEBUG: jsonrpcs
[jsonrpcs_sock.c:620]: jsonrpc_dgram_server(): buf is {"jsonrpc":
"2.0", "method": "core.ps <http://core.ps>",
"id": 1}#012 and we
have received 48 bytes
Oct 8 11:20:47 devsrv devsrv[11468]: DEBUG: jsonrpcs
[jsonrpcs_sock.c:629]: jsonrpc_dgram_server(): command executed -
result: [200] [0x18b5430]
[{#012#011"jsonrpc":#011"2.0",#012#011"result":#011[11432,
"main
process - attendant", 11434, "udp receiver child=0
sock=127.0.0.1:5060 <http://127.0.0.1:5060>", 11435, "udp receiver
child=1 sock=127.0.0.1:5060 <http://127.0.0.1:5060>", 11436, "udp
receiver child=0 sock=172.16.50.231:5060
<http://172.16.50.231:5060>", 11437, "udp receiver child=1
sock=172.16.50.231:5060 <http://172.16.50.231:5060>", 11438, "udp
receiver child=2 sock=172.16.50.231:5060
<http://172.16.50.231:5060>", 11439, "udp receiver child=3
sock=172.16.50.231:5060 <http://172.16.50.231:5060>", 11440, "udp
receiver child=4 sock=172.16.50.231:5060
<http://172.16.50.231:5060>", 11441, "udp receiver child=5
sock=172.16.50.231:5060 <http://172.16.50.231:5060>", 11442, "udp
receiver child=6 sock=172.16.50.231:5060
<http://172.16.50.231:5060>", 11443, "udp receiver child=7
sock=172.16.50.231:5060 <http://172.16.50.231:5060>", 11444, "udp
receiver child=0 sock=192.168.0.10:5060
<http://192.168.0.10:5060>", 11445, "udp receiver
child=1sock=192.168.0.10:5060 <http://192.168.0.10:5060>", 11446,
"udp receiver child=2 sock=192.168.0.10:5060
<http://192.168.0.10:5060>", 11447, "udp receiver child=3
sock=192.168.0.10:5060 <http://192.168.0.10:5060>", 11448, "udp
receiver child=4 sock=192.168.0.10:5060
<http://192.168.0.10:5060>", 11449, "udp receiver child=5
sock=192.168.0.10:5060 <http://192.168.0.10:5060>", 11450, "udp
receiver child=6 sock=192.168.0.10:5060
<http://192.168.0.10:5060>", 11451, "udp receiver child=7
sock=192.168.0.10:5060 <http://192.168.0.10:5060>", 11452, "udp
receiver child=0 sock=192.168.0.10:5090
<http://192.168.0.10:5090>", 11453, "udp receiver child=1
sock=192.168.0.10:5090 <http://192.168.0.10:5090>", 11454, "udp
receiver child=2 sock=192.168.0.10:5090
<http://192.168.0.10:5090>", 11455, "udp receiver child=3
sock=192.168.0.10:5090 <http://192.168.0.10:5090>", 11456, "udp
receiver child=4 sock=192.168.0.10:5090
<http://192.168.0.10:5090>", 11457, "udp receiver child=5
sock=192.168.0.10:5090 <http://192.168.0.10:5090>", 11458, "udp
receiver child=6 sock=192.168.0.10:5090
<http://192.168.0.10:5090>", 11459, "udp receiver child=7
sock=192.168.0.10:5090 <http://192.168.0.10:5090>", 11460, "slow
timer", 11461, "timer", 11462, "secondary timer", 11463,
"Async
Task Worker", 11464, "Async Task Worker", 11465, "Async Task
Worker", 11466, "Async Task Worker", 11467, "JSONRPCS
FIFO",
11468, "JSONRPCS DATAGRAM", 11469, "USRLOC Timer", 11470,
"USRLOC
Timer", 11471, "USRLOC Timer", 11472, "USRLOC Timer",
11473,
"Dialog Clean Timer", 11474, "ctl handler", 11475, "TIMER
UAC
REG", 11476, "SNMP AgentX", 11478, "Http Async Worker",
11479,
"tcp receiver (generic) child=0", 11480, "tcp receiver (generic)
child=1", 11481, "tcp receiver (generic) child=2", 11482, "tcp
receiver (generic) child=3", 11483, "tcp receiver (generic)
child=4", 11484, "tcp receiver (generic) child=5", 11485, "tcp
receiver (generic) child=6", 11486, "tcp receiver (generic)
child=7", 11487, "tcp main
process"],#012#011"id":#0111#012}]
I expected to receive this output back to the udp client in this case.
Also, the following request it's not getting anything even in the log
file:
echo '{"id": "1", "jsonrpc": "2.0",
"method": "ul.dump" }' >
/dev/udp/127.0.0.1/8090 <http://127.0.0.1/8090>
If there is a way to have all this data directly in the database -
that would be perfect (I already created a feature request for that,
maybe in the next versions that will be implemented).
Regards,
On Wed, 7 Oct 2020 at 21:27, Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
Hello,
Your requirement to frequently dump millions of user location
entries on a loaded production system sounds a bit odd to me.
Maybe you can elaborate a bit on your requirements. If you have
the requirement to access this data frequently, maybe getting it
from the database directly is the better idea. If you want to get
it from kamailio, choose a module that provides the option to
configure the number of children (e.g. like dgram_workers in
jsonrpcs) and set it to a sufficient number.
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
<https://skalatan.de/blog/>
Kamailio services – https://gilawa.com <https://gilawa.com/>
*From:* sr-users <sr-users-bounces(a)lists.kamailio.org
<mailto:sr-users-bounces@lists.kamailio.org>> *On Behalf Of *Ilie
Soltanici
*Sent:* Tuesday, October 6, 2020 12:29 AM
*To:* Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>>
*Subject:* [SR-Users] jsonrpcs - safest way to get data
Hello,
I'm trying to periodically get all registered contacts from the
Kamailio memory in the json format.
JSONRPCS module have 3 different types of transport to get this
data. I just wonder what will be the safest transport for the
Kamailio to get all this data? Because all this data it's not so
important, I want to gather it without affecting the main process
or with the minimal effect on the Kamailio SIP Processing module.
1. The HTTP transport - very useful, working async, but because it
depends on the xhttp module - which works synchronously I'm afraid
that for big data with contacts this type of transport can affect
the kamailio performance.
2. FIFO - this transport also is very fast, working locally - but
it's synchronous, I'm not sure how it will affect Kamailio SIP
Processing when the local database will have few millions of users
registered.
3. DGRAM - also seems to be very fast, working locally or remotely
through the UDP protocol, but again it's synchronous and I'm not
sure that this transport is returning any output for the request,
I tried - but didn't get back anything, while for changing
something it worked fine. Maybe it was a misconfiguration in my
config file.
What would be your recommendation for this case?
Thank you.
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
1511
days inactive
1515
days old
3 comments
3 participants
participants (3)
-
Daniel-Constantin Mierla -
Henning Westerholt -
Ilie Soltanici