[SR-Users] jsonrpcs - safest way to get data
Daniel-Constantin Mierla
miconda at gmail.com
Fri Oct 9 13:46:30 CEST 2020
Hello,
do you need to fetch for all users at the same time? If not, you can
fetch for one user at a time, like it is done by:
kamctl ul show user at domain
Regarding storing to database, of course it is open source and pretty
much anything can be implemented, however if you talk about millions of
active users, probably the database system will become overloaded by
doing updates to all records during the rather-short keepalive interval:
say you have 10mil active users and keepalive every 20-30secs (for
keeping nat pinhole open), then there are quire some updates performed
to db.
If someone wants to add db persistency for keepalive attributes, it is
fine for me, just make it based on a modparam.
Cheers,
Daniel
On 08.10.20 13:10, Ilie Soltanici wrote:
> Hello,
>
> OK, so the reason to dump all contacts periodically from the Kamailio
> memory is that I have a task to monitor the network latency for every
> extension, which in this case we can do with the KA function from the
> usrloc module.
> This information it's not in the DB, we can get this information
> either from the kamailio memory, or by parsing the log file. I was
> thinking that the first option can be better because for the second
> one Kamailio will have to do a lot of IO operation to the disk, which
> can affect somehow the performance, even if the local storage it's
> based on SSD's.
> I was trying to use dgram for this case - but I'm not getting any
> output from the kamailio by sending any request.
>
> For ex the following command:
>
> echo '{"id": "1", "jsonrpc": "2.0", "method": "core.psx" }' >
> /dev/udp/127.0.0.1/8090 <http://127.0.0.1/8090>
>
>
> gave the following output to the log file:
>
> Oct 8 11:20:47 devsrv devsrv[11468]: DEBUG: jsonrpcs
> [jsonrpcs_sock.c:608]: jsonrpc_dgram_server(): received
> {"jsonrpc": "2.0", "method": "core.ps <http://core.ps>", "id": 1}
> Oct 8 11:20:47 devsrv devsrv[11468]: DEBUG: jsonrpcs
> [jsonrpcs_sock.c:620]: jsonrpc_dgram_server(): buf is {"jsonrpc":
> "2.0", "method": "core.ps <http://core.ps>", "id": 1}#012 and we
> have received 48 bytes
> Oct 8 11:20:47 devsrv devsrv[11468]: DEBUG: jsonrpcs
> [jsonrpcs_sock.c:629]: jsonrpc_dgram_server(): command executed -
> result: [200] [0x18b5430]
> [{#012#011"jsonrpc":#011"2.0",#012#011"result":#011[11432, "main
> process - attendant", 11434, "udp receiver child=0
> sock=127.0.0.1:5060 <http://127.0.0.1:5060>", 11435, "udp receiver
> child=1 sock=127.0.0.1:5060 <http://127.0.0.1:5060>", 11436, "udp
> receiver child=0 sock=172.16.50.231:5060
> <http://172.16.50.231:5060>", 11437, "udp receiver child=1
> sock=172.16.50.231:5060 <http://172.16.50.231:5060>", 11438, "udp
> receiver child=2 sock=172.16.50.231:5060
> <http://172.16.50.231:5060>", 11439, "udp receiver child=3
> sock=172.16.50.231:5060 <http://172.16.50.231:5060>", 11440, "udp
> receiver child=4 sock=172.16.50.231:5060
> <http://172.16.50.231:5060>", 11441, "udp receiver child=5
> sock=172.16.50.231:5060 <http://172.16.50.231:5060>", 11442, "udp
> receiver child=6 sock=172.16.50.231:5060
> <http://172.16.50.231:5060>", 11443, "udp receiver child=7
> sock=172.16.50.231:5060 <http://172.16.50.231:5060>", 11444, "udp
> receiver child=0 sock=192.168.0.10:5060
> <http://192.168.0.10:5060>", 11445, "udp receiver
> child=1sock=192.168.0.10:5060 <http://192.168.0.10:5060>", 11446,
> "udp receiver child=2 sock=192.168.0.10:5060
> <http://192.168.0.10:5060>", 11447, "udp receiver child=3
> sock=192.168.0.10:5060 <http://192.168.0.10:5060>", 11448, "udp
> receiver child=4 sock=192.168.0.10:5060
> <http://192.168.0.10:5060>", 11449, "udp receiver child=5
> sock=192.168.0.10:5060 <http://192.168.0.10:5060>", 11450, "udp
> receiver child=6 sock=192.168.0.10:5060
> <http://192.168.0.10:5060>", 11451, "udp receiver child=7
> sock=192.168.0.10:5060 <http://192.168.0.10:5060>", 11452, "udp
> receiver child=0 sock=192.168.0.10:5090
> <http://192.168.0.10:5090>", 11453, "udp receiver child=1
> sock=192.168.0.10:5090 <http://192.168.0.10:5090>", 11454, "udp
> receiver child=2 sock=192.168.0.10:5090
> <http://192.168.0.10:5090>", 11455, "udp receiver child=3
> sock=192.168.0.10:5090 <http://192.168.0.10:5090>", 11456, "udp
> receiver child=4 sock=192.168.0.10:5090
> <http://192.168.0.10:5090>", 11457, "udp receiver child=5
> sock=192.168.0.10:5090 <http://192.168.0.10:5090>", 11458, "udp
> receiver child=6 sock=192.168.0.10:5090
> <http://192.168.0.10:5090>", 11459, "udp receiver child=7
> sock=192.168.0.10:5090 <http://192.168.0.10:5090>", 11460, "slow
> timer", 11461, "timer", 11462, "secondary timer", 11463, "Async
> Task Worker", 11464, "Async Task Worker", 11465, "Async Task
> Worker", 11466, "Async Task Worker", 11467, "JSONRPCS FIFO",
> 11468, "JSONRPCS DATAGRAM", 11469, "USRLOC Timer", 11470, "USRLOC
> Timer", 11471, "USRLOC Timer", 11472, "USRLOC Timer", 11473,
> "Dialog Clean Timer", 11474, "ctl handler", 11475, "TIMER UAC
> REG", 11476, "SNMP AgentX", 11478, "Http Async Worker", 11479,
> "tcp receiver (generic) child=0", 11480, "tcp receiver (generic)
> child=1", 11481, "tcp receiver (generic) child=2", 11482, "tcp
> receiver (generic) child=3", 11483, "tcp receiver (generic)
> child=4", 11484, "tcp receiver (generic) child=5", 11485, "tcp
> receiver (generic) child=6", 11486, "tcp receiver (generic)
> child=7", 11487, "tcp main process"],#012#011"id":#0111#012}]
>
>
> I expected to receive this output back to the udp client in this case.
>
> Also, the following request it's not getting anything even in the log
> file:
>
> echo '{"id": "1", "jsonrpc": "2.0", "method": "ul.dump" }' >
> /dev/udp/127.0.0.1/8090 <http://127.0.0.1/8090>
>
>
> If there is a way to have all this data directly in the database -
> that would be perfect (I already created a feature request for that,
> maybe in the next versions that will be implemented).
>
> Regards,
>
> On Wed, 7 Oct 2020 at 21:27, Henning Westerholt <hw at skalatan.de
> <mailto:hw at skalatan.de>> wrote:
>
> Hello,
>
>
>
> Your requirement to frequently dump millions of user location
> entries on a loaded production system sounds a bit odd to me.
> Maybe you can elaborate a bit on your requirements. If you have
> the requirement to access this data frequently, maybe getting it
> from the database directly is the better idea. If you want to get
> it from kamailio, choose a module that provides the option to
> configure the number of children (e.g. like dgram_workers in
> jsonrpcs) and set it to a sufficient number.
>
>
>
> Cheers,
>
>
>
> Henning
>
>
>
> --
>
> Henning Westerholt – https://skalatan.de/blog/
> <https://skalatan.de/blog/>
>
> Kamailio services – https://gilawa.com <https://gilawa.com/>
>
>
>
> *From:* sr-users <sr-users-bounces at lists.kamailio.org
> <mailto:sr-users-bounces at lists.kamailio.org>> *On Behalf Of *Ilie
> Soltanici
> *Sent:* Tuesday, October 6, 2020 12:29 AM
> *To:* Kamailio (SER) - Users Mailing List
> <sr-users at lists.kamailio.org <mailto:sr-users at lists.kamailio.org>>
> *Subject:* [SR-Users] jsonrpcs - safest way to get data
>
>
>
> Hello,
>
>
>
> I'm trying to periodically get all registered contacts from the
> Kamailio memory in the json format.
>
> JSONRPCS module have 3 different types of transport to get this
> data. I just wonder what will be the safest transport for the
> Kamailio to get all this data? Because all this data it's not so
> important, I want to gather it without affecting the main process
> or with the minimal effect on the Kamailio SIP Processing module.
>
>
>
> 1. The HTTP transport - very useful, working async, but because it
> depends on the xhttp module - which works synchronously I'm afraid
> that for big data with contacts this type of transport can affect
> the kamailio performance.
>
>
>
> 2. FIFO - this transport also is very fast, working locally - but
> it's synchronous, I'm not sure how it will affect Kamailio SIP
> Processing when the local database will have few millions of users
> registered.
>
>
>
> 3. DGRAM - also seems to be very fast, working locally or remotely
> through the UDP protocol, but again it's synchronous and I'm not
> sure that this transport is returning any output for the request,
> I tried - but didn't get back anything, while for changing
> something it worked fine. Maybe it was a misconfiguration in my
> config file.
>
>
>
> What would be your recommendation for this case?
>
>
>
> Thank you.
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20201009/3d260760/attachment.htm>
More information about the sr-users
mailing list