27 Feb
2019
27 Feb
'19
12:23 p.m.
Greetings,
I'm trying to use SQlite as my engine in order to improve performance. I
would like to have my Database stored in memory for faster access.
I see that Kamailio can open the SQlite database as Read-Only, but i can't
find a way to open it in memory. Is this possible using Kamailio? What's
the best way to do this?
Best Regards,
Duarte Rocha
27 Feb
27 Feb
12:26 p.m.
What sort of insufficient read performance are you seeing from SQLite on modern storage
which would lead you to this unnecessary ideation? Read performance on SQLite is insanely
fast...
—
Sent from mobile, with due apologies for brevity and errors.
On Feb 27, 2019, at 12:23 PM, Duarte Rocha
<duarterocha91(a)gmail.com> wrote:
Greetings,
I'm trying to use SQlite as my engine in order to improve performance. I would like
to have my Database stored in memory for faster access.
I see that Kamailio can open the SQlite database as Read-Only, but i can't find a way
to open it in memory. Is this possible using Kamailio? What's the best way to do
this?
Best Regards,
Duarte Rocha
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
12:56 p.m.
Greetings Alex,
At the moment i'm trying to debug some issues with the server and try some
solutions to see the impact.
With little load i'm getting some percentage of "503 Server Unavaliable"
errors from Kamailio. I made some testing without using database for
anything but Dispatcher and Accounting and the performance got way better.
Right now, on this server, under as little as 20CPS Kamailio is returning
"503" on 1-2% of all calls. However, my CPU and Memory look like this:
%Cpu(s): 20.3 us, 6.1 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 0.2 si, 0.2 st
Memory :
total used free shared buff/cache available
Mem: 1830 1172 382 109 274 375
Swap: 819 49 770
I have around 100 processes for Kamailio.
Cheers
1:27 p.m.
Okay, but at a paltry 20 CPS -- or even several multiples of that --
none of this should be an issue. There's something else going on.
On Wed, Feb 27, 2019 at 10:28:54AM -0800, Julien Chavanton wrote:
*https://www.kamailio.org/docs/modules/5.2.x/modules/db_sqlite.html#db_sqlite.p.db_set_journal_mode
<https://www.kamailio.org/docs/modules/5.2.x/modules/db_sqlite.html#db_sqlite.p.db_set_journal_mode>*
On Wed, Feb 27, 2019 at 10:27 AM Julien Chavanton <jchavanton(a)gmail.com>
wrote:
>
> Hi, you can use the WAL mode, so that readers are never delayed.
>
> Just make sure all the connections to the DB are using the WAL mode.
>
> The downside is that the writers can be delayed sometimes, similar to
> Postgres read only replica
>
> On Wed, Feb 27, 2019 at 9:57 AM Duarte Rocha <duarterocha91(a)gmail.com>
> wrote:
>
>> Greetings Alex,
>>
>> At the moment i'm trying to debug some issues with the server and try
>> some solutions to see the impact.
>> With little load i'm getting some percentage of "503 Server
Unavaliable"
>> errors from Kamailio. I made some testing without using database for
>> anything but Dispatcher and Accounting and the performance got way better.
>>
>> Right now, on this server, under as little as 20CPS Kamailio is returning
>> "503" on 1-2% of all calls. However, my CPU and Memory look like
this:
>>
>> %Cpu(s): 20.3 us, 6.1 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 0.2 si, 0.2 st
>>
>>
>> Memory :
>>
>> total used free shared buff/cache available
>> Mem: 1830 1172 382 109 274 375
>> Swap: 819 49 770
>>
>> I have around 100 processes for Kamailio.
>>
>> Cheers
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users(a)lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
1:27 p.m.
Hi, you can use the WAL mode, so that readers are never delayed.
Just make sure all the connections to the DB are using the WAL mode.
The downside is that the writers can be delayed sometimes, similar to
Postgres read only replica
On Wed, Feb 27, 2019 at 9:57 AM Duarte Rocha <duarterocha91(a)gmail.com>
wrote:
Greetings Alex,
At the moment i'm trying to debug some issues with the server and try some
solutions to see the impact.
With little load i'm getting some percentage of "503 Server Unavaliable"
errors from Kamailio. I made some testing without using database for
anything but Dispatcher and Accounting and the performance got way better.
Right now, on this server, under as little as 20CPS Kamailio is returning
"503" on 1-2% of all calls. However, my CPU and Memory look like this:
%Cpu(s): 20.3 us, 6.1 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 0.2 si, 0.2 st
Memory :
total used free shared buff/cache available
Mem: 1830 1172 382 109 274 375
Swap: 819 49 770
I have around 100 processes for Kamailio.
Cheers
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
1:28 p.m.
*https://www.kamailio.org/docs/modules/5.2.x/modules/db_sqlite.html#db_sqlite.p.db_set_journal_mode
<https://www.kamailio.org/docs/modules/5.2.x/modules/db_sqlite.html#db_sqlite.p.db_set_journal_mode>*
On Wed, Feb 27, 2019 at 10:27 AM Julien Chavanton <jchavanton(a)gmail.com>
wrote:
Hi, you can use the WAL mode, so that readers are never delayed.
Just make sure all the connections to the DB are using the WAL mode.
The downside is that the writers can be delayed sometimes, similar to
Postgres read only replica
On Wed, Feb 27, 2019 at 9:57 AM Duarte Rocha <duarterocha91(a)gmail.com>
wrote:
Greetings Alex,
At the moment i'm trying to debug some issues with the server and try
some solutions to see the impact.
With little load i'm getting some percentage of "503 Server Unavaliable"
errors from Kamailio. I made some testing without using database for
anything but Dispatcher and Accounting and the performance got way better.
Right now, on this server, under as little as 20CPS Kamailio is returning
"503" on 1-2% of all calls. However, my CPU and Memory look like this:
%Cpu(s): 20.3 us, 6.1 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 0.2 si, 0.2 st
Memory :
total used free shared buff/cache available
Mem: 1830 1172 382 109 274 375
Swap: 819 49 770
I have around 100 processes for Kamailio.
Cheers
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
1:35 p.m.
Ok, i reduced the number of processes and tried a few different values, but
the performance hasn't changed a lot.
How can i debug this and find the bottleneck in my system?
A quarta, 27/02/2019, 17:56, Duarte Rocha <duarterocha91(a)gmail.com>
escreveu:
Greetings Alex,
At the moment i'm trying to debug some issues with the server and try some
solutions to see the impact.
With little load i'm getting some percentage of "503 Server Unavaliable"
errors from Kamailio. I made some testing without using database for
anything but Dispatcher and Accounting and the performance got way better.
Right now, on this server, under as little as 20CPS Kamailio is returning
"503" on 1-2% of all calls. However, my CPU and Memory look like this:
%Cpu(s): 20.3 us, 6.1 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 0.2 si, 0.2 st
Memory :
total used free shared buff/cache available
Mem: 1830 1172 382 109 274 375
Swap: 819 49 770
I have around 100 processes for Kamailio.
Cheers
1:37 p.m.
I would say the first step is to gauge the total velocity of any given
initial request through your config.
One way to do that is:
https://kamailio.org/docs/modules/5.2.x/modules/benchmark.html
But I prefer this approach:
route {
$avp(proc_start) = $TV(Sn);
...
if(is_method("INVITE")) {
processing_etc();
$var(cur_time) = $TV(Sn);
$var(proc_diff) = (
(((
$(var(cur_time){s.select,0,.}{s.int}) -
$(avp(proc_start){s.select,0,.}{s.int})
) * 1000000)
+
(
$(var(cur_time){s.select,1,.}{s.int}) -
$(avp(proc_start){s.select,1,.}{s.int})
) / 1000) mod 1000
);
xlog("L_INFO", "[$ci] Request processing delay: $var(proc_diff)
ms\n");
if(!t_relay())
sl_reply_error();
exit;
}
}
I don't vouch for the precision/granularity of this timing scheme, but
it's good enough that it should tell you if your total request
processing volume is insane. Then you can move these breakpoints around
until you find which operation is actually so slow.
That said, I am not aware of any aspect by which Kamailio intrinsically
sends a 503 if it is overwhelmed with requests. It just drops
them/doesn't process.
-- Alex
On Wed, Feb 27, 2019 at 06:35:46PM +0000, Duarte Rocha wrote:
Ok, i reduced the number of processes and tried a few
different values, but
the performance hasn't changed a lot.
How can i debug this and find the bottleneck in my system?
A quarta, 27/02/2019, 17:56, Duarte Rocha <duarterocha91(a)gmail.com>
escreveu:
> Greetings Alex,
>
> At the moment i'm trying to debug some issues with the server and try some
> solutions to see the impact.
> With little load i'm getting some percentage of "503 Server
Unavaliable"
> errors from Kamailio. I made some testing without using database for
> anything but Dispatcher and Accounting and the performance got way better.
>
> Right now, on this server, under as little as 20CPS Kamailio is returning
> "503" on 1-2% of all calls. However, my CPU and Memory look like this:
>
> %Cpu(s): 20.3 us, 6.1 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 0.2 si, 0.2 st
>
>
> Memory :
>
> total used free shared buff/cache available
> Mem: 1830 1172 382 109 274 375
> Swap: 819 49 770
>
> I have around 100 processes for Kamailio.
>
> Cheers
>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
1:40 p.m.
On Wed, Feb 27, 2019 at 01:43:02PM -0500, Ovidiu Sas wrote:
You can do hundreds of CPS with only two workers and
the CPU will be
at 3% utilization on very old hardware.
There's something else in your config creating issues.
+1
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
1:43 p.m.
Do you use a lot of logging?
Have you enabled asynchronous logging?
If you are using the DB only for dispatcher, then you have other
issues then the db. The dispatcher module is caching the data on
memory.
You can do hundreds of CPS with only two workers and the CPU will be
at 3% utilization on very old hardware.
There's something else in your config creating issues.
Regards,
Ovidiu Sas
On Wed, Feb 27, 2019 at 1:36 PM Duarte Rocha <duarterocha91(a)gmail.com> wrote:
Ok, i reduced the number of processes and tried a few different values, but the
performance hasn't changed a lot.
How can i debug this and find the bottleneck in my system?
A quarta, 27/02/2019, 17:56, Duarte Rocha <duarterocha91(a)gmail.com> escreveu:
--
VoIP Embedded, Inc.
http://www.voipembedded.com
Greetings Alex,
At the moment i'm trying to debug some issues with the server and try some solutions
to see the impact.
With little load i'm getting some percentage of "503 Server Unavaliable"
errors from Kamailio. I made some testing without using database for anything but
Dispatcher and Accounting and the performance got way better.
Right now, on this server, under as little as 20CPS Kamailio is returning "503"
on 1-2% of all calls. However, my CPU and Memory look like this:
%Cpu(s): 20.3 us, 6.1 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 0.2 si, 0.2 st
Memory :
total used free shared buff/cache available
Mem: 1830 1172 382 109 274 375
Swap: 819 49 770
I have around 100 processes for Kamailio.
Cheers
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
1:51 p.m.
On Wed, Feb 27, 2019 at 06:53:54PM +0000, Duarte Rocha wrote:
I use database for lots of configs. On any given first
request i do around
10 queries.
A while ago i measured the processing time (using bm_timer) when i was
still using MySql and got times of around 6ms per first Invite and around
2ms for other in-dialog requests.
While i think those are big processing times, i don't think they should be
the problem at a rate as low as 20CPS.
No, those aren't big processing times. Any cumulative round trips to the
database, even a local one, will amount to a few ms at least.
It doesn't sound like that's where the issue lies...
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
1:53 p.m.
I use database for lots of configs. On any given first request i do around
10 queries.
A while ago i measured the processing time (using bm_timer) when i was
still using MySql and got times of around 6ms per first Invite and around
2ms for other in-dialog requests.
While i think those are big processing times, i don't think they should be
the problem at a rate as low as 20CPS.
Thank you for your help :)
A quarta, 27/02/2019, 18:35, Duarte Rocha <duarterocha91(a)gmail.com>
escreveu:
Ok, i reduced the number of processes and tried a few
different values,
but the performance hasn't changed a lot.
How can i debug this and find the bottleneck in my system?
A quarta, 27/02/2019, 17:56, Duarte Rocha <duarterocha91(a)gmail.com>
escreveu:
Greetings Alex,
At the moment i'm trying to debug some issues with the server and try
some solutions to see the impact.
With little load i'm getting some percentage of "503 Server Unavaliable"
errors from Kamailio. I made some testing without using database for
anything but Dispatcher and Accounting and the performance got way better.
Right now, on this server, under as little as 20CPS Kamailio is returning
"503" on 1-2% of all calls. However, my CPU and Memory look like this:
%Cpu(s): 20.3 us, 6.1 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 0.2 si, 0.2 st
Memory :
total used free shared buff/cache available
Mem: 1830 1172 382 109 274 375
Swap: 819 49 770
I have around 100 processes for Kamailio.
Cheers
1914
days inactive
1914
days old
11 comments
4 participants
participants (4)
-
Alex Balashov -
Duarte Rocha -
Julien Chavanton -
Ovidiu Sas