RTPEngine performance - How far are you pushing your systems
Dear all, Not a direct question on kamailio, but on rtpengine, so feel free to ignore :) I have a farm of rtpengine servers , supporting our set of kamailio's. On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc. I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-. Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems? kind regards and happy 2020!
Hello Davy, You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine: https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html Cheers, Henning -- Henning Westerholt – https://skalatan.de/blog/ Kamailio services – https://gilawa.com<https://gilawa.com/> From: sr-users <sr-users-bounces@lists.kamailio.org> On Behalf Of davy van de moere Sent: Monday, January 6, 2020 12:52 PM To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> Subject: [SR-Users] RTPEngine performance - How far are you pushing your systems Dear all, Not a direct question on kamailio, but on rtpengine, so feel free to ignore :) I have a farm of rtpengine servers , supporting our set of kamailio's. On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc. I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-. Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems? kind regards and happy 2020!
Ok, I'll get my SIPP warmed up then ... ;) thx for the feedback! Op ma 6 jan. 2020 om 14:45 schreef Henning Westerholt <hw@skalatan.de>:
Hello Davy,
You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine:
https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *davy van de moere *Sent:* Monday, January 6, 2020 12:52 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Subject:* [SR-Users] RTPEngine performance - How far are you pushing your systems
Dear all,
Not a direct question on kamailio, but on rtpengine, so feel free to ignore :)
I have a farm of rtpengine servers , supporting our set of kamailio's.
On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc.
I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-.
Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems?
kind regards and happy 2020!
Hey Davy, Check cat /proc/rtpengine/0/list De : sr-users <sr-users-bounces@lists.kamailio.org> De la part de davy van de moere Envoyé : lundi 6 janvier 2020 14:47 À : Henning Westerholt <hw@skalatan.de> Cc : Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> Objet : Re: [SR-Users] RTPEngine performance - How far are you pushing your systems Ok, I'll get my SIPP warmed up then ... ;) thx for the feedback! Op ma 6 jan. 2020 om 14:45 schreef Henning Westerholt <hw@skalatan.de<mailto:hw@skalatan.de>>: Hello Davy, You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine: https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html Cheers, Henning -- Henning Westerholt – https://skalatan.de/blog/ Kamailio services – https://gilawa.com<https://gilawa.com/> From: sr-users <sr-users-bounces@lists.kamailio.org<mailto:sr-users-bounces@lists.kamailio.org>> On Behalf Of davy van de moere Sent: Monday, January 6, 2020 12:52 PM To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>> Subject: [SR-Users] RTPEngine performance - How far are you pushing your systems Dear all, Not a direct question on kamailio, but on rtpengine, so feel free to ignore :) I have a farm of rtpengine servers , supporting our set of kamailio's. On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc. I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-. Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems? kind regards and happy 2020!
It will probably be a case of RTFM... quoting from: https://github.com/sipwise/rtpengine/blob/master/README.md # ensure that the table we want to use doesn't exist - usually needed after a daemon # restart, otherwise will error echo 'del 0' > /proc/rtpengine/control this part seems to be very relevant indeed! Op ma 6 jan. 2020 om 17:41 schreef Nicolas Breuer < Nicolas.Breuer@belcenter.biz>:
Hey Davy,
Check cat /proc/rtpengine/0/list
*De :* sr-users <sr-users-bounces@lists.kamailio.org> *De la part de* davy van de moere *Envoyé :* lundi 6 janvier 2020 14:47 *À :* Henning Westerholt <hw@skalatan.de> *Cc :* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Objet :* Re: [SR-Users] RTPEngine performance - How far are you pushing your systems
Ok, I'll get my SIPP warmed up then ... ;) thx for the feedback!
Op ma 6 jan. 2020 om 14:45 schreef Henning Westerholt <hw@skalatan.de>:
Hello Davy,
You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine:
https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *davy van de moere *Sent:* Monday, January 6, 2020 12:52 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Subject:* [SR-Users] RTPEngine performance - How far are you pushing your systems
Dear all,
Not a direct question on kamailio, but on rtpengine, so feel free to ignore :)
I have a farm of rtpengine servers , supporting our set of kamailio's.
On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc.
I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-.
Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems?
kind regards and happy 2020!
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
As a person who working much on developing new RTP engine for Kamailio, I should express achieving to 12k concurrent call is very strange with Kernel part of RTPEngine. Regardless of NIC's parameter, multi core and some other parameters, it is important where the hooking linux kernel function is called. The Kernel part of RTPEngine is called in NF_INET_LOCAL_IN. It means the rtp packet goes up local routing in network stack. I could not reach to 12k yet, although i call Linux kernel function in NF-INET-PRE-ROUTING On Mon, Jan 6, 2020 at 8:26 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
It will probably be a case of RTFM...
quoting from: https://github.com/sipwise/rtpengine/blob/master/README.md
# ensure that the table we want to use doesn't exist - usually needed after a daemon # restart, otherwise will error echo 'del 0' > /proc/rtpengine/control
this part seems to be very relevant indeed!
Op ma 6 jan. 2020 om 17:41 schreef Nicolas Breuer < Nicolas.Breuer@belcenter.biz>:
Hey Davy,
Check cat /proc/rtpengine/0/list
*De :* sr-users <sr-users-bounces@lists.kamailio.org> *De la part de* davy van de moere *Envoyé :* lundi 6 janvier 2020 14:47 *À :* Henning Westerholt <hw@skalatan.de> *Cc :* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Objet :* Re: [SR-Users] RTPEngine performance - How far are you pushing your systems
Ok, I'll get my SIPP warmed up then ... ;) thx for the feedback!
Op ma 6 jan. 2020 om 14:45 schreef Henning Westerholt <hw@skalatan.de>:
Hello Davy,
You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine:
https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *davy van de moere *Sent:* Monday, January 6, 2020 12:52 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Subject:* [SR-Users] RTPEngine performance - How far are you pushing your systems
Dear all,
Not a direct question on kamailio, but on rtpengine, so feel free to ignore :)
I have a farm of rtpengine servers , supporting our set of kamailio's.
On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc.
I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-.
Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems?
kind regards and happy 2020!
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S
As an update, I'm getting 2000 channels in kernel mode, with about 50% usage of 1 core. (load of the server is at 0.5), which is a 10 fold difference. The problem I was hitting, was the (probable) inability to work in kernelmode with the rtpproxy.so module, rtpengine.so works nicely. Op di 7 jan. 2020 om 10:19 schreef Mojtaba <mespio@gmail.com>:
As a person who working much on developing new RTP engine for Kamailio, I should express achieving to 12k concurrent call is very strange with Kernel part of RTPEngine. Regardless of NIC's parameter, multi core and some other parameters, it is important where the hooking linux kernel function is called. The Kernel part of RTPEngine is called in NF_INET_LOCAL_IN. It means the rtp packet goes up local routing in network stack. I could not reach to 12k yet, although i call Linux kernel function in NF-INET-PRE-ROUTING
On Mon, Jan 6, 2020 at 8:26 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
It will probably be a case of RTFM...
quoting from: https://github.com/sipwise/rtpengine/blob/master/README.md
# ensure that the table we want to use doesn't exist - usually needed after a daemon # restart, otherwise will error echo 'del 0' > /proc/rtpengine/control
this part seems to be very relevant indeed!
Op ma 6 jan. 2020 om 17:41 schreef Nicolas Breuer < Nicolas.Breuer@belcenter.biz>:
Hey Davy,
Check cat /proc/rtpengine/0/list
*De :* sr-users <sr-users-bounces@lists.kamailio.org> *De la part de* davy van de moere *Envoyé :* lundi 6 janvier 2020 14:47 *À :* Henning Westerholt <hw@skalatan.de> *Cc :* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Objet :* Re: [SR-Users] RTPEngine performance - How far are you pushing your systems
Ok, I'll get my SIPP warmed up then ... ;) thx for the feedback!
Op ma 6 jan. 2020 om 14:45 schreef Henning Westerholt <hw@skalatan.de>:
Hello Davy,
You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine:
https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *davy van de moere *Sent:* Monday, January 6, 2020 12:52 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Subject:* [SR-Users] RTPEngine performance - How far are you pushing your systems
Dear all,
Not a direct question on kamailio, but on rtpengine, so feel free to ignore :)
I have a farm of rtpengine servers , supporting our set of kamailio's.
On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc.
I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-.
Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems?
kind regards and happy 2020!
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Yes, I reached 3000 channels in kernel mode with about 25% usage of one core. Its incredible. But be notic the number of channels is not matter, What about its QoS parameter such as max-delta, jitter, lost? The max delay i have with the same result is 60. On Tue, Jan 7, 2020 at 12:56 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
As an update, I'm getting 2000 channels in kernel mode, with about 50% usage of 1 core. (load of the server is at 0.5), which is a 10 fold difference.
The problem I was hitting, was the (probable) inability to work in kernelmode with the rtpproxy.so module, rtpengine.so works nicely.
Op di 7 jan. 2020 om 10:19 schreef Mojtaba <mespio@gmail.com>:
As a person who working much on developing new RTP engine for Kamailio, I should express achieving to 12k concurrent call is very strange with Kernel part of RTPEngine. Regardless of NIC's parameter, multi core and some other parameters, it is important where the hooking linux kernel function is called. The Kernel part of RTPEngine is called in NF_INET_LOCAL_IN. It means the rtp packet goes up local routing in network stack. I could not reach to 12k yet, although i call Linux kernel function in NF-INET-PRE-ROUTING
On Mon, Jan 6, 2020 at 8:26 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
It will probably be a case of RTFM...
quoting from: https://github.com/sipwise/rtpengine/blob/master/README.md
# ensure that the table we want to use doesn't exist - usually needed after a daemon # restart, otherwise will error echo 'del 0' > /proc/rtpengine/control
this part seems to be very relevant indeed!
Op ma 6 jan. 2020 om 17:41 schreef Nicolas Breuer < Nicolas.Breuer@belcenter.biz>:
Hey Davy,
Check cat /proc/rtpengine/0/list
*De :* sr-users <sr-users-bounces@lists.kamailio.org> *De la part de* davy van de moere *Envoyé :* lundi 6 janvier 2020 14:47 *À :* Henning Westerholt <hw@skalatan.de> *Cc :* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org
*Objet :* Re: [SR-Users] RTPEngine performance - How far are you pushing your systems
Ok, I'll get my SIPP warmed up then ... ;) thx for the feedback!
Op ma 6 jan. 2020 om 14:45 schreef Henning Westerholt <hw@skalatan.de>:
Hello Davy,
You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine:
https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *davy van de moere *Sent:* Monday, January 6, 2020 12:52 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org> *Subject:* [SR-Users] RTPEngine performance - How far are you pushing your systems
Dear all,
Not a direct question on kamailio, but on rtpengine, so feel free to ignore :)
I have a farm of rtpengine servers , supporting our set of kamailio's.
On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc.
I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-.
Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems?
kind regards and happy 2020!
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S
Using wireshark, I get a jitter still smaller than 1ms. You have efficient ways to measure that? Op di 7 jan. 2020 om 10:43 schreef Mojtaba <mespio@gmail.com>:
Yes, I reached 3000 channels in kernel mode with about 25% usage of one core. Its incredible. But be notic the number of channels is not matter, What about its QoS parameter such as max-delta, jitter, lost? The max delay i have with the same result is 60.
On Tue, Jan 7, 2020 at 12:56 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
As an update, I'm getting 2000 channels in kernel mode, with about 50% usage of 1 core. (load of the server is at 0.5), which is a 10 fold difference.
The problem I was hitting, was the (probable) inability to work in kernelmode with the rtpproxy.so module, rtpengine.so works nicely.
Op di 7 jan. 2020 om 10:19 schreef Mojtaba <mespio@gmail.com>:
As a person who working much on developing new RTP engine for Kamailio, I should express achieving to 12k concurrent call is very strange with Kernel part of RTPEngine. Regardless of NIC's parameter, multi core and some other parameters, it is important where the hooking linux kernel function is called. The Kernel part of RTPEngine is called in NF_INET_LOCAL_IN. It means the rtp packet goes up local routing in network stack. I could not reach to 12k yet, although i call Linux kernel function in NF-INET-PRE-ROUTING
On Mon, Jan 6, 2020 at 8:26 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
It will probably be a case of RTFM...
quoting from: https://github.com/sipwise/rtpengine/blob/master/README.md
# ensure that the table we want to use doesn't exist - usually needed after a daemon # restart, otherwise will error echo 'del 0' > /proc/rtpengine/control
this part seems to be very relevant indeed!
Op ma 6 jan. 2020 om 17:41 schreef Nicolas Breuer < Nicolas.Breuer@belcenter.biz>:
Hey Davy,
Check cat /proc/rtpengine/0/list
*De :* sr-users <sr-users-bounces@lists.kamailio.org> *De la part de* davy van de moere *Envoyé :* lundi 6 janvier 2020 14:47 *À :* Henning Westerholt <hw@skalatan.de> *Cc :* Kamailio (SER) - Users Mailing List < sr-users@lists.kamailio.org> *Objet :* Re: [SR-Users] RTPEngine performance - How far are you pushing your systems
Ok, I'll get my SIPP warmed up then ... ;) thx for the feedback!
Op ma 6 jan. 2020 om 14:45 schreef Henning Westerholt <hw@skalatan.de
:
Hello Davy,
You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine:
https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *davy van de moere *Sent:* Monday, January 6, 2020 12:52 PM *To:* Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org
*Subject:* [SR-Users] RTPEngine performance - How far are you pushing your systems
Dear all,
Not a direct question on kamailio, but on rtpengine, so feel free to ignore :)
I have a farm of rtpengine servers , supporting our set of kamailio's.
On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc.
I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-.
Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems?
kind regards and happy 2020!
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Have you checked /proc/rtpengine ? On Tue, 7 Jan 2020 at 09:56, davy van de moere <davy.van.de.moere@gmail.com> wrote:
Using wireshark, I get a jitter still smaller than 1ms. You have efficient ways to measure that?
Op di 7 jan. 2020 om 10:43 schreef Mojtaba <mespio@gmail.com>:
Yes, I reached 3000 channels in kernel mode with about 25% usage of one core. Its incredible. But be notic the number of channels is not matter, What about its QoS parameter such as max-delta, jitter, lost? The max delay i have with the same result is 60.
On Tue, Jan 7, 2020 at 12:56 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
As an update, I'm getting 2000 channels in kernel mode, with about 50% usage of 1 core. (load of the server is at 0.5), which is a 10 fold difference.
The problem I was hitting, was the (probable) inability to work in kernelmode with the rtpproxy.so module, rtpengine.so works nicely.
Op di 7 jan. 2020 om 10:19 schreef Mojtaba <mespio@gmail.com>:
As a person who working much on developing new RTP engine for Kamailio, I should express achieving to 12k concurrent call is very strange with Kernel part of RTPEngine. Regardless of NIC's parameter, multi core and some other parameters, it is important where the hooking linux kernel function is called. The Kernel part of RTPEngine is called in NF_INET_LOCAL_IN. It means the rtp packet goes up local routing in network stack. I could not reach to 12k yet, although i call Linux kernel function in NF-INET-PRE-ROUTING
On Mon, Jan 6, 2020 at 8:26 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
It will probably be a case of RTFM...
quoting from: https://github.com/sipwise/rtpengine/blob/master/README.md
# ensure that the table we want to use doesn't exist - usually needed after a daemon # restart, otherwise will error echo 'del 0' > /proc/rtpengine/control
this part seems to be very relevant indeed!
Op ma 6 jan. 2020 om 17:41 schreef Nicolas Breuer < Nicolas.Breuer@belcenter.biz>:
Hey Davy,
Check cat /proc/rtpengine/0/list
*De :* sr-users <sr-users-bounces@lists.kamailio.org> *De la part de* davy van de moere *Envoyé :* lundi 6 janvier 2020 14:47 *À :* Henning Westerholt <hw@skalatan.de> *Cc :* Kamailio (SER) - Users Mailing List < sr-users@lists.kamailio.org> *Objet :* Re: [SR-Users] RTPEngine performance - How far are you pushing your systems
Ok, I'll get my SIPP warmed up then ... ;) thx for the feedback!
Op ma 6 jan. 2020 om 14:45 schreef Henning Westerholt <hw@skalatan.de >:
Hello Davy,
You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine:
https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *davy van de moere *Sent:* Monday, January 6, 2020 12:52 PM *To:* Kamailio (SER) - Users Mailing List < sr-users@lists.kamailio.org> *Subject:* [SR-Users] RTPEngine performance - How far are you pushing your systems
Dear all,
Not a direct question on kamailio, but on rtpengine, so feel free to ignore :)
I have a farm of rtpengine servers , supporting our set of kamailio's.
On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc.
I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-.
Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems?
kind regards and happy 2020!
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- Regards, David Villasmil email: david.villasmil.work@gmail.com phone: +34669448337
Absolutely using wireshark to capture all RTP traffic for analysing is very challengeable because the size of capture file is very huge. So i have used -C option to spilite to multiple pcap file. I shared a picture of it: https://www.linkedin.com/posts/mojtaba-esfandiari_kamailio-kamailioworld-act... The result of you get is strange! On Tue, Jan 7, 2020 at 1:24 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
Using wireshark, I get a jitter still smaller than 1ms. You have efficient ways to measure that?
Op di 7 jan. 2020 om 10:43 schreef Mojtaba <mespio@gmail.com>:
Yes, I reached 3000 channels in kernel mode with about 25% usage of one core. Its incredible. But be notic the number of channels is not matter, What about its QoS parameter such as max-delta, jitter, lost? The max delay i have with the same result is 60.
On Tue, Jan 7, 2020 at 12:56 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
As an update, I'm getting 2000 channels in kernel mode, with about 50% usage of 1 core. (load of the server is at 0.5), which is a 10 fold difference.
The problem I was hitting, was the (probable) inability to work in kernelmode with the rtpproxy.so module, rtpengine.so works nicely.
Op di 7 jan. 2020 om 10:19 schreef Mojtaba <mespio@gmail.com>:
As a person who working much on developing new RTP engine for Kamailio, I should express achieving to 12k concurrent call is very strange with Kernel part of RTPEngine. Regardless of NIC's parameter, multi core and some other parameters, it is important where the hooking linux kernel function is called. The Kernel part of RTPEngine is called in NF_INET_LOCAL_IN. It means the rtp packet goes up local routing in network stack. I could not reach to 12k yet, although i call Linux kernel function in NF-INET-PRE-ROUTING
On Mon, Jan 6, 2020 at 8:26 PM davy van de moere < davy.van.de.moere@gmail.com> wrote:
It will probably be a case of RTFM...
quoting from: https://github.com/sipwise/rtpengine/blob/master/README.md
# ensure that the table we want to use doesn't exist - usually needed after a daemon # restart, otherwise will error echo 'del 0' > /proc/rtpengine/control
this part seems to be very relevant indeed!
Op ma 6 jan. 2020 om 17:41 schreef Nicolas Breuer < Nicolas.Breuer@belcenter.biz>:
Hey Davy,
Check cat /proc/rtpengine/0/list
*De :* sr-users <sr-users-bounces@lists.kamailio.org> *De la part de* davy van de moere *Envoyé :* lundi 6 janvier 2020 14:47 *À :* Henning Westerholt <hw@skalatan.de> *Cc :* Kamailio (SER) - Users Mailing List < sr-users@lists.kamailio.org> *Objet :* Re: [SR-Users] RTPEngine performance - How far are you pushing your systems
Ok, I'll get my SIPP warmed up then ... ;) thx for the feedback!
Op ma 6 jan. 2020 om 14:45 schreef Henning Westerholt <hw@skalatan.de >:
Hello Davy,
You should look into it, sounds strange. Compare e.g. to this e-mail from 2015 – over 16k simultaneous calls on a 12 core machine:
https://lists.kamailio.org/pipermail/sr-dev/2015-October/031432.html
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces@lists.kamailio.org> *On Behalf Of *davy van de moere *Sent:* Monday, January 6, 2020 12:52 PM *To:* Kamailio (SER) - Users Mailing List < sr-users@lists.kamailio.org> *Subject:* [SR-Users] RTPEngine performance - How far are you pushing your systems
Dear all,
Not a direct question on kamailio, but on rtpengine, so feel free to ignore :)
I have a farm of rtpengine servers , supporting our set of kamailio's.
On recent decent hardware (hp dl360 gen9/10 servers), I typically push my systems upto 2500 channels per machine. Then I tend to see the load of the machine go up in a logarithmic manner. E.g. a load of 3 supports 2500 channels, but when its at 3000 channels, load can be at 8. etc.
I have the kernel module running, and I have a good impression it's being used too. No transcoding is going on, no recording, ... I've tried on different OS'es, different network cards, and I always end up at that 2500 channels +-.
Is that what I can expect from rtpengine? Or am I really overlooking something? What numbers do you push out of your systems?
kind regards and happy 2020!
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- --Mojtaba Esfandiari.S
participants (5)
-
David Villasmil -
davy van de moere -
Henning Westerholt -
Mojtaba -
Nicolas Breuer