On Jul 01, 2004 at 15:53, Adrian Georgescu ag@ag-projects.com wrote:
On Thursday 01 July 2004 14:11, you wrote:
On Jun 30, 2004 at 08:53, Ezequiel Colombo ecolombo@arcotel.net
wrote:
Hi all, i am tested the mediaproxy 1.0.1 version on a dual CPU (2.6GHz) and get at least 180 simultaneous calls ! This is a good and very scalable solution to solve NAT related problems.
Ezequiel's test result seems to be in the expected range (given our own test results).
In our tests, on a 1GHz Athlon CPU running linux 2.4.26 we got the following results:
mediaproxy reaches 95% CPU load at around 60 simultaneous calls mediaproxy gets to 100% CPU load at 80-90 calls
rtpproxy reaches 95% CPU load at 120 calls rtpproxy gets to 100% CPU load at about 150 calls
Given that we discovered that rtpproxy is only at most twice as fast as mediaproxy, we never bothered to re-implement mediaproxy in C. It's much easier to stack another box (or even more), than to spend lots of time in optimizing a program and in the end to only get it to run twice as fast.
Given that Andrei's results are very different from what our tests showed, we suspect that there were some errors in the testing procedure.
Also there is an error in his patch (see comments below for details). I attached a version of rtpgenerator that was modified to work with rtpproxy, but with this error fixed.
I tested it on a dual athlon mp2000. I've also modified rtpgenerator to work with nathelper (patch for nh version attached).
Patch has a problem. It sends packets to the same IP as the generator itself (the default is 127.0.0.1) and not to the rtpproxy returned IP. This means that in it's default configuration it sends from 127.0.0.1 to 127.0.0.1 where rtpproxy doesn't listen. We tried it and it gets flooded back with "udp port unreachable". This explains the high load on the generator you've seen and the much lower load on rtpproxy.
No, rtpproxy was listening on 127.0.0.1 (I've started it with -l 127.0.0.1). If it would have not listened on 127.0.0.1 then there would have been no load on it (cpu 0%).
BTW: there is an error in your changes: command should be: "%(mode)s %(session)s %(ip)s %(port)d dummyfrom dummyto\n" % self.__dict_
and not "%(mode)s %(session)s %(port)d %(ip)s dummyfrom dummyto\n" ...
rtpgenerator has a few drawbacks: it eats more cpu than rtproxy and cannot create more than 510 calls (the python code seems to use internally select witch doesn't work on more than 1024 file descriptors).
You can call poll3 instead of poll, but it is not designed to generate that many calls. That's why we limited it to 30 max. If it generates more it loads the cpu too much and the results are not conclusive. Instead run multiple instances with 30 calls and they should do better. Or run it on a different host that the one that runs rtpproxy.
If you run it on a different host you end up with the same problem (it will still eat the cpu faster then rtpproxy, unless you have a much faster test host). In my tests I've used a dual machine. Both rtpproxy and rtpgenerator are single processes, so each of them got in fact a cpu for them (they were not competing for the same cpu). Moreover running rtpgenerator on a different host than rtpproxy will require more changes, since rtpproxy uses udp in this case.
Another problem if you try to generate too many calls with it is that it won't be able to send the packets at exactly 20ms time intervals. In this case you will see a high load on the generator and a much smaller load on rtpproxy itself. (as we noticed the thing that generates the high load on both mediaproxy and rtpproxy is the combination of many calls with the small time interval that the packets arrive at the proxy (10-20 ms). if this time interval is increased because the generator cannot push that many data at every 20ms the load on rtpproxy will decrease very quickly while the load on the generator will sharply increase).
As a conclusion, I suggest the following to get results that are closer to reality:
- Preferably run the generator on another machine.
As I said this won't solve much and requires deeper changes to rtpgenerator (udp support). OTOH I think a dual machine is pretty good for this kind of tests.
- Never use a count higher than 50 (at least for a 1GHz machine) for a running instance of the generator. The rule is to not see the
generator eat more than 1% of CPU. If you need more calls run multiple instances of the generator to sum up the number of calls.
In my case generator always use more than 1% of the CPU (even for count=30). A good test will run different instances on different machines. However you would need a lot of them.
(The attached generator accepts up to 50 calls - increased from 30) 3. Specify the commands you used to run both the generator and the proxy so it is easy to spot problems like data being sent to 127.0.0.1 or the fact that data is only sent in one direction but never answered (we've seen this sometimes if --ip is not specified)
Ok, I forgot to include them. I've re-run the tests with similar results: ulimit -n 10240 ./rtpproxy -f -l 127.0.0.1 -s /tmp/rtp.sock chmod 666 /tmp/rtp.sock
./rtp2generator.py --ip=127.0.0.1 --proxy=/tmp/rtp.sock --g711 --count=500
- Make sure the data is passing through the proxy. With mediaproxy you
can use the sessions.py script of the media_sessions.phtml web page. With rtpproxy use iptraf or tcpdump.
Or just watch the load and the udp socket queues.
[...]
You might have a point though about the large number of calls bad behaviour of rtpgenerator. Since it seems like it's using select it might scale very badly with the number of calls (select/poll performance drops much faster then linearly with the increasing number of files descriptors).
The same problem affects rtpproxy. I think its performance would improve dramatically if it would switch from poll to sigio (linux 2.4), epoll (2.6), kqueue (*bsd) or /dev/poll (solaris).
Andrei