[Devel] Re: [Serdev] OpenSER/SER with OSP performance test results

[di-shi at transnexus.com]

Hi Greger,

Thank you for your feedback. We know the test plan can be improved and we appreciate your suggestions.

>We have an iptel.org page for performance: http://www.iptel.org/ser/doc/performance
>If you allow, I would like to make a link to the test (or you can do it yourself; the page can be edited).

Thank you for asking. Please add the link to the test.

>Some comments to the test:
>- I'm still not sure that I understand the Post-Dial Delay. You state that 20% of the calls will complete 
>on the fourth attempt. So, this means that three INVITEs will time out on 2,000 ms (as SER 2.0 now 
>has a higher timer resolution, see http://www.iptel.org/how_the_new_timer_framework_works), 
>which is 6,000 ms or 6s. Wouldn't you then expect 20% of the calls to complete in >6s ? I may 
>have completely misunderstood this, but calls completing in less than 6s would do so due to the 
>impreciseness of the old 0.9 timers?
>So, to understand what actually happens, a scatter diagram (instead of the groupings) would be better?

We agree a scatter diagram is better. We used grouping of data because this is how SIPp presents the data.
As I had mentioned in the email to Jiri, the 6 sec threshold is not a good value. If the threshold is 6.1 sec, 
we expect 0 calls complete after the threshold. We may change it next time.

>- Do you have any idea what happened when CPU > 90% and SER's call completion dropped? 
>Looks strange to me. I also noticed that debugging was turned on (which, btw, we have discussed 
>that we probably will turn off when we release). 

We did not investigate why SER dropped calls when CPU > 90%. One clue is that we used asyn syslog. It 
is possible that during certain short time maybe CPU > 100%. 
We did not realize that the debug was on when we did the test. We will doble check next time.

>- Allow me to quote Jim Dalton's (TransNexus) blog post about the test 
>(http://transnexus.blogspot.com/2007/04/openser-performance-benchmark.html):
>"If we had used all four CPU cores we expect the results would have been 800 calls per second. To be 
>conservative, we would recommend service providers to plan on maximum CPU utilization of about 60%. 
>This would establish the OpenSER planning gauideline of 500 calls per second on a server with two, dual 
>core Xeon CPUs. 
>If you assume 15% of a service provider's traffic occurs during the busy hour, a 50% Answer Seizure >
>Ratio (ASR) and a 3 minute average call duration, then 500 calls per second equates to 540 million 
>minutes of VoIP traffic per month! We think this is impressive for an open source SIP proxy running on 
>a server with a retail price of $2,967."
>(of course, when referring to openser here, I assume he really means *SER)

You are right. the example you quoted applies to *SER. I had not yet tested SER when Jim wrote that blog.

Thanks,

Di-Shi Sun.




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://openser.org/pipermail/devel/attachments/20070508/d25c91b9/attachment.htm