Re: [Serusers] Re: Fw: [Users] TM : retransmission timers

I'm sorry to nip in, but I tried to rerun the tests again and add more info into output as requested and add stable ser and CVS openser. I know that this test doesn't conform much to real life (for example generated callid/branch/tags differs only in a number, etc) but it can give at least an image about simple stateful forward. So, if anybody is interested: http://www.iptel.org/~vku/performance/tm.serXopenser.correct/ I tried the same once more with less iterations because there were some errors in log from openser speaking about low memory (I used -m to specify shared mem size but with 768M it still said errors, might be a memleak or did I anything wrong?). With 1M iterations it was without errors: http://www.iptel.org/~vku/performance/tm.serXopenser.1M/ Vaclav P.S. I have forgotten - SIPP was "Sipp v1.1, version 20060829, built Sep 5 2006, 15:07:25", I'm attaching simple patch which I have used. On Wed, Nov 22, 2006 at 12:48:12AM +0200, Daniel-Constantin Mierla wrote: >I love such "independent" and "very very useful" tests ... one selected >the versions he liked, latest development of ser with latest stable >version of openser, the details about testing scenarios are pretty >limited. However these details are very very insignificant, really. > >What matters is this particular case: what you tested is useless and >someone can better implement a tiny kernel module to perform same job >much faster that will make openser/ser trashed instantly if that is >their only usage. More important are the performances in real world >cases. I am not going to do comparison tests and reveal numbers, I will >let you do and hope make the results available. > >I will exemplify with just two common use cases: >A) ITSP where usrloc is required - to get the throughput from your tests >one needs to have over million of online users. Let me know how SER is >doing with loading them, I can bet that it takes several minutes to >start (so service down for a significat time) and lot to lookup a record >afterwards, do not forget to mention required memory. Then we will see >if the forwarding throughput is the bottleneck. >B) carrier - heavy accounting needed - take the latest cvs snapshots and >test it, look at flexibility in same time and see if the balance of >throughput and features is satisfactory. Do not forget that behind >database should be redundant for a reliable accounting storage. > >My conclusion and the point I wanted to underline is that forwarding is >not the bottleneck by far and so far in real-world deployments -- or at >least nobody reported in openser mailing lists. Once it will be, for >sure there will be effort and focus to optimize it. I don't even bother >to check the scenarios, environment and test results you had, because >makes no sense today. > >It is more important to look at the results gave, for example, here by >an independent party: >http://openser.org/pipermail/users/2006-November/007777.html > >With a real config and clustering system the performance of a box was >300calls per second -- having at least 5 database accesses!!!. If you >need double you can add one more hardware, without extra configuration >overhead, just plug and play. And that is stable version of OpenSER >since July this year (btw, for those who keep saying that OpenSER does >not focus on stability, just check the CVS and see the number of bugs >encountered with this release, maybe you can change your opinion), and >you can have a safe environment distributed geographically where each >hardware can undertake the traffic from the others on the fly. With >single box crashing because of different independent reasons (hardware >failure, power outages ...) you get no service ... with three boxes you >can serve huge number of active subscribers in peak hours and have >failover support, so service availability 100%. I am sure most of the >people look now how to build reliable platforms that scale very easy and >can be distributed around the world, with a bunch of useful features -- >simple first line replacement is not the business case for VoIP anymore. > >We didn't try at OpenSER to get a airplane when we have to drive city >streets, we looked to get feature rich and reliable application for its >use cases. I would propose to have focus on making own applications >better than trying to show the other one is worse. > >Cheers, >Daniel > >PS. You can use stateless forwarding to get even better results, the >usefulness will be the same. > >On 11/21/06 12:30, Jiri Kuthan wrote: >>Regarding the technical discussion, here are some hard numbers which show >>how SER stack outperforms derivative work. Forwarding throughput is >>clearly >>several times better under stress and consequently, variation of response >>delay is rather stable. >> >> >>>http://www.iptel.org/~vku/performance/tm.serXopenser.pulpuk/ >>> >> >>-jiri >> >> >>At 21:16 09/11/2006, Rao Ramaratnamma wrote: >> >>>Hi Weiter, >>> >>>Yeah, I have been trying to limit myself to technical observations too, >>>but the governance aspect is somewhat interesting too as a hint for >>>future development, even though I guess even this is much more >>>confusing than the technical ones. I have investigated, both projects >>>have their firms with them that pursue their commercial interests which >>>creates a risk of possibly departing from the public interest, like >>>with redhat. >>>From this angle they look quite similar. But if any worries me just a >>>little bit more than openser. Appearance at commercial shows on the >>>"open" side versus technical event on the "net" side if I take your BSD >>>parallel, marketing "open" webpage accusing "net" version bad, hiding >>>root commerical sponsors on the "open" webpage, this could be signs for >>>a redhat-like doubleedged sword. Hopefully I am oversensing because I >>>mean it is natural that everybody has SOME interest, but indisputably >>>folks on both sides have done good work, but same indisputably more >>>TRANSPARENCY would be helpful for both projects so that users can be >>>less investigative. >>> >>>But I agree the technical comparison you suggest will be very useful if >>>not most useful. This is what I am eventually upto. Anything folks have >>>to tell in this topic is most welcome like the retransmission timers in >>>subject or user loading. >>> >>>rr >>> >>>disconcerted by the fact that the more I know the more I am confused >>>and determined to get over the learning curve quickly. also excuse the >>>abuse I crossposted again but I think cross interrogation is a bit >>>painful but the more effective :-) >>> >>>----- Original Message ---- >>>From: Weiter Leiter &lt;bp4mls(a)googlemail.com&gt; >>>To: Kim Il &lt;kim_il_s(a)yahoo.com&gt; >>>Cc: users(a)openser.org >>>Sent: Thursday, November 9, 2006 1:42:29 PM >>>Subject: Re: Fw: [Users] TM : retransmission timers >>> >>>Common user barely has time to meet his boss requirements, rather than >>>playing around with different scenarios, platforms, environments. >>>I only read one email where Daniel stated that OpenSER now performs a >>>whole much better while loading users from database. SER guys put no >>>figure out yet, neither bare numbers nor comparisons. I'm just really >>>curious to see how both servers perform, that's all. >>>Even though I must maintain my SER, I kinda like OpenSER's faster >>>releases and developers' responsiveness (that I shamelessly exploit for >>>the common code left there :-), which is pretty much nonexistent with >>>iptel (at least this is the general belief here at OpenSER). But about >>>this I'll probably have to fight on SER's mailing list. I still wish >>>that one day I won't have to compare features; heck, NetSER and FreeSER >>>are still available ;-). >>>WL. >>> >>>PS. Maybe regretfully, I haven't seen any iptel booth at von this year, >>>while OpenSER guys put up a nice show. My congrats. >>> >>>On 11/9/06, Kim Il <<mailto:kim_il_s@yahoo.com>kim_il_s@yahoo.com> >>>wrote: I can see what you are hinting at, but I guess that the users >>>are the unbiased party that should do the judgment and not the parties >>>who have something to gain. >>>cheers >>> >>>Weiter Leiter <<mailto:bp4mls@googlemail.com>bp4mls@googlemail.com> >>>wrote: This features comparisons are not to last for too long, some >>>performance comparisons would also be nice. After all, there are plenty >>>of UA-level stacks out there. At least now that both projects get to >>>have stable releases after forking and some core functionality remained >>>shared. I wonder what "unbiased" organization will take up the >>>challenge. :-) >>>On 11/8/06, Kim Il <<mailto:kim_il_s@yahoo.com> kim_il_s(a)yahoo.com > >>>wrote: Mike, >>>this is a really good start and we should collect these things so as >>>to help the community to take the right choice. I would also suggest >>>that what ever ground breaking issues we list we stay at the functional >>>level (I do not think anyone is helped by using a description >>>containing "allowing carrier grade platforms" and similar marketing >>>phrases). cheers >>> >>> >>>{truncated because too large} >>> >>> >>> >>> >>>Sponsored Link >>>Talk more and pay less. Vonage can save you up to $300 a year on your >>>phone bill. >>><http://clk.atdmt.com/VON/go/yhxxxvon1080000017von/direct/01/>Sign up >>>now. >>> >>>_______________________________________________ >>>Users mailing list &gt;&gt;&gt;Users(a)openser.org >>><http://openser.org/cgi-bin/mailman/listinfo/users>http://openser.org/cgi-bin/mailman/listinfo/users >>> >>> >>>_______________________________________________ >>>Serusers mailing list &gt;&gt;&gt;Serusers(a)lists.iptel.org >>>http://lists.iptel.org/mailman/listinfo/serusers >>> >>-- >>Jiri Kuthan http://iptel.org/~jiri/ >> >>_______________________________________________ >>Serusers mailing list &gt;&gt;Serusers(a)lists.iptel.org >>http://lists.iptel.org/mailman/listinfo/serusers >> >> >_______________________________________________ >Serusers mailing list &gt;Serusers(a)lists.iptel.org >http://lists.iptel.org/mailman/listinfo/serusers > >------------------------------------------------------------------------ > >_______________________________________________ >Serusers mailing list &gt;Serusers(a)lists.iptel.org >http://lists.iptel.org/mailman/listinfo/serusers