Re: [Serusers] Re: Fw: [Users] TM : retransmission timers

I'm sorry to nip in, but I tried to rerun the tests again and add more info into output as requested and add stable ser and CVS openser. I know that this test doesn't conform much to real life (for example generated callid/branch/tags differs only in a number, etc) but it can give at least an image about simple stateful forward. So, if anybody is interested: http://www.iptel.org/~vku/performance/tm.serXopenser.correct/ I tried the same once more with less iterations because there were some errors in log from openser speaking about low memory (I used -m to specify shared mem size but with 768M it still said errors, might be a memleak or did I anything wrong?). With 1M iterations it was without errors: http://www.iptel.org/~vku/performance/tm.serXopenser.1M/ Vaclav P.S. I have forgotten - SIPP was "Sipp v1.1, version 20060829, built Sep 5 2006, 15:07:25", I'm attaching simple patch which I have used. On Wed, Nov 22, 2006 at 12:48:12AM +0200, Daniel-Constantin Mierla wrote: > I love such "independent" and "very very useful" tests ... one selected > the versions he liked, latest development of ser with latest stable > version of openser, the details about testing scenarios are pretty > limited. However these details are very very insignificant, really. > > What matters is this particular case: what you tested is useless and > someone can better implement a tiny kernel module to perform same job > much faster that will make openser/ser trashed instantly if that is > their only usage. More important are the performances in real world > cases. I am not going to do comparison tests and reveal numbers, I will > let you do and hope make the results available. > > I will exemplify with just two common use cases: > A) ITSP where usrloc is required - to get the throughput from your tests > one needs to have over million of online users. Let me know how SER is > doing with loading them, I can bet that it takes several minutes to > start (so service down for a significat time) and lot to lookup a record > afterwards, do not forget to mention required memory. Then we will see > if the forwarding throughput is the bottleneck. > B) carrier - heavy accounting needed - take the latest cvs snapshots and > test it, look at flexibility in same time and see if the balance of > throughput and features is satisfactory. Do not forget that behind > database should be redundant for a reliable accounting storage. > > My conclusion and the point I wanted to underline is that forwarding is > not the bottleneck by far and so far in real-world deployments -- or at > least nobody reported in openser mailing lists. Once it will be, for > sure there will be effort and focus to optimize it. I don't even bother > to check the scenarios, environment and test results you had, because > makes no sense today. > > It is more important to look at the results gave, for example, here by > an independent party: > http://openser.org/pipermail/users/2006-November/007777.html > > With a real config and clustering system the performance of a box was > 300calls per second -- having at least 5 database accesses!!!. If you > need double you can add one more hardware, without extra configuration > overhead, just plug and play. And that is stable version of OpenSER > since July this year (btw, for those who keep saying that OpenSER does > not focus on stability, just check the CVS and see the number of bugs > encountered with this release, maybe you can change your opinion), and > you can have a safe environment distributed geographically where each > hardware can undertake the traffic from the others on the fly. With > single box crashing because of different independent reasons (hardware > failure, power outages ...) you get no service ... with three boxes you > can serve huge number of active subscribers in peak hours and have > failover support, so service availability 100%. I am sure most of the > people look now how to build reliable platforms that scale very easy and > can be distributed around the world, with a bunch of useful features -- > simple first line replacement is not the business case for VoIP anymore. > > We didn't try at OpenSER to get a airplane when we have to drive city > streets, we looked to get feature rich and reliable application for its > use cases. I would propose to have focus on making own applications > better than trying to show the other one is worse. > > Cheers, > Daniel > > PS. You can use stateless forwarding to get even better results, the > usefulness will be the same. > > On 11/21/06 12:30, Jiri Kuthan wrote: >> Regarding the technical discussion, here are some hard numbers which show >> how SER stack outperforms derivative work. Forwarding throughput is clearly >> several times better under stress and consequently, variation of response >> delay is rather stable. >> >> >>> http://www.iptel.org/~vku/performance/tm.serXopenser.pulpuk/ >>> >> >> -jiri >> >> >> At 21:16 09/11/2006, Rao Ramaratnamma wrote: >> >>> Hi Weiter, >>> >>> Yeah, I have been trying to limit myself to technical observations too, >>> but the governance aspect is somewhat interesting too as a hint for >>> future development, even though I guess even this is much more confusing >>> than the technical ones. I have investigated, both projects have their >>> firms with them that pursue their commercial interests which creates a >>> risk of possibly departing from the public interest, like with redhat. >> >From this angle they look quite similar. But if any worries me just a >>> little bit more than openser. Appearance at commercial shows on the >>> "open" side versus technical event on the "net" side if I take your BSD >>> parallel, marketing "open" webpage accusing "net" version bad, hiding >>> root commerical sponsors on the "open" webpage, this could be signs for a >>> redhat-like doubleedged sword. Hopefully I am oversensing because I mean >>> it is natural that everybody has SOME interest, but indisputably folks on >>> both sides have done good work, but same indisputably more TRANSPARENCY >>> would be helpful for both projects so that users can be less >>> investigative. >>> >>> But I agree the technical comparison you suggest will be very useful if >>> not most useful. This is what I am eventually upto. Anything folks have >>> to tell in this topic is most welcome like the retransmission timers in >>> subject or user loading. >>> >>> rr >>> >>> disconcerted by the fact that the more I know the more I am confused and >>> determined to get over the learning curve quickly. also excuse the abuse >>> I crossposted again but I think cross interrogation is a bit painful but >>> the more effective :-) >>> >>> ----- Original Message ---- >>> From: Weiter Leiter &lt;bp4mls(a)googlemail.com&gt; >>> To: Kim Il &lt;kim_il_s(a)yahoo.com&gt; >>> Cc: users(a)openser.org >>> Sent: Thursday, November 9, 2006 1:42:29 PM >>> Subject: Re: Fw: [Users] TM : retransmission timers >>> >>> Common user barely has time to meet his boss requirements, rather than >>> playing around with different scenarios, platforms, environments. >>> I only read one email where Daniel stated that OpenSER now performs a >>> whole much better while loading users from database. SER guys put no >>> figure out yet, neither bare numbers nor comparisons. I'm just really >>> curious to see how both servers perform, that's all. >>> Even though I must maintain my SER, I kinda like OpenSER's faster >>> releases and developers' responsiveness (that I shamelessly exploit for >>> the common code left there :-), which is pretty much nonexistent with >>> iptel (at least this is the general belief here at OpenSER). But about >>> this I'll probably have to fight on SER's mailing list. I still wish that >>> one day I won't have to compare features; heck, NetSER and FreeSER are >>> still available ;-). >>> WL. >>> >>> PS. Maybe regretfully, I haven't seen any iptel booth at von this year, >>> while OpenSER guys put up a nice show. My congrats. >>> >>> On 11/9/06, Kim Il <<mailto:kim_il_s@yahoo.com>kim_il_s@yahoo.com> wrote: >>> I can see what you are hinting at, but I guess that the users are the >>> unbiased party that should do the judgment and not the parties who have >>> something to gain. >>> cheers >>> >>> Weiter Leiter <<mailto:bp4mls@googlemail.com>bp4mls@googlemail.com> >>> wrote: This features comparisons are not to last for too long, some >>> performance comparisons would also be nice. After all, there are plenty >>> of UA-level stacks out there. At least now that both projects get to have >>> stable releases after forking and some core functionality remained >>> shared. I wonder what "unbiased" organization will take up the challenge. >>> :-) >>> On 11/8/06, Kim Il <<mailto:kim_il_s@yahoo.com> kim_il_s(a)yahoo.com > >>> wrote: Mike, >>> this is a really good start and we should collect these things so as to >>> help the community to take the right choice. I would also suggest that >>> what ever ground breaking issues we list we stay at the functional level >>> (I do not think anyone is helped by using a description containing >>> "allowing carrier grade platforms" and similar marketing phrases). cheers >>> >>> >>> {truncated because too large} >>> >>> >>> >>> >>> Sponsored Link >>> Talk more and pay less. Vonage can save you up to $300 a year on your >>> phone bill. >>> <http://clk.atdmt.com/VON/go/yhxxxvon1080000017von/direct/01/>Sign up >>> now. >>> >>> _______________________________________________ >>> Users mailing list >>> Users(a)openser.org >>> <http://openser.org/cgi-bin/mailman/listinfo/users>http://openser.org/cgi-bin/mailman/listinfo/users >>> >>> >>> _______________________________________________ >>> Serusers mailing list >>> Serusers(a)lists.iptel.org >>> http://lists.iptel.org/mailman/listinfo/serusers >>> >> -- >> Jiri Kuthan http://iptel.org/~jiri/ >> >> _______________________________________________ >> Serusers mailing list >> Serusers(a)lists.iptel.org >> http://lists.iptel.org/mailman/listinfo/serusers >> >> > _______________________________________________ > Serusers mailing list > Serusers(a)lists.iptel.org > http://lists.iptel.org/mailman/listinfo/serusers > > ------------------------------------------------------------------------ > > _______________________________________________ > Serusers mailing list > Serusers(a)lists.iptel.org > http://lists.iptel.org/mailman/listinfo/serusers