Subject: Re: [Users] TM : retransmission timers
From: Vaclav Kubart <[email-masked]>
To: Daniel-Constantin Mierla <[email-masked]>
Cc: Jiri Kuthan <[email-masked]>, [email-masked], Kim Il <[email-masked]>, [email-masked]
Date: Tue, 28 Nov 2006 07:07:13 +0100

On Mon, Nov 27, 2006 at 11:16:01PM +0200, Daniel-Constantin Mierla wrote:
> As I can see, you get better and better with openser, maybe you can get 
> some training so you will be able to configure and tune it properly to 
> fit your needs and get the appropriate results (googling will reveal 
> some doing trainings for openser). So which are right, these ones, the 
> previous ones or the next testing results? You tested something, which 
> (I suppose) you are very familiar with (ser), against something that you 
> do not know properly to configure. There are some internals that differ 
> a lot and may have quite a lot of performance impact.

Might be. I have increased shared memory size to get less errors (and
timeouts) got by some openser tests which have influence on average
throughput - I wanted to have this number to be comparable with ser. In
the last one I reduced number of created transactions because 768 MB was
not sufficient.

> Just after you sent this mail, I have seen a commit to usrloc which 
> saves some problems I pointed in my previous email in this thread (so, 
> yes, we are concerned about performance, and see you need to catch up 
> now in some directions). The difference is now that ser's usrloc hash 
> table size is by default 2^14 (16384) while in openser is 2^9 (512). So 
> I guess some tests you can do now will be faster for ser, but to change 
> the hash value in ser you have to re-compile, as opposite to openser 
> where you can do it via module parameter.
 
:-) This usrloc problem is known to us for some weeks as a result of
other performance tests. It is really good to do them. ;-) I suggested
to Andrei to do hash table parameters (hash function and size)
configurable but he conviced me, that fixed solution (at least for the
size) will be sufficient...

> So please, try to make sure that the corresponding parameters have same 
> values, from number of processes, to memory, hash tables sizes, and so 
> on ... for future tests. If you are not able to do fair tests, it is 
> better to leave it for some impartial entities. With the description of 
> your tests a lot of parameters and variables are hidden, and do not 
> reflect the real capacity of the applications (I can say it now at least 
> for openser).
> 

Might be, I only tried to run with default compilation flags and very
simple config. I didn't try to play with anything else. Such tests are
good to check a small closed part of code to show if there are problems
or not - the usrloc problem was found this way...

> And just to remember, as proved, you got very good performances but 
> wrong processing. I will ask you and users: do you prefer to have *high 
> performances* tied to *invalid processing* of the requests?
> http://openser.org/pipermail/users/2006-November/007843.html
> Well, maybe for some folks it is a 'proud' to say: *ser offers very fast 
> _invalid_ functionalities* or "ser can scale to millions of _offline_ 
> users*.
> 
> You owe to say to the users that the performances were obtained with a 
> *buggy SIP transaction processing*, and correct the web pages posted at 
> the link below.

OK, I will try to verify your bug report - if it will show, that you
are right I can do the tests again as soon as it will be corrected.

	Vaclav

> Actually I replied to this mail because of some accuses that 'we spread 
> the rumor  "ser is dead"'. Nowhere you can find such statement from our 
> side. After some investigations proved that the phrase in charge is:
> "Soon after, /iptelorg.com Gmbh/ was sold to /Tekelec/, which had no 
> intention to continue the development of the public project."
> from oepnser history 
> (http://www.openser.org/index.php?option=com_content&task=view&id=40&Itemid=61).
> 
> This is quite different than "ser is dead", and many of you know the 
> statement made in "openser history" is true. I cannot name some private 
> channels I have, but there are publicly spoken ones. I paste from 
> https://mail.internet2.edu/wws/arc/sip.edu/2006-10/msg00003.html:
> 
> <snip>
> Christian says that he was surprised at the continued development of
> SER after speaking with people from Tekelec, as they mentioned that
> they were not as interested in continuing SER as an open source
> project and were more interested in integrating SER into their IMS
> offerings. He asks about the future of SER as an open source
> project. Jiri feels that there should be no issues, and the main
> contributors are still making contributions. He's hesitant to speak on
> the behalf of the company, but feels that based on past performance
> nothing should be changing.
> </snip>
> 
> I would polite ask those persons not to twist the phrases without giving 
> good proof.
> 
> Users can see now who and how tries to hoke up the reality.
> 
> Cheers,
> Daniel
> 
> 
> On 11/24/06 17:06, Vaclav Kubart wrote:
> >I'm sorry to nip in, but I tried to rerun the tests again and add more
> >info into output as requested and add stable ser and CVS openser.
> >
> >I know that this test doesn't conform much to real life (for example
> >generated callid/branch/tags differs only in a number, etc) but it can
> >give at least an image about simple stateful forward.
> >
> >So, if anybody is interested:
> >
> >http://www.iptel.org/~vku/performance/tm.serXopenser.correct/
> >
> >I tried the same once more with less iterations because there were some
> >errors in log from openser speaking about low memory (I used -m to
> >specify shared mem size but with 768M it still said errors, might be a
> >memleak or did I anything wrong?). With 1M iterations it was without
> >errors:
> >
> >http://www.iptel.org/~vku/performance/tm.serXopenser.1M/
> >
> >	Vaclav
> >
> >P.S. I have forgotten - SIPP was "Sipp v1.1, version 20060829, built Sep
> >5 2006, 15:07:25", I'm attaching simple patch which I have used.
> >
> >On Wed, Nov 22, 2006 at 12:48:12AM +0200, Daniel-Constantin Mierla wrote:
> >  
> >>I love such "independent" and "very very useful" tests ... one selected 
> >>the versions he liked, latest development of ser with latest stable 
> >>version of openser, the details about testing scenarios are pretty 
> >>limited. However these details are very very insignificant, really.
> >>
> >>What matters is this particular case: what you tested is useless and 
> >>someone can better implement a tiny kernel module to perform same job 
> >>much faster that will make openser/ser trashed instantly if that is 
> >>their only usage. More important are the performances in real world 
> >>cases. I am not going to do comparison tests and reveal numbers, I will 
> >>let you do and hope make the results available.
> >>
> >>I will exemplify with just two common use cases:
> >>A) ITSP where usrloc is required - to get the throughput from your tests 
> >>one needs to have over million of online users. Let me know how SER is 
> >>doing with loading them, I can bet that it takes several minutes to 
> >>start (so service down for a significat time) and lot to lookup a record 
> >>afterwards, do not forget to mention required memory. Then we will see 
> >>if the forwarding throughput is the bottleneck.
> >>B) carrier - heavy accounting needed - take the latest cvs snapshots and 
> >>test it, look at flexibility in same time and see if the balance of 
> >>throughput and features is satisfactory. Do not forget that behind 
> >>database should be redundant for a reliable accounting storage.
> >>
> >>My conclusion and the point I wanted to underline is that forwarding is 
> >>not the bottleneck by far and so far in real-world deployments -- or at 
> >>least nobody reported in openser mailing lists. Once it will be, for 
> >>sure there will be effort and focus to optimize it. I don't even bother 
> >>to check the scenarios, environment and test results you had, because 
> >>makes no sense today.
> >>
> >>It is more important to look at the results gave, for example, here by 
> >>an independent party:
> >>http://openser.org/pipermail/users/2006-November/007777.html
> >>
> >>With a real config and clustering system the performance of a box was 
> >>300calls per second -- having at least 5 database accesses!!!. If you 
> >>need double you can add one more hardware, without extra configuration 
> >>overhead, just plug and play. And that is stable version of OpenSER 
> >>since July this year (btw, for those who keep saying that OpenSER does 
> >>not focus on stability, just check the CVS and see the number of bugs 
> >>encountered with this release, maybe you can change your opinion), and 
> >>you can have a safe environment distributed geographically where each 
> >>hardware can undertake the traffic from the others on the fly. With 
> >>single box crashing because of different independent reasons (hardware 
> >>failure, power outages ...) you get no service ... with three boxes you 
> >>can serve huge number of active subscribers in peak hours and have 
> >>failover support, so service availability 100%. I am sure most of the 
> >>people look now how to build reliable platforms that scale very easy and 
> >>can be distributed around the world, with a bunch of useful features -- 
> >>simple first line replacement is not the business case for VoIP anymore.
> >>
> >>We didn't try at OpenSER to get a airplane when we have to drive city 
> >>streets, we looked to get feature rich and reliable application for its 
> >>use cases. I would propose to have focus on making own applications 
> >>better than trying to show the other one is worse.
> >>
> >>Cheers,
> >>Daniel
> >>
> >>PS. You can use stateless forwarding to get even better results, the 
> >>usefulness will be the same.
> >>
> >>On 11/21/06 12:30, Jiri Kuthan wrote:
> >>    
> >>>Regarding the technical discussion, here are some hard numbers which show
> >>>how SER stack outperforms derivative work. Forwarding throughput is 
> >>>clearly
> >>>several times better under stress and consequently, variation of response
> >>>delay is rather stable.
> >>>
> >>> 
> >>>      
> >>>>http://www.iptel.org/~vku/performance/tm.serXopenser.pulpuk/
> >>>>   
> >>>>        
> >>>-jiri
> >>>
> >>>
> >>>At 21:16 09/11/2006, Rao Ramaratnamma wrote:
> >>> 
> >>>      
> >>>>Hi Weiter,
> >>>>
> >>>>Yeah, I have been trying to limit myself to technical observations too, 
> >>>>but the governance aspect is somewhat interesting too as a hint for 
> >>>>future development, even though I guess even this is much more 
> >>>>confusing than the technical ones. I have investigated, both projects 
> >>>>have their firms with them that pursue their commercial interests which 
> >>>>creates a risk of possibly departing from the public interest, like 
> >>>>with redhat. 
> >>>>From this angle they look quite similar. But if any worries me just a 
> >>>      
> >>>>little bit more than openser.  Appearance at commercial shows on the 
> >>>>"open" side versus technical event on the "net" side if I take your BSD 
> >>>>parallel, marketing "open" webpage accusing "net" version bad, hiding 
> >>>>root commerical sponsors on the "open" webpage, this could be signs for 
> >>>>a redhat-like doubleedged sword.  Hopefully I am oversensing because I 
> >>>>mean it is natural that everybody has SOME interest, but indisputably 
> >>>>folks on both sides have done good work, but same indisputably more 
> >>>>TRANSPARENCY would be helpful for both projects so that users can be 
> >>>>less investigative.
> >>>>
> >>>>But I agree the technical comparison you suggest will be very useful if 
> >>>>not most useful. This is what I am eventually upto. Anything folks have 
> >>>>to tell in this topic is most welcome like the retransmission timers in 
> >>>>subject or user loading.
> >>>>
> >>>>rr
> >>>>
> >>>>disconcerted by the fact that the more I know the more I am confused 
> >>>>and determined to get over the learning curve quickly. also excuse the 
> >>>>abuse I crossposted again but I think cross interrogation is a bit 
> >>>>painful but the more effective :-)
> >>>>
> >>>>----- Original Message ----
> >>>>From: Weiter Leiter <[email-masked]>
> >>>>To: Kim Il <[email-masked]>
> >>>>Cc: [email-masked]
> >>>>Sent: Thursday, November 9, 2006 1:42:29 PM
> >>>>Subject: Re: Fw: [Users] TM : retransmission timers
> >>>>
> >>>>Common user barely has time to meet his boss requirements, rather than 
> >>>>playing around with different scenarios, platforms, environments. 
> >>>>I only read one email where Daniel stated that OpenSER now performs a 
> >>>>whole much better while loading users from database. SER guys put no 
> >>>>figure out yet, neither bare numbers nor comparisons. I'm just really 
> >>>>curious to see how both servers perform, that's all. 
> >>>>Even though I must maintain my SER, I kinda like OpenSER's faster 
> >>>>releases and developers' responsiveness (that I shamelessly exploit for 
> >>>>the common code left there :-), which is pretty much nonexistent with 
> >>>>iptel (at least this is the general belief here at OpenSER). But about 
> >>>>this I'll probably have to fight on SER's mailing list. I still wish 
> >>>>that one day I won't have to compare features; heck, NetSER and FreeSER 
> >>>>are still available ;-). 
> >>>>WL.
> >>>>
> >>>>PS. Maybe regretfully, I haven't seen any iptel booth at von this year, 
> >>>>while OpenSER guys put up a nice show. My congrats.
> >>>>
> >>>>On 11/9/06, Kim Il <<mailto:[email-masked]>[email-masked]> 
> >>>>wrote: I can see what you are hinting at, but I guess that the users 
> >>>>are the unbiased party that should do the judgment and not the parties 
> >>>>who have something to gain.
> >>>>cheers
> >>>>
> >>>>Weiter Leiter <<mailto:[email-masked]>[email-masked]> 
> >>>>wrote:  This features comparisons are not to last for too long, some 
> >>>>performance comparisons would also be nice. After all, there are plenty 
> >>>>of UA-level stacks out there. At least now that both projects get to 
> >>>>have stable releases after forking and some core functionality remained 
> >>>>shared. I wonder what "unbiased" organization will take up the 
> >>>>challenge. :-)
> >>>>On 11/8/06, Kim Il <<mailto:[email-masked]> [email-masked] > 
> >>>>wrote:  Mike,
> >>>>this is a really good start and we should collect these things  so as 
> >>>>to help the  community to take the right choice. I would also suggest 
> >>>>that what ever ground breaking issues we list we stay at the functional 
> >>>>level (I do not think anyone is helped by using a description 
> >>>>containing "allowing carrier grade platforms" and similar marketing 
> >>>>phrases). cheers
> >>>>
> >>>>
> >>>>{truncated because too large}
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>Sponsored Link 
> >>>>Talk more and pay less. Vonage can save you up to $300 a year on your 
> >>>>phone bill. 
> >>>><http://clk.atdmt.com/VON/go/yhxxxvon1080000017von/direct/01/>Sign up 
> >>>>now. 
> >>>>
> >>>>_______________________________________________
> >>>>Users mailing list
> >>>>[email-masked]
> >>>><http://openser.org/cgi-bin/mailman/listinfo/users>http://openser.org/cgi-bin/mailman/listinfo/users
> >>>>
> >>>>
> >>>>_______________________________________________
> >>>>Serusers mailing list
> >>>>[email-masked]
> >>>>http://lists.iptel.org/mailman/listinfo/serusers
> >>>>   
> >>>>        
> >>>--
> >>>Jiri Kuthan            http://iptel.org/~jiri/  
> >>>
> >>>_______________________________________________
> >>>Serusers mailing list
> >>>[email-masked]
> >>>http://lists.iptel.org/mailman/listinfo/serusers
> >>>
> >>> 
> >>>      
> >>_______________________________________________
> >>Serusers mailing list
> >>[email-masked]
> >>http://lists.iptel.org/mailman/listinfo/serusers
> >>    
> >>------------------------------------------------------------------------
> >>
> >>Index: call.cpp
> >>===================================================================
> >>--- call.cpp	(revision 32)
> >>+++ call.cpp	(working copy)
> >>@@ -1812,6 +1812,8 @@
> >>           dest += sprintf(dest, "%s", (media_ip_is_ipv6 ? "6" : "4"));
> >>         } else if(!strcmp(keyword, "call_number")) {
> >>           dest += sprintf(dest, "%lu", number);
> >>+        } else if(!strcmp(keyword, "call_number_6")) {
> >>+          dest += sprintf(dest, "%06lu", number);
> >>         } else if(!strcmp(keyword, "call_id")) {
> >>           dest += sprintf(dest, "%s", id);
> >>         } else if(!strcmp(keyword, "cseq")) {
> >>@@ -2246,6 +2248,19 @@
> >>     return;
> >>   }
> >> 
> >>+  /* quick hack for UAS and loose router - needed to use routes 
> >>+   * in the same order as Record-Routes */
> >>+  if (bRequestIncoming) {
> >>+	  dialog_route_set = (char *)calloc(1, strlen(rr) + 2);
> >>+	  sprintf(dialog_route_set, "%s", rr);
> >>+
> >>+	  if (strlen (contact)) {
> >>+		  strcpy (next_req_url, contact);
> >>+		  formatNextReqUrl (next_req_url);
> >>+	  }
> >>+	  return;
> >>+  }
> >>+  
> >>   char actual_rr[MAX_HEADER_LEN];
> >>   char targetURI[MAX_HEADER_LEN];
> >>   memset(actual_rr, 0, sizeof(actual_rr));
> >>