[Serusers] Via: branch tags from SER for CANCELs/BYEs frequently don't match value in the INVITE

Jiri Kuthan jiri at iptel.org
Fri Dec 4 23:48:09 CET 2009


While I have a lot of sympathies for your disappointment, I'm not really
sure you are tying it with the proper causes.  Let me explain my  viewpoint:

- I think the Email you have received from Andrei explains even using
   references why Via(INVITE)==Via(BYE) requirement violates standards
   and actual functionality as well. Why do you think you have not been
   getting a good advice? What's wrong with the CANCEL suggestion?

- the MD5 performance problem you are worried about has been addressed.
   I admit we have answered by classifying this as a non-problem, still this
   is our best-knowledge of the matter based on quite details profiling.
   Do you have any numbers supporting this is a real problem?

   Generally we don't think that for any given hardware performance is
   a problem with SER. Of course it can degrade for example by use of
   database, or any other expensive operations, but I'm quite confident
   that SER's thoughput is excellent and if service logic consumes more
   resources hardly anything can be done. At a point of time, more throughput
   takes more boxes but I think that this threshold is actually very high
   with SER.

   With certainty I know that SER has been used in the big and in the *biggest*
   deployments. I'm worried that this may sound a bit unfriendly towards the
   effort you guys developed or purchased as professional service, but I think
   that the presence of such deployments demonstrates scale is a non-issue
   in reasonably dimensioned environment. SER doesn't come up with dimensioning
   plans, one of the reasons being that it is non-trivial to provide generally
   valid assertions (traffic, hardware, confgiruation, dependencies on database,
   network architecture, all of these differences in actual deployments make it
   hard to provide general rules of thumb.)


- I cannot possibly comment on interoperability of "high-dollar SBCs and
   switches" to the general extent you are implying. I'm worried you can
   be dramattically disappointed if you tie all your expectations to dollars.
   I only know with certainty that in the specific cases we have encountered
   I can impossibly  assert that "high-dollar" and " brands" means knowing
   how INVITE  shall look like. In fact, we have been using SER to fix INVITEs from
   high-dollar brands to look like they are supposed to look like. Which
   is a double-edged sword, as frequently turning a message into something
   that A likes makes it hard-to-swallow for B. Unless you are in a single-vendor
   environment, the likelihood of necessity to address interop issues is,
   say, higher than noticeable.

- I agree that lack of parameter passing is a shortcoming. I agree the
   documentation is suboptimal. I'm very thankful to all participants who
   spend their valuable time and return SER's value by their contributions,
   but there is no "central contribution control" that would allow someone
   to cause the participants to address your particular wishes.

-jiri


Frank Durda IV wrote:
> Hello,
> 
> You are a bit out of date on this.  The problem was specific
> to CANCELs being ignored because the tag SER added to the
> CANCEL did not match the tag that SER added to the INVITE
> for the same call.
> 
> This flaw is in at least 2.0.0RC1  (we could not use 0.x
> versions due to even more bugs), and adding syn_branch=0
> merely changed the failure from CANCELs being ignored 100%
> of the time by commercial switches to being ignored perhaps
> 90% of the time (but at least now the messages looked more
> like what the RFC says they should look like).
> 
> This bug, where in many cases for calls sent from
> major name commercial equipment*, the CANCELs would not
> have the exact same tag as SER created for the INVITE
> (which two brands of commercial switch correctly ignored),
> was fixed by changing SER so that it creates its branch
> tag from header lines that do not legally change between
> INVITE and CANCEL.   Exactly what I stated in later posts
> in this thread.  I had to come up with the fix for SER
> myself.
> 
> * Strangely, calls from simple systems like Asterisk,
> always produced the header lines in the CANCEL that were
> identical to those in the INVITE, so the same tag was
> generated and the CANCEL from such sources worked.
> The fancier equipment opted to add legal information to
> header lines between INVITE and CANCEL and so doomed
> SER to compute a different branch tag.  SER could have
> solved this by using data for the computation only up to
> the ";" in those headers lines where data could be added
> or not using header lines that have a risk of being
> different, OR by saving a copy of the tag computed earlier.
> Yeah, yeah, I know about the stateless dream.  Whatever.
> 
> 
> My code change has been running for several million calls now
> at all our sites and solved the problem.  CANCELs are now
> accepted and acted-on by both Sonus and Alacatel-Lucent
> switches where previously both brands of switch would
> ignore most CANCEL messages passed through SER.   The calls
> were coming to us via/from ACME packet, Sonus, Nortel,
> Cisco and other switch brands, eg equipment who should know
> what an INVITE and CANCEL should look like and would have
> been accepted if SER hadn't put a mismatched tag in there.
> 
> 
> Despite solving this issue in-house, the annoyance threshold
> has been reached and we will be removing SER from the
> production call path.   SER seems fine for smaller setups,
> but shove 100-500Mbit/sec or more of calls through it around
> the clock (times several similar sites and all running on a
> number of high-high-end servers that are barely ticking over)
> and strange things start happening.
> 
> Most of that incoming traffic is coming from high-dollar
> SBCs and switches, built by people who probably have the
> standards implemented mostly right, and with enough volume,
> the "infrequent and rare" problems in SER that are known but
> no one wants to talk about or fix decide to appear frequently,
> and so SER becomes a pain.
> 
> No doubt, we set ourselves up for extra pain by electing to
> use NAT (for additional Internet isolation and for overlapping
> customer private network addressing) and rtpproxy, and worse,
> the two-interface mode of rtpproxy, which we chose for
> physical network isolation as well as to expand RTP bandwidth
> capacity.  The two-interface mode part was extremely painful,
> required many code fixes in both SER and rtpproxy to make that
> usable. as well as new functions in SER, yet bugs that
> were there in SER and rtpproxy all along remain.   On the good
> side, SER did protect a particularly unhardened and vulnerable
> commercial switch from the stupider SIP messages some of our
> clients generated, garbled or forwarded "as is" to us, as well
> as the garbage packets one has to expect coming from the
> Internet.
> 
> To, some the concept of using SER and rtpproxy only as
> cannon-fodder to take and if needed crash in response to
> the junk we were sometimes sent seems like a demeaning
> assignment for a piece of software that really is meant to
> do so much more, but we didn't need a lot of that, and the
> things we hoped we could do with SER were gradually whittled
> to a few because of the huge .cfg language restriction of not
> being able to pass variables or the contents of variables
> to built-in functions.  (Developers, you have got to
> implement a way to pass dynamic data to functions, not just
> constants.)
> 
> In our case, Everything coming in from the outside had only
> one place to go, a PSTN gateway switch sitting nearby.
> This meant our database was empty, and if there had been a way
> to disconnect mysql and eliminate two or three thousand mysql
> connections on each server doing little or nothing (other than
> whining that I needed to enable more connections), we would
> have done so.
> All SER configuration was done via the .cfg file, of which we
> had to run dozens of instances (one per SIP "trunk")
> due to limitations in SER that had to be gotten-around by
> brute force.  See the lack of variable passing mentioned above.
> In a NAT environment where you have to substitute public and
> private addresses in various headers of each message, the
> lack of variable passing means you have to hard-code
> constants for the old and substituted IP addresses everywhere
> and had to have giant if-else trees to decide which pair to
> substitute and in which direction this message was going
> so as to get the substiutions right.  We finally elected to
> run multiple separate copies SER on each server so as to not
> disturb other customers when adding or changing another.
> 
> For almost two years SER did a reasonable job doing the task of
> blocking traffic from the bad/malformed packets, and undesired
> sources, even though a few got through that killed the PSTN
> switch.  However, in the end, it was the good packets from
> desired sources that SER then turned into bad packets before
> passing them on (such as concoting different tags between
> INVITE and CANCEL messages with the same Cseq) that
> contributed to the forced retirement of SER.
> 
> 
> A probably-final suggestion for SER.   Fixing/updating the
> documentation for SER or its successor(s) would be an immense
> help to those using or considering its use.  The web site(s)
> I have tried to use are dreadful, and even a simple index that
> lists functions names and a one-liner of what they are good
> for IN THE CURRENT VERSION is tough to come by.   On one I
> got the impression that more time was spent implementing
> some cutesy HTML/javascript feature than on making the content
> usable.
> 
> There is too much expectation that someone using SER will
> troll all the source files, .c's and .h's looking for any
> comments on how functions work or  what parameters they
> accept that might have gotten left in there, or better still,
> updated to match how the code actually works today.
> Be sure to match up dates because you may find three
> description of the same function and what parameters it takes,
> and only or none of them is how it actually works today.
> I realize few people like to write documentation, but it
> desperately needs to be done.
> 
> Finally, mere mortals can't send e-mails to the SER developers
> lists!  It appears to be a closed club with no doorman.
> I gave up trying that ages ago.   That being the case, we
> "users" have to rely on the developers coming out once in a while
> to read what their users are saying out here on the "mortals"
> serusers list, OR maybe they sould open access to some list both
> parties read and write to.  Having worked on another open-source
> project (FreeBSD, back when I had spare time long ago), these
> things are handled quite differently there and so communications
> between developers and users,  particularly advanced or
> heavy-volume users, is pretty easy.
> 
> Good luck to all and thanks for the fish.
> 
> 
> Frank Durda IV - send mail to this address and remove the "LOSE"
> and adjust the month/year password accordingly:
> <uhclemLOSE.dec09%nemesis.lonestar.org>    http://nemesis.lonestar.org
>   "If you have a point, I suggest you sharpen it." - Frank Durda IV 2003
> Copyright 2009, ask before reprinting.
> 
> 
> _______________________________________________
> Serusers mailing list
> Serusers at lists.iptel.org
> http://lists.iptel.org/mailman/listinfo/serusers
> 




More information about the sr-users mailing list