Problem with worst case scenario of accessing 15 gateways over the DISPATCHER module - MAX_BRANCHES is reached
Hi,
I have a problem in my scenario where I use the dispatcher module for (round-robin based) distribution of incoming calls to 15 (!) different gateways. In idle case (= all gateways are up and running or just a couple of them does not accept calls) my solution is working fine. The solution is very primitive and based on the example of the dispatcher module's ReadMe. However, when e.g. 12 of 15 gateways are out of service (in detail: rejecting the call with "480 - Temporarily Unavailable") and only the 13th would be okay, I can not reach that gateway. This is because the TM module is interrupting the distribution with an error message indicating that the maximum number of branches (per transaction) is reached (ERROR:/tm/:/add_uac/: /maximum number of branches/ exceeded)....
My first idea for solving this problem was/is in manipulating the definition of the "max_branches" constant in the config.h file. However, I prefer using the precompiled Debian packages. Therefore that solution is not optimal. I have not found a method for decreasing the destination_set on an easy way... Does anybody know an alternative solution / workaround?
Thanks in advance, Klaus
On Sunday 03 July 2011, Klaus Feichtinger wrote:
I have a problem in my scenario where I use the dispatcher module for (round-robin based) distribution of incoming calls to 15 (!) different gateways. In idle case (= all gateways are up and running or just a couple of them does not accept calls) my solution is working fine. The solution is very primitive and based on the example of the dispatcher module's ReadMe. However, when e.g. 12 of 15 gateways are out of service (in detail: rejecting the call with "480 - Temporarily Unavailable") and only the 13th would be okay, I can not reach that gateway. This is because the TM module is interrupting the distribution with an error message indicating that the maximum number of branches (per transaction) is reached (ERROR:tm:add_uac: maximum number of branches exceeded)....
My first idea for solving this problem was/is in manipulating the definition of the "max_branches" constant in the config.h file. However, I prefer using the precompiled Debian packages. Therefore that solution is not optimal. I have not found a method for decreasing the destination_set on an easy way... Does anybody know an alternative solution / workaround?
Hi Klaus,
did I understand it correctly, you trying sequential 15 gateways in your setup? E.g.,
- try GW 1 - on error try GW 2 - ... - up to GW 15?
Normally I'd expect that you restrict the number of tries to something smaller, that you don't run into this max_branches issue and also don't have a that long call setup time for the user. And I also would think that you would notice that you running at a few percent of your capacitiy via some other alarms, and just fix the gateways?
With regards to your question, you could make the max_branches number configurable with a patch, at the moment its just a define.
Best regards,
Henning
On Sunday 03 July 2011, Klaus Feichtinger wrote:
I have a problem in my scenario where I use the dispatcher module for (round-robin based) distribution of incoming calls to 15 (!) different gateways. In idle case (= all gateways are up and running or just a couple of them does not accept calls) my solution is working fine. The solution is very primitive and based on the example of the dispatcher module's ReadMe. However, when e.g. 12 of 15 gateways are out of service (in detail: rejecting the call with "480 - Temporarily Unavailable") and only the 13th would be okay, I can not reach that gateway. This is because the TM module is interrupting the distribution with an error message indicating that the maximum number of branches (per transaction) is reached (ERROR:tm:add_uac: maximum number of branches exceeded)....
My first idea for solving this problem was/is in manipulating the definition of the "max_branches" constant in the config.h file. However, I prefer using the precompiled Debian packages. Therefore that solution is not optimal. I have not found a method for decreasing the destination_set on an easy way... Does anybody know an alternative solution / workaround?
Hi Klaus,
did I understand it correctly, you trying sequential 15 gateways in your setup? E.g.,
- try GW 1
- on error try GW 2
- ...
- up to GW 15?
Normally I'd expect that you restrict the number of tries to something smaller, that you don't run into this max_branches issue and also don't have a that long call setup time for the user. And I also would think that you would notice that you running at a few percent of your capacitiy via some other alarms, and just fix the gateways?
With regards to your question, you could make the max_branches number configurable with a patch, at the moment its just a define.
Best regards,
Henning
Hi Henning,
the described scenario is not a standard scenario. This would be realistic in emergency situations only (e.g. so many calls are coming in and the gateways are busy => e.g. responding with "486 Busy Here" ). Also the time delay would not be very high, because the fr_timer variable is set to a value of 750ms before relaying the call to the dispatcher-targets ("t_set_fr([...]))") - this is just for a scenario when the target is unreachable.
In case that the dispatcher-target is responding with a negative response code (as it was until now), all targets are sequentially addressed within a time of lower than 1 second until the error message of the TM module is interrupting that process.
However, if there is no alternative to changing the define value of MAX_BRANCHES I will use your recommendation for limiting the number of gateways that will be addressed over the failure_route. My Question now is: how can I count the number of branches / rejections (from the failure_route)? I've tried using the pseudo_variable $branch(count) (according description of the WIKI page), but the value is "0" all the time => not useful. I could create a private table and use the sqlops module. Do you have any easier alternative recommendation how the number could be limited?
Thanks in advance!
Best regards, Klaus
P.S. an excerpt of the current situation of Kamailio 3.1.4 in the scenario - the <names> are representing different [failure]routes
kamailio: INFO: -<|XLOG|>-: <route> an incoming call to a virtual ID with R-URI sip:225201@<ipaddr-Sip-Srv> kamailio: INFO: -<|XLOG|>-: <DISTGWY> is reached for target sip:225201@<ipaddr-Sip-Srv> kamailio: INFO: -<|XLOG|>-: <DISTGWY> relay to R-URI: sip:225201@<ipaddr-Sip-Srv> with DS: <null> and AVP: sip:10.180.38.76:5060 kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 100 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <REPLYDISTGWY> is reached with Reply-Code 480 From: sip:123456@<ipaddr-vgw> To: sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> is reached with Code 480 from sip:123456@<ipaddr-vgw> for sip:225201@<ipaddr-sip-srv2> kamailio: INFO: -<|XLOG|>-: <FAILDISTGWY> ds_next_dst was executed successfully with DS <null> and AVP <null> kamailio: ERROR: tm [t_fwd.c:651]: ERROR: add_uac: maximum number of branches exceeded kamailio: ERROR: tm [t_fwd.c:1528]: ERROR: t_forward_nonack: failure to add branches kamailio: ERROR: tm [tm.c:1368]: ERROR: w_t_relay_to: t_relay_to failed kamailio: WARNING: -<|XLOG|>-: <FAILDISTGWY> t_relay for GWYs failed.
On Wednesday 06 July 2011, Klaus Feichtinger wrote:
the described scenario is not a standard scenario. This would be realistic in emergency situations only (e.g. so many calls are coming in and the gateways are busy => e.g. responding with "486 Busy Here" ). Also the time delay would not be very high, because the fr_timer variable is set to a value of 750ms before relaying the call to the dispatcher-targets ("t_set_fr([...]))") - this is just for a scenario when the target is unreachable.
Hi Klaus,
i understand.
In case that the dispatcher-target is responding with a negative response code (as it was until now), all targets are sequentially addressed within a time of lower than 1 second until the error message of the TM module is interrupting that process.
1s is of course fast enough.
However, if there is no alternative to changing the define value of MAX_BRANCHES I will use your recommendation for limiting the number of gateways that will be addressed over the failure_route. My Question now is: how can I count the number of branches / rejections (from the failure_route)? I've tried using the pseudo_variable $branch(count) (according description of the WIKI page), but the value is "0" all the time => not useful. I could create a private table and use the sqlops module. Do you have any easier alternative recommendation how the number could be limited?
What about just incrementing an variable, like an avp and check this in an if case?
Best regards,
Henning
Hi Klaus,
i understand.
In case that the dispatcher-target is responding with a negative response code (as it was until now), all targets are sequentially addressed within a time of lower than 1 second until the error message of the TM module is interrupting that process.
1s is of course fast enough.
However, if there is no alternative to changing the define value of MAX_BRANCHES I will use your recommendation for limiting the number of gateways that will be addressed over the failure_route. My Question now is: how can I count the number of branches / rejections (from the failure_route)? I've tried using the pseudo_variable $branch(count) (according description of the WIKI page), but the value is "0" all the time => not useful. I could create a private table and use the sqlops module. Do you have any easier alternative recommendation how the number could be limited?
What about just incrementing an variable, like an avp and check this in an if case?
Best regards,
Henning
Hi Henning,
I decided using your recommendation with AVP. Until now I kept some distance to this mechanism (AVP in general). But because of having no alternative (std. $var can not work, avoidance of self compilation for a customer project a.s.o.) I have limited the number of hits - it is now working fine. Without any dirty hack ;-)
Thanks and best regards, Klaus
P.S. the abstract solution
route[YZ] { if (!ds_select_dst("1", "4")) { .. } $avp(i:9)=1; t_on_failure("xy"); t_on_reply("yz"); t_set_fr(3000, 750); t_relay(); exit; }
failure_route[xy] { [...] if ($avp(i:9) < 10){ $avp(i:9) = $avp(i:9) + 1; #!ifdef WITH_XLOGDEBUG xlog("L_INFO", " <FAILXY> the new value of branch counter is: $avp(i:9) \n"); #!endif } else { #!ifdef WITH_XLOGINFO xlog("L_INFO", " <FAILXY> distr. was interrupted due to too many faulty GWYs \n"); #!endif t_reply("480", "Temporarily Unavailable"); exit; } if (!ds_next_dst()) { .. } [...] }
On 04.07.2011 11:07, Henning Westerholt wrote:
With regards to your question, you could make the max_branches number configurable with a patch, at the moment its just a define.
+1
dirty (very dirty) hack: use the last branch to loop the request to Kamailio again, then you can start from the beginning :-)
regards Klaus
PS: I would increase max_branches before using such a dirty hack.
On 03.07.2011 14:01, Klaus Feichtinger wrote:
Hi,
I have a problem in my scenario where I use the dispatcher module for (round-robin based) distribution of incoming calls to 15 (!) different gateways. In idle case (= all gateways are up and running or just a couple of them does not accept calls) my solution is working fine. The solution is very primitive and based on the example of the dispatcher module's ReadMe. However, when e.g. 12 of 15 gateways are out of service (in detail: rejecting the call with "480 - Temporarily Unavailable") and only the 13th would be okay, I can not reach that gateway. This is because the TM module is interrupting the distribution with an error message indicating that the maximum number of branches (per transaction) is reached (ERROR:/tm/:/add_uac/: /maximum number of branches/ exceeded)....
My first idea for solving this problem was/is in manipulating the definition of the "max_branches" constant in the config.h file. However, I prefer using the precompiled Debian packages. Therefore that solution is not optimal. I have not found a method for decreasing the destination_set on an easy way... Does anybody know an alternative solution / workaround?
Thanks in advance, Klaus
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-
Henning Westerholt -
Klaus Darilion -
Klaus Feichtinger