Hi, kamailio 1.5.4, let's suppose the following tables for LCR:
lcr: +----+-----------+-------------+---------+----------+ | id l prefix | from_uri | grp_id | priority | +----+-----------+-------------+---------+----------+ | 1 | NULL | NULL | 1 | 1 | | 2 | NULL | NULL | 2 | 2 | +----+-----------+-------------+---------+----------+
gw: +----+--------+---------------------+------+-----------+ | id | grp_id | ip_addr | port | weight | +-----+--------+---------------------+------+-----------+ | 1 | 1 | 1.1.1.1 | 5060 | 1 | | 2 | 1 | 2.2.2.2 | 5060 | 1 | | 3 | 2 | 2.2.2.2 | 5062 | 1 | +----+--------+--------+---------------------+---------+
IMHO it's clear what *should* occur when a request arrives and load_gw() is called:
1) First the rule 1 should be executed, so gws with id 1 and 2 would be tested (when invoking next_gw() in route and failure_route). 2) In case both gws fail them the gw 3 should be used.
However this is what occurs:
DBG:lcr:do_load_gws: added matched_gws[0]=[3, 1, 2, 2460706] DBG:lcr:do_load_gws: added matched_gws[1]=[1, 1, 1, 3162440] DBG:lcr:add_gws_into_avps: added gw_uri_avp <1|0||33686018||5062|0|0> DBG:lcr:add_gws_into_avps: added gw_uri_avp <1|0||16843009||5060|0|0>
This is, just gws 1 and 2 are loaded in gw_uri_avp (or sometimes gws 1 and 3). Note that gws 2 and 3 have same IP but *different* port, so there is no reason at all not to load gw 3.
IMHO this shouldn't be the behavior, in fact it seems buggy IMHO. I know that same occurs in 3.0 version. Any comment?
BTW I strongly think that LCR module should be improved and I would like to do it. Some ideas I have in mind: - Improve the tables design by having three tables: lcr_rules, lcr_groups and lcr_gws. - Allow loading same gw twice in gw_uri_avp when it belongs to different lcr_rule/lcr_groups. - An option to enable/dissable a gw (by setting a 0 in a new "enabled" column). - Am option to allow stopping lcr_rules, this is, if the lcr_rule is choosen (by priority/prefix/from_uri) and the request fails, no other gateways from other lcr_rules would be tryed.
Well, I'll detail it further soon. Regards.
Iñaki Baz Castillo writes:
This is, just gws 1 and 2 are loaded in gw_uri_avp (or sometimes gws 1 and 3). Note that gws 2 and 3 have same IP but *different* port, so there is no reason at all not to load gw 3.
IMHO this shouldn't be the behavior, in fact it seems buggy IMHO. I know that same occurs in 3.0 version. Any comment?
inaki,
the comment is that currently uniqueness of gws is based only on gw's ip address or hostname , i.e., port is not considered. after quick look at the code, it should be quite easy to add checking of port too.
BTW I strongly think that LCR module should be improved and I would like to do it. Some ideas I have in mind:
- Improve the tables design by having three tables: lcr_rules,
lcr_groups and lcr_gws.
i cannot comment on this one unless you tell why three tables are needed.
- Allow loading same gw twice in gw_uri_avp when it belongs to
different lcr_rule/lcr_groups.
if that is the port issue discussed in above, it can be done.
- An option to enable/dissable a gw (by setting a 0 in a new "enabled"
- column).
that already exists in 3.0 (see defunct column).
- Am option to allow stopping lcr_rules, this is, if the lcr_rule is
choosen (by priority/prefix/from_uri) and the request fails, no other gateways from other lcr_rules would be tryed.
why can't you do this in the script?
-- juha
2010/5/30 Juha Heinanen jh@tutpro.com:
Iñaki Baz Castillo writes:
This is, just gws 1 and 2 are loaded in gw_uri_avp (or sometimes gws 1 and 3). Note that gws 2 and 3 have same IP but *different* port, so there is no reason at all not to load gw 3.
IMHO this shouldn't be the behavior, in fact it seems buggy IMHO. I know that same occurs in 3.0 version. Any comment?
inaki,
the comment is that currently uniqueness of gws is based only on gw's ip address or hostname , i.e., port is not considered. after quick look at the code, it should be quite easy to add checking of port too.
SInce different gws can exist in gw table with same ip:[port] but different attributes (strip, flags, etc) I think it should be possible to load gateways with same ip[:port] for same request if theses gateways match the rules.
BTW I strongly think that LCR module should be improved and I would like to do it. Some ideas I have in mind:
- Improve the tables design by having three tables: lcr_rules,
lcr_groups and lcr_gws.
i cannot comment on this one unless you tell why three tables are needed.
Main reason: current design makes *very* difficult to manage the tables content via an admin interface (i.e. a web interface). Also, if I want to include same gateway (so same IP, port and attributes) in two rules then I must to repeat same data in two different gw entries. This is not good from a design point of view.
What I'll propose is:
- table 'lcr_rules' (same as 'lcr') includes entries with 'instance_id', 'grp_id' and so. Also it would contain two new columns: 'enabled' (the rule is just loaded if it's 1), and 'stop' (no new rules are inspected when 'load_gws()' finds this rule). More about this at the end of the mail. Also I would add a new schema constain: a composite UNIQUE key for columns 'prefix', 'grp_id' and 'priority' as it makes no sense two rules with same prefix, grp_id and priority (which is chosen first? is it random?).
- table 'lcr_gws' include gateways definition. Same as current 'gw' table but *without* 'grp_id' and 'instance_id' columns.
- table 'lcr_groups' contains these columns: - id: primary key as usual. - grp_id: The group identifier. - gw_id: integer pointing to the id column of an entry under 'lcr_gws' table.º If a group contains 2 gateways then two entries would exist in this table with same 'grp_id', each one with 'gw_id' pointing to the corresponding entry under 'lcr_gws' table.
This would make management of lcr easier as it respects some basic design patterns: - Each lcr rule contains some attributes (priority, prefix...) and "belongs to" one gws group (grp_id). - Each gws group "has many" gateways (entries under lcr_gws table) and also "has_many" lcr rules (as same grp_id could appear in different lcr_rules entries). - Each gateway must be defined just *once* even if it used by various lcr groups (different gws groups sharing the same gateway without having to duplicate its information ini two places, why not?).
- An option to enable/dissable a gw (by setting a 0 in a new "enabled"
- column).
that already exists in 3.0 (see defunct column).
ok :)
- Am option to allow stopping lcr_rules, this is, if the lcr_rule is
choosen (by priority/prefix/from_uri) and the request fails, no other gateways from other lcr_rules would be tryed.
why can't you do this in the script?
Because it's ugly to do so in the script, it makes unfeasible to admin it via a web interface. A common use case: - I want a default route using gw1 and gw2 (for any prefix). - In calls to 001 (USA) I want to use *just* gw3, so I could create a lcr rule with prefix 001 and grp_id=X, and a lcr_group with id=X containing just gw3. - If gw3 is down or replies 503 (this is, if the USA lcr rule fails) I don't want to try the default route (gw1 and gw2). This is not possible with the current design as the default route (lcr rules with no prefix) would be loaded automatically. - With my proposal (a column "stop" with value 1) there is no need to coding ugly and static code in failure_route. Instead load_gws() would stop inspecting lcr_rules table when a rule with "stop=1" is found.
I suggest/propose all of this due to my real experience. I've deployed a proxy for load balancing, failover and routing (based on prefix) to some PSTN gateways and after implementing it I really miss the above features.
Regards.
PS: I'll write such specifications and proposals somewhere (maybe in the wiki) and will share here to comment.
Iñaki Baz Castillo writes:
the comment is that currently uniqueness of gws is based only on gw's ip address or hostname , i.e., port is not considered. after quick look at the code, it should be quite easy to add checking of port too.
SInce different gws can exist in gw table with same ip:[port] but different attributes (strip, flags, etc) I think it should be possible to load gateways with same ip[:port] for same request if theses gateways match the rules.
yes, i agree that it should be possible to load two gws with the same ip or hostname if port is not given or if it is given and is different in the two.
Main reason: current design makes *very* difficult to manage the tables content via an admin interface (i.e. a web interface). Also, if I want to include same gateway (so same IP, port and attributes) in two rules then I must to repeat same data in two different gw entries. This is not good from a design point of view.
ok.
What I'll propose is:
- table 'lcr_rules' (same as 'lcr') includes entries with
'instance_id', 'grp_id' and so. Also it would contain two new columns: 'enabled' (the rule is just loaded if it's 1), and 'stop' (no new rules are inspected when 'load_gws()' finds this rule). More about this at the end of the mail. Also I would add a new schema constain: a composite UNIQUE key for columns 'prefix', 'grp_id' and 'priority' as it makes no sense two rules with same prefix, grp_id and priority (which is chosen first? is it random?).
- table 'lcr_gws' include gateways definition. Same as current 'gw'
table but *without* 'grp_id' and 'instance_id' columns.
- table 'lcr_groups' contains these columns:
- id: primary key as usual.
- grp_id: The group identifier.
- gw_id: integer pointing to the id column of an entry under
'lcr_gws' table.º If a group contains 2 gateways then two entries would exist in this table with same 'grp_id', each one with 'gw_id' pointing to the corresponding entry under 'lcr_gws' table.
sounds ok.
Because it's ugly to do so in the script, it makes unfeasible to admin it via a web interface. A common use case:
- I want a default route using gw1 and gw2 (for any prefix).
- In calls to 001 (USA) I want to use *just* gw3, so I could create a
lcr rule with prefix 001 and grp_id=X, and a lcr_group with id=X containing just gw3.
- If gw3 is down or replies 503 (this is, if the USA lcr rule fails) I
don't want to try the default route (gw1 and gw2). This is not possible with the current design as the default route (lcr rules with no prefix) would be loaded automatically.
- With my proposal (a column "stop" with value 1) there is no need to
coding ugly and static code in failure_route. Instead load_gws() would stop inspecting lcr_rules table when a rule with "stop=1" is found.
does this cover the case where there are two equal priority gw groups defined for prefix 001?
I suggest/propose all of this due to my real experience. I've deployed a proxy for load balancing, failover and routing (based on prefix) to some PSTN gateways and after implementing it I really miss the above features.
thanks for your suggestions, it just remains to implement them :-).
-- juha
2010/5/31 Juha Heinanen jh@tutpro.com:
- I want a default route using gw1 and gw2 (for any prefix).
- In calls to 001 (USA) I want to use *just* gw3, so I could create a
lcr rule with prefix 001 and grp_id=X, and a lcr_group with id=X containing just gw3.
- If gw3 is down or replies 503 (this is, if the USA lcr rule fails) I
don't want to try the default route (gw1 and gw2). This is not possible with the current design as the default route (lcr rules with no prefix) would be loaded automatically.
- With my proposal (a column "stop" with value 1) there is no need to
coding ugly and static code in failure_route. Instead load_gws() would stop inspecting lcr_rules table when a rule with "stop=1" is found.
does this cover the case where there are two equal priority gw groups defined for prefix 001?
Hi, also remember the new constrain I proposed: there cannot be two lcr entries with same prefix, grp_id and priority.
I suggest/propose all of this due to my real experience. I've deployed a proxy for load balancing, failover and routing (based on prefix) to some PSTN gateways and after implementing it I really miss the above features.
thanks for your suggestions, it just remains to implement them :-).
Sure. Just let me some weeks until I get some time to make the specifications and start coding it.
Regards.
Iñaki Baz Castillo writes:
does this cover the case where there are two equal priority gw groups defined for prefix 001?
Hi, also remember the new constrain I proposed: there cannot be two lcr entries with same prefix, grp_id and priority.
what i tried to ask is that there is two different gw groups for u.s. prefix 001. they may have the same or different priority, but stopping must not be done before both of them are tried if first one does not work.
-- juha
2010/5/31 Juha Heinanen jh@tutpro.com:
Iñaki Baz Castillo writes:
does this cover the case where there are two equal priority gw groups defined for prefix 001?
Hi, also remember the new constrain I proposed: there cannot be two lcr entries with same prefix, grp_id and priority.
what i tried to ask is that there is two different gw groups for u.s. prefix 001. they may have the same or different priority, but stopping must not be done before both of them are tried if first one does not work.
Well, I would speack about "two different lcr rules with same prefix" rather than "two different groups with prefix 001" (prefix belongs to lcr rules rather than gws groups). In the case you mean, the stopper should be set in the lcr rule (for prefix 001) with lower priority (highest value). This is:
rule 1) - prefix: 001 - grp_id: 1 - priority: 1 - stop: no
rule 2) - prefix: 001 - grp_id: 2 - priority: 2 - stop: yes
Iñaki Baz Castillo writes:
rule 1)
- prefix: 001
- grp_id: 1
- priority: 1
- stop: no
rule 2)
- prefix: 001
- grp_id: 2
- priority: 2
- stop: yes
this would not cover the case, where the two rules have the same priority, which they currently can have. it would be best if "stop" could be defined on the length of the prefix, i.e., "do not consider shorter prefixes".
-- juha
2010/5/31 Juha Heinanen jh@tutpro.com:
Iñaki Baz Castillo writes:
rule 1)
- prefix: 001
- grp_id: 1
- priority: 1
- stop: no
rule 2)
- prefix: 001
- grp_id: 2
- priority: 2
- stop: yes
this would not cover the case, where the two rules have the same priority, which they currently can have.
Right. Please let me re-definice the constrain I suggested as it was wrong. It should be (for a future I mean):
There cannot be two lcr rules with same instance_id, prefix, priority and from_uri. This would solve your case, right?
it would be best if "stop" could be defined on the length of the prefix, i.e., "do not consider shorter prefixes".
But where to configure such option? is it a module option or a per lcr rule option (table column)?
Regards.
-- juha
Iñaki Baz Castillo writes:
There cannot be two lcr rules with same instance_id, prefix, priority and from_uri. This would solve your case, right?
i don't think so, because currently there can be more than one gw group for the same instance_id, prefix, priority, and from_uri. if priority is the same, it just means that the user does not care, which gw group is tried first.
it would be best if "stop" could be defined on the length of the prefix, i.e., "do not consider shorter prefixes".
But where to configure such option? is it a module option or a per lcr rule option (table column)?
i haven't thought about that yet. perhaps "stop" in a rule could be interpreted to mean "stop" trying shorted prefixes, or then there would need to be one more (prefix) table.
-- juha
Juha Heinanen writes:
i haven't thought about that yet. perhaps "stop" in a rule could be interpreted to mean "stop" trying shorted prefixes, or then there would need to be one more (prefix) table.
one possibility might be to include a "stop" character at the end of prefix, e.g.
001$
which would mean, if prefix 001 matches, do not try any shorter ones.
-- juha
2010/5/31 Juha Heinanen jh@tutpro.com:
Juha Heinanen writes:
i haven't thought about that yet. perhaps "stop" in a rule could be interpreted to mean "stop" trying shorted prefixes, or then there would need to be one more (prefix) table.
one possibility might be to include a "stop" character at the end of prefix, e.g.
001$
which would mean, if prefix 001 matches, do not try any shorter ones.
It would be ok. However from a design point of view IMHO it's simpler to have a column "stop" rather than requiring special syntax in 'prefix' column. Content of lcr table is loaded into memory so I don't see the benefict of using 001$ rather than "stop=1". In order to design an admin interface is easier to handle a single column rather than inspecting the syntax of the 'prefix' column.
Iñaki Baz Castillo writes:
It would be ok. However from a design point of view IMHO it's simpler to have a column "stop" rather than requiring special syntax in 'prefix' column. Content of lcr table is loaded into memory so I don't see the benefict of using 001$ rather than "stop=1". In order to design an admin interface is easier to handle a single column rather than inspecting the syntax of the 'prefix' column.
what bothers me is that stopping business has nothing to do with a particular row in rules table, but with a particular prefix. may be a clean solution requires one more table.
-- juha
2010/5/31 Juha Heinanen jh@tutpro.com:
Iñaki Baz Castillo writes:
It would be ok. However from a design point of view IMHO it's simpler to have a column "stop" rather than requiring special syntax in 'prefix' column. Content of lcr table is loaded into memory so I don't see the benefict of using 001$ rather than "stop=1". In order to design an admin interface is easier to handle a single column rather than inspecting the syntax of the 'prefix' column.
what bothers me is that stopping business has nothing to do with a particular row in rules table, but with a particular prefix. may be a clean solution requires one more table.
And this would be better from a design point of view :)
So let's imagine a new table 'lcr_stopper_rules' with the following columns:
- id: Primary key as usual. - rule_id: Integer pointing to an entry under 'lcr_rules' ('lcr') table.
Iñaki Baz Castillo writes:
So let's imagine a new table 'lcr_stopper_rules' with the following columns:
- id: Primary key as usual.
- rule_id: Integer pointing to an entry under 'lcr_rules' ('lcr') table.
inaki,
the above does not tie stopping to a certain prefix, but rules. what determines if a given rule applies is:
lcr_id prefix from_uri
stopping should be tied to these three values, which would result in a table containing
id lcr_id prefix from_uri stop
then there would be a table
id id_from_above_table grp_id priority
perhaps we could handle the rest offline?
-- juha
2010/5/31 Juha Heinanen jh@tutpro.com:
Iñaki Baz Castillo writes:
So let's imagine a new table 'lcr_stopper_rules' with the following columns:
- id: Primary key as usual.
- rule_id: Integer pointing to an entry under 'lcr_rules' ('lcr') table.
inaki,
the above does not tie stopping to a certain prefix, but rules. what determines if a given rule applies is:
lcr_id prefix from_uri
stopping should be tied to these three values, which would result in a table containing
id lcr_id prefix from_uri stop
then there would be a table
id id_from_above_table grp_id priority
Juha, you are 100% right and you gave me the key of the current problem: lcr rule definitions and lcr rule steps are merged now in same table 'lcr'. They should be splited in two tables:
table 'lcr_rules': - id: Id of the rule. - lcr_id: instance of LCR. - prefix - from_uri - stopper: 1/0 - enabled: 1/0
table 'lcr_rule_steps': - id: Primary key as usual. - rule_id: Intenger pointing to 'lcr_rules' entry. - proirity: Priority of this step within its rule. - gw_grp_id: Integer pointing to an entry in 'lcr_gw_groups'.
tables 'lcr_gw_groups' and 'lcr_gws' as defined in a previous mail.
perhaps we could handle the rest offline?
Ok, I'm connected to sip-router IRC channel right now. Or let me how to conntact you whenever it's ok for you.
Regards.
2010/5/31 Juha Heinanen jh@tutpro.com:
There cannot be two lcr rules with same instance_id, prefix, priority and from_uri. This would solve your case, right?
i don't think so, because currently there can be more than one gw group for the same instance_id, prefix, priority, and from_uri. if priority is the same, it just means that the user does not care, which gw group is tried first.
Yes but in my proposal it wouldn't be valid. Those rules should have a different priority value (constrain). IMHO it makes sense as the random behavior should just occur for gateways within same group (based on weight).
Iñaki Baz Castillo writes:
i don't think so, because currently there can be more than one gw group for the same instance_id, prefix, priority, and from_uri. if priority is the same, it just means that the user does not care, which gw group is tried first.
Yes but in my proposal it wouldn't be valid. Those rules should have a different priority value (constrain). IMHO it makes sense as the random behavior should just occur for gateways within same group (based on weight).
we cannot make invalid rules that previously have been allowed. what comes to randomizing the rules, yes, it could be added.
-- juha
2010/5/31 Juha Heinanen jh@tutpro.com:
yes, i agree that it should be possible to load two gws with the same ip or hostname if port is not given or if it is given and is different in the two.
Note that with my proposal there is no need to inspect ip[:port] in loaded gateways. Instead it's just required to inspect the id field of lcr_gws table, so each gw would be loaded just once even if it appears in different gws groups.
Iñaki Baz Castillo writes:
Note that with my proposal there is no need to inspect ip[:port] in loaded gateways. Instead it's just required to inspect the id field of lcr_gws table, so each gw would be loaded just once even if it appears in different gws groups.
ok.
-- juha
On Sunday 30 May 2010, Iñaki Baz Castillo wrote:
[..]
- Am option to allow stopping lcr_rules, this is, if the lcr_rule is
choosen (by priority/prefix/from_uri) and the request fails, no other gateways from other lcr_rules would be tryed.
why can't you do this in the script?
Because it's ugly to do so in the script, it makes unfeasible to admin it via a web interface. A common use case:
- I want a default route using gw1 and gw2 (for any prefix).
- In calls to 001 (USA) I want to use *just* gw3, so I could create a
lcr rule with prefix 001 and grp_id=X, and a lcr_group with id=X containing just gw3.
- If gw3 is down or replies 503 (this is, if the USA lcr rule fails) I
don't want to try the default route (gw1 and gw2). This is not possible with the current design as the default route (lcr rules with no prefix) would be loaded automatically.
- With my proposal (a column "stop" with value 1) there is no need to
coding ugly and static code in failure_route. Instead load_gws() would stop inspecting lcr_rules table when a rule with "stop=1" is found.
Hi Iñaki,
maybe you want to take a look to the cr module (at least for some additional inspiration..), where we've added functionality to match for the prefix, arbitrary flags or for reply codes and decide with this data if the next rule/ which rule should be executed as a next try. You'll find some examples in the docs (1.21, 1.22). The specific table is named "carrierfailureroute". Perhaps it provides a bit more generic solution to the usecase you described, if i understand it correctly.
Cheers,
Henning
2010/6/1 Henning Westerholt henning.westerholt@1und1.de:
Hi Iñaki,
maybe you want to take a look to the cr module (at least for some additional inspiration..), where we've added functionality to match for the prefix, arbitrary flags or for reply codes and decide with this data if the next rule/ which rule should be executed as a next try. You'll find some examples in the docs (1.21, 1.22). The specific table is named "carrierfailureroute". Perhaps it provides a bit more generic solution to the usecase you described, if i understand it correctly.
Thanks Henning.
Yes, I want also to learn about cr module :) For now Juha and me have started a wiki page with new additions and changes to LCR module: http://www.kamailio.org/dokuwiki/doku.php/modules-new-design:lcr-module-desi...
Thanks a lot.
-
Henning Westerholt -
Iñaki Baz Castillo -
Juha Heinanen