[SR-Users] Active/active keepalived - Kamailio includes non-local virtual IP in "myself" which breaks DMQ call routing
Rhys Hanrahan
rhys at nexusone.com.au
Wed Dec 22 11:59:17 CET 2021
Hi Everyone,
I have just opened a feature request and a patch which addresses the request, here: https://github.com/kamailio/kamailio/issues/2984
I haven't gone to the extent of doing proper commits and a pull request yet, as I'm not sure if anything I've done would get outright rejected. But if it seems roughly OK then I will go ahead and do that!
I would really appreciate some feedback on if my approach is OK as this is my first time attempting any changes to Kamailio.
Given the depth of the issue occurring, I felt that a change at the code level was the best way to address my issues, as I didn't know how many crazy edge cases I might have to deal with if trying to fix this in the config alone - this way seemed a lot simpler and neater.
This so far seems to address all the issues I raised below - in my testing so far I can at least confirm it allows lookup() to behave correctly with PATH support, when a virtual IP is active on one node but not the other.
Thanks!
Rhys.
-----Original Message-----
From: sr-users [mailto:sr-users-bounces at lists.kamailio.org] On Behalf Of Rhys Hanrahan
Sent: Friday, 17 December 2021 2:01 AM
To: Kamailio (SER) - Users Mailing List <sr-users at lists.kamailio.org>
Subject: Re: [SR-Users] Active/active keepalived - Kamailio includes non-local virtual IP in "myself" which breaks DMQ call routing
Hi Arnd (and list),
I'm just wondering, have you had to deal with this situation of having an in-active floating IP before? Was it reasonable to handle?
I tried your suggestion, and that part worked great. But the reason I ask the above is because I think my issues are much more complicated than I first thought. I didn't realise it at first but it looks like the "uri==myself" in my config is not the issue currently, and in-fact the issue is with lookup(). Lookup uses check_self() in the code, which is the same as uri==myself, and my virtual Ips completely break PATH support because check_self returns always returns a match.
I could of course implement my own checks for PATH in my config after doing a lookup, but realising that there's C code using check_self() everywhere, I feel like this is just the first of many problems I will run into with this attempted configuration.
Maybe I am over-thinking this, and it won't be so bad? But to me it seems, the only reasonable solutions would be the following - though I would love to be told I'm wrong and it's easier than I think!
1) See if I can patch check_self() (or really grep_sock_info()) to skip matching sockets that are not currently active locally.
2) Move to a full any-cast pair of nodes as Igor had suggested. Anycast has it's own challenges of syncing dialogs and such (which I wasn't planning to do) but I guess it would avoid this check_self() issue because the IP is actually local on both nodes at the same time.
3) Give up on active/active and just live with Keepalived active/passive, which again avoids this issue. And seems to be a more common setup. As much as I want active/active, it may not be worth the effort - I'm undecided so far.
I'm keen to hear your (or anyone's) thoughts on this.
Thanks!
Rhys.
On 15/12/21, 6:07 pm, "sr-users on behalf of Arnd Schmitter" <sr-users-bounces at lists.kamailio.org on behalf of aschmitter at megasat.de> wrote:
Hi,
For the RPC command, take a look at https://kamailio.org/docs/modules/5.5.x/modules/pv.html#pv.rpc.shvSet
Regards
Arnd
Am Mi, Dez 15, 2021 um 04:59:06 schrieb Rhys Hanrahan:
> Hi Guys,
>
> That's awesome - thanks to you both for the suggestions. I had overlooked anycast because my Ips are only active on one box at a time, but it makes sense that I'm basically dealing with the same challenges as an anycast setup, thanks Igor.
>
> And thanks Arnd! This is what I was thinking of as a quick fix, but I couldn't find anything useful to change via RPC commands. A shared variable makes perfect sense!
>
> I will work through these ideas and post back with whatever I manage to come up with.
>
> Thanks!
> Rhys.
>
> On 15/12/21, 12:19 am, "sr-users on behalf of Arnd Schmitter" <sr-users-bounces at lists.kamailio.org on behalf of aschmitter at megasat.de> wrote:
>
> Hello Rhys,
>
>
> For a quick solution, to can use notify scripts in keepalived which will alter a shared variable via RPC call. Then you can test the content of this variable to see if this instance is master or not.
>
>
> Regards,
> Arnd
>
>
> Am Di, Dez 14, 2021 um 14:03:27 schrieb Igor Olhovskiy:
> > Rhys,
> >
> > Seems you're looking into something called "anycast". If it's the case, have
> > you checked https://github.com/kamailio/kamailio/blob/master/misc/examples/mixed/kamailio-minimal-anycast.cfg
> > ?
> >
> > But overall you are correct, myself is not enough clever to get if interface
> > is active or not.
> >
> > Regards,
> > Igor
> >
> > On 14.12.2021 05:50, Rhys Hanrahan wrote:
> > >
> > > Hi Everyone,
> > >
> > > I have Kamailio acting as a registrar, using DMQ to replicate usrloc
> > > between nodes. I’m relaying INVITEs to the home registrar, but skipping
> > > this if (uri == myself) is true.
> > >
> > > In my lab I just setup my SBCs in an active/active mode, similar to the
> > > link below. So I have 2x VIPs and both SBCs listen on both, but the
> > > first VIP is only master on SBC01 and the second VIP is only master on
> > > SBC02.
> > >
> > > https://docs.nginx.com/nginx/admin-guide/high-availability/ha-keepalived-nodes/ <https://docs.nginx.com/nginx/admin-guide/high-availability/ha-keepalived-nodes/>
> > >
> > >
> > > My issue is that now “myself” will always return true even if the
> > > virtual IP is actually not active on this node at the moment, so the
> > > invite is never relayed properly if it’s on another node. I guess
> > > “myself” is just blindly looking at the list of Ips that are configured
> > > with either listen= or alias= and doesn’t care if the IP actually exists
> > > on the server at the time, or not?
> > >
> > > I am thinking I may have to ditch relying on “myself” entirely and come
> > > up with some other way of checking for local requests, but I am
> > > wondering if there’s a good way of approaching this? I was thinking of
> > > perhaps triggering an add/remove of listen directives (or something)
> > > when a failover of a VIP happens, though this seems kind of heavy
> > > handed. Hoping there is a better way.
> > >
> > > Has anyone else tried doing active/active Kamailio instances like this?
> > >
> > > Thanks!
> > >
> > >
> > > *Rhys Hanrahan* | Chief Information Officer
> > > *e:* rhys at nexusone.com.au <mailto:rhys at nexusone.com.au>
> > >
> > > www.nexusone.com.au <http://www.nexusone.com.au/>signature_132622389
> > > <http://www.fusiontech.com.au/>
> > > *
> > > **NEXUS ONE****|** FUSION TECHNOLOGY SOLUTIONS**
> > > **p:* 1800 NEXUS1 (1800 639 871) or 1800 565 845 *|* *a:* Suite 12.03
> > > Level 12, 227 Elizabeth Street, Sydney NSW 2000
> > > www.nexusone.com.au <http://www.nexusone.com.au/> *|*
> > > www.fusiontech.com.au <http://www.fusiontech.com.au/>
> > >
> > > /The information in this email and any accompanying attachments may
> > > contain; a. Confidential information of Fusion Technology Solutions Pty
> > > Ltd, Nexus One Pty Ltd or third parties; b. Legally privileged
> > > information of Fusion Technology Solutions Pty Ltd, Nexus One Pty Ltd or
> > > third parties; and or c. Copyright material Fusion Technology Solutions
> > > Pty Ltd, Nexus One Pty Ltd or third parties. If you have received this
> > > email in error, please notify the sender immediately and delete this
> > > message. Fusion Technology Solutions Pty Ltd, Nexus One Pty Ltd does not
> > > accept any responsibility for loss or damage arising from the use or
> > > distribution of this email./
> > >
> > > /Please consider the environment before printing this email./
> > >
> > >
> > > __________________________________________________________
> > > Kamailio - Users Mailing List - Non Commercial Discussions
> > > * sr-users at lists.kamailio.org
> > > Important: keep the mailing list in the recipients, do not reply only to the sender!
> > > Edit mailing list options or unsubscribe:
> > > * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
> > __________________________________________________________
> > Kamailio - Users Mailing List - Non Commercial Discussions
> > * sr-users at lists.kamailio.org
> > Important: keep the mailing list in the recipients, do not reply only to the sender!
> > Edit mailing list options or unsubscribe:
> > * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
> __________________________________________________________
> Kamailio - Users Mailing List - Non Commercial Discussions
> * sr-users at lists.kamailio.org
> Important: keep the mailing list in the recipients, do not reply only to the sender!
> Edit mailing list options or unsubscribe:
> * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
> __________________________________________________________
> Kamailio - Users Mailing List - Non Commercial Discussions
> * sr-users at lists.kamailio.org
> Important: keep the mailing list in the recipients, do not reply only to the sender!
> Edit mailing list options or unsubscribe:
> * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
* sr-users at lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
* https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
* sr-users at lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
* https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
More information about the sr-users
mailing list