Hello my friends,
I wonder if someone has done this before? i would like to implement 2 Kamailio servers with redundancy, something like the following:
SERVER_A is working as the primary sip proxy (virtual IP as the sip signaling), if it fails, the other server (sleeping) should UP the virtual IP and takes all the traffic....i guess there is many Linux implementations thah can do this, but i would like to know if there is someone that has done this before and hear some recomendations...
I've read some difficulty in the synchronisation of registrations because Kamailio works best when it stores registrations in memory and registrations are constantly changing - they expire and are renewed, as well as new ones joining and old ones leaving. To make the failover solution function seamlessly, it is necessary to synchronise the in-memory registrations between the primary and the backup server . This can be done by forking a copy of the registration request to the backup server, but there are some practical problems in doing this, has anyone do something with this?
Thanks in advance!
Am 27.01.2011 11:21, schrieb Danny Dias:
Hello my friends,
I wonder if someone has done this before? i would like to implement 2 Kamailio servers with redundancy, something like the following:
SERVER_A is working as the primary sip proxy (virtual IP as the sip signaling), if it fails, the other server (sleeping) should UP the virtual IP and takes all the traffic....i guess there is many Linux implementations thah can do this, but i would like to know if there is someone that has done this before and hear some recomendations...
I've read some difficulty in the synchronisation of registrations because Kamailio works best when it stores registrations in memory and registrations are constantly changing - they expire and are renewed, as well as new ones joining and old ones leaving. To make the failover solution function seamlessly, it is necessary to synchronise the in-memory registrations between the primary and the backup server . This can be done by forking a copy of the registration request to the backup server, but there are some practical problems in doing this, has anyone do something with this?
Yes - the problem with SIP based replication is that both proxies must be running. This is a problem as Kamailio binds to the virtual IP at start up - thus adding the virtual IP address to the backup server does not make Backup-Kamailio listening to the new IP address - you would have to restart the backup Kamailio.
I think most people either have a database (which is highly-available by itself) which is used by both proxies, or every proxy has a local database and the synchronization is on DB level (e.g. master-slave replication, btw: does somebody know if usrloc DB queries are suitable for master-master replication?)
regards klaus
On Thursday 27 January 2011, Klaus Darilion wrote:
Am 27.01.2011 11:21, schrieb Danny Dias:
I've read some difficulty in the synchronisation of registrations because Kamailio works best when it stores registrations in memory and registrations are constantly changing - they expire and are renewed, as well as new ones joining and old ones leaving. To make the failover solution function seamlessly, it is necessary to synchronise the in-memory registrations between the primary and the backup server . This can be done by forking a copy of the registration request to the backup server, but there are some practical problems in doing this, has anyone do something with this?
What problems are you referring to? I use this for some years now without any problems.
Yes - the problem with SIP based replication is that both proxies must be running. This is a problem as Kamailio binds to the virtual IP at start up - thus adding the virtual IP address to the backup server does not make Backup-Kamailio listening to the new IP address - you would have to restart the backup Kamailio.
Just bind kamailio to the HA IP on both servers and do REGISTER replication between the two (on SIP level). Then if the IP migrates to the other server, it will take over the rgistrar function with no loss of records. No restart needed.
I think most people either have a database (which is highly-available by itself) which is used by both proxies, or every proxy has a local database and the synchronization is on DB level (e.g. master-slave replication, btw: does somebody know if usrloc DB queries are suitable for master-master replication?)
Last time i tried, they are not, at least not in writeback mode. One proxy is expiring records from the DB which the other proxy is trying to update. Maybe DB-only mode will work, but that has some practical (performcance) problems.
Thanks Alex...
2011/1/27 Alex Hermann alex@speakup.nl
On Thursday 27 January 2011, Klaus Darilion wrote:
Am 27.01.2011 11:21, schrieb Danny Dias:
I've read some difficulty in the synchronisation of registrations
because
Kamailio works best when it stores registrations in memory and registrations are constantly changing - they expire and are renewed, as well as new ones joining and old ones leaving. To make the failover solution function seamlessly, it is necessary to synchronise the in-memory registrations between the primary and the backup server .
This
can be done by forking a copy of the registration request to the backup server, but there are some practical problems in doing this, has anyone do something with this?
What problems are you referring to? I use this for some years now without any problems.
I checked for some problems here:
http://www.smartvox.co.uk/astfaq_ha_failover_ideas.htm
Yes - the problem with SIP based replication is that both proxies must be running. This is a problem as Kamailio binds to the virtual IP at start up - thus adding the virtual IP address to the backup server does not make Backup-Kamailio listening to the new IP address - you would have to restart the backup Kamailio.
Just bind kamailio to the HA IP on both servers and do REGISTER replication between the two (on SIP level). Then if the IP migrates to the other server, it will take over the rgistrar function with no loss of records. No restart needed.
Do you mean that both Kamailio-1 and Kamailio-2 will be as primary server? and the clients will register in the 2 machines? and also they will bind to the ip of the HA? sorry my friend but i do not understand very well, i'm quite new with redundant systems, could you please explain a little?
I think most people either have a database (which is highly-available by itself) which is used by both proxies, or every proxy has a local database and the synchronization is on DB level (e.g. master-slave replication, btw: does somebody know if usrloc DB queries are suitable for master-master replication?)
Last time i tried, they are not, at least not in writeback mode. One proxy is expiring records from the DB which the other proxy is trying to update. Maybe DB-only mode will work, but that has some practical (performcance) problems. -- Greetings,
Alex Hermann
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
On 01/27/2011 04:15 PM, Danny Dias wrote:
Thanks Alex...
2011/1/27 Alex Hermann <alex@speakup.nl mailto:alex@speakup.nl>
On Thursday 27 January 2011, Klaus Darilion wrote: > Am 27.01.2011 11:21, schrieb Danny Dias: > > I've read some difficulty in the synchronisation of registrations because > > Kamailio works best when it stores registrations in memory and > > registrations are constantly changing - they expire and are renewed, as > > well as new ones joining and old ones leaving. To make the failover > > solution function seamlessly, it is necessary to synchronise the > > in-memory registrations between the primary and the backup server . This > > can be done by forking a copy of the registration request to the backup > > server, but there are some practical problems in doing this, has anyone > > do something with this? What problems are you referring to? I use this for some years now without any problems.
I checked for some problems here:
http://www.smartvox.co.uk/astfaq_ha_failover_ideas.htm
> Yes - the problem with SIP based replication is that both proxies must > be running. This is a problem as Kamailio binds to the virtual IP at > start up - thus adding the virtual IP address to the backup server does > not make Backup-Kamailio listening to the new IP address - you would > have to restart the backup Kamailio. Just bind kamailio to the HA IP on both servers and do REGISTER replication between the two (on SIP level). Then if the IP migrates to the other server, it will take over the rgistrar function with no loss of records. No restart needed.
Do you mean that both Kamailio-1 and Kamailio-2 will be as primary server? and the clients will register in the 2 machines? and also they will bind to the ip of the HA? sorry my friend but i do not understand very well, i'm quite new with redundant systems, could you please explain a little?
Hello
This week, I have added the p_usrloc module to K master branch, that allows partitioned user location service for Kamailio. This has the benefits of redundancy, failover and load balancing to user location service for K. Along with Henning Westerholt, we will also present some strategies for partioned user location to the upcoming FOSDEM meeting in Brussels.(more info here http://www.fosdem.org/2011/schedule/event/kamailiolocationservices)
You can check the README of the module (master branch modules_k/p_usrloc) for some strategies for partitioned user location.
Marius
> I think most people either have a database (which is highly-available by > itself) which is used by both proxies, or every proxy has a local > database and the synchronization is on DB level (e.g. master-slave > replication, btw: does somebody know if usrloc DB queries are suitable > for master-master replication?) Last time i tried, they are not, at least not in writeback mode. One proxy is expiring records from the DB which the other proxy is trying to update. Maybe DB-only mode will work, but that has some practical (performcance) problems. -- Greetings, Alex Hermann _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Ing. Danny Dias www.DannTEL.net http://www.DannTEL.net
Am 27.01.2011 13:05, schrieb Alex Hermann:
Yes - the problem with SIP based replication is that both proxies must be running. This is a problem as Kamailio binds to the virtual IP at start up - thus adding the virtual IP address to the backup server does not make Backup-Kamailio listening to the new IP address - you would have to restart the backup Kamailio.
Just bind kamailio to the HA IP on both servers and do REGISTER replication between the two (on SIP level). Then if the IP migrates to the other server, it will take over the rgistrar function with no loss of records. No restart needed.
Is it possible to bind Kamailio to an IP address which is not active? (e.g. start Kamailio on the backup server)
klaus
Try to set this option to bind to an non existing ip:
net.ipv4.ip_nonlocal_bind = 1
does work for SSH and thefore should also work with kamailio
-----Ursprüngliche Nachricht----- Von: sr-users-bounces@lists.sip-router.org [mailto:sr-users-bounces@lists.sip-router.org] Im Auftrag von Klaus Darilion Gesendet: Donnerstag, 27. Januar 2011 15:18 An: Alex Hermann Cc: sr-users@lists.sip-router.org Betreff: Re: [SR-Users] Redundancy between 2 Kamailio servers
Am 27.01.2011 13:05, schrieb Alex Hermann:
Yes - the problem with SIP based replication is that both proxies must be running. This is a problem as Kamailio binds to the virtual IP at start up - thus adding the virtual IP address to the backup server does not make Backup-Kamailio listening to the new IP address - you would have to restart the backup Kamailio.
Just bind kamailio to the HA IP on both servers and do REGISTER replication between the two (on SIP level). Then if the IP migrates to the other server, it will take over the rgistrar function with no loss of records. No restart needed.
Is it possible to bind Kamailio to an IP address which is not active? (e.g. start Kamailio on the backup server)
klaus
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
2011/1/27 Klaus Darilion klaus.mailinglists@pernau.at:
Is it possible to bind Kamailio to an IP address which is not active? (e.g. start Kamailio on the backup server)
Yes, it depends on a sysctl option of the kernel (I don't remember the name, sorry). With such option set you can make a server to listen in a IP which doesn't exist in the server.
Am 27.01.2011 13:05, schrieb Alex Hermann:
On Thursday 27 January 2011, Klaus Darilion wrote:
Am 27.01.2011 11:21, schrieb Danny Dias:
I've read some difficulty in the synchronisation of registrations because Kamailio works best when it stores registrations in memory and registrations are constantly changing - they expire and are renewed, as well as new ones joining and old ones leaving. To make the failover solution function seamlessly, it is necessary to synchronise the in-memory registrations between the primary and the backup server . This can be done by forking a copy of the registration request to the backup server, but there are some practical problems in doing this, has anyone do something with this?
What problems are you referring to? I use this for some years now without any problems.
Alex, do you also do NAT keep-alive from the proxies? If yes, are you sending them from both servers at the same time?
regards klaus
On Thursday 27 January 2011, Klaus Darilion wrote:
Alex, do you also do NAT keep-alive from the proxies? If yes, are you sending them from both servers at the same time?
No, we require clients to sent nat-keepalives. It is much more efficient. In addition, the registrars are not directly accessible by clients, they go via load-balancers. Clients keep the NAT binding with the balancer, not the registrar.
Do you mean that both Kamailio-1 and Kamailio-2 will be as primary server? and the clients will register in the 2 machines? and also they will bind to the ip of the HA? sorry my friend but i do not understand very well, i'm quite new with redundant systems, could you please explain a little please?
2011/1/27 Alex Hermann alex@speakup.nl
On Thursday 27 January 2011, Klaus Darilion wrote:
Alex, do you also do NAT keep-alive from the proxies? If yes, are you sending them from both servers at the same time?
No, we require clients to sent nat-keepalives. It is much more efficient. In addition, the registrars are not directly accessible by clients, they go via load-balancers. Clients keep the NAT binding with the balancer, not the registrar.
-- Greetings,
Alex Hermann
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
On Thursday 27 January 2011 17:18:22 Danny Dias wrote:
Do you mean that both Kamailio-1 and Kamailio-2 will be as primary server? and the clients will register in the 2 machines? and also they will bind to the ip of the HA? sorry my friend but i do not understand very well, i'm quite new with redundant systems, could you please explain a little please?
Clients will register to the HA ip. The registrars are bound to the HA ip and to another IP on which the REGISTERs will be replicated between the registrars. Outbound request are forced through the right socket to the HA ip so they follow the NAT binding of the client.
2011/1/27 Danny Dias ing.diasdanny@gmail.com:
Do you mean that both Kamailio-1 and Kamailio-2 will be as primary server? and the clients will register in the 2 machines? and also they will bind to the ip of the HA? sorry my friend but i do not understand very well, i'm quite new with redundant systems, could you please explain a little please?
Two Kamailios in a HeartBeat cluster which manages the kamailio service along with a virtual IP in which kamailios are supposed to listen. Just one kamailio is running (HA manages them).
Regsitration can be done in a shared database with db_mode=3 (or 2) so no locations are lost when HA stops the running instance of kamailio (or the server is down) and starts kamailio in the other cluster node.
Another option without using realtime DB storage is replicating the REGISTER from one Kamailio to the other (t_replicate method) but it requires both kamailios being running at the same time (so net.ipv4.ip_nonlocal_bind must be 1) and kamailios must NOT be managed by HA. Also it requires some other considerations.
Thanks Iñaki,
2011/1/27 Iñaki Baz Castillo ibc@aliax.net
2011/1/27 Danny Dias ing.diasdanny@gmail.com:
Do you mean that both Kamailio-1 and Kamailio-2 will be as primary
server?
and the clients will register in the 2 machines? and also they will bind
to
the ip of the HA? sorry my friend but i do not understand very well, i'm quite new with redundant systems, could you please explain a little
please?
Two Kamailios in a HeartBeat cluster which manages the kamailio service along with a virtual IP in which kamailios are supposed to listen. Just one kamailio is running (HA manages them).
So, the heartbeat cluster shall manage that both are ok and also check that the virtual ip and the kamailio service in the primary server is OK....if something fails it will activate the virtual IP address and the kamailio process in the other server? so this heartbeat cluster is installed in both kamailio servers?
which HA software do you recommend?
Regsitration can be done in a shared database with db_mode=3 (or 2) so no locations are lost when HA stops the running instance of kamailio (or the server is down) and starts kamailio in the other cluster node.
So, the database of the kamailios should be dedicated and externalised server?
Another option without using realtime DB storage is replicating the REGISTER from one Kamailio to the other (t_replicate method) but it requires both kamailios being running at the same time (so net.ipv4.ip_nonlocal_bind must be 1) and kamailios must NOT be managed by HA. Also it requires some other considerations.
-- Iñaki Baz Castillo ibc@aliax.net
2011/1/27 Danny Dias ing.diasdanny@gmail.com:
Two Kamailios in a HeartBeat cluster which manages the kamailio service along with a virtual IP in which kamailios are supposed to listen. Just one kamailio is running (HA manages them).
So, the heartbeat cluster shall manage that both are ok and also check that the virtual ip and the kamailio service in the primary server is OK....if something fails it will activate the virtual IP address and the kamailio process in the other server? so this heartbeat cluster is installed in both kamailio servers? which HA software do you recommend?
As I said at the top of my previous mail: HeartBeat (as it is the only I'm used to).
Regsitration can be done in a shared database with db_mode=3 (or 2) so no locations are lost when HA stops the running instance of kamailio (or the server is down) and starts kamailio in the other cluster node.
So, the database of the kamailios should be dedicated and externalised server?
Could be, or not. It doesn't matter too much.
Hello,
I was thinking that if i store the contacts that are registered into my proxy in a database, like this:
. .
# ----- usrloc params ----- modparam("usrloc", "db_mode", 3) . . if (is_method("REGISTER")) { if (!save("location")) sl_reply_error(); exit; } . .
This way i would store the contacts in table "location", and then i could make a backup of the DB and restore on the other server...is that ok? or i'm missing something?
Thanks in advance for your help!
2011/1/27 Iñaki Baz Castillo ibc@aliax.net
2011/1/27 Danny Dias ing.diasdanny@gmail.com:
Two Kamailios in a HeartBeat cluster which manages the kamailio service along with a virtual IP in which kamailios are supposed to listen. Just one kamailio is running (HA manages them).
So, the heartbeat cluster shall manage that both are ok and also check
that
the virtual ip and the kamailio service in the primary server is OK....if something fails it will activate the virtual IP address and the kamailio process in the other server? so this heartbeat cluster is installed in
both
kamailio servers? which HA software do you recommend?
As I said at the top of my previous mail: HeartBeat (as it is the only I'm used to).
Regsitration can be done in a shared database with db_mode=3 (or 2) so no locations are lost when HA stops the running instance of kamailio (or the server is down) and starts kamailio in the other cluster node.
So, the database of the kamailios should be dedicated and externalised server?
Could be, or not. It doesn't matter too much.
-- Iñaki Baz Castillo ibc@aliax.net
Would be possible.
Have a look at DRBD!
Von: sr-users-bounces@lists.sip-router.org [mailto:sr-users-bounces@lists.sip-router.org] Im Auftrag von Danny Dias Gesendet: Mittwoch, 2. Februar 2011 11:20 An: kamailio Betreff: Re: [SR-Users] Redundancy between 2 Kamailio servers
Hello,
I was thinking that if i store the contacts that are registered into my proxy in a database, like this:
. .
# ----- usrloc params ----- modparam("usrloc", "db_mode", 3) . . if (is_method("REGISTER")) { if (!save("location")) sl_reply_error(); exit; } . .
This way i would store the contacts in table "location", and then i could make a backup of the DB and restore on the other server...is that ok? or i'm missing something?
Thanks in advance for your help!
2011/1/27 Iñaki Baz Castillo ibc@aliax.net 2011/1/27 Danny Dias ing.diasdanny@gmail.com:
Two Kamailios in a HeartBeat cluster which manages the kamailio service along with a virtual IP in which kamailios are supposed to listen. Just one kamailio is running (HA manages them).
So, the heartbeat cluster shall manage that both are ok and also check that the virtual ip and the kamailio service in the primary server is OK....if something fails it will activate the virtual IP address and the kamailio process in the other server? so this heartbeat cluster is installed in both kamailio servers? which HA software do you recommend?
As I said at the top of my previous mail: HeartBeat (as it is the only I'm used to).
Regsitration can be done in a shared database with db_mode=3 (or 2) so no locations are lost when HA stops the running instance of kamailio (or the server is down) and starts kamailio in the other cluster node.
So, the database of the kamailios should be dedicated and externalised server?
Could be, or not. It doesn't matter too much.
-- Iñaki Baz Castillo ibc@aliax.net
On 02/02/2011 12:20 PM, Danny Dias wrote:
Hello,
I was thinking that if i store the contacts that are registered into my proxy in a database, like this:
. .
# ----- usrloc params ----- modparam("usrloc", "db_mode", 3) . . if (is_method("REGISTER")) { if (!save("location")) sl_reply_error(); exit; } . .
This way i would store the contacts in table "location", and then i could make a backup of the DB and restore on the other server...is that ok? or i'm missing something?
Or you can use the p_usrloc module that does automatically the db replication and failover + data partitioning (or you can combine them to suite your needs). New in master branch (</shameless advertisement>)
But seriously, we might want to give it a try. The config syntax remains the same as above you just need to load p_usrloc instead of usrloc and setup the databases.
Marius
Thanks in advance for your help!
2011/1/27 Iñaki Baz Castillo <ibc@aliax.net mailto:ibc@aliax.net>
2011/1/27 Danny Dias <ing.diasdanny@gmail.com <mailto:ing.diasdanny@gmail.com>>: >> Two Kamailios in a HeartBeat cluster which manages the kamailio >> service along with a virtual IP in which kamailios are supposed to >> listen. Just one kamailio is running (HA manages them). >> > > So, the heartbeat cluster shall manage that both are ok and also check that > the virtual ip and the kamailio service in the primary server is OK....if > something fails it will activate the virtual IP address and the kamailio > process in the other server? so this heartbeat cluster is installed in both > kamailio servers? > which HA software do you recommend? As I said at the top of my previous mail: HeartBeat (as it is the only I'm used to). >> Regsitration can be done in a shared database with db_mode=3 (or 2) so >> no locations are lost when HA stops the running instance of kamailio >> (or the server is down) and starts kamailio in the other cluster node. >> > > So, the database of the kamailios should be dedicated and externalised > server? Could be, or not. It doesn't matter too much. -- Iñaki Baz Castillo <ibc@aliax.net <mailto:ibc@aliax.net>>
Am 27.01.2011 13:05, schrieb Alex Hermann:
On Thursday 27 January 2011, Klaus Darilion wrote:
Am 27.01.2011 11:21, schrieb Danny Dias:
I've read some difficulty in the synchronisation of registrations because Kamailio works best when it stores registrations in memory and registrations are constantly changing - they expire and are renewed, as well as new ones joining and old ones leaving. To make the failover solution function seamlessly, it is necessary to synchronise the in-memory registrations between the primary and the backup server . This can be done by forking a copy of the registration request to the backup server, but there are some practical problems in doing this, has anyone do something with this?
What problems are you referring to? I use this for some years now without any problems.
Alex, what happens if one server is down. There will be lots of "replication transactions" which will timeout. Can this cause any problems if there are too many open transactions (retransmissions ...)?
btw: I just found in tm readme:
* t_replicate should be done more cleanly--Vias, Routes, etc. should be removed from a message prior to replicating it (well, does not matter any longer so much as there is a new replication module).
Is there really a dedicated replication module somewhere?
klaus
On Wednesday 02 February 2011, you wrote:
Am 27.01.2011 13:05, schrieb Alex Hermann:
On Thursday 27 January 2011, Klaus Darilion wrote:
Am 27.01.2011 11:21, schrieb Danny Dias:
I've read some difficulty in the synchronisation of registrations because Kamailio works best when it stores registrations in memory and registrations are constantly changing - they expire and are renewed, as well as new ones joining and old ones leaving. To make the failover solution function seamlessly, it is necessary to synchronise the in-memory registrations between the primary and the backup server . This can be done by forking a copy of the registration request to the backup server, but there are some practical problems in doing this, has anyone do something with this?
What problems are you referring to? I use this for some years now without any problems.
Alex, what happens if one server is down. There will be lots of "replication transactions" which will timeout. Can this cause any problems if there are too many open transactions (retransmissions ...)?
I just forward() them.
btw: I just found in tm readme:
- t_replicate should be done more cleanly--Vias, Routes, etc. should be removed from a message prior to replicating it (well, does not matter any longer so much as there is a new replication module).
Is there really a dedicated replication module somewhere?
Never seen one.
Am 02.02.2011 15:24, schrieb Alex Hermann:
On Wednesday 02 February 2011, you wrote:
Am 27.01.2011 13:05, schrieb Alex Hermann:
On Thursday 27 January 2011, Klaus Darilion wrote:
Am 27.01.2011 11:21, schrieb Danny Dias:
I've read some difficulty in the synchronisation of registrations because Kamailio works best when it stores registrations in memory and registrations are constantly changing - they expire and are renewed, as well as new ones joining and old ones leaving. To make the failover solution function seamlessly, it is necessary to synchronise the in-memory registrations between the primary and the backup server . This can be done by forking a copy of the registration request to the backup server, but there are some practical problems in doing this, has anyone do something with this?
What problems are you referring to? I use this for some years now without any problems.
Alex, what happens if one server is down. There will be lots of "replication transactions" which will timeout. Can this cause any problems if there are too many open transactions (retransmissions ...)?
I just forward() them.
Interesting.
So, do you have some logic to drop the response from the "backup" registrar? Or will the client receive 2 responses? Or will the "backup" server not response at all?
I guess the last option would be the simplest one...
regards klaus
On Wednesday 02 February 2011, Klaus Darilion wrote:
Am 02.02.2011 15:24, schrieb Alex Hermann:
On Wednesday 02 February 2011, you wrote:
Alex, what happens if one server is down. There will be lots of "replication transactions" which will timeout. Can this cause any problems if there are too many open transactions (retransmissions ...)?
I just forward() them.
Interesting.
So, do you have some logic to drop the response from the "backup" registrar? Or will the client receive 2 responses? Or will the "backup" server not response at all?
I guess the last option would be the simplest one...
The "backup" server does not respond at all. The first does all the authentication and verification, the backup just stores it in usrloc.