Hi Experts,

We have been using Kamailio in an Active/Standby Pair(with Keepalived under the rugs moving the single Virtual IP to access the Active Kamailio) for sometime now. Kamailio also acts as a Registrar for our webrtc endpoints. 
It has been serving the purpose pretty well and now we have a requirement where we need to be syncing the Registration DB between Two Pairs of Kamailios.

 Kamailio-Active(Pair 1) -------- Kamailio-Standby(Pair 1)
                                          ||
 Kamailio-Active(Pair 2) -------- Kamailio-Standby(Pair 2)    

We generally keep the counterpart in the same Pair as a notifier( modparam("dmq", "notification_address", "sip:PEER1_IP:5060") ) so as to sync the dialogs and the userloc data too( modparam("dmq_usrloc", "enable", 1) ). 
In order to achieve the said requirement with the other Pair, we added another "notification_address" in the kamailio cfg. At this point, we ran into weird issues.

1. With Kamailio ver 5.3.2, the subsequent notification_address line in the cfg file, seemed to be overriding the previous one. Hence we see only the latter peer in the dmq list nodes.
    Example:
            modparam("dmq", "notification_address", "sip:172.27.45.77:5090")
             modparam("dmq", "notification_address", "sip:172.27.45.200:5090")         
 
        In this case, the "kamcmd dmq.list_nodes" would show the local Machine and 172.27.45.200 as the only nodes in the output, ie 172.27.45.77 is not showing up at all, which is problematic, since the local machine(172.27.45.243 in our case) would not been able to send any dmq sync info to its peer(172.27.45.77) in the same Pair.

  To see if the above issue might have been addressed in later release, we upgraded to the latest Kamailio ver 5.6.0
To our respite, the above issue no longer exists in the new version(though not sure which immediate release after v 5.3.2 it would have been initially fixed.)
This is where we have a new issue explained below:

2. The registration data does get synced to the peer Kamailio in the same Pair, and also to the Kamailio instances in the other Pair. However, the Socket Parameter in "kamctl ul show" output shows [not set] even on the side where the websocket connection actually exists. 

[root@localhost ~]# kamctl ul show
{
  "jsonrpc":  "2.0",
  "result": {
    "Domains":  [{
        "Domain": {
          "Domain": "location",
          "Size": 1024,
          "AoRs": [{
              "Info": {
                "AoR":  "9008077221",
                "HashID": 1952082106,
                "Contacts": [{
                    "Contact":  {
                      "Address":  "sip:Harneet_qifir@172.24.58.210",
                      "Expires":  159,
                      "Q":  -1,
                      "Call-ID":  "vfli2uv8du3ppda73q5ppe",
                      "CSeq": 106,
                      "User-Agent": "EngageDigital",
                      "Received": "sip:172.27.44.252:60070;transport=ws",
                      "Path": "[not set]",
                      "State":  "CS_NEW",
                      "Flags":  0,
                      "CFlags": 0,
                      "Socket": "[not set]",   <<<<<<<<<<<<<<<<<<<<<<
                      "Methods":  7071,
                      "Ruid": "uloc-62f26e3b-2677-1",
                      "Instance": "<urn:uuid:4fbd3e96-b4de-497b-886d-1ca8ffa016a4>",
                      "Reg-Id": 1,
                      "Server-Id":  0,
                      "Tcpconn-Id": -1,
                      "Keepalive":  0,
                      "Last-Keepalive": 1660063892,
                      "KA-Roundtrip": 0,
                      "Last-Modified":  1660063892
                    }
                  }]
              }
            }
  ],

In order to confirm that the socket actually exists on this Kamailio instance, I am pasting the below outputs from the same machine, where ws dump and even the native netstat confirms that.

[root@localhost ~]# netstat -tunelap | grep 60070
tcp        0      0 172.27.45.199:8080      172.27.44.252:60070     ESTABLISHED 994        17322355   16230/kamailio      
[root@localhost ~]# kamcmd ws.dump
{
connections: {
1: ws:172.27.44.252:60070 -> ws:172.27.45.199:8080 (state: OPEN,  last used 24s ago, sub-protocol: sip)
}
info: {
wscounter: 1
truncated: no
}
}

We do need to distinguish the actual Kamailio instance where the websocket connection actually exists, so as to route the call ahead from the same instance, or if it does not exist(and it's merely a sync'ed registration data received over DMQ Channel), then the Kamailio should route the call ahead to the Kamailio instance in the other Pair, which can then route it ahead to the Registered webrtc endpoint. We were hoping to use the Socket Parameter output, but for the said problem, unable to use the same as an indicator. 

So what would be the best way to identify which Kamailio has the websocket connection with the Actual endpoint? Should we rely on the output of netstat or ws.dump to infer that? I mean this needs to be done in kamailio.cfg for each call, so want to know the best way, or if there is completely different approach that can be suggested?

Apologies for the long email, but any pointers will be much helpful.

Thanks & Regards,
Harneet Singh
                      
--
"Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth" - Sir Arthur Conan Doyle