[SR-Users] Incorrect Socket and Tcpconn-Id with Mid-Registrar and WebSockets

[Grant Bagdasarian]

Hi,

We're attempting to implement a mid-registrar for websocket connections but
are running into some issues where the tcpconn-id and socket attributes are
not stored correctly in memory (usrloc).
REGISTER Flow: UA1 -> P1 (Mid-Registrar) ~> P2 (Core Proxy/Registrar)
INVITE Flow: UA2 -> P2 -> P1 -> UA1

UA1: baresip cli connecting to P1 using websockets
UA2: softphone

Everything is running locally on a dev machine, and we're using Docker to
launch P1 and P2.

The below code snippet is what we use on P1, and has been simplified.
The general idea is as following:
1) UA1 sets up the websocket connection and sends a REGISTER request to P1
2) P1 checks if this request has already been sent to P2
    2.a) If not, then relay the request to P2. Upon receiving a 200 OK from
P2 we save the registration in usrloc
    2.b) If so, then absorb the request (save) and generate a reply
3) UA2 tries to call UA1 via P1 and P2

The problem occurs at 2.a since the REGISTER request is forwarded using UDP
and the 200 OK is also received using UDP.
When the registration is stored in reply route, the Socket
(udp:x.x.x.x:xxxx) and Tcpconn-Id (-1) are not set to what we would prefer
(see Output 1)
TCP was also tried as the transport between P1 and P2. Upon receiving the
reply from P2 the Tcpconn-Id and Socket of the connection between P1 and P2
was saved in the binding.
When 2.b occurs the Socket and Tcpconn-Id are set to the correct values
(see Output 2)

Making a call when the binding state is at 2.a we receive the following
error: "TCP/TLS connection (id: 0) for WebSocket could not be found"
Making a call when the binding state is at 2.b works perfectly fine since
the correct Tcpconn-Id is set.

We did find a solution, or rather a workaround to this issue:
1) Always call the save function in request_route
2) If P2 responds with an error (non-200 OK) use the unregister function to
remove the binding

We also tried the tcpops tcp_set_otcpid and tcp_set_otcpid_flag functions
after lookup, but this didn't do anything.

KEMI Script
def ksr_request_route(self, msg):
    if kemi.is_REGISTER():
        aor = "aor"

        if result := kemi.htable.sht_is_null("registrations", aor) == 1:
            kemi.tm.t_on_reply("ksr_register_reply_route")
            kemi.dispatcher.ds_select_dst(1, 4)
            kemi.tm.t_relay()
            return 1

        kemi.registrar.save("registrations", 1)

        return 1

    if kemi.is_INVITE():
        aor = "aor"
        if result := kemi.registrar.lookup_uri("registrations", aor):
            kemi.tm.t_relay()
            return 1

def ksr_register_reply_route(self, msg):
    if kemi.tm.t_check_status("200") == 1:
        kemi.registrar.save("registrations", 1)

        aor = "aor"

        kemi.htable.sht_sets("registrations", aor, "")

        return 1

Output 1: Binding state when request is being forwarded
kamcmd ul.dump
{
Domains: {
Domain: {
Domain: registrations
Size: 1024
AoRs: {
Info: {
AoR: grant at 127.0.0.1
HashID: 409622189
Contacts: {
Contact: {
Address: sip:grant-0x7fe915a79138 at 172.16.99.12:9;transport=ws
Expires: 13
Q: 1.000000
Call-ID: 7cdfbf93f23bb54f
CSeq: 34818
User-Agent: n/a
Received: sip:192.168.80.1:62184;transport=ws
Path: [not set]
State: CS_NEW
Flags: 1
CFlags: 20
Socket: udp:10.2.0.15:5060
Methods: -1
Ruid: uloc-626a80d3-46-1
Instance: <urn:uuid:c67df38d-d5e0-e834-1d98-a8df2d3f291b>
Reg-Id: 0
Server-Id: 0
Tcpconn-Id: -1
Keepalive: 0
Last-Keepalive: 1651146965
KA-Roundtrip: 0
Last-Modified: 1651146965
}
}
}
}
Stats: {
Records: 1
Max-Slots: 1
}
}
}
}
kamcmd core.tcp_list
{
id: 1
type: WS
state: CONN_OK
timeout: 2
lifetime: 120
ref_count: 2
src_ip: 192.168.80.1
src_port: 62184
dst_ip: 192.168.80.15
dst_port: 80
}
kamcmd ws.dump
{
connections: {
1: ws:192.168.80.1:62184 -> ws:192.168.80.15:80 (state: OPEN,  last used 3s
ago, sub-protocol: sip)
}
info: {
wscounter: 1
truncated: no
}
}

Output 2: Binding state when request is being absorbed
kamcmd ul.dump
{
Domains: {
Domain: {
Domain: registrations
Size: 1024
AoRs: {
Info: {
AoR: grant at 127.0.0.1
HashID: 409622189
Contacts: {
Contact: {
Address: sip:grant-0x7fe915a79138 at 172.16.99.12:9;transport=ws
Expires: 9
Q: 1.000000
Call-ID: 7cdfbf93f23bb54f
CSeq: 34819
User-Agent: baresip v2.0.2 (x86_64/darwin)
Received: sip:192.168.80.1:62184;transport=ws
Path: [not set]
State: CS_NEW
Flags: 1
CFlags: 20
Socket: tcp:192.168.80.15:80
Methods: 5087
Ruid: uloc-626a80d3-46-1
Instance: <urn:uuid:c67df38d-d5e0-e834-1d98-a8df2d3f291b>
Reg-Id: 0
Server-Id: 0
Tcpconn-Id: 1
Keepalive: 0
Last-Keepalive: 1651146979
KA-Roundtrip: 0
Last-Modified: 1651146979
}
}
}
}
Stats: {
Records: 1
Max-Slots: 1
}
}
}
}
kamcmd core.tcp_list
{
id: 1
type: WS
state: CONN_OK
timeout: 3
lifetime: 120
ref_count: 2
src_ip: 192.168.80.1
src_port: 62184
dst_ip: 192.168.80.15
dst_port: 80
}
kamcmd ws.dump
{
connections: {
1: ws:192.168.80.1:62184 -> ws:192.168.80.15:80 (state: OPEN,  last used 1s
ago, sub-protocol: sip)
}
info: {
wscounter: 1
truncated: no
}
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20220429/02681969/attachment.htm>