[sr-dev] Proxy protocol issue

[Federico Cabiddu]

Hi Arsen,
the issue is exactly that kamailio behind HA LB DON'T reuse the same
connection because there is no tcp alias for CLIENT_IP/LOCAL_KAMAILIO_IP,
because the kamailio local socket has been overwritten by the balancer IP.
I've read the specification and nowhere is written that the load balancer
IP must overwrite the local socket information. With the actual
implementation it is impossible to send back a message (not a reply, a
brand new one) to the client, because there is no matching tcp alias (I've
done a quite deep debug). Proxy protocol was mainly taught for
unidirectional flows (http), not for SIP. IMHO it is useless , if not
dangerous, that the local socket is rewritten with an IP that doesn't
belong to kamailio. Commenting the dst ip overwriting makes kamailio create
the correct alias for the tcp connection and reuse it to send, in example,
a BYE message from the callee to the caller.

Cheers,

Federico

On Fri, Oct 15, 2021 at 12:40 PM Arsen Semenov <arsperger at gmail.com> wrote:

> Hi Federico,
>
> I was facing the same issues when I was playing with the proxy protocol
> some time ago.
> As per my understanding the way how proxy protocol(PP) is implemented in
> Kamailio is outlined in the de-facto standard doc:
> https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
>
> "
> The PROXY protocol's goal is to fill the server's internal structures with
> the
> information collected by the proxy that the server would have been able to
> get
> by itself if the client was connecting directly to the server instead of
> via a
> proxy. The information carried by the protocol are the ones the server
> would
> get using getsockname() and getpeername() :
>   - address family (AF_INET for IPv4, AF_INET6 for IPv6, AF_UNIX)
>   - socket protocol (SOCK_STREAM for TCP, SOCK_DGRAM for UDP)
>   - layer 3 source and destination addresses
>   - layer 4 source and destination ports if any
> "
>
> So that internally Kamailio will have info of Client
> source_ip:source_port/LB_external_IP:LB_external_port
> You can confirm that by executing core.tcp_list command.
>
> For a call initiated by a client behind LB with proxy protocol, responses
> will re-use existing TCP connection and will reach the client. But for
> example if the call will be terminated from the callee side, (i.e new
> transaction) Kamailio will fail to find existing TCP connection since it
> does not know nothing about it and will try to open a new one, which, in
> turn will fail either because of LB reject - if so_useport is disabled, or
> if so_reuseport=yes because of the fact that in the OS there is already
> opened TCP connection with the same tuple (ie. between LB and Kamailio
> host)
> So the way to use proxy protocol in Kamailio is DSR (direct server return)
> scenarios.
>
> here you can find related conversation:
> https://github.com/kamailio/kamailio/issues/2103
>
>
> Regards,
>
> On Fri, Oct 15, 2021 at 2:43 PM Federico Cabiddu <
> federico.cabiddu at gmail.com> wrote:
>
>> Hi all,
>> I've been recently testing kamailio support for proxy protocol which was
>> introduced by https://github.com/kamailio/kamailio/issues/1757. As
>> reported by others, even if kamailio is able to decode the proxy protocol
>> and get the client's original IP address, it is unable to send SIP messages
>> to the client which initiated the connection through the HA load balancer
>> (nginx in my case). After investigation I've found that there is no alias
>> added to the tcp connection aliases list for the tuple
>> CLIENT_IP:CLIENT_PORT/LOCAL_KAMAILIO_IP:KAMAILIO_PORT. This means that when
>> trying to forward a message to the originating client kamailio won't use
>> the existing connection with the load balancer/proxy but will try to
>> establish a new connection. The fact is that the function which parses the
>> proxy header overwrites the dst ip/port of the connection with the
>> "Destination IP" and "Destination Port" fields of the proxy header (
>> https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L989
>> for v2,
>> https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L1071
>> for v1). This fields contain the IP/port of the Load Balancer, not the
>> kamailio IP/Port, and kamailio will fail to find a tcp connection toward
>> the client's src IP since the Load Balancer IP is not a kamailio's local
>> socket.
>> I think that the destination IP of the connection shouldn't be rewritten
>> with the load balancer IP, unless I'm missing something.
>> Hopefully I've been clear enough explaining the issue :)
>> If you agree with the analysis I can prepare a PR for it.
>>
>> Have you all a great weekend,
>>
>> Federico Cabiddu
>>
>> _______________________________________________
>> Kamailio (SER) - Development Mailing List
>> sr-dev at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
>>
>
>
> --
> Arsen Semenov
>
> _______________________________________________
> Kamailio (SER) - Development Mailing List
> sr-dev at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-dev/attachments/20211015/e0a94f80/attachment-0001.htm>