[sr-dev] Proxy protocol issue

[Sergey Safarov]

Hi Federico
Here is also another issue here.
I do know how to HAproxy UDP messages but let talk about TCP/TLS
connections.
Imagine you know which socket ned to use to establish a new connection.

But you are not able to do it. Because HAproxy do not provide the ability
to establish a connection from the backend server to the client.
So real socket knowled do not help you. This do work with HAproxy servers.

What you can, establish a direct connection from Kamailio to the client
using a different socket with a different "advertise" keyword.

You anyway need another socket.
Any socket that you want.

Maybe you need anycast Kamailio installation?

Sergey

On Fri, Oct 15, 2021 at 2:27 PM Federico Cabiddu <federico.cabiddu at gmail.com>
wrote:

> Hi Arsen,
> the issue is exactly that kamailio behind HA LB DON'T reuse the same
> connection because there is no tcp alias for CLIENT_IP/LOCAL_KAMAILIO_IP,
> because the kamailio local socket has been overwritten by the balancer IP.
> I've read the specification and nowhere is written that the load balancer
> IP must overwrite the local socket information. With the actual
> implementation it is impossible to send back a message (not a reply, a
> brand new one) to the client, because there is no matching tcp alias (I've
> done a quite deep debug). Proxy protocol was mainly taught for
> unidirectional flows (http), not for SIP. IMHO it is useless , if not
> dangerous, that the local socket is rewritten with an IP that doesn't
> belong to kamailio. Commenting the dst ip overwriting makes kamailio create
> the correct alias for the tcp connection and reuse it to send, in example,
> a BYE message from the callee to the caller.
>
> Cheers,
>
> Federico
>
> On Fri, Oct 15, 2021 at 12:40 PM Arsen Semenov <arsperger at gmail.com>
> wrote:
>
>> Hi Federico,
>>
>> I was facing the same issues when I was playing with the proxy protocol
>> some time ago.
>> As per my understanding the way how proxy protocol(PP) is implemented in
>> Kamailio is outlined in the de-facto standard doc:
>> https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
>>
>> "
>> The PROXY protocol's goal is to fill the server's internal structures
>> with the
>> information collected by the proxy that the server would have been able
>> to get
>> by itself if the client was connecting directly to the server instead of
>> via a
>> proxy. The information carried by the protocol are the ones the server
>> would
>> get using getsockname() and getpeername() :
>>   - address family (AF_INET for IPv4, AF_INET6 for IPv6, AF_UNIX)
>>   - socket protocol (SOCK_STREAM for TCP, SOCK_DGRAM for UDP)
>>   - layer 3 source and destination addresses
>>   - layer 4 source and destination ports if any
>> "
>>
>> So that internally Kamailio will have info of Client
>> source_ip:source_port/LB_external_IP:LB_external_port
>> You can confirm that by executing core.tcp_list command.
>>
>> For a call initiated by a client behind LB with proxy protocol, responses
>> will re-use existing TCP connection and will reach the client. But for
>> example if the call will be terminated from the callee side, (i.e new
>> transaction) Kamailio will fail to find existing TCP connection since it
>> does not know nothing about it and will try to open a new one, which, in
>> turn will fail either because of LB reject - if so_useport is disabled, or
>> if so_reuseport=yes because of the fact that in the OS there is already
>> opened TCP connection with the same tuple (ie. between LB and Kamailio
>> host)
>> So the way to use proxy protocol in Kamailio is DSR (direct server
>> return) scenarios.
>>
>> here you can find related conversation:
>> https://github.com/kamailio/kamailio/issues/2103
>>
>>
>> Regards,
>>
>> On Fri, Oct 15, 2021 at 2:43 PM Federico Cabiddu <
>> federico.cabiddu at gmail.com> wrote:
>>
>>> Hi all,
>>> I've been recently testing kamailio support for proxy protocol which was
>>> introduced by https://github.com/kamailio/kamailio/issues/1757. As
>>> reported by others, even if kamailio is able to decode the proxy protocol
>>> and get the client's original IP address, it is unable to send SIP messages
>>> to the client which initiated the connection through the HA load balancer
>>> (nginx in my case). After investigation I've found that there is no alias
>>> added to the tcp connection aliases list for the tuple
>>> CLIENT_IP:CLIENT_PORT/LOCAL_KAMAILIO_IP:KAMAILIO_PORT. This means that when
>>> trying to forward a message to the originating client kamailio won't use
>>> the existing connection with the load balancer/proxy but will try to
>>> establish a new connection. The fact is that the function which parses the
>>> proxy header overwrites the dst ip/port of the connection with the
>>> "Destination IP" and "Destination Port" fields of the proxy header (
>>> https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L989
>>> for v2,
>>> https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L1071
>>> for v1). This fields contain the IP/port of the Load Balancer, not the
>>> kamailio IP/Port, and kamailio will fail to find a tcp connection toward
>>> the client's src IP since the Load Balancer IP is not a kamailio's local
>>> socket.
>>> I think that the destination IP of the connection shouldn't be rewritten
>>> with the load balancer IP, unless I'm missing something.
>>> Hopefully I've been clear enough explaining the issue :)
>>> If you agree with the analysis I can prepare a PR for it.
>>>
>>> Have you all a great weekend,
>>>
>>> Federico Cabiddu
>>>
>>> _______________________________________________
>>> Kamailio (SER) - Development Mailing List
>>> sr-dev at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
>>>
>>
>>
>> --
>> Arsen Semenov
>>
>> _______________________________________________
>> Kamailio (SER) - Development Mailing List
>> sr-dev at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
>>
> _______________________________________________
> Kamailio (SER) - Development Mailing List
> sr-dev at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-dev/attachments/20211015/078c2952/attachment.htm>