[sr-dev] Proxy protocol issue
Daniel-Constantin Mierla
miconda at gmail.com
Mon Oct 18 18:56:41 CEST 2021
Hello,
if it is needed/useful, then new fields can be added in the internal
structures (e.g., tcp connection, received info) to keep the haproxy
address/port.
Otherwise, I haven't needed to haproxy so far to become familiar with,
so cannot comment much, but adding an alias seems to be the right approach.
Cheers,
Daniel
On 18.10.21 16:42, Federico Cabiddu wrote:
> Hi Arsen and Sergey,
> actually the protocol's specs say that the destination ip and port
> fields of the protocol have to be considered as "the ones the server
> would get using getsockname()", so Arsen is right and the
> implementation in kamailio is strictly correct.
> However, as I said, this implementation is preventing kamailio from
> re-using the already established tcp connection for sending messages
> to the connection initiator, e.g. in-dialog messages for a call.
> I can force the tcp connection to be used for such messages by means
> of tcp_set_otcipd
> (https://kamailio.org/docs/modules/devel/modules/tcpops.html#tcpops.f.tcp_set_otcpid
> <https://kamailio.org/docs/modules/devel/modules/tcpops.html#tcpops.f.tcp_set_otcpid>),
> e.g. saving the tcp connection id into a Record-Route parameter to be
> retrieved during loose routed messages handling, I think that a tcp
> alias reflecting the original destination ip/port could be added to
> the aliases' list, to make the re-usage transparent.
> I'll prepare a PR for this.
> Thanks for the feedback.
>
> Federico
>
>
>
> On Fri, Oct 15, 2021 at 2:25 PM Arsen Semenov <arsperger at gmail.com
> <mailto:arsperger at gmail.com>> wrote:
>
> Hi Sergey,
> What Frederico talks about wouldn't lead to opening a new
> connection towards LB/HAproxy, instead Kamailio should find a
> connection by alias and re-use it.
>
> Frederico, I did not test with commenting the dst ip overwriting,
> in my tests I was trying to find a connection by using 0 for
> local_ip for calls originated from behind proxy protocol, it was
> working, but I'm not sure this is the best option here.
>
> I am wondering if there is anyone who actually uses proxy protocol
> in DSR scenarios?
> maybe really need to change it in order to make "sending msg back"
> working.
>
> On Fri, Oct 15, 2021 at 4:44 PM Sergey Safarov
> <s.safarov at gmail.com <mailto:s.safarov at gmail.com>> wrote:
>
> Hi Federico
> Here is also another issue here.
> I do know how to HAproxy UDP messages but let talk about
> TCP/TLS connections.
> Imagine you know which socket ned to use to establish a new
> connection.
>
> But you are not able to do it. Because HAproxy do not provide
> the ability to establish a connection from the backend server
> to the client.
> So real socket knowled do not help you. This do work with
> HAproxy servers.
>
> What you can, establish a direct connection from Kamailio to
> the client using a different socket with a different
> "advertise" keyword.
>
> You anyway need another socket.
> Any socket that you want.
>
> Maybe you need anycast Kamailio installation?
>
> Sergey
>
> On Fri, Oct 15, 2021 at 2:27 PM Federico Cabiddu
> <federico.cabiddu at gmail.com
> <mailto:federico.cabiddu at gmail.com>> wrote:
>
> Hi Arsen,
> the issue is exactly that kamailio behind HA LB DON'T
> reuse the same connection because there is no tcp alias
> for CLIENT_IP/LOCAL_KAMAILIO_IP, because the kamailio
> local socket has been overwritten by the balancer IP. I've
> read the specification and nowhere is written that the
> load balancer IP must overwrite the local socket
> information. With the actual implementation it is
> impossible to send back a message (not a reply, a brand
> new one) to the client, because there is no matching tcp
> alias (I've done a quite deep debug). Proxy protocol was
> mainly taught for unidirectional flows (http), not for
> SIP. IMHO it is useless , if not dangerous, that the local
> socket is rewritten with an IP that doesn't belong to
> kamailio. Commenting the dst ip overwriting makes kamailio
> create the correct alias for the tcp connection and reuse
> it to send, in example, a BYE message from the callee to
> the caller.
>
> Cheers,
>
> Federico
>
> On Fri, Oct 15, 2021 at 12:40 PM Arsen Semenov
> <arsperger at gmail.com <mailto:arsperger at gmail.com>> wrote:
>
> Hi Federico,
>
> I was facing the same issues when I was playing with
> the proxy protocol some time ago.
> As per my understanding the way how proxy protocol(PP)
> is implemented in Kamailio is outlined in the de-facto
> standard doc:
> https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
> <https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt>
>
> "
> The PROXY protocol's goal is to fill the server's
> internal structures with the
> information collected by the proxy that the server
> would have been able to get
> by itself if the client was connecting directly to the
> server instead of via a
> proxy. The information carried by the protocol are the
> ones the server would
> get using getsockname() and getpeername() :
> - address family (AF_INET for IPv4, AF_INET6 for
> IPv6, AF_UNIX)
> - socket protocol (SOCK_STREAM for TCP, SOCK_DGRAM
> for UDP)
> - layer 3 source and destination addresses
> - layer 4 source and destination ports if any
> "
>
> So that internally Kamailio will have info of Client
> source_ip:source_port/LB_external_IP:LB_external_port
> You can confirm that by executing core.tcp_list command.
>
> For a call initiated by a client behind LB with proxy
> protocol, responses will re-use existing TCP
> connection and will reach the client. But for example
> if the call will be terminated from the callee side,
> (i.e new transaction) Kamailio will fail to find
> existing TCP connection since it does not know nothing
> about it and will try to open a new one, which, in
> turn will fail either because of LB reject - if
> so_useport is disabled, or if so_reuseport=yes because
> of the fact that in the OS there is already opened TCP
> connection with the same tuple (ie. between LB and
> Kamailio host)
> So the way to use proxy protocol in Kamailio is DSR
> (direct server return) scenarios.
>
> here you can find related
> conversation: https://github.com/kamailio/kamailio/issues/2103
> <https://github.com/kamailio/kamailio/issues/2103>
>
>
> Regards,
>
> On Fri, Oct 15, 2021 at 2:43 PM Federico Cabiddu
> <federico.cabiddu at gmail.com
> <mailto:federico.cabiddu at gmail.com>> wrote:
>
> Hi all,
> I've been recently testing kamailio support for
> proxy protocol which was introduced
> by https://github.com/kamailio/kamailio/issues/1757
> <https://github.com/kamailio/kamailio/issues/1757>.
> As reported by others, even if kamailio is able to
> decode the proxy protocol and get the client's
> original IP address, it is unable to send SIP
> messages to the client which initiated the
> connection through the HA load balancer (nginx in
> my case). After investigation I've found that
> there is no alias added to the tcp connection
> aliases list for the tuple
> CLIENT_IP:CLIENT_PORT/LOCAL_KAMAILIO_IP:KAMAILIO_PORT.
> This means that when trying to forward a message
> to the originating client kamailio won't use the
> existing connection with the load balancer/proxy
> but will try to establish a new connection. The
> fact is that the function which parses the proxy
> header overwrites the dst ip/port of the
> connection with the "Destination IP" and
> "Destination Port" fields of the proxy header
> (https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L989
> <https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L989>
> for
> v2, https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L1071
> <https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L1071>
> for v1). This fields contain the IP/port of the
> Load Balancer, not the kamailio IP/Port, and
> kamailio will fail to find a tcp connection toward
> the client's src IP since the Load Balancer IP is
> not a kamailio's local socket.
> I think that the destination IP of the connection
> shouldn't be rewritten with the load balancer IP,
> unless I'm missing something.
> Hopefully I've been clear enough explaining the
> issue :)
> If you agree with the analysis I can prepare a PR
> for it.
>
> Have you all a great weekend,
>
> Federico Cabiddu
>
> _______________________________________________
> Kamailio (SER) - Development Mailing List
> sr-dev at lists.kamailio.org
> <mailto:sr-dev at lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
> <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev>
>
>
>
> --
> Arsen Semenov
>
> _______________________________________________
> Kamailio (SER) - Development Mailing List
> sr-dev at lists.kamailio.org
> <mailto:sr-dev at lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
> <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev>
>
> _______________________________________________
> Kamailio (SER) - Development Mailing List
> sr-dev at lists.kamailio.org <mailto:sr-dev at lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
> <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev>
>
> _______________________________________________
> Kamailio (SER) - Development Mailing List
> sr-dev at lists.kamailio.org <mailto:sr-dev at lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
> <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev>
>
>
>
> --
> Arsen Semenov
>
> _______________________________________________
> Kamailio (SER) - Development Mailing List
> sr-dev at lists.kamailio.org <mailto:sr-dev at lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
> <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev>
>
>
> _______________________________________________
> Kamailio (SER) - Development Mailing List
> sr-dev at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - Online
Nov 08-11, 2021 (Europe Timezone) - Nov 22-25, 2021 (America Timezone)
* https://www.asipto.com/sw/kamailio-advanced-training-online/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-dev/attachments/20211018/9345428a/attachment-0001.htm>
More information about the sr-dev
mailing list