<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello,</p>
<p>if it is needed/useful, then new fields can be added in the
internal structures (e.g., tcp connection, received info) to keep
the haproxy address/port.</p>
<p>Otherwise, I haven't needed to haproxy so far to become familiar
with, so cannot comment much, but adding an alias seems to be the
right approach.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div class="moz-cite-prefix">On 18.10.21 16:42, Federico Cabiddu
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAFOaF_gtMEG7KyqzG0MBpLXiLqQs9jN4PjLSDsVJuz9q-Lf++Q@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Hi Arsen and Sergey,
<div>actually the protocol's specs say that the destination ip
and port fields of the protocol have to be considered as "<span style="color:rgb(0,0,0);white-space:pre-wrap">the ones the server would </span><span style="color:rgb(0,0,0);white-space:pre-wrap">get using getsockname()", so Arsen is right and the implementation in kamailio is strictly correct.</span></div>
<div>However, as I said, this implementation is preventing
kamailio from re-using the already established tcp connection
for sending messages to the connection initiator, e.g.
in-dialog messages for a call.</div>
<div>I can force the tcp connection to be used for such messages
by means of tcp_set_otcipd (<a
href="https://kamailio.org/docs/modules/devel/modules/tcpops.html#tcpops.f.tcp_set_otcpid"
target="_blank" moz-do-not-send="true">https://kamailio.org/docs/modules/devel/modules/tcpops.html#tcpops.f.tcp_set_otcpid</a>),
e.g. saving the tcp connection id into a Record-Route
parameter to be retrieved during loose routed messages
handling, I think that a tcp alias reflecting the original
destination ip/port could be added to the aliases' list, to
make the re-usage transparent.</div>
<div>I'll prepare a PR for this.</div>
<div>Thanks for the feedback.</div>
<div><br>
</div>
<div>Federico</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Oct 15, 2021 at 2:25
PM Arsen Semenov <<a href="mailto:arsperger@gmail.com"
target="_blank" moz-do-not-send="true">arsperger@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi Sergey,<br>
What Frederico talks about wouldn't lead to opening a new
connection towards LB/HAproxy, instead Kamailio should find
a connection by alias and re-use it.
<div><br>
</div>
<div>Frederico, I did not test with commenting the dst ip
overwriting, in my tests I was trying to find a connection
by using 0 for local_ip for calls originated from behind
proxy protocol, it was working, but I'm not sure this is
the best option here. <br>
<br>
I am wondering if there is anyone who actually uses proxy
protocol in DSR scenarios? <br>
maybe really need to change it in order to make "sending
msg back" working. <br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Oct 15, 2021 at
4:44 PM Sergey Safarov <<a
href="mailto:s.safarov@gmail.com" target="_blank"
moz-do-not-send="true">s.safarov@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div dir="ltr">Hi Federico<br>
Here is also another issue here.</div>
<div>I do know how to HAproxy UDP messages but let talk
about TCP/TLS connections.</div>
<div>Imagine you know which socket ned to use to
establish a new connection.</div>
<div><br>
</div>
<div>But you are not able to do it. Because HAproxy do
not provide the ability to establish a connection from
the backend server to the client.</div>
<div>So real socket knowled do not help you. This do
work with HAproxy servers.</div>
<div><br>
</div>
<div>What you can, establish a direct connection from
Kamailio to the client using a different socket with a
different "advertise" keyword.</div>
<div><br>
</div>
<div>You anyway need another socket.<br>
Any socket that you want.</div>
<div><br>
</div>
<div>Maybe you need anycast Kamailio installation?</div>
<div><br>
</div>
<div>Sergey</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Oct 15, 2021
at 2:27 PM Federico Cabiddu <<a
href="mailto:federico.cabiddu@gmail.com"
target="_blank" moz-do-not-send="true">federico.cabiddu@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi Arsen,
<div>the issue is exactly that kamailio behind HA
LB DON'T reuse the same connection because there
is no tcp alias for CLIENT_IP/LOCAL_KAMAILIO_IP,
because the kamailio local socket has been
overwritten by the balancer IP. I've read the
specification and nowhere is written that the
load balancer IP must overwrite the local socket
information. With the actual implementation it
is impossible to send back a message (not a
reply, a brand new one) to the client, because
there is no matching tcp alias (I've done a
quite deep debug). Proxy protocol was mainly
taught for unidirectional flows (http), not for
SIP. IMHO it is useless , if not dangerous, that
the local socket is rewritten with an IP that
doesn't belong to kamailio. Commenting the dst
ip overwriting makes kamailio create the correct
alias for the tcp connection and reuse it to
send, in example, a BYE message from the callee
to the caller.</div>
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Federico</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Oct 15,
2021 at 12:40 PM Arsen Semenov <<a
href="mailto:arsperger@gmail.com"
target="_blank" moz-do-not-send="true">arsperger@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>Hi Federico,</div>
<div><br>
</div>
I was facing the same issues when I was
playing with the proxy protocol some time ago.<br>
As per my understanding the way how proxy
protocol(PP) is implemented in Kamailio is
outlined in the de-facto standard doc: <a
href="https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt"
target="_blank" moz-do-not-send="true">https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt</a><br>
<br>
"<br>
The PROXY protocol's goal is to fill the
server's internal structures with the<br>
information collected by the proxy that the
server would have been able to get<br>
by itself if the client was connecting
directly to the server instead of via a<br>
proxy. The information carried by the protocol
are the ones the server would<br>
get using getsockname() and getpeername() :<br>
- address family (AF_INET for IPv4, AF_INET6
for IPv6, AF_UNIX)<br>
- socket protocol (SOCK_STREAM for TCP,
SOCK_DGRAM for UDP)<br>
- layer 3 source and destination addresses<br>
- layer 4 source and destination ports if
any<br>
"<br>
<br>
So that internally Kamailio will have info of
Client
source_ip:source_port/LB_external_IP:LB_external_port<br>
You can confirm that by executing
core.tcp_list command.<br>
<br>
For a call initiated by a client behind LB
with proxy protocol, responses will re-use
existing TCP connection and will reach the
client. But for example if the call will be
terminated from the callee side, (i.e new
transaction) Kamailio will fail to find
existing TCP connection since it does not know
nothing about it and will try to open a new
one, which, in turn will fail either because
of LB reject - if so_useport is disabled, or
if so_reuseport=yes because of the fact that
in the OS there is already opened TCP
connection with the same tuple (ie. between LB
and Kamailio host) <br>
So the way to use proxy protocol in Kamailio
is DSR (direct server return) scenarios.
<div><br>
<div>here you can find related
conversation: <a
href="https://github.com/kamailio/kamailio/issues/2103"
target="_blank" moz-do-not-send="true">https://github.com/kamailio/kamailio/issues/2103</a></div>
<div><br>
</div>
<div><br>
</div>
<div>Regards, </div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Oct
15, 2021 at 2:43 PM Federico Cabiddu <<a
href="mailto:federico.cabiddu@gmail.com"
target="_blank" moz-do-not-send="true">federico.cabiddu@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi all,
<div>I've been recently testing kamailio
support for proxy protocol which was
introduced by <a
href="https://github.com/kamailio/kamailio/issues/1757"
target="_blank" moz-do-not-send="true">https://github.com/kamailio/kamailio/issues/1757</a>.
As reported by others, even if kamailio
is able to decode the proxy protocol and
get the client's original IP address, it
is unable to send SIP messages to the
client which initiated the connection
through the HA load balancer (nginx in
my case). After investigation I've found
that there is no alias added to the tcp
connection aliases list for the tuple
CLIENT_IP:CLIENT_PORT/LOCAL_KAMAILIO_IP:KAMAILIO_PORT.
This means that when trying to forward a
message to the originating client
kamailio won't use the existing
connection with the load balancer/proxy
but will try to establish a new
connection. The fact is that the
function which parses the proxy header
overwrites the dst ip/port of the
connection with the "Destination IP" and
"Destination Port" fields of the proxy
header (<a
href="https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L989"
target="_blank" moz-do-not-send="true">https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L989</a>
for v2, <a
href="https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L1071"
target="_blank" moz-do-not-send="true">https://github.com/kamailio/kamailio/blob/f677dea597db6ceaa66a2755dd6e9e738855dc35/src/core/tcp_main.c#L1071</a>
for v1). This fields contain the IP/port
of the Load Balancer, not the kamailio
IP/Port, and kamailio will fail to find
a tcp connection toward the client's src
IP since the Load Balancer IP is not a
kamailio's local socket.</div>
<div>I think that the destination IP of
the connection shouldn't be rewritten
with the load balancer IP, unless I'm
missing something.</div>
<div>Hopefully I've been clear enough
explaining the issue :)</div>
<div>If you agree with the analysis I can
prepare a PR for it.</div>
<div><br>
</div>
<div>Have you all a great weekend,</div>
<div><br>
</div>
<div>Federico Cabiddu</div>
<div><br>
</div>
</div>
_______________________________________________<br>
Kamailio (SER) - Development Mailing List<br>
<a href="mailto:sr-dev@lists.kamailio.org"
target="_blank" moz-do-not-send="true">sr-dev@lists.kamailio.org</a><br>
<a
href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev</a><br>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>Arsen Semenov<br>
</div>
<div><br>
</div>
</div>
</div>
_______________________________________________<br>
Kamailio (SER) - Development Mailing List<br>
<a href="mailto:sr-dev@lists.kamailio.org"
target="_blank" moz-do-not-send="true">sr-dev@lists.kamailio.org</a><br>
<a
href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev</a><br>
</blockquote>
</div>
_______________________________________________<br>
Kamailio (SER) - Development Mailing List<br>
<a href="mailto:sr-dev@lists.kamailio.org"
target="_blank" moz-do-not-send="true">sr-dev@lists.kamailio.org</a><br>
<a
href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev</a><br>
</blockquote>
</div>
</div>
_______________________________________________<br>
Kamailio (SER) - Development Mailing List<br>
<a href="mailto:sr-dev@lists.kamailio.org" target="_blank"
moz-do-not-send="true">sr-dev@lists.kamailio.org</a><br>
<a
href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev</a><br>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>Arsen Semenov<br>
</div>
<div><br>
</div>
</div>
</div>
_______________________________________________<br>
Kamailio (SER) - Development Mailing List<br>
<a href="mailto:sr-dev@lists.kamailio.org" target="_blank"
moz-do-not-send="true">sr-dev@lists.kamailio.org</a><br>
<a
href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev</a><br>
</blockquote>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Kamailio (SER) - Development Mailing List
<a class="moz-txt-link-abbreviated" href="mailto:sr-dev@lists.kamailio.org">sr-dev@lists.kamailio.org</a>
<a class="moz-txt-link-freetext" href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla -- <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/miconda">www.twitter.com/miconda</a> -- <a class="moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - Online
Nov 08-11, 2021 (Europe Timezone) - Nov 22-25, 2021 (America Timezone)
* <a class="moz-txt-link-freetext" href="https://www.asipto.com/sw/kamailio-advanced-training-online/">https://www.asipto.com/sw/kamailio-advanced-training-online/</a></pre>
</body>
</html>