21 Sep
2017
21 Sep
'17
1:06 p.m.
Hello,
I have a use case where I need to have kamailio bind to a VRF device. The
configuration in question is similar to the example below, where eth1 is a
slave to the VRF-lite device:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Both the main routing table and "vrf-green" routing table have a default
route.
What I need to be able to do is have kamailio bind to both interfaces:
listen=eth0:5060
listen=vrf-green:5060
And additionally be able to use force_send_socket to select an interface,
for example:
force_send_socket(udp:2.2.2.2:5060);
However, I can't get this to work. The above configuration fails because
there is no listen directive for 2.2.2.2. Also, kamailio doesn't process
packets received on the VRF with the above listen directives, it behaves as
if it doesn't listen on 2.2.2.2 indeed.
In addition using either of the below:
listen=udp:2.2.2.2:5060
or
listen=eth1:5060
fails with an error upon starting kamailio.
According to the kernel documentation:
Applications that are to work within a VRF need to bind their socket to the
VRF device:
setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
or to specify the output device using cmsg and IP_PKTINFO.
The question is, is VRF useable with kamailio right now? Or is
development needed? Thanks!
BR,
George
25 Sep
25 Sep
7:10 p.m.
Sorry to bump this, but it would be very useful if I knew whether there's
any point in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello,
I have a use case where I need to have kamailio bind to a VRF device. The
configuration in question is similar to the example below, where eth1 is a
slave to the VRF-lite device:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Both the main routing table and "vrf-green" routing table have a default
route.
What I need to be able to do is have kamailio bind to both interfaces:
listen=eth0:5060
listen=vrf-green:5060
And additionally be able to use force_send_socket to select an interface,
for example:
force_send_socket(udp:2.2.2.2:5060);
However, I can't get this to work. The above configuration fails because
there is no listen directive for 2.2.2.2. Also, kamailio doesn't process
packets received on the VRF with the above listen directives, it behaves as
if it doesn't listen on 2.2.2.2 indeed.
In addition using either of the below:
listen=udp:2.2.2.2:5060
or
listen=eth1:5060
fails with an error upon starting kamailio.
According to the kernel documentation:
Applications that are to work within a VRF need to bind their socket to the
VRF device:
setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
or to specify the output device using cmsg and IP_PKTINFO.
The question is, is VRF useable with kamailio right now? Or is development needed?
Thanks!
BR,
George
26 Sep
26 Sep
8:52 a.m.
Hello,
I haven't worked with vrf here, so no first hand experience ...
What happens if you try with next option?
listen=vrf-green:5060 advertise 2.2.2.2:5060
Cheers,
Daniel
On 25.09.17 19:10, George Diamantopoulos wrote:
Sorry to bump this, but it would be very useful if I
knew whether
there's any point in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hello,
I have a use case where I need to have kamailio bind to a VRF
device. The configuration in question is similar to the example
below, where eth1 is a slave to the VRF-lite device:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Both the main routing table and "vrf-green" routing table have a
default route.
What I need to be able to do is have kamailio bind to both interfaces:
listen=eth0:5060
listen=vrf-green:5060
And additionally be able to use force_send_socket to select an
interface, for example:
force_send_socket(udp:2.2.2.2:5060 <http://2.2.2.2:5060>);
However, I can't get this to work. The above configuration fails
because there is no listen directive for 2.2.2.2. Also, kamailio
doesn't process packets received on the VRF with the above listen
directives, it behaves as if it doesn't listen on 2.2.2.2 indeed.
In addition using either of the below:
listen=udp:2.2.2.2:5060 <http://2.2.2.2:5060>
or
listen=eth1:5060
fails with an error upon starting kamailio.
According to the kernel documentation:
Applications that are to work within a VRF need to bind their socket to the
VRF device:
setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
or to specify the output device using cmsg and IP_PKTINFO.
The question is, is VRF useable with kamailio right now? Or is development needed?
Thanks!
BR,
George
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com
11 Oct
11 Oct
11:07 p.m.
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the "listen"
directive will result in kamaiio no longer failing to start when using
force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the "advertise"
option.
However, this doesn't make much of a difference in that kamailio still
won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
OPTIONS coming with sngrep, but nothing in the logs, no replies, only
retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface
with mhomed=1, this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the
transaction fails with 477 Unfortunately error on sending to next hop
occurred. In the log, kamailio prints something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16): Invalid
argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060) has
been called before t_relay()
Also, I additionally found out that if the networking configuration is as
follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060
listen=vrf-red:5060 advertise 3.3.3.3:5060
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and
127.0.0.2) for kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus slack
channel (cumulus contributed the VRF code to the linux kernel): "any app
that has a command line or option switch to call SO_BINDTODEVICE on the
sockets can be configured to use any VRF".
I guess the problem lies in that kamailio binds to several sockets and
there are multiple routing tables to consult, so mhomed doesn't work as
expected (if every table has a default route and nothing more specific than
that). But still, this doesn't really explain why binding to the VRF master
net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
ping)...
I don't think VRF support is very useful for most users though, I just had
one corner case I needed to tackle and VRF seemed like an easy fix. I'll
think of an alternative at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <miconda(a)gmail.com>
wrote:
Hello,
I haven't worked with vrf here, so no first hand experience ...
What happens if you try with next option?
listen=vrf-green:5060 advertise 2.2.2.2:5060
Cheers,
Daniel
On 25.09.17 19:10, George Diamantopoulos wrote:
Sorry to bump this, but it would be very useful if I knew whether there's
any
point in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
>
> Hello,
>
> I have a use case where I need to have kamailio bind to a VRF device.
The
configuration in question is similar to the example below, where eth1
is a slave to the VRF-lite device:
>
> +----------+ +-------------------+
> | eth0 | | vrf-green |
> | 1.1.1.1 | | 127.0.0.1 |
> +----------+ +-------------------+
> |
> +----------+
> | eth1 |
> | 2.2.2.2 |
> +----------+
>
> Both the main routing table and "vrf-green" routing table have a default
route.
>
> What I need to be able to do is have kamailio bind to both interfaces:
>
> listen=eth0:5060
> listen=vrf-green:5060
>
> And additionally be able to use force_send_socket to select an
interface, for
example:
>
> force_send_socket(udp:2.2.2.2:5060);
>
> However, I can't get this to work. The above configuration fails because
there is no listen directive for 2.2.2.2. Also, kamailio doesn't process
packets received on the VRF with the above listen directives, it behaves as
if it doesn't listen on 2.2.2.2 indeed.
>
> In addition using either of the below:
>
> listen=udp:2.2.2.2:5060
> or
> listen=eth1:5060
>
> fails with an error upon starting kamailio.
>
> According to the kernel documentation:
>
> Applications that are to work within a VRF need to bind their socket to
the
> VRF device:
>
> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>
> or to specify the output device using cmsg and IP_PKTINFO.
>
> The question is, is VRF useable with kamailio right now? Or is
development
needed? Thanks!
BR,
George
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com 
11:26 p.m.
Hi All
Just to dive in here with something which may be related. I was about to do a post with a
much simpler problem where I have one real NIC and one VPN created NIC. If I listen to
the public IP all’s well. If I listen to the IP of the VPN NIC Kamailio doesn’t see any
traffic. I can tcpdump on either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
On 11 Oct 2017, at 22:07, George Diamantopoulos
<georgediam(a)gmail.com> wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the "listen"
directive will result in kamaiio no longer failing to start when using
force_send_socket(2.2.2.2:5060 <http://2.2.2.2:5060/>) with 2.2.2.2 being used in
the "advertise" option.
However, this doesn't make much of a difference in that kamailio still won't
process IP packets arriving on eth1 at IP 2.2.2.2... I can see OPTIONS coming with sngrep,
but nothing in the logs, no replies, only retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface with mhomed=1,
this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the transaction fails
with 477 Unfortunately error on sending to next hop occurred. In the log, kamailio prints
something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060
<http://9.9.9.9:5060/>,16): Invalid argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060
<http://2.2.2.2:5060/>) has been called before t_relay()
Also, I additionally found out that if the networking configuration is as follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
listen=vrf-red:5060 advertise 3.3.3.3:5060 <http://3.3.3.3:5060/>
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and 127.0.0.2) for
kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus slack
channel (cumulus contributed the VRF code to the linux kernel): "any app that has a
command line or option switch to call SO_BINDTODEVICE on the sockets can be configured to
use any VRF".
I guess the problem lies in that kamailio binds to several sockets and there are multiple
routing tables to consult, so mhomed doesn't work as expected (if every table has a
default route and nothing more specific than that). But still, this doesn't really
explain why binding to the VRF master net devices doesn't work (I mean, "ping -I
vrf-green" works just fine for ping)...
I don't think VRF support is very useful for most users though, I just had one corner
case I needed to tackle and VRF seemed like an easy fix. I'll think of an alternative
at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>> wrote:
Hello,
I haven't worked with vrf here, so no first hand experience ...
What happens if you try with next option?
listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
Cheers,
Daniel
On 25.09.17 19:10, George Diamantopoulos wrote:
Sorry to bump this, but it would be very useful if I knew whether there's any point
in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos <georgediam(a)gmail.com
<mailto:georgediam@gmail.com>> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hello,
I have a use case where I need to have kamailio bind to a VRF device. The configuration
in question is similar to the example below, where eth1 is a slave to the VRF-lite
device:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Both the main routing table and "vrf-green" routing table have a default
route.
What I need to be able to do is have kamailio bind to both interfaces:
listen=eth0:5060
listen=vrf-green:5060
And additionally be able to use force_send_socket to select an interface, for example:
force_send_socket(udp:2.2.2.2:5060 <http://2.2.2.2:5060/>);
However, I can't get this to work. The above configuration fails because there is no
listen directive for 2.2.2.2. Also, kamailio doesn't process packets received on the
VRF with the above listen directives, it behaves as if it doesn't listen on 2.2.2.2
indeed.
In addition using either of the below:
listen=udp:2.2.2.2:5060 <http://2.2.2.2:5060/>
or
listen=eth1:5060
fails with an error upon starting kamailio.
According to the kernel documentation:
Applications that are to work within a VRF need to bind their socket to the
VRF device:
setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
or to specify the output device using cmsg and IP_PKTINFO.
The question is, is VRF useable with kamailio right now? Or is development needed?
Thanks!
BR,
George
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
--
Daniel-Constantin Mierla
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Kamailio Advanced Training - www.asipto.com <http://www.asipto.com/>
Kamailio World Conference - www.kamailioworld.com <http://www.kamailioworld.com/>
12 Oct
12 Oct
4:56 p.m.
Hello,
That seems strange... What kind of device is that VPN device? Like a ppp0
interface? What is the output of "netstat -plunt | grep kamailio"? Are you
using a "private" IP for the VPN interface and what's your main routing
table like?
Also what if you omit the "listen" directive in kamailio configuration?
Does it listen on the VPN interface then? Kamailio will bind by default to
all interfaces if no "listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com> wrote:
Hi All
Just to dive in here with something which may be related. I was about to
do a post with a much simpler problem where I have one real NIC and one VPN
created NIC. If I listen to the public IP all’s well. If I listen to the
IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on
either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the "listen"
directive will result in kamaiio no longer failing to start when using
force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
"advertise" option.
However, this doesn't make much of a difference in that kamailio still
won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
OPTIONS coming with sngrep, but nothing in the logs, no replies, only
retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface
with mhomed=1, this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the
transaction fails with 477 Unfortunately error on sending to next hop
occurred. In the log, kamailio prints something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16): Invalid
argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060)
has been called before t_relay()
Also, I additionally found out that if the networking configuration is as
follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060
listen=vrf-red:5060 advertise 3.3.3.3:5060
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and
127.0.0.2) for kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus
slack channel (cumulus contributed the VRF code to the linux kernel): "any
app that has a command line or option switch to call SO_BINDTODEVICE on the
sockets can be configured to use any VRF".
I guess the problem lies in that kamailio binds to several sockets and
there are multiple routing tables to consult, so mhomed doesn't work as
expected (if every table has a default route and nothing more specific than
that). But still, this doesn't really explain why binding to the VRF master
net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
ping)...
I don't think VRF support is very useful for most users though, I just had
one corner case I needed to tackle and VRF seemed like an easy fix. I'll
think of an alternative at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <miconda(a)gmail.com>
wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hello,
I haven't worked with vrf here, so no first hand experience ...
What happens if you try with next option?
listen=vrf-green:5060 advertise 2.2.2.2:5060
Cheers,
Daniel
On 25.09.17 19:10, George Diamantopoulos wrote:
Sorry to bump this, but it would be very useful if I knew whether
there's any
point in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos <
georgediam(a)gmail.com>
wrote:
>
> Hello,
>
> I have a use case where I need to have kamailio bind to a VRF device.
The
configuration in question is similar to the example below, where eth1
is a slave to the VRF-lite device:
>
> +----------+ +-------------------+
> | eth0 | | vrf-green |
> | 1.1.1.1 | | 127.0.0.1 |
> +----------+ +-------------------+
> |
> +----------+
> | eth1 |
> | 2.2.2.2 |
> +----------+
>
> Both the main routing table and "vrf-green" routing table have a
default route.
>
> What I need to be able to do is have kamailio bind to both interfaces:
>
> listen=eth0:5060
> listen=vrf-green:5060
>
> And additionally be able to use force_send_socket to select an
interface, for
example:
>
> force_send_socket(udp:2.2.2.2:5060);
>
> However, I can't get this to work. The above configuration fails
because
there is no listen directive for 2.2.2.2. Also, kamailio doesn't
process packets received on the VRF with the above listen directives, it
behaves as if it doesn't listen on 2.2.2.2 indeed.
>
> In addition using either of the below:
>
> listen=udp:2.2.2.2:5060
> or
> listen=eth1:5060
>
> fails with an error upon starting kamailio.
>
> According to the kernel documentation:
>
> Applications that are to work within a VRF need to bind their socket to
the
> VRF device:
>
> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>
> or to specify the output device using cmsg and IP_PKTINFO.
>
> The question is, is VRF useable with kamailio right now? Or is
development
needed? Thanks!
BR,
George
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com
5:26 p.m.
Hi George
(IP / MACs below anonymised a bit)
I’m working on a https://www.zerotier.com <https://www.zerotier.com/> VPN endpoint
which presents as a new interface;
Real Interface
ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2508723344 (2.5 GB) TX bytes:1149701606 (1.1 GB)
loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
ZeroTier
zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
This is actually a SIPCapture/Homer node so the config is;
modparam("sipcapture", "capture_on", 1)
modparam("sipcapture", "hep_capture_on", 1)
listen=udp:10.0.0.1:9060
The port bind looks ok;
# netstat -plunt | grep kamailio
udp 0 0 10.0.0.1:9060 0.0.0.0:*
19592/kamailio
I’m tracking packets at the top of "route {“
route {
xlog("route entered\r\n”);
If I tcpdump the ZeroTier Interface ;
tcpdump -i zt0 -p -X port 9060
15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@..?.J..
0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 0216 ...r..#d.m......
0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 4b20 ....\.5#.J..ACK.
… but nothing seen in kamailio.
If I swap over to sending to / listening on the public IP everything works as expected.
Cheers
Mark
On 12 Oct 2017, at 15:56, George Diamantopoulos
<georgediam(a)gmail.com> wrote:
Hello,
That seems strange... What kind of device is that VPN device? Like a ppp0 interface? What
is the output of "netstat -plunt | grep kamailio"? Are you using a
"private" IP for the VPN interface and what's your main routing table like?
Also what if you omit the "listen" directive in kamailio configuration? Does it
listen on the VPN interface then? Kamailio will bind by default to all interfaces if no
"listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com
<mailto:mark@darkorigins.com>> wrote:
Hi All
Just to dive in here with something which may be related. I was about to do a post with
a much simpler problem where I have one real NIC and one VPN created NIC. If I listen to
the public IP all’s well. If I listen to the IP of the VPN NIC Kamailio doesn’t see any
traffic. I can tcpdump on either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
On 11 Oct 2017, at 22:07, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the "listen"
directive will result in kamaiio no longer failing to start when using
force_send_socket(2.2.2.2:5060 <http://2.2.2.2:5060/>) with 2.2.2.2 being used in
the "advertise" option.
However, this doesn't make much of a difference in that kamailio still won't
process IP packets arriving on eth1 at IP 2.2.2.2... I can see OPTIONS coming with sngrep,
but nothing in the logs, no replies, only retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface with mhomed=1,
this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the transaction fails
with 477 Unfortunately error on sending to next hop occurred. In the log, kamailio prints
something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060
<http://9.9.9.9:5060/>,16): Invalid argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060
<http://2.2.2.2:5060/>) has been called before t_relay()
Also, I additionally found out that if the networking configuration is as follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
listen=vrf-red:5060 advertise 3.3.3.3:5060 <http://3.3.3.3:5060/>
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and 127.0.0.2) for
kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus slack
channel (cumulus contributed the VRF code to the linux kernel): "any app that has a
command line or option switch to call SO_BINDTODEVICE on the sockets can be configured to
use any VRF".
I guess the problem lies in that kamailio binds to several sockets and there are multiple
routing tables to consult, so mhomed doesn't work as expected (if every table has a
default route and nothing more specific than that). But still, this doesn't really
explain why binding to the VRF master net devices doesn't work (I mean, "ping -I
vrf-green" works just fine for ping)...
I don't think VRF support is very useful for most users though, I just had one corner
case I needed to tackle and VRF seemed like an easy fix. I'll think of an alternative
at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hello,
I haven't worked with vrf here, so no first hand experience ...
What happens if you try with next option?
listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
Cheers,
Daniel
On 25.09.17 19:10, George Diamantopoulos wrote:
Sorry to bump this, but it would be very useful if I knew whether there's any point
in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos <georgediam(a)gmail.com
<mailto:georgediam@gmail.com>> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
Hello,
I have a use case where I need to have kamailio bind to a VRF device. The configuration
in question is similar to the example below, where eth1 is a slave to the VRF-lite
device:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Both the main routing table and "vrf-green" routing table have a default
route.
What I need to be able to do is have kamailio bind to both interfaces:
listen=eth0:5060
listen=vrf-green:5060
And additionally be able to use force_send_socket to select an interface, for example:
force_send_socket(udp:2.2.2.2:5060 <http://2.2.2.2:5060/>);
However, I can't get this to work. The above configuration fails because there is no
listen directive for 2.2.2.2. Also, kamailio doesn't process packets received on the
VRF with the above listen directives, it behaves as if it doesn't listen on 2.2.2.2
indeed.
In addition using either of the below:
listen=udp:2.2.2.2:5060 <http://2.2.2.2:5060/>
or
listen=eth1:5060
fails with an error upon starting kamailio.
According to the kernel documentation:
Applications that are to work within a VRF need to bind their socket to the
VRF device:
setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
or to specify the output device using cmsg and IP_PKTINFO.
The question is, is VRF useable with kamailio right now? Or is development needed?
Thanks!
BR,
George
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
--
Daniel-Constantin Mierla
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Kamailio Advanced Training - www.asipto.com <http://www.asipto.com/>
Kamailio World Conference - www.kamailioworld.com <http://www.kamailioworld.com/>
6:21 p.m.
Hmm... Everything looks fine indeed, I can't see any obvious reason why
this isn't working...
Just to be clear, have you tried listening on all interfaces:
listen=udp:0.0.0.0:9060
And if yes, can you capture traffic arriving on the ZeroTier interface in
this case? If it works, is there any reason why listening on all interfaces
is not acceptable in your use case?
Regardless, it might be a good idea to start a new thread about this, maybe
it's related to the inner workings of the ZeroTier device and how it
interacts with the kernel...
On 12 October 2017 at 18:26, Mark Boyce <mark(a)darkorigins.com> wrote:
Hi George
(IP / MACs below anonymised a bit)
I’m working on a https://www.zerotier.com VPN endpoint which presents as
a new interface;
Real Interface
ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2508723344 (2.5 GB) TX bytes:1149701606 (1.1 GB)
loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
ZeroTier
zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
This is actually a SIPCapture/Homer node so the config is;
modparam("sipcapture", "capture_on", 1)
modparam("sipcapture", "hep_capture_on", 1)
listen=udp:10.0.0.1:9060
The port bind looks ok;
# netstat -plunt | grep kamailio
udp 0 0 10.0.0.1:9060 0.0.0.0:*
19592/kamailio
I’m tracking packets at the top of "route {“
route {
xlog("route entered\r\n”);
If I tcpdump the ZeroTier Interface ;
tcpdump -i zt0 -p -X port 9060
15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@..?.J..
0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 0216 ...r..#d.m......
0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 4b20 ....\.5#.J..ACK.
… but nothing seen in kamailio.
If I swap over to sending to / listening on the public IP everything works
as expected.
Cheers
Mark
On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello,
That seems strange... What kind of device is that VPN device? Like a ppp0
interface? What is the output of "netstat -plunt | grep kamailio"? Are you
using a "private" IP for the VPN interface and what's your main routing
table like?
Also what if you omit the "listen" directive in kamailio configuration?
Does it listen on the VPN interface then? Kamailio will bind by default to
all interfaces if no "listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com> wrote:
Hi All
Just to dive in here with something which may be related. I was about to
do a post with a much simpler problem where I have one real NIC and one VPN
created NIC. If I listen to the public IP all’s well. If I listen to the
IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on
either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the "listen"
directive will result in kamaiio no longer failing to start when using
force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
"advertise" option.
However, this doesn't make much of a difference in that kamailio still
won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
OPTIONS coming with sngrep, but nothing in the logs, no replies, only
retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface
with mhomed=1, this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the
transaction fails with 477 Unfortunately error on sending to next hop
occurred. In the log, kamailio prints something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16): Invalid
argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060)
has been called before t_relay()
Also, I additionally found out that if the networking configuration is as
follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060
listen=vrf-red:5060 advertise 3.3.3.3:5060
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and
127.0.0.2) for kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus
slack channel (cumulus contributed the VRF code to the linux kernel): "any
app that has a command line or option switch to call SO_BINDTODEVICE on the
sockets can be configured to use any VRF".
I guess the problem lies in that kamailio binds to several sockets and
there are multiple routing tables to consult, so mhomed doesn't work as
expected (if every table has a default route and nothing more specific than
that). But still, this doesn't really explain why binding to the VRF master
net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
ping)...
I don't think VRF support is very useful for most users though, I just
had one corner case I needed to tackle and VRF seemed like an easy fix.
I'll think of an alternative at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
miconda(a)gmail.com> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hello,
I haven't worked with vrf here, so no first hand experience ...
What happens if you try with next option?
listen=vrf-green:5060 advertise 2.2.2.2:5060
Cheers,
Daniel
On 25.09.17 19:10, George Diamantopoulos wrote:
Sorry to bump this, but it would be very useful if I knew whether
there's any
point in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos <
georgediam(a)gmail.com>
wrote:
>
> Hello,
>
> I have a use case where I need to have kamailio bind to a VRF device.
The
configuration in question is similar to the example below, where eth1
is a slave to the VRF-lite device:
>
> +----------+ +-------------------+
> | eth0 | | vrf-green |
> | 1.1.1.1 | | 127.0.0.1 |
> +----------+ +-------------------+
> |
> +----------+
> | eth1 |
> | 2.2.2.2 |
> +----------+
>
> Both the main routing table and "vrf-green" routing table have a
default route.
>
> What I need to be able to do is have kamailio bind to both interfaces:
>
> listen=eth0:5060
> listen=vrf-green:5060
>
> And additionally be able to use force_send_socket to select an
interface, for
example:
>
> force_send_socket(udp:2.2.2.2:5060);
>
> However, I can't get this to work. The above configuration fails
because
there is no listen directive for 2.2.2.2. Also, kamailio doesn't
process packets received on the VRF with the above listen directives, it
behaves as if it doesn't listen on 2.2.2.2 indeed.
>
> In addition using either of the below:
>
> listen=udp:2.2.2.2:5060
> or
> listen=eth1:5060
>
> fails with an error upon starting kamailio.
>
> According to the kernel documentation:
>
> Applications that are to work within a VRF need to bind their socket
to the
> VRF device:
>
> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>
> or to specify the output device using cmsg and IP_PKTINFO.
>
> The question is, is VRF useable with kamailio right now? Or is
development
needed? Thanks!
BR,
George
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com
7:26 p.m.
Hi
Now using listen=udp:0.0.0.0:9060
net stat agrees;
netstat -plunt | grep kamailio
udp 0 0 0.0.0.0:9060 0.0.0.0:*
20486/kamailio
TCPDump of Sending / Receiving to zero tier port
17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@..N.J..
0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 0211 ...r..#d..&.....
0x0020: 13c4 13d8 894a abd8 894a abd8 494e 5649 .....J...J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
Nothing in Kamailio Logs
Swap to using the real IP and I’m seeing the log line from the top of the main route;
Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
and the matching tcpdump from the public interface;
17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
0x0000: 4510 03d8 863c 0000 3911 6109 894a abd8 E....<..9.a..J..
0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 0216 3.-...#d........
0x0020: 07de 13c5 5c17 3523 894a abd8 494e 5649 ....\.5#.J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
Loading the dumped packets in to wireshark I can’t see any obvious difference.
Will start up a new thread … thanks for the loan of yours :-)
Mark
On 12 Oct 2017, at 17:21, George Diamantopoulos
<georgediam(a)gmail.com> wrote:
Hmm... Everything looks fine indeed, I can't see any obvious reason why this
isn't working...
Just to be clear, have you tried listening on all interfaces:
listen=udp:0.0.0.0:9060 <http://0.0.0.0:9060/>
And if yes, can you capture traffic arriving on the ZeroTier interface in this case? If
it works, is there any reason why listening on all interfaces is not acceptable in your
use case?
Regardless, it might be a good idea to start a new thread about this, maybe it's
related to the inner workings of the ZeroTier device and how it interacts with the
kernel...
On 12 October 2017 at 18:26, Mark Boyce <mark(a)darkorigins.com
<mailto:mark@darkorigins.com>> wrote:
Hi George
(IP / MACs below anonymised a bit)
I’m working on a https://www.zerotier.com <https://www.zerotier.com/> VPN endpoint
which presents as a new interface;
Real Interface
ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2508723344 (2.5 GB) TX bytes:1149701606 (1.1 GB)
loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
ZeroTier
zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
This is actually a SIPCapture/Homer node so the config is;
modparam("sipcapture", "capture_on", 1)
modparam("sipcapture", "hep_capture_on", 1)
listen=udp:10.0.0.1:9060 <http://10.0.0.1:9060/>
The port bind looks ok;
# netstat -plunt | grep kamailio
udp 0 0 10.0.0.1:9060 <http://10.0.0.1:9060/> 0.0.0.0:*
19592/kamailio
I’m tracking packets at the top of "route {“
route {
xlog("route entered\r\n”);
If I tcpdump the ZeroTier Interface ;
tcpdump -i zt0 -p -X port 9060
15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@..?.J..
0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 0216 ...r..#d.m......
0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 4b20 ....\.5#.J..ACK.
… but nothing seen in kamailio.
If I swap over to sending to / listening on the public IP everything works as expected.
Cheers
Mark
On 12 Oct 2017, at 15:56, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hello,
That seems strange... What kind of device is that VPN device? Like a ppp0 interface? What
is the output of "netstat -plunt | grep kamailio"? Are you using a
"private" IP for the VPN interface and what's your main routing table like?
Also what if you omit the "listen" directive in kamailio configuration? Does it
listen on the VPN interface then? Kamailio will bind by default to all interfaces if no
"listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com
<mailto:mark@darkorigins.com>> wrote:
Hi All
Just to dive in here with something which may be related. I was about to do a post with
a much simpler problem where I have one real NIC and one VPN created NIC. If I listen to
the public IP all’s well. If I listen to the IP of the VPN NIC Kamailio doesn’t see any
traffic. I can tcpdump on either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users On 11 Oct 2017, at 22:07, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the "listen"
directive will result in kamaiio no longer failing to start when using
force_send_socket(2.2.2.2:5060 <http://2.2.2.2:5060/>) with 2.2.2.2 being used in
the "advertise" option.
However, this doesn't make much of a difference in that kamailio still won't
process IP packets arriving on eth1 at IP 2.2.2.2... I can see OPTIONS coming with sngrep,
but nothing in the logs, no replies, only retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface with mhomed=1,
this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the transaction fails
with 477 Unfortunately error on sending to next hop occurred. In the log, kamailio prints
something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060
<http://9.9.9.9:5060/>,16): Invalid argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060
<http://2.2.2.2:5060/>) has been called before t_relay()
Also, I additionally found out that if the networking configuration is as follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
listen=vrf-red:5060 advertise 3.3.3.3:5060 <http://3.3.3.3:5060/>
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and 127.0.0.2) for
kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus slack
channel (cumulus contributed the VRF code to the linux kernel): "any app that has a
command line or option switch to call SO_BINDTODEVICE on the sockets can be configured to
use any VRF".
I guess the problem lies in that kamailio binds to several sockets and there are multiple
routing tables to consult, so mhomed doesn't work as expected (if every table has a
default route and nothing more specific than that). But still, this doesn't really
explain why binding to the VRF master net devices doesn't work (I mean, "ping -I
vrf-green" works just fine for ping)...
I don't think VRF support is very useful for most users though, I just had one corner
case I needed to tackle and VRF seemed like an easy fix. I'll think of an alternative
at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
Hello,
I haven't worked with vrf here, so no first hand experience ...
What happens if you try with next option?
listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
Cheers,
Daniel
On 25.09.17 19:10, George Diamantopoulos wrote:
Sorry to bump this, but it would be very useful if I knew whether there's any point
in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos <georgediam(a)gmail.com
<mailto:georgediam@gmail.com>> wrote:
>
> Hello,
>
> I have a use case where I need to have kamailio bind to a VRF device. The
configuration in question is similar to the example below, where eth1 is a slave to the
VRF-lite device:
>
> +----------+ +-------------------+
> | eth0 | | vrf-green |
> | 1.1.1.1 | | 127.0.0.1 |
> +----------+ +-------------------+
> |
> +----------+
> | eth1 |
> | 2.2.2.2 |
> +----------+
>
> Both the main routing table and "vrf-green" routing table have a default
route.
>
> What I need to be able to do is have kamailio bind to both interfaces:
>
> listen=eth0:5060
> listen=vrf-green:5060
>
> And additionally be able to use force_send_socket to select an interface, for
example:
>
> force_send_socket(udp:2.2.2.2:5060 <http://2.2.2.2:5060/>);
>
> However, I can't get this to work. The above configuration fails because there is
no listen directive for 2.2.2.2. Also, kamailio doesn't process packets received on
the VRF with the above listen directives, it behaves as if it doesn't listen on
2.2.2.2 indeed.
>
> In addition using either of the below:
>
> listen=udp:2.2.2.2:5060 <http://2.2.2.2:5060/>
> or
> listen=eth1:5060
>
> fails with an error upon starting kamailio.
>
> According to the kernel documentation:
>
> Applications that are to work within a VRF need to bind their socket to the
> VRF device:
>
> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>
> or to specify the output device using cmsg and IP_PKTINFO.
>
> The question is, is VRF useable with kamailio right now? Or is development needed?
Thanks!
>
> BR,
>
> George
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
--
Daniel-Constantin Mierla
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Kamailio Advanced Training - www.asipto.com <http://www.asipto.com/>
Kamailio World Conference - www.kamailioworld.com <http://www.kamailioworld.com/>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
7:31 p.m.
Just one last thing, are you absolutely certain there's no firewall doing
the damage there? It's quite common for firewall-blocked traffic to show up
with tcpdump, but never reach the application...
On 12 October 2017 at 20:26, Mark Boyce <mark(a)darkorigins.com> wrote:
Hi
Now using listen=udp:0.0.0.0:9060
net stat agrees;
netstat -plunt | grep kamailio
udp 0 0 0.0.0.0:9060 0.0.0.0:*
20486/kamailio
TCPDump of Sending / Receiving to zero tier port
17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@..N.J..
0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 0211 ...r..#d..&.....
0x0020: 13c4 13d8 894a abd8 894a abd8 494e 5649 .....J...J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
Nothing in Kamailio Logs
Swap to using the real IP and I’m seeing the log line from the top of the
main route;
Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
and the matching tcpdump from the public interface;
17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
0x0000: 4510 03d8 863c 0000 3911 6109 894a abd8 E....<..9.a..J..
0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 0216 3.-...#d........
0x0020: 07de 13c5 5c17 3523 894a abd8 494e 5649 ....\.5#.J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
Loading the dumped packets in to wireshark I can’t see any obvious
difference.
Will start up a new thread … thanks for the loan of yours :-)
Mark
On 12 Oct 2017, at 17:21, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hmm... Everything looks fine indeed, I can't see any obvious reason why
this isn't working...
Just to be clear, have you tried listening on all interfaces:
listen=udp:0.0.0.0:9060
And if yes, can you capture traffic arriving on the ZeroTier interface in
this case? If it works, is there any reason why listening on all interfaces
is not acceptable in your use case?
Regardless, it might be a good idea to start a new thread about this,
maybe it's related to the inner workings of the ZeroTier device and how it
interacts with the kernel...
On 12 October 2017 at 18:26, Mark Boyce <mark(a)darkorigins.com> wrote:
Hi George
(IP / MACs below anonymised a bit)
I’m working on a https://www.zerotier.com VPN endpoint which presents as
a new interface;
Real Interface
ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2508723344 (2.5 GB) TX bytes:1149701606 (1.1 GB)
loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
ZeroTier
zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
This is actually a SIPCapture/Homer node so the config is;
modparam("sipcapture", "capture_on", 1)
modparam("sipcapture", "hep_capture_on", 1)
listen=udp:10.0.0.1:9060
The port bind looks ok;
# netstat -plunt | grep kamailio
udp 0 0 10.0.0.1:9060 0.0.0.0:*
19592/kamailio
I’m tracking packets at the top of "route {“
route {
xlog("route entered\r\n”);
If I tcpdump the ZeroTier Interface ;
tcpdump -i zt0 -p -X port 9060
15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@
..?.J..
0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 0216 ...r..#d.m......
0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 4b20 ....\.5#.J..ACK.
… but nothing seen in kamailio.
If I swap over to sending to / listening on the public IP everything
works as expected.
Cheers
Mark
On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello,
That seems strange... What kind of device is that VPN device? Like a ppp0
interface? What is the output of "netstat -plunt | grep kamailio"? Are you
using a "private" IP for the VPN interface and what's your main routing
table like?
Also what if you omit the "listen" directive in kamailio configuration?
Does it listen on the VPN interface then? Kamailio will bind by default to
all interfaces if no "listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi All
Just to dive in here with something which may be related. I was about
to do a post with a much simpler problem where I have one real NIC and one
VPN created NIC. If I listen to the public IP all’s well. If I listen to
the IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on
either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the
"listen" directive will result in kamaiio no longer failing to start when
using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
"advertise" option.
However, this doesn't make much of a difference in that kamailio still
won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
OPTIONS coming with sngrep, but nothing in the logs, no replies, only
retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface
with mhomed=1, this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the
transaction fails with 477 Unfortunately error on sending to next hop
occurred. In the log, kamailio prints something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16): Invalid
argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060)
has been called before t_relay()
Also, I additionally found out that if the networking configuration is
as follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060
listen=vrf-red:5060 advertise 3.3.3.3:5060
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and
127.0.0.2) for kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus
slack channel (cumulus contributed the VRF code to the linux kernel): "any
app that has a command line or option switch to call SO_BINDTODEVICE on the
sockets can be configured to use any VRF".
I guess the problem lies in that kamailio binds to several sockets and
there are multiple routing tables to consult, so mhomed doesn't work as
expected (if every table has a default route and nothing more specific than
that). But still, this doesn't really explain why binding to the VRF master
net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
ping)...
I don't think VRF support is very useful for most users though, I just
had one corner case I needed to tackle and VRF seemed like an easy fix.
I'll think of an alternative at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
miconda(a)gmail.com> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hello,
I haven't worked with vrf here, so no first hand experience ...
What happens if you try with next option?
listen=vrf-green:5060 advertise 2.2.2.2:5060
Cheers,
Daniel
On 25.09.17 19:10, George Diamantopoulos wrote:
Sorry to bump this, but it would be very useful if I knew whether
there's any
point in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos <
georgediam(a)gmail.com>
wrote:
>
> Hello,
>
> I have a use case where I need to have kamailio bind to a VRF device.
The
configuration in question is similar to the example below, where eth1
is a slave to the VRF-lite device:
>
> +----------+ +-------------------+
> | eth0 | | vrf-green |
> | 1.1.1.1 | | 127.0.0.1 |
> +----------+ +-------------------+
> |
> +----------+
> | eth1 |
> | 2.2.2.2 |
> +----------+
>
> Both the main routing table and "vrf-green" routing table have a
default route.
>
> What I need to be able to do is have kamailio bind to both interfaces:
>
> listen=eth0:5060
> listen=vrf-green:5060
>
> And additionally be able to use force_send_socket to select an
interface, for
example:
>
> force_send_socket(udp:2.2.2.2:5060);
>
> However, I can't get this to work. The above configuration fails
because
there is no listen directive for 2.2.2.2. Also, kamailio doesn't
process packets received on the VRF with the above listen directives, it
behaves as if it doesn't listen on 2.2.2.2 indeed.
>
> In addition using either of the below:
>
> listen=udp:2.2.2.2:5060
> or
> listen=eth1:5060
>
> fails with an error upon starting kamailio.
>
> According to the kernel documentation:
>
> Applications that are to work within a VRF need to bind their socket
to the
> VRF device:
>
> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>
> or to specify the output device using cmsg and IP_PKTINFO.
>
> The question is, is VRF useable with kamailio right now? Or is
development
needed? Thanks!
>
> BR,
>
> George
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
8:12 p.m.
Hi
I had the same thought .. but if there is I can’t see it. Running tcpdump with the -p
Don’t Use Promiscuous Mode still sees the traffic.
Mark
On 12 Oct 2017, at 18:31, George Diamantopoulos
<georgediam(a)gmail.com> wrote:
Just one last thing, are you absolutely certain there's no firewall doing the damage
there? It's quite common for firewall-blocked traffic to show up with tcpdump, but
never reach the application...
On 12 October 2017 at 20:26, Mark Boyce <mark(a)darkorigins.com
<mailto:mark@darkorigins.com>> wrote:
Hi
Now using listen=udp:0.0.0.0:9060 <http://0.0.0.0:9060/>
net stat agrees;
netstat -plunt | grep kamailio
udp 0 0 0.0.0.0:9060 <http://0.0.0.0:9060/> 0.0.0.0:*
20486/kamailio
TCPDump of Sending / Receiving to zero tier port
17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@..N.J..
0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 0211 ...r..#d..&.....
0x0020: 13c4 13d8 894a abd8 894a abd8 494e 5649 .....J...J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
Nothing in Kamailio Logs
Swap to using the real IP and I’m seeing the log line from the top of the main route;
Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
and the matching tcpdump from the public interface;
17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
0x0000: 4510 03d8 863c 0000 3911 6109 894a abd8 E....<..9.a..J..
0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 0216 3.-...#d........
0x0020: 07de 13c5 5c17 3523 894a abd8 494e 5649 ....\.5#.J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
<mailto:2@sbc-1.w>
Loading the dumped packets in to wireshark I can’t see any obvious difference.
Will start up a new thread … thanks for the loan of yours :-)
Mark
On 12 Oct 2017, at 17:21, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hmm... Everything looks fine indeed, I can't see any obvious reason why this
isn't working...
Just to be clear, have you tried listening on all interfaces:
listen=udp:0.0.0.0:9060 <http://0.0.0.0:9060/>
And if yes, can you capture traffic arriving on the ZeroTier interface in this case? If
it works, is there any reason why listening on all interfaces is not acceptable in your
use case?
Regardless, it might be a good idea to start a new thread about this, maybe it's
related to the inner workings of the ZeroTier device and how it interacts with the
kernel...
On 12 October 2017 at 18:26, Mark Boyce <mark(a)darkorigins.com
<mailto:mark@darkorigins.com>> wrote:
Hi George
(IP / MACs below anonymised a bit)
I’m working on a https://www.zerotier.com <https://www.zerotier.com/> VPN endpoint
which presents as a new interface;
Real Interface
ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2508723344 (2.5 GB) TX bytes:1149701606 (1.1 GB)
loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
ZeroTier
zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
This is actually a SIPCapture/Homer node so the config is;
modparam("sipcapture", "capture_on", 1)
modparam("sipcapture", "hep_capture_on", 1)
listen=udp:10.0.0.1:9060 <http://10.0.0.1:9060/>
The port bind looks ok;
# netstat -plunt | grep kamailio
udp 0 0 10.0.0.1:9060 <http://10.0.0.1:9060/> 0.0.0.0:*
19592/kamailio
I’m tracking packets at the top of "route {“
route {
xlog("route entered\r\n”);
If I tcpdump the ZeroTier Interface ;
tcpdump -i zt0 -p -X port 9060
15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@..?.J..
0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 0216 ...r..#d.m......
0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 4b20 ....\.5#.J..ACK.
… but nothing seen in kamailio.
If I swap over to sending to / listening on the public IP everything works as expected.
Cheers
Mark
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users On 12 Oct 2017, at 15:56, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hello,
That seems strange... What kind of device is that VPN device? Like a ppp0 interface? What
is the output of "netstat -plunt | grep kamailio"? Are you using a
"private" IP for the VPN interface and what's your main routing table like?
Also what if you omit the "listen" directive in kamailio configuration? Does it
listen on the VPN interface then? Kamailio will bind by default to all interfaces if no
"listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com
<mailto:mark@darkorigins.com>> wrote:
Hi All
Just to dive in here with something which may be related. I was about to do a post with
a much simpler problem where I have one real NIC and one VPN created NIC. If I listen to
the public IP all’s well. If I listen to the IP of the VPN NIC Kamailio doesn’t see any
traffic. I can tcpdump on either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users> On 11 Oct 2017, at 22:07, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the "listen"
directive will result in kamaiio no longer failing to start when using
force_send_socket(2.2.2.2:5060 <http://2.2.2.2:5060/>) with 2.2.2.2 being used in
the "advertise" option.
However, this doesn't make much of a difference in that kamailio still won't
process IP packets arriving on eth1 at IP 2.2.2.2... I can see OPTIONS coming with sngrep,
but nothing in the logs, no replies, only retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface with mhomed=1,
this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the transaction fails
with 477 Unfortunately error on sending to next hop occurred. In the log, kamailio prints
something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060
<http://9.9.9.9:5060/>,16): Invalid argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060
<http://2.2.2.2:5060/>) has been called before t_relay()
Also, I additionally found out that if the networking configuration is as follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
listen=vrf-red:5060 advertise 3.3.3.3:5060 <http://3.3.3.3:5060/>
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and 127.0.0.2) for
kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus slack
channel (cumulus contributed the VRF code to the linux kernel): "any app that has a
command line or option switch to call SO_BINDTODEVICE on the sockets can be configured to
use any VRF".
I guess the problem lies in that kamailio binds to several sockets and there are multiple
routing tables to consult, so mhomed doesn't work as expected (if every table has a
default route and nothing more specific than that). But still, this doesn't really
explain why binding to the VRF master net devices doesn't work (I mean, "ping -I
vrf-green" works just fine for ping)...
I don't think VRF support is very useful for most users though, I just had one corner
case I needed to tackle and VRF seemed like an easy fix. I'll think of an alternative
at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>> wrote:
>
> Hello,
>
> I haven't worked with vrf here, so no first hand experience ...
>
> What happens if you try with next option?
>
> listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
>
> Cheers,
> Daniel
>
>
> On 25.09.17 19:10, George Diamantopoulos wrote:
>
> Sorry to bump this, but it would be very useful if I knew whether there's any
point in pursuing this or not. Any hints?
>
> On 21 September 2017 at 14:06, George Diamantopoulos <georgediam(a)gmail.com
<mailto:georgediam@gmail.com>> wrote:
>>
>> Hello,
>>
>> I have a use case where I need to have kamailio bind to a VRF device. The
configuration in question is similar to the example below, where eth1 is a slave to the
VRF-lite device:
>>
>> +----------+ +-------------------+
>> | eth0 | | vrf-green |
>> | 1.1.1.1 | | 127.0.0.1 |
>> +----------+ +-------------------+
>> |
>> +----------+
>> | eth1 |
>> | 2.2.2.2 |
>> +----------+
>>
>> Both the main routing table and "vrf-green" routing table have a
default route.
>>
>> What I need to be able to do is have kamailio bind to both interfaces:
>>
>> listen=eth0:5060
>> listen=vrf-green:5060
>>
>> And additionally be able to use force_send_socket to select an interface, for
example:
>>
>> force_send_socket(udp:2.2.2.2:5060 <http://2.2.2.2:5060/>);
>>
>> However, I can't get this to work. The above configuration fails because
there is no listen directive for 2.2.2.2. Also, kamailio doesn't process packets
received on the VRF with the above listen directives, it behaves as if it doesn't
listen on 2.2.2.2 indeed.
>>
>> In addition using either of the below:
>>
>> listen=udp:2.2.2.2:5060 <http://2.2.2.2:5060/>
>> or
>> listen=eth1:5060
>>
>> fails with an error upon starting kamailio.
>>
>> According to the kernel documentation:
>>
>> Applications that are to work within a VRF need to bind their socket to the
>> VRF device:
>>
>> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>>
>> or to specify the output device using cmsg and IP_PKTINFO.
>>
>> The question is, is VRF useable with kamailio right now? Or is development
needed? Thanks!
>>
>> BR,
>>
>> George
>
>
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>
>
> --
> Daniel-Constantin Mierla
> www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
> Kamailio Advanced Training - www.asipto.com <http://www.asipto.com/>
> Kamailio World Conference - www.kamailioworld.com
<http://www.kamailioworld.com/>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users> 
8:59 p.m.
This may be NAT/iptables issue if used some type of virtualization
Check UNDEPLIED connection using conntrack
Sergey
чт, 12 окт. 2017 г. в 20:27, Mark Boyce <mark(a)darkorigins.com>om>:
Hi
Now using listen=udp:0.0.0.0:9060
net stat agrees;
netstat -plunt | grep kamailio
udp 0 0 0.0.0.0:9060 0.0.0.0:*
20486/kamailio
TCPDump of Sending / Receiving to zero tier port
17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@..N.J..
0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 0211 ...r..#d..&.....
0x0020: 13c4 13d8 894a abd8 894a abd8 494e 5649 .....J...J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
Nothing in Kamailio Logs
Swap to using the real IP and I’m seeing the log line from the top of the
main route;
Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
and the matching tcpdump from the public interface;
17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
0x0000: 4510 03d8 863c 0000 3911 6109 894a abd8 E....<..9.a..J..
0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 0216 3.-...#d........
0x0020: 07de 13c5 5c17 3523 894a abd8 494e 5649 ....\.5#.J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
Loading the dumped packets in to wireshark I can’t see any obvious
difference.
Will start up a new thread … thanks for the loan of yours :-)
Mark
On 12 Oct 2017, at 17:21, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hmm... Everything looks fine indeed, I can't see any obvious reason why
this isn't working...
Just to be clear, have you tried listening on all interfaces:
listen=udp:0.0.0.0:9060
And if yes, can you capture traffic arriving on the ZeroTier interface in
this case? If it works, is there any reason why listening on all interfaces
is not acceptable in your use case?
Regardless, it might be a good idea to start a new thread about this,
maybe it's related to the inner workings of the ZeroTier device and how it
interacts with the kernel...
On 12 October 2017 at 18:26, Mark Boyce <mark(a)darkorigins.com> wrote:
Hi George
(IP / MACs below anonymised a bit)
I’m working on a https://www.zerotier.com VPN endpoint which presents as
a new interface;
Real Interface
ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2508723344 <(250)%20872-3344> (2.5 GB) TX
bytes:1149701606 (1.1 GB)
loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
ZeroTier
zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
This is actually a SIPCapture/Homer node so the config is;
modparam("sipcapture", "capture_on", 1)
modparam("sipcapture", "hep_capture_on", 1)
listen=udp:10.0.0.1:9060
The port bind looks ok;
# netstat -plunt | grep kamailio
udp 0 0 10.0.0.1:9060 0.0.0.0:*
19592/kamailio
I’m tracking packets at the top of "route {“
route {
xlog("route entered\r\n”);
If I tcpdump the ZeroTier Interface ;
tcpdump -i zt0 -p -X port 9060
15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@
..?.J..
0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 0216 ...r..#d.m......
0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 4b20 ....\.5#.J..ACK.
… but nothing seen in kamailio.
If I swap over to sending to / listening on the public IP everything
works as expected.
Cheers
Mark
On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello,
That seems strange... What kind of device is that VPN device? Like a ppp0
interface? What is the output of "netstat -plunt | grep kamailio"? Are you
using a "private" IP for the VPN interface and what's your main routing
table like?
Also what if you omit the "listen" directive in kamailio configuration?
Does it listen on the VPN interface then? Kamailio will bind by default to
all interfaces if no "listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi All
Just to dive in here with something which may be related. I was about
to do a post with a much simpler problem where I have one real NIC and one
VPN created NIC. If I listen to the public IP all’s well. If I listen to
the IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on
either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the
"listen" directive will result in kamaiio no longer failing to start when
using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
"advertise" option.
However, this doesn't make much of a difference in that kamailio still
won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
OPTIONS coming with sngrep, but nothing in the logs, no replies, only
retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface
with mhomed=1, this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the
transaction fails with 477 Unfortunately error on sending to next hop
occurred. In the log, kamailio prints something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16): Invalid
argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060)
has been called before t_relay()
Also, I additionally found out that if the networking configuration is
as follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060
listen=vrf-red:5060 advertise 3.3.3.3:5060
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and
127.0.0.2) for kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus
slack channel (cumulus contributed the VRF code to the linux kernel): "any
app that has a command line or option switch to call SO_BINDTODEVICE on the
sockets can be configured to use any VRF".
I guess the problem lies in that kamailio binds to several sockets and
there are multiple routing tables to consult, so mhomed doesn't work as
expected (if every table has a default route and nothing more specific than
that). But still, this doesn't really explain why binding to the VRF master
net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
ping)...
I don't think VRF support is very useful for most users though, I just
had one corner case I needed to tackle and VRF seemed like an easy fix.
I'll think of an alternative at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
miconda(a)gmail.com> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hello,
I haven't worked with vrf here, so no first hand experience ...
What happens if you try with next option?
listen=vrf-green:5060 advertise 2.2.2.2:5060
Cheers,
Daniel
On 25.09.17 19:10, George Diamantopoulos wrote:
Sorry to bump this, but it would be very useful if I knew whether
there's any
point in pursuing this or not. Any hints?
On 21 September 2017 at 14:06, George Diamantopoulos <
georgediam(a)gmail.com>
wrote:
>
> Hello,
>
> I have a use case where I need to have kamailio bind to a VRF device.
The
configuration in question is similar to the example below, where eth1
is a slave to the VRF-lite device:
>
> +----------+ +-------------------+
> | eth0 | | vrf-green |
> | 1.1.1.1 | | 127.0.0.1 |
> +----------+ +-------------------+
> |
> +----------+
> | eth1 |
> | 2.2.2.2 |
> +----------+
>
> Both the main routing table and "vrf-green" routing table have a
default route.
>
> What I need to be able to do is have kamailio bind to both interfaces:
>
> listen=eth0:5060
> listen=vrf-green:5060
>
> And additionally be able to use force_send_socket to select an
interface, for
example:
>
> force_send_socket(udp:2.2.2.2:5060);
>
> However, I can't get this to work. The above configuration fails
because
there is no listen directive for 2.2.2.2. Also, kamailio doesn't
process packets received on the VRF with the above listen directives, it
behaves as if it doesn't listen on 2.2.2.2 indeed.
>
> In addition using either of the below:
>
> listen=udp:2.2.2.2:5060
> or
> listen=eth1:5060
>
> fails with an error upon starting kamailio.
>
> According to the kernel documentation:
>
> Applications that are to work within a VRF need to bind their socket
to the
> VRF device:
>
> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>
> or to specify the output device using cmsg and IP_PKTINFO.
>
> The question is, is VRF useable with kamailio right now? Or is
development
needed? Thanks!
>
> BR,
>
> George
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
9:06 p.m.
Hi
I *think* I may have found it … although still not sure what’s doing the filtering.
If I force the sending IP to be the the VPN interface on the sender by adding;
modparam("siptrace", "force_send_sock", "sip:10.0.0.2:5060”)
So packets now arrive at the receiver from the 10. IP rather than from the public IP of
the sender
… and magically it starts working.
So something in Ubuntu 16.04 / zerotier is “filtering” packets which arrive on the VPN
interface but with a no VPN IP
Cheers
Mark
On 12 Oct 2017, at 19:59, Sergey Safarov
<s.safarov(a)gmail.com> wrote:
This may be NAT/iptables issue if used some type of virtualization
Check UNDEPLIED connection using conntrack
Sergey
чт, 12 окт. 2017 г. в 20:27, Mark Boyce <mark(a)darkorigins.com
<mailto:mark@darkorigins.com>>:
Hi
Now using listen=udp:0.0.0.0:9060 <http://0.0.0.0:9060/>
net stat agrees;
netstat -plunt | grep kamailio
udp 0 0 0.0.0.0:9060 <http://0.0.0.0:9060/> 0.0.0.0:*
20486/kamailio
TCPDump of Sending / Receiving to zero tier port
17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@..N.J..
0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 0211 ...r..#d..&.....
0x0020: 13c4 13d8 894a abd8 894a abd8 494e 5649 .....J...J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
Nothing in Kamailio Logs
Swap to using the real IP and I’m seeing the log line from the top of the main route;
Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
and the matching tcpdump from the public interface;
17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
0x0000: 4510 03d8 863c 0000 3911 6109 894a abd8 E....<..9.a..J..
0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 0216 3.-...#d........
0x0020: 07de 13c5 5c17 3523 894a abd8 494e 5649 ....\.5#.J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w
<mailto:2@sbc-1.w>
Loading the dumped packets in to wireshark I can’t see any obvious difference.
Will start up a new thread … thanks for the loan of yours :-)
Mark
On 12 Oct 2017, at 17:21, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hmm... Everything looks fine indeed, I can't see any obvious reason why this
isn't working...
Just to be clear, have you tried listening on all interfaces:
listen=udp:0.0.0.0:9060 <http://0.0.0.0:9060/>
And if yes, can you capture traffic arriving on the ZeroTier interface in this case? If
it works, is there any reason why listening on all interfaces is not acceptable in your
use case?
Regardless, it might be a good idea to start a new thread about this, maybe it's
related to the inner workings of the ZeroTier device and how it interacts with the
kernel...
On 12 October 2017 at 18:26, Mark Boyce <mark(a)darkorigins.com
<mailto:mark@darkorigins.com>> wrote:
Hi George
(IP / MACs below anonymised a bit)
I’m working on a https://www.zerotier.com <https://www.zerotier.com/> VPN endpoint
which presents as a new interface;
Real Interface
ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2508723344 <tel:(250)%20872-3344> (2.5 GB) TX bytes:1149701606
(1.1 GB)
loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
ZeroTier
zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
This is actually a SIPCapture/Homer node so the config is;
modparam("sipcapture", "capture_on", 1)
modparam("sipcapture", "hep_capture_on", 1)
listen=udp:10.0.0.1:9060 <http://10.0.0.1:9060/>
The port bind looks ok;
# netstat -plunt | grep kamailio
udp 0 0 10.0.0.1:9060 <http://10.0.0.1:9060/> 0.0.0.0:*
19592/kamailio
I’m tracking packets at the top of "route {“
route {
xlog("route entered\r\n”);
If I tcpdump the ZeroTier Interface ;
tcpdump -i zt0 -p -X port 9060
15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@..?.J..
0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 0216 ...r..#d.m......
0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 4b20 ....\.5#.J..ACK.
… but nothing seen in kamailio.
If I swap over to sending to / listening on the public IP everything works as expected.
Cheers
Mark
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users On 12 Oct 2017, at 15:56, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hello,
That seems strange... What kind of device is that VPN device? Like a ppp0 interface? What
is the output of "netstat -plunt | grep kamailio"? Are you using a
"private" IP for the VPN interface and what's your main routing table like?
Also what if you omit the "listen" directive in kamailio configuration? Does it
listen on the VPN interface then? Kamailio will bind by default to all interfaces if no
"listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com
<mailto:mark@darkorigins.com>> wrote:
Hi All
Just to dive in here with something which may be related. I was about to do a post with
a much simpler problem where I have one real NIC and one VPN created NIC. If I listen to
the public IP all’s well. If I listen to the IP of the VPN NIC Kamailio doesn’t see any
traffic. I can tcpdump on either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users> On 11 Oct 2017, at 22:07, George Diamantopoulos
<georgediam(a)gmail.com <mailto:georgediam@gmail.com>> wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the "listen"
directive will result in kamaiio no longer failing to start when using
force_send_socket(2.2.2.2:5060 <http://2.2.2.2:5060/>) with 2.2.2.2 being used in
the "advertise" option.
However, this doesn't make much of a difference in that kamailio still won't
process IP packets arriving on eth1 at IP 2.2.2.2... I can see OPTIONS coming with sngrep,
but nothing in the logs, no replies, only retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface with mhomed=1,
this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the transaction fails
with 477 Unfortunately error on sending to next hop occurred. In the log, kamailio prints
something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060
<http://9.9.9.9:5060/>,16): Invalid argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060
<http://2.2.2.2:5060/>) has been called before t_relay()
Also, I additionally found out that if the networking configuration is as follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
listen=vrf-red:5060 advertise 3.3.3.3:5060 <http://3.3.3.3:5060/>
One needs to use different IPs for the MASTER devices (say 127.0.0.1 and 127.0.0.2) for
kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus slack
channel (cumulus contributed the VRF code to the linux kernel): "any app that has a
command line or option switch to call SO_BINDTODEVICE on the sockets can be configured to
use any VRF".
I guess the problem lies in that kamailio binds to several sockets and there are multiple
routing tables to consult, so mhomed doesn't work as expected (if every table has a
default route and nothing more specific than that). But still, this doesn't really
explain why binding to the VRF master net devices doesn't work (I mean, "ping -I
vrf-green" works just fine for ping)...
I don't think VRF support is very useful for most users though, I just had one corner
case I needed to tackle and VRF seemed like an easy fix. I'll think of an alternative
at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>> wrote:
>
> Hello,
>
> I haven't worked with vrf here, so no first hand experience ...
>
> What happens if you try with next option?
>
> listen=vrf-green:5060 advertise 2.2.2.2:5060 <http://2.2.2.2:5060/>
>
> Cheers,
> Daniel
>
>
> On 25.09.17 19:10, George Diamantopoulos wrote:
>
> Sorry to bump this, but it would be very useful if I knew whether there's any
point in pursuing this or not. Any hints?
>
> On 21 September 2017 at 14:06, George Diamantopoulos <georgediam(a)gmail.com
<mailto:georgediam@gmail.com>> wrote:
>>
>> Hello,
>>
>> I have a use case where I need to have kamailio bind to a VRF device. The
configuration in question is similar to the example below, where eth1 is a slave to the
VRF-lite device:
>>
>> +----------+ +-------------------+
>> | eth0 | | vrf-green |
>> | 1.1.1.1 | | 127.0.0.1 |
>> +----------+ +-------------------+
>> |
>> +----------+
>> | eth1 |
>> | 2.2.2.2 |
>> +----------+
>>
>> Both the main routing table and "vrf-green" routing table have a
default route.
>>
>> What I need to be able to do is have kamailio bind to both interfaces:
>>
>> listen=eth0:5060
>> listen=vrf-green:5060
>>
>> And additionally be able to use force_send_socket to select an interface, for
example:
>>
>> force_send_socket(udp:2.2.2.2:5060 <http://2.2.2.2:5060/>);
>>
>> However, I can't get this to work. The above configuration fails because
there is no listen directive for 2.2.2.2. Also, kamailio doesn't process packets
received on the VRF with the above listen directives, it behaves as if it doesn't
listen on 2.2.2.2 indeed.
>>
>> In addition using either of the below:
>>
>> listen=udp:2.2.2.2:5060 <http://2.2.2.2:5060/>
>> or
>> listen=eth1:5060
>>
>> fails with an error upon starting kamailio.
>>
>> According to the kernel documentation:
>>
>> Applications that are to work within a VRF need to bind their socket to the
>> VRF device:
>>
>> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>>
>> or to specify the output device using cmsg and IP_PKTINFO.
>>
>> The question is, is VRF useable with kamailio right now? Or is development
needed? Thanks!
>>
>> BR,
>>
>> George
>
>
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>
>
> --
> Daniel-Constantin Mierla
> www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
> Kamailio Advanced Training - www.asipto.com <http://www.asipto.com/>
> Kamailio World Conference - www.kamailioworld.com
<http://www.kamailioworld.com/>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users> 
14 Oct
14 Oct
12:22 a.m.
Hi George,
Did you have any luck getting VRFs working with Kamailio?
Thanks
Matthew
On Thu, Oct 12, 2017 at 8:06 PM, Mark Boyce <mark(a)darkorigins.com> wrote:
Hi
I *think* I may have found it … although still not sure what’s doing the
filtering.
If I force the sending IP to be the the VPN interface on the sender by
adding;
modparam("siptrace", "force_send_sock", "sip:10.0.0.2:5060”)
So packets now arrive at the receiver from the 10. IP rather than from the
public IP of the sender
… and magically it starts working.
So something in Ubuntu 16.04 / zerotier is “filtering” packets which
arrive on the VPN interface but with a no VPN IP
Cheers
Mark
On 12 Oct 2017, at 19:59, Sergey Safarov <s.safarov(a)gmail.com> wrote:
This may be NAT/iptables issue if used some type of virtualization
Check UNDEPLIED connection using conntrack
Sergey
чт, 12 окт. 2017 г. в 20:27, Mark Boyce <mark(a)darkorigins.com>om>:
Hi
Now using listen=udp:0.0.0.0:9060
net stat agrees;
netstat -plunt | grep kamailio
udp 0 0 0.0.0.0:9060 0.0.0.0:*
20486/kamailio
TCPDump of Sending / Receiving to zero tier port
17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@
..N.J..
0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 0211 ...r..#d..&.....
0x0020: 13c4 13d8 894a abd8 894a abd8 494e 5649 .....J...J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
2(a)sbc-1.w
Nothing in Kamailio Logs
Swap to using the real IP and I’m seeing the log line from the top of the
main route;
Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
and the matching tcpdump from the public interface;
17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
0x0000: 4510 03d8 863c 0000 3911 6109 894a abd8 E....<..9.a..J..
0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 0216 3.-...#d........
0x0020: 07de 13c5 5c17 3523 894a abd8 494e 5649 ....\.5#.J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
2(a)sbc-1.w
Loading the dumped packets in to wireshark I can’t see any obvious
difference.
Will start up a new thread … thanks for the loan of yours :-)
Mark
On 12 Oct 2017, at 17:21, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hmm... Everything looks fine indeed, I can't see any obvious reason why
this isn't working...
Just to be clear, have you tried listening on all interfaces:
listen=udp:0.0.0.0:9060
And if yes, can you capture traffic arriving on the ZeroTier interface in
this case? If it works, is there any reason why listening on all interfaces
is not acceptable in your use case?
Regardless, it might be a good idea to start a new thread about this,
maybe it's related to the inner workings of the ZeroTier device and how it
interacts with the kernel...
On 12 October 2017 at 18:26, Mark Boyce <mark(a)darkorigins.com> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi George
(IP / MACs below anonymised a bit)
I’m working on a https://www.zerotier.com VPN endpoint which presents
as a new interface;
Real Interface
ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2508723344 <(250)%20872-3344> (2.5 GB) TX
bytes:1149701606 (1.1 GB)
loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
ZeroTier
zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
This is actually a SIPCapture/Homer node so the config is;
modparam("sipcapture", "capture_on", 1)
modparam("sipcapture", "hep_capture_on", 1)
listen=udp:10.0.0.1:9060
The port bind looks ok;
# netstat -plunt | grep kamailio
udp 0 0 10.0.0.1:9060 0.0.0.0:*
19592/kamailio
I’m tracking packets at the top of "route {“
route {
xlog("route entered\r\n”);
If I tcpdump the ZeroTier Interface ;
tcpdump -i zt0 -p -X port 9060
15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@
..?.J..
0x0010: 0a05 0172 13c4 2364 016d e2e2 0110
0216 ...r..#d.m......
0x0020: 07a8 13c5 5c17 3523 894a abd8 4143
4b20 ....\.5#.J..ACK.
… but nothing seen in kamailio.
If I swap over to sending to / listening on the public IP everything
works as expected.
Cheers
Mark
On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello,
That seems strange... What kind of device is that VPN device? Like a
ppp0 interface? What is the output of "netstat -plunt | grep kamailio"? Are
you using a "private" IP for the VPN interface and what's your main
routing
table like?
Also what if you omit the "listen" directive in kamailio configuration?
Does it listen on the VPN interface then? Kamailio will bind by default to
all interfaces if no "listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi All
Just to dive in here with something which may be related. I was about
to do a post with a much simpler problem where I have one real NIC and one
VPN created NIC. If I listen to the public IP all’s well. If I listen to
the IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on
either NIC and see the traffic arriving.
Anyone seen / solved this before?
Cheers
Mark
On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello Daniel,
I'm including the original sketch for clarity:
+----------+ +-------------------+
| eth0 | | vrf-green |
| 1.1.1.1 | | 127.0.0.1 |
+----------+ +-------------------+
|
+----------+
| eth1 |
| 2.2.2.2 |
+----------+
Thanks for the reply. Indeed, using the advertise option for the
"listen" directive will result in kamaiio no longer failing to start when
using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
"advertise" option.
However, this doesn't make much of a difference in that kamailio still
won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
OPTIONS coming with sngrep, but nothing in the logs, no replies, only
retransmissions from the originator.
In addition, while without VRF kamailio will select the right interface
with mhomed=1, this is no longer the case.
Also, If I try to send something to a host with force_send_socket, the
transaction fails with 477 Unfortunately error on sending to next hop
occurred. In the log, kamailio prints something along the lines of:
udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16):
Invalid argument(22)
udp_send(): invalid sendtoparameters
msg_send_buffer(): udp_send failed
where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060)
has been called before t_relay()
Also, I additionally found out that if the networking configuration is
as follows:
+----------+ +-------------------+ +-------------------+
| eth0 | | vrf-green | | vrf-red |
| 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
+----------+ +-------------------+ +-------------------+
| |
+----------+ +----------+
| eth1 | | eth2 |
| 2.2.2.2 | | 3.3.3.3 |
+----------+ +----------+
Then this won't work for vrf-red:
listen=vrf-green:5060 advertise 2.2.2.2:5060
listen=vrf-red:5060 advertise 3.3.3.3:5060
One needs to use different IPs for the MASTER devices (say 127.0.0.1
and 127.0.0.2) for kamailio to startup with the above listen directives.
It's a bit surprising it doesn't work. According to a user on cumulus
slack channel (cumulus contributed the VRF code to the linux kernel): "any
app that has a command line or option switch to call SO_BINDTODEVICE on the
sockets can be configured to use any VRF".
I guess the problem lies in that kamailio binds to several sockets and
there are multiple routing tables to consult, so mhomed doesn't work as
expected (if every table has a default route and nothing more specific than
that). But still, this doesn't really explain why binding to the VRF master
net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
ping)...
I don't think VRF support is very useful for most users though, I just
had one corner case I needed to tackle and VRF seemed like an easy fix.
I'll think of an alternative at some point, I guess.
On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
miconda(a)gmail.com> wrote:
>
> Hello,
>
> I haven't worked with vrf here, so no first hand experience ...
>
> What happens if you try with next option?
>
> listen=vrf-green:5060 advertise 2.2.2.2:5060
>
> Cheers,
> Daniel
>
>
> On 25.09.17 19:10, George Diamantopoulos wrote:
>
> Sorry to bump this, but it would be very useful if I knew whether
there's any point in pursuing this or not. Any hints?
>
> On 21 September 2017 at 14:06, George Diamantopoulos <
georgediam(a)gmail.com> wrote:
>>
>> Hello,
>>
>> I have a use case where I need to have kamailio bind to a VRF
device. The configuration in question is similar to the example below,
where eth1 is a slave to the VRF-lite device:
>>
>> +----------+ +-------------------+
>> | eth0 | | vrf-green |
>> | 1.1.1.1 | | 127.0.0.1 |
>> +----------+ +-------------------+
>> |
>> +----------+
>> | eth1 |
>> | 2.2.2.2 |
>> +----------+
>>
>> Both the main routing table and "vrf-green" routing table have a
default route.
>>
>> What I need to be able to do is have kamailio bind to both
interfaces:
>>
>> listen=eth0:5060
>> listen=vrf-green:5060
>>
>> And additionally be able to use force_send_socket to select an
interface, for example:
>>
>> force_send_socket(udp:2.2.2.2:5060);
>>
>> However, I can't get this to work. The above configuration fails
because there is no listen directive for 2.2.2.2. Also, kamailio doesn't
process packets received on the VRF with the above listen directives, it
behaves as if it doesn't listen on 2.2.2.2 indeed.
>>
>> In addition using either of the below:
>>
>> listen=udp:2.2.2.2:5060
>> or
>> listen=eth1:5060
>>
>> fails with an error upon starting kamailio.
>>
>> According to the kernel documentation:
>>
>> Applications that are to work within a VRF need to bind their socket
to the
>> VRF device:
>>
>> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>>
>> or to specify the output device using cmsg and IP_PKTINFO.
>>
>> The question is, is VRF useable with kamailio right now? Or is
development needed? Thanks!
>>
>> BR,
>>
>> George
>
>
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
> --
> Daniel-Constantin Mierla
> www.twitter.com/miconda -- www.linkedin.com/in/miconda
> Kamailio Advanced Training - www.asipto.com
> Kamailio World Conference - www.kamailioworld.com
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
15 Oct
15 Oct
2:53 p.m.
Hello,
No, I gave up in the end. Fortunately I was able to do what I needed with
policy-based routing, which is not the same as VRF, but it did the trick in
my scenario...
BR,
George
On 14 October 2017 at 01:22, Marrold <kamailio(a)marrold.co.uk> wrote:
Hi George,
Did you have any luck getting VRFs working with Kamailio?
Thanks
Matthew
On Thu, Oct 12, 2017 at 8:06 PM, Mark Boyce <mark(a)darkorigins.com> wrote:
Hi
I *think* I may have found it … although still not sure what’s doing the
filtering.
If I force the sending IP to be the the VPN interface on the sender by
adding;
modparam("siptrace", "force_send_sock", "sip:10.0.0.2:5060”)
So packets now arrive at the receiver from the 10. IP rather than from
the public IP of the sender
… and magically it starts working.
So something in Ubuntu 16.04 / zerotier is “filtering” packets which
arrive on the VPN interface but with a no VPN IP
Cheers
Mark
On 12 Oct 2017, at 19:59, Sergey Safarov <s.safarov(a)gmail.com> wrote:
This may be NAT/iptables issue if used some type of virtualization
Check UNDEPLIED connection using conntrack
Sergey
чт, 12 окт. 2017 г. в 20:27, Mark Boyce <mark(a)darkorigins.com>om>:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi
Now using listen=udp:0.0.0.0:9060
net stat agrees;
netstat -plunt | grep kamailio
udp 0 0 0.0.0.0:9060 0.0.0.0:*
20486/kamailio
TCPDump of Sending / Receiving to zero tier port
17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@
..N.J..
0x0010: 0a05 0172 13c4 2364 04dd 261b 0110
0211 ...r..#d..&.....
0x0020: 13c4 13d8 894a abd8 894a abd8 494e
5649 .....J...J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
2(a)sbc-1.w
Nothing in Kamailio Logs
Swap to using the real IP and I’m seeing the log line from the top of
the main route;
Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
and the matching tcpdump from the public interface;
17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
0x0000: 4510 03d8 863c 0000 3911 6109 894a
abd8 E....<..9.a..J..
0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110
0216 3.-...#d........
0x0020: 07de 13c5 5c17 3523 894a abd8 494e
5649 ....\.5#.J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
2(a)sbc-1.w
Loading the dumped packets in to wireshark I can’t see any obvious
difference.
Will start up a new thread … thanks for the loan of yours :-)
Mark
On 12 Oct 2017, at 17:21, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hmm... Everything looks fine indeed, I can't see any obvious reason why
this isn't working...
Just to be clear, have you tried listening on all interfaces:
listen=udp:0.0.0.0:9060
And if yes, can you capture traffic arriving on the ZeroTier interface
in this case? If it works, is there any reason why listening on all
interfaces is not acceptable in your use case?
Regardless, it might be a good idea to start a new thread about this,
maybe it's related to the inner workings of the ZeroTier device and how it
interacts with the kernel...
On 12 October 2017 at 18:26, Mark Boyce <mark(a)darkorigins.com> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi George
(IP / MACs below anonymised a bit)
I’m working on a https://www.zerotier.com VPN endpoint which presents
as a new interface;
Real Interface
ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2508723344 <(250)%20872-3344> (2.5 GB) TX
bytes:1149701606 (1.1 GB)
loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
ZeroTier
zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
This is actually a SIPCapture/Homer node so the config is;
modparam("sipcapture", "capture_on", 1)
modparam("sipcapture", "hep_capture_on", 1)
listen=udp:10.0.0.1:9060
The port bind looks ok;
# netstat -plunt | grep kamailio
udp 0 0 10.0.0.1:9060 0.0.0.0:*
19592/kamailio
I’m tracking packets at the top of "route {“
route {
xlog("route entered\r\n”);
If I tcpdump the ZeroTier Interface ;
tcpdump -i zt0 -p -X port 9060
15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@
..?.J..
0x0010: 0a05 0172 13c4 2364 016d e2e2 0110
0216 ...r..#d.m......
0x0020: 07a8 13c5 5c17 3523 894a abd8 4143
4b20 ....\.5#.J..ACK.
… but nothing seen in kamailio.
If I swap over to sending to / listening on the public IP everything
works as expected.
Cheers
Mark
On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hello,
That seems strange... What kind of device is that VPN device? Like a
ppp0 interface? What is the output of "netstat -plunt | grep kamailio"? Are
you using a "private" IP for the VPN interface and what's your main
routing
table like?
Also what if you omit the "listen" directive in kamailio configuration?
Does it listen on the VPN interface then? Kamailio will bind by default to
all interfaces if no "listen" directive is used...
On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com> wrote:
> Hi All
>
> Just to dive in here with something which may be related. I was about
> to do a post with a much simpler problem where I have one real NIC and one
> VPN created NIC. If I listen to the public IP all’s well. If I listen to
> the IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on
> either NIC and see the traffic arriving.
>
> Anyone seen / solved this before?
>
> Cheers
> Mark
>
> On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam(a)gmail.com>
> wrote:
>
> Hello Daniel,
>
> I'm including the original sketch for clarity:
>
> +----------+ +-------------------+
> | eth0 | | vrf-green |
> | 1.1.1.1 | | 127.0.0.1 |
> +----------+ +-------------------+
> |
> +----------+
> | eth1 |
> | 2.2.2.2 |
> +----------+
>
> Thanks for the reply. Indeed, using the advertise option for the
> "listen" directive will result in kamaiio no longer failing to start when
> using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
> "advertise" option.
>
> However, this doesn't make much of a difference in that kamailio still
> won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
> OPTIONS coming with sngrep, but nothing in the logs, no replies, only
> retransmissions from the originator.
> In addition, while without VRF kamailio will select the right
> interface with mhomed=1, this is no longer the case.
> Also, If I try to send something to a host with force_send_socket, the
> transaction fails with 477 Unfortunately error on sending to next hop
> occurred. In the log, kamailio prints something along the lines of:
> udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16):
> Invalid argument(22)
> udp_send(): invalid sendtoparameters
> msg_send_buffer(): udp_send failed
> where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060)
> has been called before t_relay()
>
> Also, I additionally found out that if the networking configuration is
> as follows:
>
> +----------+ +-------------------+ +-------------------+
> | eth0 | | vrf-green | | vrf-red |
> | 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
> +----------+ +-------------------+ +-------------------+
> | |
> +----------+ +----------+
> | eth1 | | eth2 |
> | 2.2.2.2 | | 3.3.3.3 |
> +----------+ +----------+
>
> Then this won't work for vrf-red:
> listen=vrf-green:5060 advertise 2.2.2.2:5060
> listen=vrf-red:5060 advertise 3.3.3.3:5060
> One needs to use different IPs for the MASTER devices (say 127.0.0.1
> and 127.0.0.2) for kamailio to startup with the above listen directives.
>
> It's a bit surprising it doesn't work. According to a user on cumulus
> slack channel (cumulus contributed the VRF code to the linux kernel): "any
> app that has a command line or option switch to call SO_BINDTODEVICE on the
> sockets can be configured to use any VRF".
>
> I guess the problem lies in that kamailio binds to several sockets and
> there are multiple routing tables to consult, so mhomed doesn't work as
> expected (if every table has a default route and nothing more specific than
> that). But still, this doesn't really explain why binding to the VRF master
> net devices doesn't work (I mean, "ping -I vrf-green" works just fine
for
> ping)...
>
> I don't think VRF support is very useful for most users though, I just
> had one corner case I needed to tackle and VRF seemed like an easy fix.
> I'll think of an alternative at some point, I guess.
>
> On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
> miconda(a)gmail.com> wrote:
> >
> > Hello,
> >
> > I haven't worked with vrf here, so no first hand experience ...
> >
> > What happens if you try with next option?
> >
> > listen=vrf-green:5060 advertise 2.2.2.2:5060
> >
> > Cheers,
> > Daniel
> >
> >
> > On 25.09.17 19:10, George Diamantopoulos wrote:
> >
> > Sorry to bump this, but it would be very useful if I knew whether
> there's any point in pursuing this or not. Any hints?
> >
> > On 21 September 2017 at 14:06, George Diamantopoulos <
> georgediam(a)gmail.com> wrote:
> >>
> >> Hello,
> >>
> >> I have a use case where I need to have kamailio bind to a VRF
> device. The configuration in question is similar to the example below,
> where eth1 is a slave to the VRF-lite device:
> >>
> >> +----------+ +-------------------+
> >> | eth0 | | vrf-green |
> >> | 1.1.1.1 | | 127.0.0.1 |
> >> +----------+ +-------------------+
> >> |
> >> +----------+
> >> | eth1 |
> >> | 2.2.2.2 |
> >> +----------+
> >>
> >> Both the main routing table and "vrf-green" routing table have a
> default route.
> >>
> >> What I need to be able to do is have kamailio bind to both
> interfaces:
> >>
> >> listen=eth0:5060
> >> listen=vrf-green:5060
> >>
> >> And additionally be able to use force_send_socket to select an
> interface, for example:
> >>
> >> force_send_socket(udp:2.2.2.2:5060);
> >>
> >> However, I can't get this to work. The above configuration fails
> because there is no listen directive for 2.2.2.2. Also, kamailio doesn't
> process packets received on the VRF with the above listen directives, it
> behaves as if it doesn't listen on 2.2.2.2 indeed.
> >>
> >> In addition using either of the below:
> >>
> >> listen=udp:2.2.2.2:5060
> >> or
> >> listen=eth1:5060
> >>
> >> fails with an error upon starting kamailio.
> >>
> >> According to the kernel documentation:
> >>
> >> Applications that are to work within a VRF need to bind their
> socket to the
> >> VRF device:
> >>
> >> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
> >>
> >> or to specify the output device using cmsg and IP_PKTINFO.
> >>
> >> The question is, is VRF useable with kamailio right now? Or is
> development needed? Thanks!
> >>
> >> BR,
> >>
> >> George
> >
> >
> >
> >
> > _______________________________________________
> > Kamailio (SER) - Users Mailing List
> > sr-users(a)lists.kamailio.org
> > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
> >
> >
> > --
> > Daniel-Constantin Mierla
> > www.twitter.com/miconda -- www.linkedin.com/in/miconda
> > Kamailio Advanced Training - www.asipto.com
> > Kamailio World Conference - www.kamailioworld.com
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
3:39 p.m.
You can easy implement vrf using docker and linux network namespace
I not see before programms that have VRF support. This requres such support
by programm design.
вс, 15 окт. 2017 г., 15:55 George Diamantopoulos <georgediam(a)gmail.com>om>:
Hello,
No, I gave up in the end. Fortunately I was able to do what I needed with
policy-based routing, which is not the same as VRF, but it did the trick in
my scenario...
BR,
George
On 14 October 2017 at 01:22, Marrold <kamailio(a)marrold.co.uk> wrote:
Hi George,
Did you have any luck getting VRFs working with Kamailio?
Thanks
Matthew
On Thu, Oct 12, 2017 at 8:06 PM, Mark Boyce <mark(a)darkorigins.com> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi
I *think* I may have found it … although still not sure what’s doing the
filtering.
If I force the sending IP to be the the VPN interface on the sender by
adding;
modparam("siptrace", "force_send_sock", "sip:10.0.0.2:5060”)
So packets now arrive at the receiver from the 10. IP rather than from
the public IP of the sender
… and magically it starts working.
So something in Ubuntu 16.04 / zerotier is “filtering” packets which
arrive on the VPN interface but with a no VPN IP
Cheers
Mark
On 12 Oct 2017, at 19:59, Sergey Safarov <s.safarov(a)gmail.com> wrote:
This may be NAT/iptables issue if used some type of virtualization
Check UNDEPLIED connection using conntrack
Sergey
чт, 12 окт. 2017 г. в 20:27, Mark Boyce <mark(a)darkorigins.com>om>:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi
Now using listen=udp:0.0.0.0:9060
net stat agrees;
netstat -plunt | grep kamailio
udp 0 0 0.0.0.0:9060 0.0.0.0:*
20486/kamailio
TCPDump of Sending / Receiving to zero tier port
17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@
..N.J..
0x0010: 0a05 0172 13c4 2364 04dd 261b 0110
0211 ...r..#d..&.....
0x0020: 13c4 13d8 894a abd8 894a abd8 494e
5649 .....J...J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
2(a)sbc-1.w
Nothing in Kamailio Logs
Swap to using the real IP and I’m seeing the log line from the top of
the main route;
Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
and the matching tcpdump from the public interface;
17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
0x0000: 4510 03d8 863c 0000 3911 6109 894a
abd8 E....<..9.a..J..
0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110
0216 3.-...#d........
0x0020: 07de 13c5 5c17 3523 894a abd8 494e
5649 ....\.5#.J..INVI
0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
2(a)sbc-1.w
Loading the dumped packets in to wireshark I can’t see any obvious
difference.
Will start up a new thread … thanks for the loan of yours :-)
Mark
On 12 Oct 2017, at 17:21, George Diamantopoulos <georgediam(a)gmail.com>
wrote:
Hmm... Everything looks fine indeed, I can't see any obvious reason why
this isn't working...
Just to be clear, have you tried listening on all interfaces:
listen=udp:0.0.0.0:9060
And if yes, can you capture traffic arriving on the ZeroTier interface
in this case? If it works, is there any reason why listening on all
interfaces is not acceptable in your use case?
Regardless, it might be a good idea to start a new thread about this,
maybe it's related to the inner workings of the ZeroTier device and how it
interacts with the kernel...
On 12 October 2017 at 18:26, Mark Boyce <mark(a)darkorigins.com> wrote:
> Hi George
>
> (IP / MACs below anonymised a bit)
>
>
> I’m working on a https://www.zerotier.com VPN endpoint which presents
> as a new interface;
>
>
> Real Interface
>
> ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
> inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
> inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
> TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:2508723344 <(250)%20872-3344> (2.5 GB) TX
> bytes:1149701606 (1.1 GB)
>
> loopback
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:65536 Metric:1
> RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
> TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1
> RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
>
> ZeroTier
>
> zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
> inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
> RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
>
>
> This is actually a SIPCapture/Homer node so the config is;
>
> modparam("sipcapture", "capture_on", 1)
> modparam("sipcapture", "hep_capture_on", 1)
> listen=udp:10.0.0.1:9060
>
> The port bind looks ok;
>
> # netstat -plunt | grep kamailio
> udp 0 0 10.0.0.1:9060 0.0.0.0:*
> 19592/kamailio
>
>
> I’m tracking packets at the top of "route {“
>
> route {
> xlog("route entered\r\n”);
>
>
>
> If I tcpdump the ZeroTier Interface ;
>
> tcpdump -i zt0 -p -X port 9060
>
> 15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
> 0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@
> ..?.J..
> 0x0010: 0a05 0172 13c4 2364 016d e2e2 0110
> 0216 ...r..#d.m......
> 0x0020: 07a8 13c5 5c17 3523 894a abd8 4143
> 4b20 ....\.5#.J..ACK.
>
>
> … but nothing seen in kamailio.
>
> If I swap over to sending to / listening on the public IP everything
> works as expected.
>
>
>
> Cheers
> Mark
>
>
> On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam(a)gmail.com>
> wrote:
>
> Hello,
>
> That seems strange... What kind of device is that VPN device? Like a
> ppp0 interface? What is the output of "netstat -plunt | grep kamailio"?
Are
> you using a "private" IP for the VPN interface and what's your main
routing
> table like?
>
> Also what if you omit the "listen" directive in kamailio
> configuration? Does it listen on the VPN interface then? Kamailio will bind
> by default to all interfaces if no "listen" directive is used...
>
> On 12 October 2017 at 00:26, Mark Boyce <mark(a)darkorigins.com> wrote:
>
>> Hi All
>>
>> Just to dive in here with something which may be related. I was
>> about to do a post with a much simpler problem where I have one real NIC
>> and one VPN created NIC. If I listen to the public IP all’s well. If I
>> listen to the IP of the VPN NIC Kamailio doesn’t see any traffic. I can
>> tcpdump on either NIC and see the traffic arriving.
>>
>> Anyone seen / solved this before?
>>
>> Cheers
>> Mark
>>
>> On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam(a)gmail.com>
>> wrote:
>>
>> Hello Daniel,
>>
>> I'm including the original sketch for clarity:
>>
>> +----------+ +-------------------+
>> | eth0 | | vrf-green |
>> | 1.1.1.1 | | 127.0.0.1 |
>> +----------+ +-------------------+
>> |
>> +----------+
>> | eth1 |
>> | 2.2.2.2 |
>> +----------+
>>
>> Thanks for the reply. Indeed, using the advertise option for the
>> "listen" directive will result in kamaiio no longer failing to start
when
>> using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
>> "advertise" option.
>>
>> However, this doesn't make much of a difference in that kamailio
>> still won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
>> OPTIONS coming with sngrep, but nothing in the logs, no replies, only
>> retransmissions from the originator.
>> In addition, while without VRF kamailio will select the right
>> interface with mhomed=1, this is no longer the case.
>> Also, If I try to send something to a host with force_send_socket,
>> the transaction fails with 477 Unfortunately error on sending to next hop
>> occurred. In the log, kamailio prints something along the lines of:
>> udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16):
>> Invalid argument(22)
>> udp_send(): invalid sendtoparameters
>> msg_send_buffer(): udp_send failed
>> where 9.9.9.9 is the $dd and presumably force_send_socket(
>> 2.2.2.2:5060) has been called before t_relay()
>>
>> Also, I additionally found out that if the networking configuration
>> is as follows:
>>
>> +----------+ +-------------------+ +-------------------+
>> | eth0 | | vrf-green | | vrf-red |
>> | 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
>> +----------+ +-------------------+ +-------------------+
>> | |
>> +----------+ +----------+
>> | eth1 | | eth2 |
>> | 2.2.2.2 | | 3.3.3.3 |
>> +----------+ +----------+
>>
>> Then this won't work for vrf-red:
>> listen=vrf-green:5060 advertise 2.2.2.2:5060
>> listen=vrf-red:5060 advertise 3.3.3.3:5060
>> One needs to use different IPs for the MASTER devices (say 127.0.0.1
>> and 127.0.0.2) for kamailio to startup with the above listen directives.
>>
>> It's a bit surprising it doesn't work. According to a user on cumulus
>> slack channel (cumulus contributed the VRF code to the linux kernel): "any
>> app that has a command line or option switch to call SO_BINDTODEVICE on the
>> sockets can be configured to use any VRF".
>>
>> I guess the problem lies in that kamailio binds to several sockets
>> and there are multiple routing tables to consult, so mhomed doesn't work as
>> expected (if every table has a default route and nothing more specific than
>> that). But still, this doesn't really explain why binding to the VRF master
>> net devices doesn't work (I mean, "ping -I vrf-green" works just
fine for
>> ping)...
>>
>> I don't think VRF support is very useful for most users though, I
>> just had one corner case I needed to tackle and VRF seemed like an easy
>> fix. I'll think of an alternative at some point, I guess.
>>
>> On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
>> miconda(a)gmail.com> wrote:
>> >
>> > Hello,
>> >
>> > I haven't worked with vrf here, so no first hand experience ...
>> >
>> > What happens if you try with next option?
>> >
>> > listen=vrf-green:5060 advertise 2.2.2.2:5060
>> >
>> > Cheers,
>> > Daniel
>> >
>> >
>> > On 25.09.17 19:10, George Diamantopoulos wrote:
>> >
>> > Sorry to bump this, but it would be very useful if I knew whether
>> there's any point in pursuing this or not. Any hints?
>> >
>> > On 21 September 2017 at 14:06, George Diamantopoulos <
>> georgediam(a)gmail.com> wrote:
>> >>
>> >> Hello,
>> >>
>> >> I have a use case where I need to have kamailio bind to a VRF
>> device. The configuration in question is similar to the example below,
>> where eth1 is a slave to the VRF-lite device:
>> >>
>> >> +----------+ +-------------------+
>> >> | eth0 | | vrf-green |
>> >> | 1.1.1.1 | | 127.0.0.1 |
>> >> +----------+ +-------------------+
>> >> |
>> >> +----------+
>> >> | eth1 |
>> >> | 2.2.2.2 |
>> >> +----------+
>> >>
>> >> Both the main routing table and "vrf-green" routing table have
a
>> default route.
>> >>
>> >> What I need to be able to do is have kamailio bind to both
>> interfaces:
>> >>
>> >> listen=eth0:5060
>> >> listen=vrf-green:5060
>> >>
>> >> And additionally be able to use force_send_socket to select an
>> interface, for example:
>> >>
>> >> force_send_socket(udp:2.2.2.2:5060);
>> >>
>> >> However, I can't get this to work. The above configuration fails
>> because there is no listen directive for 2.2.2.2. Also, kamailio doesn't
>> process packets received on the VRF with the above listen directives, it
>> behaves as if it doesn't listen on 2.2.2.2 indeed.
>> >>
>> >> In addition using either of the below:
>> >>
>> >> listen=udp:2.2.2.2:5060
>> >> or
>> >> listen=eth1:5060
>> >>
>> >> fails with an error upon starting kamailio.
>> >>
>> >> According to the kernel documentation:
>> >>
>> >> Applications that are to work within a VRF need to bind their
>> socket to the
>> >> VRF device:
>> >>
>> >> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev,
>> strlen(dev)+1);
>> >>
>> >> or to specify the output device using cmsg and IP_PKTINFO.
>> >>
>> >> The question is, is VRF useable with kamailio right now? Or is
>> development needed? Thanks!
>> >>
>> >> BR,
>> >>
>> >> George
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Kamailio (SER) - Users Mailing List
>> > sr-users(a)lists.kamailio.org
>> > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>> >
>> >
>> > --
>> > Daniel-Constantin Mierla
>> > www.twitter.com/miconda -- www.linkedin.com/in/miconda
>> > Kamailio Advanced Training - www.asipto.com
>> > Kamailio World Conference - www.kamailioworld.com
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users(a)lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>>
>>
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users(a)lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
2392
days inactive
2416
days old
15 comments
5 participants
participants (5)
-
Daniel-Constantin Mierla -
George Diamantopoulos -
Mark Boyce -
Marrold -
Sergey Safarov