[SR-Users] Having kamailio bind to a VRF device

George Diamantopoulos georgediam at gmail.com
Sun Oct 15 14:53:42 CEST 2017


Hello,

No, I gave up in the end. Fortunately I was able to do what I needed with
policy-based routing, which is not the same as VRF, but it did the trick in
my scenario...

BR,
George

On 14 October 2017 at 01:22, Marrold <kamailio at marrold.co.uk> wrote:

> Hi George,
>
> Did you have any luck getting VRFs working with Kamailio?
>
> Thanks
>
> Matthew
>
> On Thu, Oct 12, 2017 at 8:06 PM, Mark Boyce <mark at darkorigins.com> wrote:
>
>> Hi
>>
>> I *think* I may have found it … although still not sure what’s doing the
>> filtering.
>>
>> If I force the sending IP to be the the VPN interface on the sender by
>> adding;
>>
>> modparam("siptrace", "force_send_sock", "sip:10.0.0.2:5060”)
>>
>> So packets now arrive at the receiver from the 10. IP rather than from
>> the public IP of the sender
>>
>> … and magically it starts working.
>>
>> So something in Ubuntu 16.04 / zerotier is “filtering” packets which
>> arrive on the VPN interface but with a no VPN IP
>>
>> Cheers
>>
>> Mark
>>
>>
>> On 12 Oct 2017, at 19:59, Sergey Safarov <s.safarov at gmail.com> wrote:
>>
>> This may be NAT/iptables issue if used some type of virtualization
>> Check UNDEPLIED connection using conntrack
>>
>> Sergey
>>
>> чт, 12 окт. 2017 г. в 20:27, Mark Boyce <mark at darkorigins.com>:
>>
>>> Hi
>>>
>>> Now using listen=udp:0.0.0.0:9060
>>>
>>> net stat agrees;
>>>
>>> netstat -plunt | grep kamailio
>>> udp        0      0 0.0.0.0:9060            0.0.0.0:*
>>>         20486/kamailio
>>>
>>>
>>> TCPDump of Sending / Receiving to zero tier port
>>>
>>> 17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
>>>         0x0000:  4510 04f1 a504 0000 4011 904e 894a abd8  E.......@
>>> ..N.J..
>>>         0x0010:  0a05 0172 13c4 2364 04dd 261b 0110
>>> 0211  ...r..#d..&.....
>>>         0x0020:  13c4 13d8 894a abd8 894a abd8 494e
>>> 5649  .....J...J..INVI
>>>         0x0030:  5445 2073 6970 3a32 4073 6263 2d31 2e77  TE.sip:
>>> 2 at sbc-1.w
>>>
>>> Nothing in Kamailio Logs
>>>
>>> Swap to using the real IP and I’m seeing the log line from the top of
>>> the main route;
>>>
>>> Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
>>>
>>> and the matching tcpdump from the public interface;
>>>
>>> 17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
>>>         0x0000:  4510 03d8 863c 0000 3911 6109 894a
>>> abd8  E....<..9.a..J..
>>>         0x0010:  33ff 2d9e 13c4 2364 03c4 f5a8 0110
>>> 0216  3.-...#d........
>>>         0x0020:  07de 13c5 5c17 3523 894a abd8 494e
>>> 5649  ....\.5#.J..INVI
>>>         0x0030:  5445 2073 6970 3a32 4073 6263 2d31 2e77  TE.sip:
>>> 2 at sbc-1.w
>>>
>>>
>>> Loading the dumped packets in to wireshark I can’t see any obvious
>>> difference.
>>>
>>>
>>> Will start up a new thread … thanks for the loan of yours :-)
>>>
>>> Mark
>>>
>>>
>>> On 12 Oct 2017, at 17:21, George Diamantopoulos <georgediam at gmail.com>
>>> wrote:
>>>
>>> Hmm... Everything looks fine indeed, I can't see any obvious reason why
>>> this isn't working...
>>>
>>> Just to be clear, have you tried listening on all interfaces:
>>> listen=udp:0.0.0.0:9060
>>>
>>> And if yes, can you capture traffic arriving on the ZeroTier interface
>>> in this case? If it works, is there any reason why listening on all
>>> interfaces is not acceptable in your use case?
>>>
>>> Regardless, it might be a good idea to start a new thread about this,
>>> maybe it's related to the inner workings of the ZeroTier device and how it
>>> interacts with the kernel...
>>>
>>>
>>> On 12 October 2017 at 18:26, Mark Boyce <mark at darkorigins.com> wrote:
>>>
>>>> Hi George
>>>>
>>>> (IP / MACs below anonymised a bit)
>>>>
>>>>
>>>> I’m working on a https://www.zerotier.com VPN endpoint which presents
>>>> as a new interface;
>>>>
>>>>
>>>> Real Interface
>>>>
>>>> ens3      Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx
>>>>           inet addr: 1.2.3.4  Bcast: 1.2.3.4  Mask:255.255.255.255
>>>>           inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
>>>>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>           RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:1000
>>>>           RX bytes:2508723344 <(250)%20872-3344> (2.5 GB)  TX
>>>> bytes:1149701606 (1.1 GB)
>>>>
>>>> loopback
>>>>
>>>> lo        Link encap:Local Loopback
>>>>           inet addr:127.0.0.1  Mask:255.0.0.0
>>>>           inet6 addr: ::1/128 Scope:Host
>>>>           UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>>>           RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:1
>>>>           RX bytes:12470726 (12.4 MB)  TX bytes:12470726 (12.4 MB)
>>>>
>>>> ZeroTier
>>>>
>>>> zt0       Link encap:Ethernet  HWaddr yy:yy:yy:yy:yy:yy
>>>>           inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
>>>>           UP BROADCAST RUNNING MULTICAST  MTU:2800  Metric:1
>>>>           RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:1000
>>>>           RX bytes:3760555 (3.7 MB)  TX bytes:584324 (584.3 KB)
>>>>
>>>>
>>>> This is actually a SIPCapture/Homer node so the config is;
>>>>
>>>> modparam("sipcapture", "capture_on", 1)
>>>> modparam("sipcapture", "hep_capture_on", 1)
>>>> listen=udp:10.0.0.1:9060
>>>>
>>>> The port bind looks ok;
>>>>
>>>> # netstat -plunt | grep kamailio
>>>> udp        0      0 10.0.0.1:9060         0.0.0.0:*
>>>>         19592/kamailio
>>>>
>>>>
>>>> I’m tracking packets at the top of "route {“
>>>>
>>>> route {
>>>>         xlog("route entered\r\n”);
>>>>
>>>>
>>>>
>>>> If I tcpdump the ZeroTier Interface ;
>>>>
>>>> tcpdump -i zt0 -p -X  port 9060
>>>>
>>>> 15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
>>>>         0x0000:  4510 0181 7883 0000 4011 c03f 894a abd8  E...x...@
>>>> ..?.J..
>>>>         0x0010:  0a05 0172 13c4 2364 016d e2e2 0110
>>>> 0216  ...r..#d.m......
>>>>         0x0020:  07a8 13c5 5c17 3523 894a abd8 4143
>>>> 4b20  ....\.5#.J..ACK.
>>>>
>>>>
>>>> … but nothing seen in kamailio.
>>>>
>>>> If I swap over to sending to / listening on the public IP everything
>>>> works as expected.
>>>>
>>>>
>>>>
>>>> Cheers
>>>> Mark
>>>>
>>>>
>>>> On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam at gmail.com>
>>>> wrote:
>>>>
>>>> Hello,
>>>>
>>>> That seems strange... What kind of device is that VPN device? Like a
>>>> ppp0 interface? What is the output of "netstat -plunt | grep kamailio"? Are
>>>> you using a "private" IP for the VPN interface and what's your main routing
>>>> table like?
>>>>
>>>> Also what if you omit the "listen" directive in kamailio configuration?
>>>> Does it listen on the VPN interface then? Kamailio will bind by default to
>>>> all interfaces if no "listen" directive is used...
>>>>
>>>> On 12 October 2017 at 00:26, Mark Boyce <mark at darkorigins.com> wrote:
>>>>
>>>>> Hi All
>>>>>
>>>>> Just to dive in here with something which may be related.  I was about
>>>>> to do a post with a much simpler problem where I have one real NIC and one
>>>>> VPN created NIC.  If I listen to the public IP all’s well.  If I listen to
>>>>> the IP of the VPN NIC Kamailio doesn’t see any traffic.   I can tcpdump on
>>>>> either NIC and see the traffic arriving.
>>>>>
>>>>> Anyone seen / solved this before?
>>>>>
>>>>> Cheers
>>>>> Mark
>>>>>
>>>>> On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam at gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hello Daniel,
>>>>>
>>>>> I'm including the original sketch for clarity:
>>>>>
>>>>>  +----------+      +-------------------+
>>>>>  |   eth0   |      |    vrf-green      |
>>>>>  | 1.1.1.1  |      |    127.0.0.1      |
>>>>>  +----------+      +-------------------+
>>>>>                             |
>>>>>                       +----------+
>>>>>                       |   eth1   |
>>>>>                       | 2.2.2.2  |
>>>>>                       +----------+
>>>>>
>>>>> Thanks for the reply. Indeed, using the advertise option for the
>>>>> "listen" directive will result in kamaiio no longer failing to start when
>>>>> using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
>>>>> "advertise" option.
>>>>>
>>>>> However, this doesn't make much of a difference in that kamailio still
>>>>> won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
>>>>> OPTIONS coming with sngrep, but nothing in the logs, no replies, only
>>>>> retransmissions from the originator.
>>>>> In addition, while without VRF kamailio will select the right
>>>>> interface with mhomed=1, this is no longer the case.
>>>>> Also, If I try to send something to a host with force_send_socket, the
>>>>> transaction fails with 477 Unfortunately error on sending to next hop
>>>>> occurred. In the log, kamailio prints something along the lines of:
>>>>> udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16):
>>>>> Invalid argument(22)
>>>>> udp_send(): invalid sendtoparameters
>>>>> msg_send_buffer(): udp_send failed
>>>>> where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060)
>>>>> has been called before t_relay()
>>>>>
>>>>> Also, I additionally found out that if the networking configuration is
>>>>> as follows:
>>>>>
>>>>>  +----------+      +-------------------+   +-------------------+
>>>>>  |   eth0   |      |    vrf-green      |   |    vrf-red        |
>>>>>  | 1.1.1.1  |      |    127.0.0.1      |   |    127.0.0.1      |
>>>>>  +----------+      +-------------------+   +-------------------+
>>>>>                             |                       |
>>>>>                       +----------+            +----------+
>>>>>                       |   eth1   |            |   eth2   |
>>>>>                       | 2.2.2.2  |            | 3.3.3.3  |
>>>>>                       +----------+            +----------+
>>>>>
>>>>> Then this won't work for vrf-red:
>>>>> listen=vrf-green:5060 advertise 2.2.2.2:5060
>>>>> listen=vrf-red:5060 advertise 3.3.3.3:5060
>>>>> One needs to use different IPs for the MASTER devices (say 127.0.0.1
>>>>> and 127.0.0.2) for kamailio to startup with the above listen directives.
>>>>>
>>>>> It's a bit surprising it doesn't work. According to a user on cumulus
>>>>> slack channel (cumulus contributed the VRF code to the linux kernel): "any
>>>>> app that has a command line or option switch to call SO_BINDTODEVICE on the
>>>>> sockets can be configured to use any VRF".
>>>>>
>>>>> I guess the problem lies in that kamailio binds to several sockets and
>>>>> there are multiple routing tables to consult, so mhomed doesn't work as
>>>>> expected (if every table has a default route and nothing more specific than
>>>>> that). But still, this doesn't really explain why binding to the VRF master
>>>>> net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
>>>>> ping)...
>>>>>
>>>>> I don't think VRF support is very useful for most users though, I just
>>>>> had one corner case I needed to tackle and VRF seemed like an easy fix.
>>>>> I'll think of an alternative at some point, I guess.
>>>>>
>>>>> On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
>>>>> miconda at gmail.com> wrote:
>>>>> >
>>>>> > Hello,
>>>>> >
>>>>> > I haven't worked with vrf here, so no first hand experience ...
>>>>> >
>>>>> > What happens if you try with next option?
>>>>> >
>>>>> > listen=vrf-green:5060 advertise 2.2.2.2:5060
>>>>> >
>>>>> > Cheers,
>>>>> > Daniel
>>>>> >
>>>>> >
>>>>> > On 25.09.17 19:10, George Diamantopoulos wrote:
>>>>> >
>>>>> > Sorry to bump this, but it would be very useful if I knew whether
>>>>> there's any point in pursuing this or not. Any hints?
>>>>> >
>>>>> > On 21 September 2017 at 14:06, George Diamantopoulos <
>>>>> georgediam at gmail.com> wrote:
>>>>> >>
>>>>> >> Hello,
>>>>> >>
>>>>> >> I have a use case where I need to have kamailio bind to a VRF
>>>>> device. The configuration in question is similar to the example below,
>>>>> where eth1 is a slave to the VRF-lite device:
>>>>> >>
>>>>> >>  +----------+      +-------------------+
>>>>> >>  |   eth0   |      |    vrf-green      |
>>>>> >>  | 1.1.1.1  |      |    127.0.0.1      |
>>>>> >>  +----------+      +-------------------+
>>>>> >>                             |
>>>>> >>                       +----------+
>>>>> >>                       |   eth1   |
>>>>> >>                       | 2.2.2.2  |
>>>>> >>                       +----------+
>>>>> >>
>>>>> >> Both the main routing table and "vrf-green" routing table have a
>>>>> default route.
>>>>> >>
>>>>> >> What I need to be able to do is have kamailio bind to both
>>>>> interfaces:
>>>>> >>
>>>>> >> listen=eth0:5060
>>>>> >> listen=vrf-green:5060
>>>>> >>
>>>>> >> And additionally be able to use force_send_socket to select an
>>>>> interface, for example:
>>>>> >>
>>>>> >> force_send_socket(udp:2.2.2.2:5060);
>>>>> >>
>>>>> >> However, I can't get this to work. The above configuration fails
>>>>> because there is no listen directive for 2.2.2.2. Also, kamailio doesn't
>>>>> process packets received on the VRF with the above listen directives, it
>>>>> behaves as if it doesn't listen on 2.2.2.2 indeed.
>>>>> >>
>>>>> >> In addition using either of the below:
>>>>> >>
>>>>> >> listen=udp:2.2.2.2:5060
>>>>> >> or
>>>>> >> listen=eth1:5060
>>>>> >>
>>>>> >> fails with an error upon starting kamailio.
>>>>> >>
>>>>> >> According to the kernel documentation:
>>>>> >>
>>>>> >> Applications that are to work within a VRF need to bind their
>>>>> socket to the
>>>>> >> VRF device:
>>>>> >>
>>>>> >>     setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>>>>> >>
>>>>> >> or to specify the output device using cmsg and IP_PKTINFO.
>>>>> >>
>>>>> >> The question is, is VRF useable with kamailio right now? Or is
>>>>> development needed? Thanks!
>>>>> >>
>>>>> >> BR,
>>>>> >>
>>>>> >> George
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > Kamailio (SER) - Users Mailing List
>>>>> > sr-users at lists.kamailio.org
>>>>> > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Daniel-Constantin Mierla
>>>>> > www.twitter.com/miconda -- www.linkedin.com/in/miconda
>>>>> > Kamailio Advanced Training - www.asipto.com
>>>>> > Kamailio World Conference - www.kamailioworld.com
>>>>>
>>>>> _______________________________________________
>>>>> Kamailio (SER) - Users Mailing List
>>>>> sr-users at lists.kamailio.org
>>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Kamailio (SER) - Users Mailing List
>>>>> sr-users at lists.kamailio.org
>>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Kamailio (SER) - Users Mailing List
>>>> sr-users at lists.kamailio.org
>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Kamailio (SER) - Users Mailing List
>>>> sr-users at lists.kamailio.org
>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>
>>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>>
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20171015/4f4e7b7d/attachment.html>


More information about the sr-users mailing list