[SR-Users] Having kamailio bind to a VRF device
Sergey Safarov
s.safarov at gmail.com
Sun Oct 15 15:39:18 CEST 2017
You can easy implement vrf using docker and linux network namespace
I not see before programms that have VRF support. This requres such support
by programm design.
вс, 15 окт. 2017 г., 15:55 George Diamantopoulos <georgediam at gmail.com>:
> Hello,
>
> No, I gave up in the end. Fortunately I was able to do what I needed with
> policy-based routing, which is not the same as VRF, but it did the trick in
> my scenario...
>
> BR,
> George
>
> On 14 October 2017 at 01:22, Marrold <kamailio at marrold.co.uk> wrote:
>
>> Hi George,
>>
>> Did you have any luck getting VRFs working with Kamailio?
>>
>> Thanks
>>
>> Matthew
>>
>> On Thu, Oct 12, 2017 at 8:06 PM, Mark Boyce <mark at darkorigins.com> wrote:
>>
>>> Hi
>>>
>>> I *think* I may have found it … although still not sure what’s doing the
>>> filtering.
>>>
>>> If I force the sending IP to be the the VPN interface on the sender by
>>> adding;
>>>
>>> modparam("siptrace", "force_send_sock", "sip:10.0.0.2:5060”)
>>>
>>> So packets now arrive at the receiver from the 10. IP rather than from
>>> the public IP of the sender
>>>
>>> … and magically it starts working.
>>>
>>> So something in Ubuntu 16.04 / zerotier is “filtering” packets which
>>> arrive on the VPN interface but with a no VPN IP
>>>
>>> Cheers
>>>
>>> Mark
>>>
>>>
>>> On 12 Oct 2017, at 19:59, Sergey Safarov <s.safarov at gmail.com> wrote:
>>>
>>> This may be NAT/iptables issue if used some type of virtualization
>>> Check UNDEPLIED connection using conntrack
>>>
>>> Sergey
>>>
>>> чт, 12 окт. 2017 г. в 20:27, Mark Boyce <mark at darkorigins.com>:
>>>
>>>> Hi
>>>>
>>>> Now using listen=udp:0.0.0.0:9060
>>>>
>>>> net stat agrees;
>>>>
>>>> netstat -plunt | grep kamailio
>>>> udp 0 0 0.0.0.0:9060 0.0.0.0:*
>>>> 20486/kamailio
>>>>
>>>>
>>>> TCPDump of Sending / Receiving to zero tier port
>>>>
>>>> 17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
>>>> 0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@
>>>> ..N.J..
>>>> 0x0010: 0a05 0172 13c4 2364 04dd 261b 0110
>>>> 0211 ...r..#d..&.....
>>>> 0x0020: 13c4 13d8 894a abd8 894a abd8 494e
>>>> 5649 .....J...J..INVI
>>>> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
>>>> 2 at sbc-1.w
>>>>
>>>> Nothing in Kamailio Logs
>>>>
>>>> Swap to using the real IP and I’m seeing the log line from the top of
>>>> the main route;
>>>>
>>>> Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
>>>>
>>>> and the matching tcpdump from the public interface;
>>>>
>>>> 17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
>>>> 0x0000: 4510 03d8 863c 0000 3911 6109 894a
>>>> abd8 E....<..9.a..J..
>>>> 0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110
>>>> 0216 3.-...#d........
>>>> 0x0020: 07de 13c5 5c17 3523 894a abd8 494e
>>>> 5649 ....\.5#.J..INVI
>>>> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
>>>> 2 at sbc-1.w
>>>>
>>>>
>>>> Loading the dumped packets in to wireshark I can’t see any obvious
>>>> difference.
>>>>
>>>>
>>>> Will start up a new thread … thanks for the loan of yours :-)
>>>>
>>>> Mark
>>>>
>>>>
>>>> On 12 Oct 2017, at 17:21, George Diamantopoulos <georgediam at gmail.com>
>>>> wrote:
>>>>
>>>> Hmm... Everything looks fine indeed, I can't see any obvious reason why
>>>> this isn't working...
>>>>
>>>> Just to be clear, have you tried listening on all interfaces:
>>>> listen=udp:0.0.0.0:9060
>>>>
>>>> And if yes, can you capture traffic arriving on the ZeroTier interface
>>>> in this case? If it works, is there any reason why listening on all
>>>> interfaces is not acceptable in your use case?
>>>>
>>>> Regardless, it might be a good idea to start a new thread about this,
>>>> maybe it's related to the inner workings of the ZeroTier device and how it
>>>> interacts with the kernel...
>>>>
>>>>
>>>> On 12 October 2017 at 18:26, Mark Boyce <mark at darkorigins.com> wrote:
>>>>
>>>>> Hi George
>>>>>
>>>>> (IP / MACs below anonymised a bit)
>>>>>
>>>>>
>>>>> I’m working on a https://www.zerotier.com VPN endpoint which presents
>>>>> as a new interface;
>>>>>
>>>>>
>>>>> Real Interface
>>>>>
>>>>> ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
>>>>> inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
>>>>> inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
>>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>>>>> RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
>>>>> TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
>>>>> collisions:0 txqueuelen:1000
>>>>> RX bytes:2508723344 <(250)%20872-3344> (2.5 GB) TX
>>>>> bytes:1149701606 (1.1 GB)
>>>>>
>>>>> loopback
>>>>>
>>>>> lo Link encap:Local Loopback
>>>>> inet addr:127.0.0.1 Mask:255.0.0.0
>>>>> inet6 addr: ::1/128 Scope:Host
>>>>> UP LOOPBACK RUNNING MTU:65536 Metric:1
>>>>> RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
>>>>> TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
>>>>> collisions:0 txqueuelen:1
>>>>> RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
>>>>>
>>>>> ZeroTier
>>>>>
>>>>> zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
>>>>> inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
>>>>> UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
>>>>> RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
>>>>> TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
>>>>> collisions:0 txqueuelen:1000
>>>>> RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
>>>>>
>>>>>
>>>>> This is actually a SIPCapture/Homer node so the config is;
>>>>>
>>>>> modparam("sipcapture", "capture_on", 1)
>>>>> modparam("sipcapture", "hep_capture_on", 1)
>>>>> listen=udp:10.0.0.1:9060
>>>>>
>>>>> The port bind looks ok;
>>>>>
>>>>> # netstat -plunt | grep kamailio
>>>>> udp 0 0 10.0.0.1:9060 0.0.0.0:*
>>>>> 19592/kamailio
>>>>>
>>>>>
>>>>> I’m tracking packets at the top of "route {“
>>>>>
>>>>> route {
>>>>> xlog("route entered\r\n”);
>>>>>
>>>>>
>>>>>
>>>>> If I tcpdump the ZeroTier Interface ;
>>>>>
>>>>> tcpdump -i zt0 -p -X port 9060
>>>>>
>>>>> 15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
>>>>> 0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@
>>>>> ..?.J..
>>>>> 0x0010: 0a05 0172 13c4 2364 016d e2e2 0110
>>>>> 0216 ...r..#d.m......
>>>>> 0x0020: 07a8 13c5 5c17 3523 894a abd8 4143
>>>>> 4b20 ....\.5#.J..ACK.
>>>>>
>>>>>
>>>>> … but nothing seen in kamailio.
>>>>>
>>>>> If I swap over to sending to / listening on the public IP everything
>>>>> works as expected.
>>>>>
>>>>>
>>>>>
>>>>> Cheers
>>>>> Mark
>>>>>
>>>>>
>>>>> On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam at gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> That seems strange... What kind of device is that VPN device? Like a
>>>>> ppp0 interface? What is the output of "netstat -plunt | grep kamailio"? Are
>>>>> you using a "private" IP for the VPN interface and what's your main routing
>>>>> table like?
>>>>>
>>>>> Also what if you omit the "listen" directive in kamailio
>>>>> configuration? Does it listen on the VPN interface then? Kamailio will bind
>>>>> by default to all interfaces if no "listen" directive is used...
>>>>>
>>>>> On 12 October 2017 at 00:26, Mark Boyce <mark at darkorigins.com> wrote:
>>>>>
>>>>>> Hi All
>>>>>>
>>>>>> Just to dive in here with something which may be related. I was
>>>>>> about to do a post with a much simpler problem where I have one real NIC
>>>>>> and one VPN created NIC. If I listen to the public IP all’s well. If I
>>>>>> listen to the IP of the VPN NIC Kamailio doesn’t see any traffic. I can
>>>>>> tcpdump on either NIC and see the traffic arriving.
>>>>>>
>>>>>> Anyone seen / solved this before?
>>>>>>
>>>>>> Cheers
>>>>>> Mark
>>>>>>
>>>>>> On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hello Daniel,
>>>>>>
>>>>>> I'm including the original sketch for clarity:
>>>>>>
>>>>>> +----------+ +-------------------+
>>>>>> | eth0 | | vrf-green |
>>>>>> | 1.1.1.1 | | 127.0.0.1 |
>>>>>> +----------+ +-------------------+
>>>>>> |
>>>>>> +----------+
>>>>>> | eth1 |
>>>>>> | 2.2.2.2 |
>>>>>> +----------+
>>>>>>
>>>>>> Thanks for the reply. Indeed, using the advertise option for the
>>>>>> "listen" directive will result in kamaiio no longer failing to start when
>>>>>> using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
>>>>>> "advertise" option.
>>>>>>
>>>>>> However, this doesn't make much of a difference in that kamailio
>>>>>> still won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
>>>>>> OPTIONS coming with sngrep, but nothing in the logs, no replies, only
>>>>>> retransmissions from the originator.
>>>>>> In addition, while without VRF kamailio will select the right
>>>>>> interface with mhomed=1, this is no longer the case.
>>>>>> Also, If I try to send something to a host with force_send_socket,
>>>>>> the transaction fails with 477 Unfortunately error on sending to next hop
>>>>>> occurred. In the log, kamailio prints something along the lines of:
>>>>>> udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16):
>>>>>> Invalid argument(22)
>>>>>> udp_send(): invalid sendtoparameters
>>>>>> msg_send_buffer(): udp_send failed
>>>>>> where 9.9.9.9 is the $dd and presumably force_send_socket(
>>>>>> 2.2.2.2:5060) has been called before t_relay()
>>>>>>
>>>>>> Also, I additionally found out that if the networking configuration
>>>>>> is as follows:
>>>>>>
>>>>>> +----------+ +-------------------+ +-------------------+
>>>>>> | eth0 | | vrf-green | | vrf-red |
>>>>>> | 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
>>>>>> +----------+ +-------------------+ +-------------------+
>>>>>> | |
>>>>>> +----------+ +----------+
>>>>>> | eth1 | | eth2 |
>>>>>> | 2.2.2.2 | | 3.3.3.3 |
>>>>>> +----------+ +----------+
>>>>>>
>>>>>> Then this won't work for vrf-red:
>>>>>> listen=vrf-green:5060 advertise 2.2.2.2:5060
>>>>>> listen=vrf-red:5060 advertise 3.3.3.3:5060
>>>>>> One needs to use different IPs for the MASTER devices (say 127.0.0.1
>>>>>> and 127.0.0.2) for kamailio to startup with the above listen directives.
>>>>>>
>>>>>> It's a bit surprising it doesn't work. According to a user on cumulus
>>>>>> slack channel (cumulus contributed the VRF code to the linux kernel): "any
>>>>>> app that has a command line or option switch to call SO_BINDTODEVICE on the
>>>>>> sockets can be configured to use any VRF".
>>>>>>
>>>>>> I guess the problem lies in that kamailio binds to several sockets
>>>>>> and there are multiple routing tables to consult, so mhomed doesn't work as
>>>>>> expected (if every table has a default route and nothing more specific than
>>>>>> that). But still, this doesn't really explain why binding to the VRF master
>>>>>> net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
>>>>>> ping)...
>>>>>>
>>>>>> I don't think VRF support is very useful for most users though, I
>>>>>> just had one corner case I needed to tackle and VRF seemed like an easy
>>>>>> fix. I'll think of an alternative at some point, I guess.
>>>>>>
>>>>>> On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
>>>>>> miconda at gmail.com> wrote:
>>>>>> >
>>>>>> > Hello,
>>>>>> >
>>>>>> > I haven't worked with vrf here, so no first hand experience ...
>>>>>> >
>>>>>> > What happens if you try with next option?
>>>>>> >
>>>>>> > listen=vrf-green:5060 advertise 2.2.2.2:5060
>>>>>> >
>>>>>> > Cheers,
>>>>>> > Daniel
>>>>>> >
>>>>>> >
>>>>>> > On 25.09.17 19:10, George Diamantopoulos wrote:
>>>>>> >
>>>>>> > Sorry to bump this, but it would be very useful if I knew whether
>>>>>> there's any point in pursuing this or not. Any hints?
>>>>>> >
>>>>>> > On 21 September 2017 at 14:06, George Diamantopoulos <
>>>>>> georgediam at gmail.com> wrote:
>>>>>> >>
>>>>>> >> Hello,
>>>>>> >>
>>>>>> >> I have a use case where I need to have kamailio bind to a VRF
>>>>>> device. The configuration in question is similar to the example below,
>>>>>> where eth1 is a slave to the VRF-lite device:
>>>>>> >>
>>>>>> >> +----------+ +-------------------+
>>>>>> >> | eth0 | | vrf-green |
>>>>>> >> | 1.1.1.1 | | 127.0.0.1 |
>>>>>> >> +----------+ +-------------------+
>>>>>> >> |
>>>>>> >> +----------+
>>>>>> >> | eth1 |
>>>>>> >> | 2.2.2.2 |
>>>>>> >> +----------+
>>>>>> >>
>>>>>> >> Both the main routing table and "vrf-green" routing table have a
>>>>>> default route.
>>>>>> >>
>>>>>> >> What I need to be able to do is have kamailio bind to both
>>>>>> interfaces:
>>>>>> >>
>>>>>> >> listen=eth0:5060
>>>>>> >> listen=vrf-green:5060
>>>>>> >>
>>>>>> >> And additionally be able to use force_send_socket to select an
>>>>>> interface, for example:
>>>>>> >>
>>>>>> >> force_send_socket(udp:2.2.2.2:5060);
>>>>>> >>
>>>>>> >> However, I can't get this to work. The above configuration fails
>>>>>> because there is no listen directive for 2.2.2.2. Also, kamailio doesn't
>>>>>> process packets received on the VRF with the above listen directives, it
>>>>>> behaves as if it doesn't listen on 2.2.2.2 indeed.
>>>>>> >>
>>>>>> >> In addition using either of the below:
>>>>>> >>
>>>>>> >> listen=udp:2.2.2.2:5060
>>>>>> >> or
>>>>>> >> listen=eth1:5060
>>>>>> >>
>>>>>> >> fails with an error upon starting kamailio.
>>>>>> >>
>>>>>> >> According to the kernel documentation:
>>>>>> >>
>>>>>> >> Applications that are to work within a VRF need to bind their
>>>>>> socket to the
>>>>>> >> VRF device:
>>>>>> >>
>>>>>> >> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev,
>>>>>> strlen(dev)+1);
>>>>>> >>
>>>>>> >> or to specify the output device using cmsg and IP_PKTINFO.
>>>>>> >>
>>>>>> >> The question is, is VRF useable with kamailio right now? Or is
>>>>>> development needed? Thanks!
>>>>>> >>
>>>>>> >> BR,
>>>>>> >>
>>>>>> >> George
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > Kamailio (SER) - Users Mailing List
>>>>>> > sr-users at lists.kamailio.org
>>>>>> > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > Daniel-Constantin Mierla
>>>>>> > www.twitter.com/miconda -- www.linkedin.com/in/miconda
>>>>>> > Kamailio Advanced Training - www.asipto.com
>>>>>> > Kamailio World Conference - www.kamailioworld.com
>>>>>>
>>>>>> _______________________________________________
>>>>>> Kamailio (SER) - Users Mailing List
>>>>>> sr-users at lists.kamailio.org
>>>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Kamailio (SER) - Users Mailing List
>>>>>> sr-users at lists.kamailio.org
>>>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Kamailio (SER) - Users Mailing List
>>>>> sr-users at lists.kamailio.org
>>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Kamailio (SER) - Users Mailing List
>>>>> sr-users at lists.kamailio.org
>>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Kamailio (SER) - Users Mailing List
>>>> sr-users at lists.kamailio.org
>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>
>>>>
>>>> _______________________________________________
>>>> Kamailio (SER) - Users Mailing List
>>>> sr-users at lists.kamailio.org
>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>>
>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>>
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20171015/288aa77b/attachment.html>
More information about the sr-users
mailing list