[SR-Users] Having kamailio bind to a VRF device
Marrold
kamailio at marrold.co.uk
Sat Oct 14 00:22:16 CEST 2017
Hi George,
Did you have any luck getting VRFs working with Kamailio?
Thanks
Matthew
On Thu, Oct 12, 2017 at 8:06 PM, Mark Boyce <mark at darkorigins.com> wrote:
> Hi
>
> I *think* I may have found it … although still not sure what’s doing the
> filtering.
>
> If I force the sending IP to be the the VPN interface on the sender by
> adding;
>
> modparam("siptrace", "force_send_sock", "sip:10.0.0.2:5060”)
>
> So packets now arrive at the receiver from the 10. IP rather than from the
> public IP of the sender
>
> … and magically it starts working.
>
> So something in Ubuntu 16.04 / zerotier is “filtering” packets which
> arrive on the VPN interface but with a no VPN IP
>
> Cheers
>
> Mark
>
>
> On 12 Oct 2017, at 19:59, Sergey Safarov <s.safarov at gmail.com> wrote:
>
> This may be NAT/iptables issue if used some type of virtualization
> Check UNDEPLIED connection using conntrack
>
> Sergey
>
> чт, 12 окт. 2017 г. в 20:27, Mark Boyce <mark at darkorigins.com>:
>
>> Hi
>>
>> Now using listen=udp:0.0.0.0:9060
>>
>> net stat agrees;
>>
>> netstat -plunt | grep kamailio
>> udp 0 0 0.0.0.0:9060 0.0.0.0:*
>> 20486/kamailio
>>
>>
>> TCPDump of Sending / Receiving to zero tier port
>>
>> 17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
>> 0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@
>> ..N.J..
>> 0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 0211 ...r..#d..&.....
>> 0x0020: 13c4 13d8 894a abd8 894a abd8 494e 5649 .....J...J..INVI
>> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
>> 2 at sbc-1.w
>>
>> Nothing in Kamailio Logs
>>
>> Swap to using the real IP and I’m seeing the log line from the top of the
>> main route;
>>
>> Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
>>
>> and the matching tcpdump from the public interface;
>>
>> 17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
>> 0x0000: 4510 03d8 863c 0000 3911 6109 894a abd8 E....<..9.a..J..
>> 0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 0216 3.-...#d........
>> 0x0020: 07de 13c5 5c17 3523 894a abd8 494e 5649 ....\.5#.J..INVI
>> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:
>> 2 at sbc-1.w
>>
>>
>> Loading the dumped packets in to wireshark I can’t see any obvious
>> difference.
>>
>>
>> Will start up a new thread … thanks for the loan of yours :-)
>>
>> Mark
>>
>>
>> On 12 Oct 2017, at 17:21, George Diamantopoulos <georgediam at gmail.com>
>> wrote:
>>
>> Hmm... Everything looks fine indeed, I can't see any obvious reason why
>> this isn't working...
>>
>> Just to be clear, have you tried listening on all interfaces:
>> listen=udp:0.0.0.0:9060
>>
>> And if yes, can you capture traffic arriving on the ZeroTier interface in
>> this case? If it works, is there any reason why listening on all interfaces
>> is not acceptable in your use case?
>>
>> Regardless, it might be a good idea to start a new thread about this,
>> maybe it's related to the inner workings of the ZeroTier device and how it
>> interacts with the kernel...
>>
>>
>> On 12 October 2017 at 18:26, Mark Boyce <mark at darkorigins.com> wrote:
>>
>>> Hi George
>>>
>>> (IP / MACs below anonymised a bit)
>>>
>>>
>>> I’m working on a https://www.zerotier.com VPN endpoint which presents
>>> as a new interface;
>>>
>>>
>>> Real Interface
>>>
>>> ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
>>> inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
>>> inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>>> RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:1000
>>> RX bytes:2508723344 <(250)%20872-3344> (2.5 GB) TX
>>> bytes:1149701606 (1.1 GB)
>>>
>>> loopback
>>>
>>> lo Link encap:Local Loopback
>>> inet addr:127.0.0.1 Mask:255.0.0.0
>>> inet6 addr: ::1/128 Scope:Host
>>> UP LOOPBACK RUNNING MTU:65536 Metric:1
>>> RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:1
>>> RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
>>>
>>> ZeroTier
>>>
>>> zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
>>> inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
>>> UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
>>> RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
>>> TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:1000
>>> RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
>>>
>>>
>>> This is actually a SIPCapture/Homer node so the config is;
>>>
>>> modparam("sipcapture", "capture_on", 1)
>>> modparam("sipcapture", "hep_capture_on", 1)
>>> listen=udp:10.0.0.1:9060
>>>
>>> The port bind looks ok;
>>>
>>> # netstat -plunt | grep kamailio
>>> udp 0 0 10.0.0.1:9060 0.0.0.0:*
>>> 19592/kamailio
>>>
>>>
>>> I’m tracking packets at the top of "route {“
>>>
>>> route {
>>> xlog("route entered\r\n”);
>>>
>>>
>>>
>>> If I tcpdump the ZeroTier Interface ;
>>>
>>> tcpdump -i zt0 -p -X port 9060
>>>
>>> 15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
>>> 0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@
>>> ..?.J..
>>> 0x0010: 0a05 0172 13c4 2364 016d e2e2 0110
>>> 0216 ...r..#d.m......
>>> 0x0020: 07a8 13c5 5c17 3523 894a abd8 4143
>>> 4b20 ....\.5#.J..ACK.
>>>
>>>
>>> … but nothing seen in kamailio.
>>>
>>> If I swap over to sending to / listening on the public IP everything
>>> works as expected.
>>>
>>>
>>>
>>> Cheers
>>> Mark
>>>
>>>
>>> On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam at gmail.com>
>>> wrote:
>>>
>>> Hello,
>>>
>>> That seems strange... What kind of device is that VPN device? Like a
>>> ppp0 interface? What is the output of "netstat -plunt | grep kamailio"? Are
>>> you using a "private" IP for the VPN interface and what's your main routing
>>> table like?
>>>
>>> Also what if you omit the "listen" directive in kamailio configuration?
>>> Does it listen on the VPN interface then? Kamailio will bind by default to
>>> all interfaces if no "listen" directive is used...
>>>
>>> On 12 October 2017 at 00:26, Mark Boyce <mark at darkorigins.com> wrote:
>>>
>>>> Hi All
>>>>
>>>> Just to dive in here with something which may be related. I was about
>>>> to do a post with a much simpler problem where I have one real NIC and one
>>>> VPN created NIC. If I listen to the public IP all’s well. If I listen to
>>>> the IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on
>>>> either NIC and see the traffic arriving.
>>>>
>>>> Anyone seen / solved this before?
>>>>
>>>> Cheers
>>>> Mark
>>>>
>>>> On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam at gmail.com>
>>>> wrote:
>>>>
>>>> Hello Daniel,
>>>>
>>>> I'm including the original sketch for clarity:
>>>>
>>>> +----------+ +-------------------+
>>>> | eth0 | | vrf-green |
>>>> | 1.1.1.1 | | 127.0.0.1 |
>>>> +----------+ +-------------------+
>>>> |
>>>> +----------+
>>>> | eth1 |
>>>> | 2.2.2.2 |
>>>> +----------+
>>>>
>>>> Thanks for the reply. Indeed, using the advertise option for the
>>>> "listen" directive will result in kamaiio no longer failing to start when
>>>> using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
>>>> "advertise" option.
>>>>
>>>> However, this doesn't make much of a difference in that kamailio still
>>>> won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
>>>> OPTIONS coming with sngrep, but nothing in the logs, no replies, only
>>>> retransmissions from the originator.
>>>> In addition, while without VRF kamailio will select the right interface
>>>> with mhomed=1, this is no longer the case.
>>>> Also, If I try to send something to a host with force_send_socket, the
>>>> transaction fails with 477 Unfortunately error on sending to next hop
>>>> occurred. In the log, kamailio prints something along the lines of:
>>>> udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16):
>>>> Invalid argument(22)
>>>> udp_send(): invalid sendtoparameters
>>>> msg_send_buffer(): udp_send failed
>>>> where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060)
>>>> has been called before t_relay()
>>>>
>>>> Also, I additionally found out that if the networking configuration is
>>>> as follows:
>>>>
>>>> +----------+ +-------------------+ +-------------------+
>>>> | eth0 | | vrf-green | | vrf-red |
>>>> | 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
>>>> +----------+ +-------------------+ +-------------------+
>>>> | |
>>>> +----------+ +----------+
>>>> | eth1 | | eth2 |
>>>> | 2.2.2.2 | | 3.3.3.3 |
>>>> +----------+ +----------+
>>>>
>>>> Then this won't work for vrf-red:
>>>> listen=vrf-green:5060 advertise 2.2.2.2:5060
>>>> listen=vrf-red:5060 advertise 3.3.3.3:5060
>>>> One needs to use different IPs for the MASTER devices (say 127.0.0.1
>>>> and 127.0.0.2) for kamailio to startup with the above listen directives.
>>>>
>>>> It's a bit surprising it doesn't work. According to a user on cumulus
>>>> slack channel (cumulus contributed the VRF code to the linux kernel): "any
>>>> app that has a command line or option switch to call SO_BINDTODEVICE on the
>>>> sockets can be configured to use any VRF".
>>>>
>>>> I guess the problem lies in that kamailio binds to several sockets and
>>>> there are multiple routing tables to consult, so mhomed doesn't work as
>>>> expected (if every table has a default route and nothing more specific than
>>>> that). But still, this doesn't really explain why binding to the VRF master
>>>> net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
>>>> ping)...
>>>>
>>>> I don't think VRF support is very useful for most users though, I just
>>>> had one corner case I needed to tackle and VRF seemed like an easy fix.
>>>> I'll think of an alternative at some point, I guess.
>>>>
>>>> On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
>>>> miconda at gmail.com> wrote:
>>>> >
>>>> > Hello,
>>>> >
>>>> > I haven't worked with vrf here, so no first hand experience ...
>>>> >
>>>> > What happens if you try with next option?
>>>> >
>>>> > listen=vrf-green:5060 advertise 2.2.2.2:5060
>>>> >
>>>> > Cheers,
>>>> > Daniel
>>>> >
>>>> >
>>>> > On 25.09.17 19:10, George Diamantopoulos wrote:
>>>> >
>>>> > Sorry to bump this, but it would be very useful if I knew whether
>>>> there's any point in pursuing this or not. Any hints?
>>>> >
>>>> > On 21 September 2017 at 14:06, George Diamantopoulos <
>>>> georgediam at gmail.com> wrote:
>>>> >>
>>>> >> Hello,
>>>> >>
>>>> >> I have a use case where I need to have kamailio bind to a VRF
>>>> device. The configuration in question is similar to the example below,
>>>> where eth1 is a slave to the VRF-lite device:
>>>> >>
>>>> >> +----------+ +-------------------+
>>>> >> | eth0 | | vrf-green |
>>>> >> | 1.1.1.1 | | 127.0.0.1 |
>>>> >> +----------+ +-------------------+
>>>> >> |
>>>> >> +----------+
>>>> >> | eth1 |
>>>> >> | 2.2.2.2 |
>>>> >> +----------+
>>>> >>
>>>> >> Both the main routing table and "vrf-green" routing table have a
>>>> default route.
>>>> >>
>>>> >> What I need to be able to do is have kamailio bind to both
>>>> interfaces:
>>>> >>
>>>> >> listen=eth0:5060
>>>> >> listen=vrf-green:5060
>>>> >>
>>>> >> And additionally be able to use force_send_socket to select an
>>>> interface, for example:
>>>> >>
>>>> >> force_send_socket(udp:2.2.2.2:5060);
>>>> >>
>>>> >> However, I can't get this to work. The above configuration fails
>>>> because there is no listen directive for 2.2.2.2. Also, kamailio doesn't
>>>> process packets received on the VRF with the above listen directives, it
>>>> behaves as if it doesn't listen on 2.2.2.2 indeed.
>>>> >>
>>>> >> In addition using either of the below:
>>>> >>
>>>> >> listen=udp:2.2.2.2:5060
>>>> >> or
>>>> >> listen=eth1:5060
>>>> >>
>>>> >> fails with an error upon starting kamailio.
>>>> >>
>>>> >> According to the kernel documentation:
>>>> >>
>>>> >> Applications that are to work within a VRF need to bind their socket
>>>> to the
>>>> >> VRF device:
>>>> >>
>>>> >> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>>>> >>
>>>> >> or to specify the output device using cmsg and IP_PKTINFO.
>>>> >>
>>>> >> The question is, is VRF useable with kamailio right now? Or is
>>>> development needed? Thanks!
>>>> >>
>>>> >> BR,
>>>> >>
>>>> >> George
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Kamailio (SER) - Users Mailing List
>>>> > sr-users at lists.kamailio.org
>>>> > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>> >
>>>> >
>>>> > --
>>>> > Daniel-Constantin Mierla
>>>> > www.twitter.com/miconda -- www.linkedin.com/in/miconda
>>>> > Kamailio Advanced Training - www.asipto.com
>>>> > Kamailio World Conference - www.kamailioworld.com
>>>>
>>>> _______________________________________________
>>>> Kamailio (SER) - Users Mailing List
>>>> sr-users at lists.kamailio.org
>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Kamailio (SER) - Users Mailing List
>>>> sr-users at lists.kamailio.org
>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>
>>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>>
>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20171013/0dad8379/attachment.html>
More information about the sr-users
mailing list