[SR-Users] Having kamailio bind to a VRF device
Sergey Safarov
s.safarov at gmail.com
Thu Oct 12 20:59:25 CEST 2017
This may be NAT/iptables issue if used some type of virtualization
Check UNDEPLIED connection using conntrack
Sergey
чт, 12 окт. 2017 г. в 20:27, Mark Boyce <mark at darkorigins.com>:
> Hi
>
> Now using listen=udp:0.0.0.0:9060
>
> net stat agrees;
>
> netstat -plunt | grep kamailio
> udp 0 0 0.0.0.0:9060 0.0.0.0:*
> 20486/kamailio
>
>
> TCPDump of Sending / Receiving to zero tier port
>
> 17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
> 0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E....... at ..N.J..
> 0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 0211 ...r..#d..&.....
> 0x0020: 13c4 13d8 894a abd8 894a abd8 494e 5649 .....J...J..INVI
> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2 at sbc-1.w
>
> Nothing in Kamailio Logs
>
> Swap to using the real IP and I’m seeing the log line from the top of the
> main route;
>
> Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered
>
> and the matching tcpdump from the public interface;
>
> 17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP
> 0x0000: 4510 03d8 863c 0000 3911 6109 894a abd8 E....<..9.a..J..
> 0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 0216 3.-...#d........
> 0x0020: 07de 13c5 5c17 3523 894a abd8 494e 5649 ....\.5#.J..INVI
> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2 at sbc-1.w
>
>
> Loading the dumped packets in to wireshark I can’t see any obvious
> difference.
>
>
> Will start up a new thread … thanks for the loan of yours :-)
>
> Mark
>
>
> On 12 Oct 2017, at 17:21, George Diamantopoulos <georgediam at gmail.com>
> wrote:
>
> Hmm... Everything looks fine indeed, I can't see any obvious reason why
> this isn't working...
>
> Just to be clear, have you tried listening on all interfaces:
> listen=udp:0.0.0.0:9060
>
> And if yes, can you capture traffic arriving on the ZeroTier interface in
> this case? If it works, is there any reason why listening on all interfaces
> is not acceptable in your use case?
>
> Regardless, it might be a good idea to start a new thread about this,
> maybe it's related to the inner workings of the ZeroTier device and how it
> interacts with the kernel...
>
>
> On 12 October 2017 at 18:26, Mark Boyce <mark at darkorigins.com> wrote:
>
>> Hi George
>>
>> (IP / MACs below anonymised a bit)
>>
>>
>> I’m working on a https://www.zerotier.com VPN endpoint which presents as
>> a new interface;
>>
>>
>> Real Interface
>>
>> ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
>> inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255
>> inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:2508723344 <(250)%20872-3344> (2.5 GB) TX
>> bytes:1149701606 (1.1 GB)
>>
>> loopback
>>
>> lo Link encap:Local Loopback
>> inet addr:127.0.0.1 Mask:255.0.0.0
>> inet6 addr: ::1/128 Scope:Host
>> UP LOOPBACK RUNNING MTU:65536 Metric:1
>> RX packets:58936 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1
>> RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)
>>
>> ZeroTier
>>
>> zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
>> inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
>> UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1
>> RX packets:11148 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)
>>
>>
>> This is actually a SIPCapture/Homer node so the config is;
>>
>> modparam("sipcapture", "capture_on", 1)
>> modparam("sipcapture", "hep_capture_on", 1)
>> listen=udp:10.0.0.1:9060
>>
>> The port bind looks ok;
>>
>> # netstat -plunt | grep kamailio
>> udp 0 0 10.0.0.1:9060 0.0.0.0:*
>> 19592/kamailio
>>
>>
>> I’m tracking packets at the top of "route {“
>>
>> route {
>> xlog("route entered\r\n”);
>>
>>
>>
>> If I tcpdump the ZeroTier Interface ;
>>
>> tcpdump -i zt0 -p -X port 9060
>>
>> 15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP
>> 0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@
>> ..?.J..
>> 0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 0216 ...r..#d.m......
>> 0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 4b20 ....\.5#.J..ACK.
>>
>>
>> … but nothing seen in kamailio.
>>
>> If I swap over to sending to / listening on the public IP everything
>> works as expected.
>>
>>
>>
>> Cheers
>> Mark
>>
>>
>> On 12 Oct 2017, at 15:56, George Diamantopoulos <georgediam at gmail.com>
>> wrote:
>>
>> Hello,
>>
>> That seems strange... What kind of device is that VPN device? Like a ppp0
>> interface? What is the output of "netstat -plunt | grep kamailio"? Are you
>> using a "private" IP for the VPN interface and what's your main routing
>> table like?
>>
>> Also what if you omit the "listen" directive in kamailio configuration?
>> Does it listen on the VPN interface then? Kamailio will bind by default to
>> all interfaces if no "listen" directive is used...
>>
>> On 12 October 2017 at 00:26, Mark Boyce <mark at darkorigins.com> wrote:
>>
>>> Hi All
>>>
>>> Just to dive in here with something which may be related. I was about
>>> to do a post with a much simpler problem where I have one real NIC and one
>>> VPN created NIC. If I listen to the public IP all’s well. If I listen to
>>> the IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on
>>> either NIC and see the traffic arriving.
>>>
>>> Anyone seen / solved this before?
>>>
>>> Cheers
>>> Mark
>>>
>>> On 11 Oct 2017, at 22:07, George Diamantopoulos <georgediam at gmail.com>
>>> wrote:
>>>
>>> Hello Daniel,
>>>
>>> I'm including the original sketch for clarity:
>>>
>>> +----------+ +-------------------+
>>> | eth0 | | vrf-green |
>>> | 1.1.1.1 | | 127.0.0.1 |
>>> +----------+ +-------------------+
>>> |
>>> +----------+
>>> | eth1 |
>>> | 2.2.2.2 |
>>> +----------+
>>>
>>> Thanks for the reply. Indeed, using the advertise option for the
>>> "listen" directive will result in kamaiio no longer failing to start when
>>> using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the
>>> "advertise" option.
>>>
>>> However, this doesn't make much of a difference in that kamailio still
>>> won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see
>>> OPTIONS coming with sngrep, but nothing in the logs, no replies, only
>>> retransmissions from the originator.
>>> In addition, while without VRF kamailio will select the right interface
>>> with mhomed=1, this is no longer the case.
>>> Also, If I try to send something to a host with force_send_socket, the
>>> transaction fails with 477 Unfortunately error on sending to next hop
>>> occurred. In the log, kamailio prints something along the lines of:
>>> udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16): Invalid
>>> argument(22)
>>> udp_send(): invalid sendtoparameters
>>> msg_send_buffer(): udp_send failed
>>> where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060)
>>> has been called before t_relay()
>>>
>>> Also, I additionally found out that if the networking configuration is
>>> as follows:
>>>
>>> +----------+ +-------------------+ +-------------------+
>>> | eth0 | | vrf-green | | vrf-red |
>>> | 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 |
>>> +----------+ +-------------------+ +-------------------+
>>> | |
>>> +----------+ +----------+
>>> | eth1 | | eth2 |
>>> | 2.2.2.2 | | 3.3.3.3 |
>>> +----------+ +----------+
>>>
>>> Then this won't work for vrf-red:
>>> listen=vrf-green:5060 advertise 2.2.2.2:5060
>>> listen=vrf-red:5060 advertise 3.3.3.3:5060
>>> One needs to use different IPs for the MASTER devices (say 127.0.0.1 and
>>> 127.0.0.2) for kamailio to startup with the above listen directives.
>>>
>>> It's a bit surprising it doesn't work. According to a user on cumulus
>>> slack channel (cumulus contributed the VRF code to the linux kernel): "any
>>> app that has a command line or option switch to call SO_BINDTODEVICE on the
>>> sockets can be configured to use any VRF".
>>>
>>> I guess the problem lies in that kamailio binds to several sockets and
>>> there are multiple routing tables to consult, so mhomed doesn't work as
>>> expected (if every table has a default route and nothing more specific than
>>> that). But still, this doesn't really explain why binding to the VRF master
>>> net devices doesn't work (I mean, "ping -I vrf-green" works just fine for
>>> ping)...
>>>
>>> I don't think VRF support is very useful for most users though, I just
>>> had one corner case I needed to tackle and VRF seemed like an easy fix.
>>> I'll think of an alternative at some point, I guess.
>>>
>>> On 26 September 2017 at 09:52, Daniel-Constantin Mierla <
>>> miconda at gmail.com> wrote:
>>> >
>>> > Hello,
>>> >
>>> > I haven't worked with vrf here, so no first hand experience ...
>>> >
>>> > What happens if you try with next option?
>>> >
>>> > listen=vrf-green:5060 advertise 2.2.2.2:5060
>>> >
>>> > Cheers,
>>> > Daniel
>>> >
>>> >
>>> > On 25.09.17 19:10, George Diamantopoulos wrote:
>>> >
>>> > Sorry to bump this, but it would be very useful if I knew whether
>>> there's any point in pursuing this or not. Any hints?
>>> >
>>> > On 21 September 2017 at 14:06, George Diamantopoulos <
>>> georgediam at gmail.com> wrote:
>>> >>
>>> >> Hello,
>>> >>
>>> >> I have a use case where I need to have kamailio bind to a VRF device.
>>> The configuration in question is similar to the example below, where eth1
>>> is a slave to the VRF-lite device:
>>> >>
>>> >> +----------+ +-------------------+
>>> >> | eth0 | | vrf-green |
>>> >> | 1.1.1.1 | | 127.0.0.1 |
>>> >> +----------+ +-------------------+
>>> >> |
>>> >> +----------+
>>> >> | eth1 |
>>> >> | 2.2.2.2 |
>>> >> +----------+
>>> >>
>>> >> Both the main routing table and "vrf-green" routing table have a
>>> default route.
>>> >>
>>> >> What I need to be able to do is have kamailio bind to both interfaces:
>>> >>
>>> >> listen=eth0:5060
>>> >> listen=vrf-green:5060
>>> >>
>>> >> And additionally be able to use force_send_socket to select an
>>> interface, for example:
>>> >>
>>> >> force_send_socket(udp:2.2.2.2:5060);
>>> >>
>>> >> However, I can't get this to work. The above configuration fails
>>> because there is no listen directive for 2.2.2.2. Also, kamailio doesn't
>>> process packets received on the VRF with the above listen directives, it
>>> behaves as if it doesn't listen on 2.2.2.2 indeed.
>>> >>
>>> >> In addition using either of the below:
>>> >>
>>> >> listen=udp:2.2.2.2:5060
>>> >> or
>>> >> listen=eth1:5060
>>> >>
>>> >> fails with an error upon starting kamailio.
>>> >>
>>> >> According to the kernel documentation:
>>> >>
>>> >> Applications that are to work within a VRF need to bind their socket
>>> to the
>>> >> VRF device:
>>> >>
>>> >> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
>>> >>
>>> >> or to specify the output device using cmsg and IP_PKTINFO.
>>> >>
>>> >> The question is, is VRF useable with kamailio right now? Or is
>>> development needed? Thanks!
>>> >>
>>> >> BR,
>>> >>
>>> >> George
>>> >
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > Kamailio (SER) - Users Mailing List
>>> > sr-users at lists.kamailio.org
>>> > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>> >
>>> >
>>> > --
>>> > Daniel-Constantin Mierla
>>> > www.twitter.com/miconda -- www.linkedin.com/in/miconda
>>> > Kamailio Advanced Training - www.asipto.com
>>> > Kamailio World Conference - www.kamailioworld.com
>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>>
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20171012/75e55567/attachment-0001.html>
More information about the sr-users
mailing list