<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi<div class=""><br class=""></div><div class="">Now using listen=udp:0.0.0.0:9060</div><div class=""><br class=""></div><div class="">net stat agrees;</div><div class=""><br class=""></div><div class="">netstat -plunt | grep kamailio<br class="">udp 0 0 0.0.0.0:9060 0.0.0.0:* 20486/kamailio </div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">TCPDump of Sending / Receiving to zero tier port</div><div class=""><br class=""></div><div class="">17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP<br class=""> 0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@..N.J..<br class=""> 0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 0211 ...r..#d..&.....<br class=""> 0x0020: 13c4 13d8 894a abd8 894a abd8 494e 5649 .....J...J..INVI<br class=""> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:2@sbc-1.w<br class=""><br class=""><div class="">Nothing in Kamailio Logs</div><div class=""><br class=""></div><div class="">Swap to using the real IP and I’m seeing the log line from the top of the main route;</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered</span></div></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class="">and the matching tcpdump from the public interface;</span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class="">17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP<br class=""> 0x0000: 4510 03d8 863c 0000 3911 6109 894a abd8 E....<..9.a..J..<br class=""> 0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 0216 3.-...#d........<br class=""> 0x0020: 07de 13c5 5c17 3523 894a abd8 494e 5649 ....\.5#.J..INVI<br class=""> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip:<a href="mailto:2@sbc-1.w" class="">2@sbc-1.w</a><br class=""> </span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Loading the dumped packets in to wireshark I can’t see any obvious difference.</span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div class=""><br class="webkit-block-placeholder"></div><div class="">Will start up a new thread … thanks for the loan of yours :-)</div><div class=""><br class=""></div><div class="">
<div class="">Mark<br class=""><br class=""></div>
</div>
<div><br class=""><blockquote type="cite" class=""><div class="">On 12 Oct 2017, at 17:21, George Diamantopoulos <<a href="mailto:georgediam@gmail.com" class="">georgediam@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">Hmm... Everything looks fine indeed, I can't see any obvious reason why this isn't working...<br class=""><br class=""></div>Just to be clear, have you tried listening on all interfaces:<br class=""><div style="margin-left:40px" class="">listen=udp:<a href="http://0.0.0.0:9060/" class="">0.0.0.0:9060</a></div><br class=""><div class=""><div class="">And if yes, can you capture traffic arriving on the ZeroTier interface in this case? If it works, is there any reason why listening on all interfaces is not acceptable in your use case?</div><div class=""><br class=""></div><div class="">Regardless, it might be a good idea to start a new thread about this, maybe it's related to the inner workings of the ZeroTier device and how it interacts with the kernel...<br class=""></div><div class=""><br class=""></div></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On 12 October 2017 at 18:26, Mark Boyce <span dir="ltr" class=""><<a href="mailto:mark@darkorigins.com" target="_blank" class="">mark@darkorigins.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space" class="">Hi George<div class=""><br class=""></div><div class="">(IP / MACs below anonymised a bit)</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">I’m working on a <a href="https://www.zerotier.com/" target="_blank" class="">https://www.zerotier.com</a> VPN endpoint which presents as a new interface;</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Real Interface</div><div class=""><br class=""></div><div class="">ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx <br class=""> inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:<wbr class="">255.255.255.255</div><div class=""> inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link<br class=""> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<br class=""> RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0<br class=""> TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0<br class=""> collisions:0 txqueuelen:1000 <br class=""> RX bytes:2508723344 (2.5 GB) TX bytes:1149701606 (1.1 GB)<br class=""><br class="">loopback</div><div class=""><br class="">lo Link encap:Local Loopback <br class=""> inet addr:127.0.0.1 Mask:255.0.0.0<br class=""> inet6 addr: ::1/128 Scope:Host<br class=""> UP LOOPBACK RUNNING MTU:65536 Metric:1<br class=""> RX packets:58936 errors:0 dropped:0 overruns:0 frame:0<br class=""> TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0<br class=""> collisions:0 txqueuelen:1 <br class=""> RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB)<br class=""><br class="">ZeroTier</div><div class=""><br class="">zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy</div><div class=""> inet addr:10.0.0.1 Bcast:10.0.0.<wbr class="">255 Mask:255.255.255.0<br class=""> UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1<br class=""> RX packets:11148 errors:0 dropped:0 overruns:0 frame:0<br class=""> TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0<br class=""> collisions:0 txqueuelen:1000 <br class=""> RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB)<br class=""><br class=""></div><div class=""><br class=""></div><div class="">This is actually a SIPCapture/Homer node so the config is;</div><div class=""><br class=""></div><div class="">modparam("sipcapture", "capture_on", 1)<br class="">modparam("sipcapture", "hep_capture_on", 1)<br class="">listen=udp:<a href="http://10.0.0.1:9060/" target="_blank" class="">10.0.0.1:9060</a></div><div class=""><br class=""></div><div class="">The port bind looks ok;</div><div class=""><br class=""></div><div class=""># netstat -plunt | grep kamailio<br class="">udp 0 0 <a href="http://10.0.0.1:9060/" target="_blank" class="">10.0.0.1:9060</a> 0.0.0.0:* 19592/kamailio</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">I’m tracking packets at the top of "route {“</div><div class=""><br class=""></div><div class="">route { <br class=""> xlog("route entered\r\n”);<br class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">If I tcpdump the ZeroTier Interface ;</div><div class=""><br class=""></div><div class="">tcpdump -i zt0 -p -X port 9060</div><div class=""><br class=""></div><div class="">15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP<br class=""> 0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@..?.J..<br class=""> 0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 0216 ...r..#d.m......<br class=""> 0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 4b20 ....\.5#.J..ACK.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">… but nothing seen in kamailio.</div><div class=""><br class=""></div><div class="">If I swap over to sending to / listening on the public IP everything works as expected.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""><div class="">
<div class="">Cheers</div><span class="HOEnZb"><font color="#888888" class=""><div class="">Mark<br class=""><br class=""></div>
</font></span></div><div class=""><div class="h5">
<div class=""><br class=""><blockquote type="cite" class=""><div class="">On 12 Oct 2017, at 15:56, George Diamantopoulos <<a href="mailto:georgediam@gmail.com" target="_blank" class="">georgediam@gmail.com</a>> wrote:</div><br class="m_5243249516259637050Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class=""><div class="">Hello,<br class=""><br class=""></div>That seems strange... What kind of device is that VPN device? Like a ppp0 interface? What is the output of "netstat -plunt | grep kamailio"? Are you using a "private" IP for the VPN interface and what's your main routing table like?</div><div class=""><br class=""></div>Also what if you omit the "listen" directive in kamailio configuration? Does it listen on the VPN interface then? Kamailio will bind by default to all interfaces if no "listen" directive is used...<br class=""></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On 12 October 2017 at 00:26, Mark Boyce <span dir="ltr" class=""><<a href="mailto:mark@darkorigins.com" target="_blank" class="">mark@darkorigins.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class="">Hi All<div class=""><br class=""></div><div class="">Just to dive in here with something which may be related. I was about to do a post with a much simpler problem where I have one real NIC and one VPN created NIC. If I listen to the public IP all’s well. If I listen to the IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on either NIC and see the traffic arriving.</div><div class=""><br class=""></div><div class="">Anyone seen / solved this before?</div><div class=""><br class=""></div><div class="">Cheers</div><span class="m_5243249516259637050HOEnZb"><font color="#888888" class=""><div class="">Mark</div></font></span><div class=""><div class="m_5243249516259637050h5"><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On 11 Oct 2017, at 22:07, George Diamantopoulos <<a href="mailto:georgediam@gmail.com" target="_blank" class="">georgediam@gmail.com</a>> wrote:</div><br class="m_5243249516259637050m_-782933797216803214Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">Hello Daniel,<br class=""><br class="">I'm including the original sketch for clarity:<br class=""><span style="font-family:monospace,monospace" class=""><br class=""> +----------+ +-------------------+<br class=""> | eth0 | | vrf-green |<br class=""> | 1.1.1.1 | | 127.0.0.1 |<br class=""> +----------+ +-------------------+<br class=""> | <br class=""> +----------+ <br class=""> | eth1 | <br class=""> | 2.2.2.2 | <br class=""> +----------+</span><br class=""><br class="">Thanks for the reply. Indeed, using the advertise option for the "listen" directive will result in kamaiio no longer failing to start when using force_send_socket(<a href="http://2.2.2.2:5060/" target="_blank" class="">2.2.2.2:5060</a><wbr class="">) with 2.2.2.2 being used in the "advertise" option.<br class=""><br class="">However, this doesn't make much of a difference in that kamailio still won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see OPTIONS coming with sngrep, but nothing in the logs, no replies, only retransmissions from the originator.<br class=""></div><div class="">In addition, while without VRF kamailio will select the right interface with mhomed=1, this is no longer the case.</div><div class="">Also, If I try to send something to a host with force_send_socket, the transaction fails with 477 Unfortunately error on sending to next hop occurred. In the log, kamailio prints something along the lines of:</div><div style="margin-left:40px" class="">udp_send(): sendto(sock,0x7f5c4f937d08,188<wbr class="">8,0,<a href="http://9.9.9.9:5060/" target="_blank" class="">9.9.9.9:5060</a>,16): Invalid argument(22)</div><div style="margin-left:40px" class="">udp_send(): invalid sendtoparameters</div><div style="margin-left:40px" class="">msg_send_buffer(): udp_send failed</div><div class=""></div><div class="">where 9.9.9.9 is the $dd and presumably force_send_socket(<a href="http://2.2.2.2:5060/" target="_blank" class="">2.2.2.2:5060</a><wbr class="">) has been called before t_relay()<br class=""></div><div class=""><br class=""></div><div class=""></div>Also, I additionally found out that if the networking configuration is as follows:<br class=""><span style="font-family:monospace,monospace" class=""><br class=""> +----------+ +-------------------+ +-------------------+ <br class=""> | eth0 | | vrf-green | | vrf-red | <br class=""> | 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 | <br class=""> +----------+ +-------------------+ +-------------------+ <br class=""> | | <br class=""> +----------+ +----------+ <br class=""> | eth1 | | eth2 | <br class=""> | 2.2.2.2 | | 3.3.3.3 | <br class=""> +----------+ +----------+ </span><br class=""><div class=""><br class=""></div><div class="">Then this won't work for vrf-red:</div><div class=""></div><div class=""><div style="margin-left:40px" class="">listen=vrf-green:5060 advertise <a href="http://2.2.2.2:5060/" target="_blank" class="">2.2.2.2:5060</a><br class="">listen=vrf-red:5060 advertise <a href="http://3.3.3.3:5060/" target="_blank" class="">3.3.3.3:5060</a><br class=""></div>One needs to use different IPs for the MASTER devices (say 127.0.0.1 and 127.0.0.2) for kamailio to startup with the above listen directives.</div><div class=""><br class=""></div><div class="">It's a bit surprising it doesn't work. According to a user on cumulus slack channel (cumulus contributed the VRF code to the linux kernel): "any app that has a command line or option switch to call SO_BINDTODEVICE on the sockets can be configured to use any VRF".</div><div class=""><br class=""></div><div class="">I guess the problem lies in that kamailio binds to several sockets and there are multiple routing tables to consult, so mhomed doesn't work as expected (if every table has a default route and nothing more specific than that). But still, this doesn't really explain why binding to the VRF master net devices doesn't work (I mean, "ping -I vrf-green" works just fine for ping)...</div><div class=""><br class=""></div><div class="">I don't think VRF support is very useful for most users though, I just had one corner case I needed to tackle and VRF seemed like an easy fix. I'll think of an alternative at some point, I guess.<br class=""></div><div class=""><br class=""></div><div class="">On 26 September 2017 at 09:52, Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com" target="_blank" class="">miconda@gmail.com</a>> wrote:<br class="">><br class="">> Hello,<br class="">><br class="">> I haven't worked with vrf here, so no first hand experience ...<br class="">><br class="">> What happens if you try with next option?<br class="">><br class="">> listen=vrf-green:5060 advertise <a href="http://2.2.2.2:5060/" target="_blank" class="">2.2.2.2:5060</a><br class="">><br class="">> Cheers,<br class="">> Daniel<br class="">><br class="">><br class="">> On 25.09.17 19:10, George Diamantopoulos wrote:<br class="">><br class="">> Sorry to bump this, but it would be very useful if I knew whether there's any point in pursuing this or not. Any hints?<br class="">><br class="">> On 21 September 2017 at 14:06, George Diamantopoulos <<a href="mailto:georgediam@gmail.com" target="_blank" class="">georgediam@gmail.com</a>> wrote:<br class="">>><br class="">>> Hello,<br class="">>><br class="">>> I have a use case where I need to have kamailio bind to a VRF device. The configuration in question is similar to the example below, where eth1 is a slave to the VRF-lite device:<br class="">>><br class="">>> +----------+ +-------------------+<br class="">>> | eth0 | | vrf-green |<br class="">>> | 1.1.1.1 | | 127.0.0.1 |<br class="">>> +----------+ +-------------------+<br class="">>> | <br class="">>> +----------+ <br class="">>> | eth1 | <br class="">>> | 2.2.2.2 | <br class="">>> +----------+<br class="">>><br class="">>> Both the main routing table and "vrf-green" routing table have a default route.<br class="">>><br class="">>> What I need to be able to do is have kamailio bind to both interfaces:<br class="">>><br class="">>> listen=eth0:5060<br class="">>> listen=vrf-green:5060<br class="">>><br class="">>> And additionally be able to use force_send_socket to select an interface, for example:<br class="">>><br class="">>> force_send_socket(udp:<a href="http://2.2.2.2:5060/" target="_blank" class="">2.2.2.2:<wbr class="">5060</a>);<br class="">>><br class="">>> However, I can't get this to work. The above configuration fails because there is no listen directive for 2.2.2.2. Also, kamailio doesn't process packets received on the VRF with the above listen directives, it behaves as if it doesn't listen on 2.2.2.2 indeed.<br class="">>><br class="">>> In addition using either of the below:<br class="">>><br class="">>> listen=udp:<a href="http://2.2.2.2:5060/" target="_blank" class="">2.2.2.2:5060</a><br class="">>> or<br class="">>> listen=eth1:5060<br class="">>><br class="">>> fails with an error upon starting kamailio.<br class="">>><br class="">>> According to the kernel documentation:<br class="">>><br class="">>> Applications that are to work within a VRF need to bind their socket to the<br class="">>> VRF device:<br class="">>><br class="">>> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);<br class="">>><br class="">>> or to specify the output device using cmsg and IP_PKTINFO.<br class="">>><br class="">>> The question is, is VRF useable with kamailio right now? Or is development needed? Thanks!<br class="">>><br class="">>> BR,<br class="">>><br class="">>> George<br class="">><br class="">><br class="">><br class="">><br class="">> ______________________________<wbr class="">_________________<br class="">> Kamailio (SER) - Users Mailing List<br class="">> <a href="mailto:sr-users@lists.kamailio.org" target="_blank" class="">sr-users@lists.kamailio.org</a><br class="">> <a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank" class="">https://lists.kamailio.org/cgi<wbr class="">-bin/mailman/listinfo/sr-users</a><br class="">><br class="">><br class="">> -- <br class="">> Daniel-Constantin Mierla<br class="">> <a href="http://www.twitter.com/miconda" target="_blank" class="">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank" class="">www.linkedin.com/in/miconda</a><br class="">> Kamailio Advanced Training - <a href="http://www.asipto.com/" target="_blank" class="">www.asipto.com</a><br class="">> Kamailio World Conference - <a href="http://www.kamailioworld.com/" target="_blank" class="">www.kamailioworld.com</a><br class=""><br class=""></div></div>
______________________________<wbr class="">_________________<br class="">Kamailio (SER) - Users Mailing List<br class=""><a href="mailto:sr-users@lists.kamailio.org" target="_blank" class="">sr-users@lists.kamailio.org</a><br class=""><a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank" class="">https://lists.kamailio.org/cgi<wbr class="">-bin/mailman/listinfo/sr-users</a><br class=""></div></blockquote></div><br class=""><div class="">
<div class=""><div class=""><br class=""></div></div></div></div></div></div></div><br class="">______________________________<wbr class="">_________________<br class="">
Kamailio (SER) - Users Mailing List<br class="">
<a href="mailto:sr-users@lists.kamailio.org" target="_blank" class="">sr-users@lists.kamailio.org</a><br class="">
<a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" rel="noreferrer" target="_blank" class="">https://lists.kamailio.org/cgi<wbr class="">-bin/mailman/listinfo/sr-users</a><br class="">
<br class=""></blockquote></div><br class=""></div>
______________________________<wbr class="">_________________<br class="">Kamailio (SER) - Users Mailing List<br class=""><a href="mailto:sr-users@lists.kamailio.org" target="_blank" class="">sr-users@lists.kamailio.org</a><br class=""><a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank" class="">https://lists.kamailio.org/<wbr class="">cgi-bin/mailman/listinfo/sr-<wbr class="">users</a><br class=""></div></blockquote></div><br class=""></div></div></div></div><br class="">______________________________<wbr class="">_________________<br class="">
Kamailio (SER) - Users Mailing List<br class="">
<a href="mailto:sr-users@lists.kamailio.org" class="">sr-users@lists.kamailio.org</a><br class="">
<a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" rel="noreferrer" target="_blank" class="">https://lists.kamailio.org/<wbr class="">cgi-bin/mailman/listinfo/sr-<wbr class="">users</a><br class="">
<br class=""></blockquote></div><br class=""></div>
_______________________________________________<br class="">Kamailio (SER) - Users Mailing List<br class=""><a href="mailto:sr-users@lists.kamailio.org" class="">sr-users@lists.kamailio.org</a><br class="">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users<br class=""></div></blockquote></div><br class=""></div></body></html>