[SR-Users] unable to receive UDP
dem at htsweb.com
dem at htsweb.com
Thu Oct 14 00:08:05 CEST 2021
Hi Fred,
We'll, problem solved...
I was able to duplicate the problem with the IP configured on em3 using # nc
-lvu 10.102.88.81 5060. No traffic.
As stated before, the IP works on interface em1(10g), not em3(1g). Centos 7,
Dell R630. Firewalld correct.
I started digging into the Dell Broadcom BCM57800 card. Numerous reports of
issues with the ports not working. Which comes as quite a surprise, considering
we have 8 x R630's running the cards since January!
I've never seen a network interface not pass the packet to the stack - kind of
the whole point. Ordering replacement Dell/Intel cards now.
Thanks to all for your help!!! Anxious to put Kamailio in production to handle
STIR/SHAKEN (and maybe some cool routing).
Thanks,
Dan
Fred Posner wrote on 10/13/21 9:39 AM:
> I think at this point, it would be best to rule out your
> system/environment before debugging code.
>
> Have you see if another application layer program can see the SIP
> traffic (such as sipp)?
>
> If a netstat shows kamailio listening on the IP/port, and kamailio
> didn't indicate any errors binding to the IP provided, we must further
> rule out something on the system.
>
> Fred Posner | palner.com
> Matrix: @fred:matrix.lod.com
> o: +1 (212) 937-7844
>
> On 10/13/21 7:58 AM, dem at htsweb.com wrote:
>> Hi Sergey,
>>
>> Only testing with the SIP INVITE from one of our production SIP trunk
>> providers. The 'length's are listed below for the tcpdump. Same packet
>> size/contents for all tests. Only change was moving IP from interface
>> em3 to em1. Doesn't look like packet size is the issue.
>>
>> Looking at some of the socket code. Could be related to socket structs,
>> maybe an array enumeration issue. We have 4 interfaces: em1 ((10g up),
>> em2(10g down), em3 (1g up), em4 (1g up). Just switched IP from 3rd
>> interface (2nd up) to 1st interface (1st up) - it worked.
>>
>> If somebody would like to provide a debug message/location, I could
>> recompile to test the array bucket theory. Quick way to check.
>>
>> Note, the config below is the entire test config. No changes. Only
>> purpose was to receive SIP INVITE to test socket.
>>
>> Thanks,
>> Dan
>>
>> Sergey Safarov wrote on 10/13/21 4:08 AM:
>>> you also need to check UDP packets are fragmented or not.
>>> you must receive all fragments before Kamailio is able to parse the
>>> message.
>>>
>>> On Wed, Oct 13, 2021 at 1:28 AM David Villasmil
>>> <david.villasmil.work at gmail.com
>>> <mailto:david.villasmil.work at gmail.com>> wrote:
>>>
>>> use the
>>>
>>> listen=udp:YOUR-LOCAL-IP:5060
>>>
>>> then do a netstat to see if it's listening on 5060
>>>
>>>
>>> Regards,
>>>
>>> David Villasmil
>>> email: david.villasmil.work at gmail.com
>>> <mailto:david.villasmil.work at gmail.com>
>>> phone: +34669448337
>>>
>>>
>>> On Tue, Oct 12, 2021 at 11:17 PM <dem at htsweb.com
>>> <mailto:dem at htsweb.com>> wrote:
>>>
>>> Okay, think I discovered something....
>>>
>>> I'm attempting to run the kamailio on interface em3 - no joy.
>>> I moved the 10.102.88.81 ip to the em1 interface. kamailio is
>>> showing the UDP packets as received (via my horribly simple
>>> config below).
>>>
>>> This test also answers the firewall question, UDP port is allowed.
>>>
>>> So, that prompts the question... how do you use kamailio with
>>> UDP on a specific interface ? I'm thinking it might not do it
>>> currently ?
>>>
>>> Also, forgive me if I messed up the prior reply to the list.
>>>
>>> Thanks,
>>> Dan
>>>
>>> dem at htsweb.com <mailto:dem at htsweb.com> wrote on 10/12/21 5:28 PM:
>>>> Hello,
>>>>
>>>> The tcpdump (below) shows packet reaching em3 interface. The
>>>> firewalld command (below) shows UDP port 5060 open.
>>>>
>>>> netcat doesn't do specific interfaces. Can't do ALL
>>>> interfaces, since em4 is a pcap of hundreds of thousands of
>>>> live UDP 5060.
>>>>
>>>> I'll have to write a standalone to prove that the firewall
>>>> allows UDP 5060 through the em3 interface, post firewall...
>>>> (betting it does). Should have test prog tomorrow.
>>>>
>>>> Thanks,
>>>> Dan
>>>>
>>>> Daniel-Constantin Mierla wrote on 10/12/21 4:27 PM:
>>>>> Hello,
>>>>>
>>>>> tcpdump and other network sniffers (e.g., ngrep, sngrep)
>>>>> capture the sip traffic before firewall, so even if you see
>>>>> packets with them, the application can still not get them
>>>>> due to firewall.
>>>>>
>>>>> To test, you can replace kamailio with netcat listening on
>>>>> UDP port 5060 and see if it receives the sip traffic.
>>>>>
>>>>> Cheers,
>>>>> Daniel
>>>>>
>>>>> On 12.10.21 19:08, dem at htsweb.com <mailto:dem at htsweb.com> wrote:
>>>>>> After 4 (long) days of trying to receive any UDP SIP, I'm
>>>>>> hoping someone can point out what I'm doing wrong. I've
>>>>>> looked at the kamailio code. I'm beginning to think it's
>>>>>> the 'polling' code, but it looks fine. It could be Centos
>>>>>> or a Dell network driver, but not seeing any errors anywhere.
>>>>>>
>>>>>> - stock build/compile...
>>>>>> Version: kamailio 5.5.2 (x86_64/linux) 0d53d9
>>>>>> Compile flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS,
>>>>>> USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK,
>>>>>> SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC,
>>>>>> DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT,
>>>>>> USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR,
>>>>>> USE_DST_BLOCKLIST, HAVE_RESOLV_RES
>>>>>> MAX_RECV_BUFFER_SIZE=262144
>>>>>> MAX_URI_SIZE=1024
>>>>>> BUF_SIZE=65535
>>>>>> DEFAULT PKG_SIZE=8MB
>>>>>> DEFAULT SHM_SIZE=64MB
>>>>>> ADAPTIVE_WAIT_LOOPS=1024
>>>>>> TCP poll methods: poll, epoll_lt, epoll_et, sigio_rt, select
>>>>>> Source code revision ID: 0d53d9
>>>>>> Compiled with: gcc 4.8.5
>>>>>> Compiled architecture: x86_64
>>>>>> Compiled on: 08:44:19 Oct 11 2021
>>>>>>
>>>>>> Running Centos 7 all updated. Dell R630.
>>>>>> # sestatus
>>>>>> SELinux status: disabled
>>>>>> # firewall-cmd --list-all
>>>>>> ports: 5666/tcp 873/tcp 873/udp 80/tcp 443/tcp 5060/tcp
>>>>>> 5060/udp 5061/tcp 5061/udp
>>>>>>
>>>>>> - Just trying to send SIP INVITE from trunk provider to
>>>>>> R630 kamailio...
>>>>>> (SIP trunk) UDP 172.200.200.202:5060
>>>>>> <http://172.200.200.202:5060> -> (firewall) UDP
>>>>>> 172.250.250.162:5060 <http://172.250.250.162:5060> -> 1:1
>>>>>> NAT -> (R630) UDP 10.102.88.81:5060 <http://10.102.88.81:5060>
>>>>>>
>>>>>> - I can 'see' the packet hit the em3 network...
>>>>>> # tcpdump -i em3 -e -vv -n port 5060
>>>>>> 11:53:33.260892 99:99:ef:3e:69:c1 > 99:99:1c:48:4e:e3,
>>>>>> ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 54, id
>>>>>> 20430, offset 0, flags [none], proto UDP (17), length 1352)
>>>>>> 172.200.200.202.sip > 10.102.88.81.sip: [udp sum ok]
>>>>>> SIP, length: 1324
>>>>>> INVITE sip:+18005551234 at 172.250.250.162:5060
>>>>>> <mailto:sip:+18005551234 at 172.250.250.162:5060> SIP/2.0
>>>>>> ...
>>>>>>
>>>>>> At this point, I just need to see a SIP INVITE received.
>>>>>> Then I can go back to working on a real config.
>>>>>>
>>>>>> I've tried with TCP on/off, listening on all networks,
>>>>>> alias on/off, strace and everything else I could think
>>>>>> of.Truly appreciate any guidance.
>>>>>>
>>>>>> Tried every sample config possible, but I just need to test
>>>>>> receiving the SIP UDP packet, so this 'should' be minimum
>>>>>> config...
>>>>>>
>>>>>> #!KAMAILIO
>>>>>> debug=9
>>>>>> log_stderror=no
>>>>>> log_facility=LOG_LOCAL0
>>>>>> log_prefix="{$mt $hdr(CSeq) $ci} "
>>>>>> children=2
>>>>>> disable_tcp=yes
>>>>>> udp4_raw=on
>>>>>> auto_aliases=no
>>>>>> alias="sbc01.mysite.com <http://sbc01.mysite.com>"
>>>>>> listen=udp:10.102.88.81:5060
>>>>>>
>>>>>> loadmodule "debugger.so"
>>>>>> loadmodule "jsonrpcs.so"
>>>>>> loadmodule "kex.so"
>>>>>> loadmodule "corex.so"
>>>>>> loadmodule "sl.so"
>>>>>> loadmodule "rr.so"
>>>>>> loadmodule "pv.so"
>>>>>> loadmodule "textops.so"
>>>>>> loadmodule "siputils.so"
>>>>>> loadmodule "xlog.so"
>>>>>> loadmodule "sanity.so"
>>>>>> loadmodule "ctl.so"
>>>>>> loadmodule "cfg_rpc.so"
>>>>>>
>>>>>> modparam("debugger", "cfgtrace", 1)
>>>>>>
>>>>>> request_route {
>>>>>> xlog("SIP received xxxxxxxxxxxxxxxxxxx\n");
>>>>>> }
>>>>>>
>>>>>> # kamcmd ps
>>>>>> 11578 main process - attendant
>>>>>> 11579 udp receiver child=0 sock=10.102.88.81:5060
>>>>>> <http://10.102.88.81:5060>
>>>>>> 11580 udp receiver child=1 sock=10.102.88.81:5060
>>>>>> <http://10.102.88.81:5060>
>>>>>> 11581 slow timer
>>>>>> 11583 timer
>>>>>> 11584 secondary timer
>>>>>> 11585 JSONRPCS FIFO
>>>>>> 11586 JSONRPCS DATAGRAM
>>>>>> 11587 ctl handler
>>>>>>
>>>>>> - I get plenty of logging, but the logging just waits here...
>>>>>> Oct 12 11:38:04 sbc01 /usr/local/sbin/kamailio[10060]:
>>>>>> INFO: ctl [io_listener.c:210]: io_listen_loop():
>>>>>> io_listen_loop: using epoll_lt as the io watch method (auto
>>>>>> detected)
>>>>>> Oct 12 11:38:04 sbc01 /usr/local/sbin/kamailio[10060]:
>>>>>> DEBUG: <core> [core/mem/q_malloc.c:374]: qm_malloc():
>>>>>> qm_malloc(0x7fd2a8bae010, 4800) called from core:
>>>>>> core/io_wait.c: init_io_wait(469)
>>>>>> Oct 12 11:38:04 sbc01 /usr/local/sbin/kamailio[10059]:
>>>>>> DEBUG: <core> [core/mem/q_malloc.c:419]: qm_malloc():
>>>>>> qm_malloc(0x7fd2a8bae010, 65456) returns address
>>>>>> 0x7fd2a8c652d8 frag. 0x7fd2a8c652a0 (size=65456) on 1 -th hit
>>>>>> Oct 12 11:38:04 sbc01 /usr/local/sbin/kamailio[10060]:
>>>>>> DEBUG: <core> [core/mem/q_malloc.c:419]: qm_malloc():
>>>>>> qm_malloc(0x7fd2a8bae010, 4800) returns address
>>>>>> 0x7fd2a8c652d8 frag. 0x7fd2a8c652a0 (size=4800) on 1 -th hit
>>>>>> Oct 12 11:38:04 sbc01 /usr/local/sbin/kamailio[10060]:
>>>>>> DEBUG: <core> [core/mem/q_malloc.c:374]: qm_malloc():
>>>>>> qm_malloc(0x7fd2a8bae010, 2400) called from core:
>>>>>> core/io_wait.c: init_io_wait(516)
>>>>>> Oct 12 11:38:04 sbc01 /usr/local/sbin/kamailio[10060]:
>>>>>> DEBUG: <core> [core/mem/q_malloc.c:419]: qm_malloc():
>>>>>> qm_malloc(0x7fd2a8bae010, 2400) returns address
>>>>>> 0x7fd2a8c66600 frag. 0x7fd2a8c665c8 (size=2400) on 1 -th hit
>>>>>> Oct 12 11:38:04 sbc01 /usr/local/sbin/kamailio[10060]:
>>>>>> DEBUG: ctl [io_listener.c:246]: io_listen_loop():
>>>>>> io_listen_loop: adding socket 8, type 2, transport 3
>>>>>> (/var/run/kamailio//kamailio_ctl)
>>>>>> Oct 12 11:38:04 sbc01 /usr/local/sbin/kamailio[10060]:
>>>>>> DEBUG: ctl [../../core/io_wait.h:375]: io_watch_add(): DBG:
>>>>>> io_watch_add(0x7fd2a70a5360, 8, 2, 0x2439960), fd_no=0
>>>>>>
>>>>>> - The only other thing that I see is transport 3. Isn't
>>>>>> that a UNIXS_SOCK?
>>>>>>
>>>>>>
>>>>>>
>>>>>> __________________________________________________________
>>>>>> Kamailio - Users Mailing List - Non Commercial Discussions
>>>>>> * sr-users at lists.kamailio.org <mailto:sr-users at lists.kamailio.org>
>>>>>> Important: keep the mailing list in the recipients, do not reply only to the sender!
>>>>>> Edit mailing list options or unsubscribe:
>>>>>> * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>>>>> --
>>>>> Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com>
>>>>> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
>>>>> Kamailio Advanced Training - Online
>>>>> Nov 08-11, 2021 (Europe Timezone) - Nov 22-25, 2021 (America Timezone)
>>>>> * https://www.asipto.com/sw/kamailio-advanced-training-online/ <https://www.asipto.com/sw/kamailio-advanced-training-online/>
>>> __________________________________________________________
>>> Kamailio - Users Mailing List - Non Commercial Discussions
>>> * sr-users at lists.kamailio.org
>>> <mailto:sr-users at lists.kamailio.org>
>>> Important: keep the mailing list in the recipients, do not
>>> reply only to the sender!
>>> Edit mailing list options or unsubscribe:
>>> *
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>> <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>>>
>>> __________________________________________________________
>>> Kamailio - Users Mailing List - Non Commercial Discussions
>>> * sr-users at lists.kamailio.org <mailto:sr-users at lists.kamailio.org>
>>> Important: keep the mailing list in the recipients, do not reply
>>> only to the sender!
>>> Edit mailing list options or unsubscribe:
>>> * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>> <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>>>
>>
>> __________________________________________________________
>> Kamailio - Users Mailing List - Non Commercial Discussions
>> * sr-users at lists.kamailio.org
>> Important: keep the mailing list in the recipients, do not reply only to the sender!
>> Edit mailing list options or unsubscribe:
>> * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
> __________________________________________________________
> Kamailio - Users Mailing List - Non Commercial Discussions
> * sr-users at lists.kamailio.org
> Important: keep the mailing list in the recipients, do not reply only to the sender!
> Edit mailing list options or unsubscribe:
> * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20211013/96a4caa7/attachment.htm>
More information about the sr-users
mailing list