Hi Sérgio
I've dealt with these sorts of problems before. This can get very
complicated/confusing. It doesn't make sense to me that _enabling_ the
RPF fixed it; I'd expect it to be the other way around: disabling it
should help.
I've noticed Linux machines doing funny things in the past with
answering ARP queries from "wrong" interfaces. If I have ip1 on eth1
and ip2 on eth2, and an ARP comes for ip2 on eth1, then I've seen a
response there.
It's possible that your two firewalls send traffic to different
network interfaces on the machine, and that's what leads to the issue.
That's my best guess. It might be simply a matter of time before you
see the same problem on the machine that has had no problems yet.
If I was to troubleshoot this further, I'd set up network captures
separately (for ARP and SIP) on the three network interfaces and see
what happens. That might tell you more, and you might see traffic on a
"wrong" interface. It can be complicated if VLANs are involved or if a
subnet mask is set incorrectly by mistake, etc.
(Before you mentioned the cause, I could have suspected fragmentation,
as Henning mentioned. An ALG might wreck a message so much that the
UDP length or UDP checksum might be wrong, so I would capture traffic
and check those in Wireshark. If the checksum is wrong, then kamailio
will never even see the message.)
James
On Thu, 19 Sept 2024 at 14:29, Sergio Charrua via sr-users
<sr-users(a)lists.kamailio.org> wrote:
Hi all!
Apparently, the issue was not within the Firewall.
As it happens, all 3 nodes have 3 different NICs, each with different subnets but in the
same IP range, meaning, subnet 1 is 10.10.10.3/27, subnet 2 is 10.10.10.67/27 and subnet3
is 10.10.10.94/27 (for example). Have no idea why they kept the same IP range but
different subnets...
What we found, just by a lucky guess, is that the rp_filter is turned off. The OS
parameter is: net.ipv4.conf.all.rp_filter
What we did is to turn this parameter on (setting net.ipv4.conf.all.rp_filter = 1) and
bingo! Kamailio started to process SIP messages!
But still, there are some really weird questions to answer:
1 - the original setup has been working with 2 nodes since December 2018!!! without
issues!!! Why now?!
2 - Node #2 never had issues. Node #1 started to have issues since client changed his
firewall from CISCO for Fortinet. Why?!
3 - Node #2 was cloned to Node #3 and the issue also appeared on Node #3, even though
Node #3 is a carbon copy of Node #2 that always worked!!! In fact, Node #2 hasn't
stopped working yet!!! Why the different behaviour?!
4 - Node #2 is still working and net.ipv4.conf.all.rp_filter is still OFF
(net.ipv4.conf.all.rp_filter=0). So why Node #1 and Node #3 had to turn the value to 1 ?!
5 - all nodes have the same routing table, the same routes defined. Why Node #1 and Node
#3 have different behaviours?!
6 - Firewall has the exact same rules for *all* nodes! So why different behaviours
between different nodes?!
For anyone who has a Kamailio server with 3 network interfaces and facing issues that the
packets are discarded from the OS because, apparently, the OS doesn't know how to
route packets even though the routes are defined, just set net.ipv4.conf.all.rp_filter =
1
This is new stuff for me....
rp_filter - Understanding the reverse path filter in Linus for your LPIC-3
(
theurbanpenguin.com)
Atenciosamente / Kind Regards / Cordialement / Un saludo,
Sérgio Charrua
On Thu, Sep 19, 2024 at 9:29 AM Sergio Charrua <sergio.charrua(a)voip.pt> wrote:
Thank you both for your replies.
Client followed some of mine guidelines and has proven that the issue is with the
Fortinet F5 firewall: he made a snapshot of the kamailio node #2 that we know is 100%
working, installed the snapshot, modified the IPs, and made tests, and the behaviour is
identical to node #1. The relevant question is why node #2, who has the same rules as node
#1, is working fine, but not nodes #1 and #3 ?! doesn't make much sense...
Meanwhile, client has been dealing with the issue with Fortinet's engineering team.
Looks like a firmware issue...
Atenciosamente / Kind Regards / Cordialement / Un saludo,
Sérgio Charrua
On Wed, Sep 18, 2024 at 10:35 PM Karsten Horsmann <khorsmann(a)gmail.com> wrote:
>
> Hi Sergio,
>
> You answer the right questions. Works it before, yes. Changed the Kamailio system,
no. So it's an Fortinet fault.
>
> By painful experience with Cisco and Fortinet stuff I know that you need to tweak the
Fortnite (SIP ALG, SIP POLICY and so one).
>
> Go in that direction cause it works with Cisco before and the Fortigate seems to
break it.
>
> Kind regards
>
> Sergio Charrua via sr-users <sr-users(a)lists.kamailio.org> schrieb am Mi., 18.
Sept. 2024, 10:19:
>>
>> Hi all!
>>
>> got a client with 2 Kamailio servers using Pacemaker/Corosync with a VIP.
>> Kamailio is setup with dispatcher, and has been working since 2018.
>>
>> This previous week, client migrated from a Cisco ASA firewall to a Fortinet F5
firewall.
>> One of the Kamailio server is, apparently, working fine, receiving SIP invites
and processing them correctly.
>> When moving the VIP to the second Kamailio node, Kamailio is ignoring any SIP
message that is not OPTIONS, despite configuration (kamailio.cfg) being exactly the same
as in the node #1.
>> Also, note that using SNGrep, we can see clearly that SIP messages are reaching
the server, but Kamailio is just not processing any INVITE. In fact, I added an output to
log file right at the very beginning of the main route and the line is added to log file
ONLY if the SIP message is OPTIONS. When the SIP message is INVITE, nothing is processed
on Kamailio's side.
>>
>> The listen parameter is set to 0.0.0.0 port 5060 and advertised IP set to the
Public IP.
>>
>> Again, the main route is only :
>>
>> route{
>> xlog(. ......);
>> }
>>
>> Kamailio version is 5.1.6 (old, I know....) but it has been working since Dec
2018!
>> What could be the issue?!
>> Why only OPTIONS are being , somehow, recognized by Kamailio and all other SIP
messages just ignored?
>>
>> Also, to add some more weirdness to the mess, connecting an Asterisk server on
the same network, and sending a call to Kamailio node #2, it works!! SIP messages of any
kind are processed correctly by Kamailio on node #2.
>> So, why doesn't it work with SIP messages received via Public IP address,
even though the messages reach the server correctly (confirmed using SNGrep) ???
>>
>> I spent 5h this afternoon trying to understand what is wrong, but I'm stuck
.... and no clue so far, apart from suspecting of the new Gateway, which I have no access
to configurations.... but if OPTIONS are received & processed OK, it wouldn't make
sense that received INVITES would be ignored by Kamailio, right?
>>
>> Atenciosamente / Kind Regards / Cordialement / Un saludo,
>>
>>
>> Sérgio Charrua
>>
>>
>> __________________________________________________________
>> Kamailio - Users Mailing List - Non Commercial Discussions
>> To unsubscribe send an email to sr-users-leave(a)lists.kamailio.org
>> Important: keep the mailing list in the recipients, do not reply only to the
sender!
>> Edit mailing list options or unsubscribe:
__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
To unsubscribe send an email to sr-users-leave(a)lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe: