UDP fragmentation issues
Hi, I have setup Kamailio on Centos 7 as a proxy in front of Asterisk in the same datacenter, using the dispatcher module. I have noted that large INVITEs sent by Kamailio do not arrive at the Asterisk box. It seems the network is dropping fragmented UDP traffic. This does not happen however when I contact the Asterisk box directly from outside the datacenter. Then large INVITES are arriving are Asterisk. So the problem is either in Kamailio or in the datacenter. I have analysed the problem with ngrep and tcpdump/wireshark. In normal mode (udp4_raw=no setting) Kamailio is producing UDP packets above 1500 bytes and leaves all fragmentation to the operating system. Enabling raw udp in Kamailio and lowering the MTU in the server didnt help. I even disabled offloading of UDP fragmentation to the NIC to get to the bottom of this. However nothing helps. Only forcing TCP in the dispatcher works when having large INVITES going to Asterisk. In any case, it appears that there is no guarantee that fragmented UDP arrives at all. As far as I am aware, the recommendation is to use TCP if messages are bigger than 1400 bytes. If so, how can I force Kamailio to send messages over 1400 bytes only in TCP mode? Or is there some other way how to deal with this? Thanks for any hints. Gerry
Hello Gerry, not sure if you already figured it out.. Have you tried to just check for the message length with the PV $ml in the relevant route, and then send over TCP if it is larger as 1400 bytes? Cheers, Henning -- Henning Westerholt - https://skalatan.de/blog/ Kamailio services - https://skalatan.de/services -----Original Message----- From: sr-users <sr-users-bounces@lists.kamailio.org> On Behalf Of Gerry | x164 Sent: Friday, June 21, 2019 12:33 PM To: sr-users@lists.kamailio.org Subject: [SR-Users] UDP fragmentation issues Hi, I have setup Kamailio on Centos 7 as a proxy in front of Asterisk in the same datacenter, using the dispatcher module. I have noted that large INVITEs sent by Kamailio do not arrive at the Asterisk box. It seems the network is dropping fragmented UDP traffic. This does not happen however when I contact the Asterisk box directly from outside the datacenter. Then large INVITES are arriving are Asterisk. So the problem is either in Kamailio or in the datacenter. I have analysed the problem with ngrep and tcpdump/wireshark. In normal mode (udp4_raw=no setting) Kamailio is producing UDP packets above 1500 bytes and leaves all fragmentation to the operating system. Enabling raw udp in Kamailio and lowering the MTU in the server didnt help. I even disabled offloading of UDP fragmentation to the NIC to get to the bottom of this. However nothing helps. Only forcing TCP in the dispatcher works when having large INVITES going to Asterisk. In any case, it appears that there is no guarantee that fragmented UDP arrives at all. As far as I am aware, the recommendation is to use TCP if messages are bigger than 1400 bytes. If so, how can I force Kamailio to send messages over 1400 bytes only in TCP mode? Or is there some other way how to deal with this? Thanks for any hints. Gerry _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi Henning, a great idea thanks! I found that the datacenter had some wrong configuration which blocked large UDP packets. They have now fixed that. However the principal problem remains that such blocking can happen elsewhere outside my control. What are best practices regarding large UDP traffic? I have never encountered this issue in the last 10 years so I am really at a loss whether I need to deal with this. Best Gerry
On 12 Jul 2019, at 23:43, Henning Westerholt <hw@skalatan.de> wrote:
Hello Gerry,
not sure if you already figured it out.. Have you tried to just check for the message length with the PV $ml in the relevant route, and then send over TCP if it is larger as 1400 bytes?
Cheers,
Henning
-- Henning Westerholt - https://skalatan.de/blog/ Kamailio services - https://skalatan.de/services
-----Original Message----- From: sr-users <sr-users-bounces@lists.kamailio.org> On Behalf Of Gerry | x164 Sent: Friday, June 21, 2019 12:33 PM To: sr-users@lists.kamailio.org Subject: [SR-Users] UDP fragmentation issues
Hi,
I have setup Kamailio on Centos 7 as a proxy in front of Asterisk in the same datacenter, using the dispatcher module. I have noted that large INVITEs sent by Kamailio do not arrive at the Asterisk box. It seems the network is dropping fragmented UDP traffic. This does not happen however when I contact the Asterisk box directly from outside the datacenter. Then large INVITES are arriving are Asterisk. So the problem is either in Kamailio or in the datacenter.
I have analysed the problem with ngrep and tcpdump/wireshark. In normal mode (udp4_raw=no setting) Kamailio is producing UDP packets above 1500 bytes and leaves all fragmentation to the operating system. Enabling raw udp in Kamailio and lowering the MTU in the server didnt help. I even disabled offloading of UDP fragmentation to the NIC to get to the bottom of this. However nothing helps. Only forcing TCP in the dispatcher works when having large INVITES going to Asterisk.
In any case, it appears that there is no guarantee that fragmented UDP arrives at all. As far as I am aware, the recommendation is to use TCP if messages are bigger than 1400 bytes. If so, how can I force Kamailio to send messages over 1400 bytes only in TCP mode?
Or is there some other way how to deal with this?
Thanks for any hints.
Gerry
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi Gerry, On Mon, Jul 15, 2019 at 10:56:12AM +0300, Gerry | x164 wrote:
What are best practices regarding large UDP traffic? I have never encountered this issue in the last 10 years so I am really at a loss whether I need to deal with this.
At the risk of self-plugging, I discuss this in an article: http://www.evaristesys.com/blog/sip-udp-fragmentation-and-kamailio-the-sip-h... -- Alex -- Alex Balashov | Principal | Evariste Systems LLC Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
participants (3)
-
Alex Balashov -
Gerry | x164 -
Henning Westerholt