[sr-dev] [kamailio/kamailio] Path MTU handling; implicit IPv4 assumption, suggested solution for IPv6 (Issue #3119)

[vanrein]

### Description

Larger SIP frames get dropped when sent over UDP and IPv6.  The sending side has MTU 1500 and the receiving side has MTU 1492.  This is an IPv6-only setup, so this is problematic.  Also, pulling down the MTU of a general server for a tunneled peer would smear like an oil stain.  The suggestion to fallback on TCP feels like a hack.

### Troubleshooting

#### Reproduction

Send a SIP message to a network interface with a lower MTU than the submitted frame size.

#### Debugging Data

None, transmission works fine.

#### Log Messages

None, transmission works fine.

#### SIP Traffic

Irrelevant.

#### Code Investigation

I explored the Kamailio source code for MTU handling.

https://github.com/kamailio/kamailio/blob/81265e41b52cfda9a284233c93683522a98f0a64/src/core/udp_server.c#L340-L349

  * This defaults to switching off the Don't Fragment bit on IPv4 frames, to allow them to be broken up downstream.
  * This should check for an IPv4 socket, because (at least) Linux has a separate symbolic value `IPV6_MTU_DISCOVER`
  * Note that IPv6 is never broken up by routers; they always return ICMPv6 message Packet Too Big
  * Note that IPv6 Path MTU discovery by the kernel is automatic only for connected UDP sockets
  * Since Kamailio uses unconnected UDP sockets, Path MTU issues cause packets to be dropped
  * Note that such packet drops depend on a somewhat dynamic SIP message size, causing random behaviour
  * I therefore suggest that Kamailio is lacking in some of its IPv6 logic

https://github.com/kamailio/kamailio/blob/81265e41b52cfda9a284233c93683522a98f0a64/src/core/udp_server.c#L331-L339

  * This enables the reporting of ICMP errors, including Path MTU but also other useful things like Host Not Found
  * This should check for an IPv4 socket, because (at least) Linux has a separate symbolic value `IPV6_RECVERR`
  * Note that the same behaviour is available on BSD, so it need not be specific to Linux
  * Note that handling errors is done with `recvmsg()` with a flag `MSG_ERRQUEUE`, which is not used in Kamailio
  * Note that the ancillary data from `recvmsg()` holds data to cleverly handle ICMP or ICMPv6 responses
  * I therefore suggest that Kamailio is lacking a fair chunk of its ICMP and ICMPv6 logic
  * I expect that this may bring efficiency gains due to faster closing of transactions

### Possible Solutions

I have been thinking about ways to lower MTU values only for some peers.

 1. Using connected sockets might work, possibly as an alternative when Path MTU problems arise.  It might not scale however.
 2. Every socket could have an extra sending socket set to a lower MTU.  The use of `SO_REUSEADDR` seems to allow for that.
 3. Before falling back on an extra socket, the desired MTU could be set.  Alternatively, as for IPv6, an MTU of 1280 might be considered in  many cases:

      - *If you can carry 6 out of 8 coffee mugs from the kitchen, you need to walk twice, and 4+4 is easier than 6+2*
      - Anything over the MTU splits into at least 2 frames
      - The headers added are 40 bytes IPv6 header and 8 bytes Fragment Extension Header
      - 2 frames can hold a single frame of 2*1280-40-8 = 2512 bytes
      - Up to 2512 bytes original MTU, breakup in 1280 byte frames will be fine
      - 3 frames are going to be useful for larger frames, then a similar style can be used
      - This might help to decide whether 1280 or a higher MTU is most desirable

### Additional Information

```
Kamailio 5.2.1 from Debian stable
```

* **Operating System**:

```
Linux Debian stable on kernel 4.19.0
```

(I don't suppose it matters, this code has been around for ages.  I used permalinks for stable reference).


-- 
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/3119
You are receiving this because you are subscribed to this thread.

Message ID: <kamailio/kamailio/issues/3119 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-dev/attachments/20220520/b65c39e0/attachment-0001.htm>