[SR-Users] Kamailio 1.5.4 and beyond - mhomed issue under Linux

[Sean O'Donnell]

Hi all,

I just started deploying Kamailio release 1.5.4, and I think there's an issue
with how Kamailio identifies an outgoing interface when mhomed is enabled under
Linux.  

I use Kamailio as a call distributor/proxy between a soft-switch/SBC and a
voicemail platform.  It
runs on a CentOS 5.3 (Linux 2.6 kernel) host with two network interfaces and is
configured such that it listens on both interfaces.  One interface (public
interface) handles traffic with the SBC, the other (private interface) handles
with the VM platform. The 'mhomed' option is enabled.

After upgrading from 1.5.3 to 1.5.4, I started noticing problems with UDP
packets coming out of the public interface.  After looking at some ngrep
captures on that interface, I noticed that some packets had the source IP
address of the private interface and also had Record-Route and Via headers for
the private interface only - no headers for the public interface were there.  

Usually when I see the wrong source IP in a UDP packet, it's an issue with how
routes are set up on the host.  However, I had our network engineer double check
them, and they seem fine (no ambiguous routes).  The fact that I captured these
messages on the public interface also indicates to me that the kernel is routing
the message correctly.  The missing Record-Route and Via for the public
interface, however, lead me to believe that the proxy didn't correctly identify
the outgoing interface in the first place.

After looking at the ChangeLog for 1.5.4, I noticed that the some new logic was
put in to improve performance when mhomed is enabled (r5971) in forward.c, and I
think this is the issue.

As I understand it, prior to 1.5.4, when mhomed was enabled, Kamailio determined
the outgoing interface by creating a temporary UDP socket, invoking connect() on
the socket with the packet destination, then checking the source IP of the
socket that the kernel assigned using getsockname().  After the source address
was determined, the temp socket was closed closed. As of 1.5.4, this was
modified to reuse the temporary socket and just re-invoking connect() with a new
destination address.

The problem with the enhancement is that Linux (again, at least in the 2.6
kernels I'm using)
doesn't seem to rebind a new source address to the socket when connect() is
called more than once on
a UDP socket.  Instead, it keeps the original one, and thus the wrong interface
is assumed.

I wrote a small program to confirm this - basically creates a UDP socket, calls
connect()/getsockname()
multiple times using different destination addresses.  I ran it on several 2.6
kernels, including
Centos4.x and Centos5.  The result was always that the source address of the
socket wasn't changed after the first connect(), regardless of the destination
address.  The only way I could get it work as
required was to first do a connect() using a zero'd out AF_UNSPEC address before
doing the
connect() to the remote address.  I also ran it on Solaris and it worked.  Go
figure.

I've downloaded the latest stable release (3.1.2) but I think the issue is still
there, and I don't see
anything in the user groups that addresses this.

Any help would be appreciated.

Thanks,

Sean