After enabling and deciphering debugging it appears there may be a bug. I also reviewed https://tools.ietf.org/html/rfc5658#section-6

I cross-referenced my pcap to ensure I was looking at the reINVITE and see:

Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: rr [loose.c:785]: after_loose(): Topmost route URI: 'sip:xx.xxx.x.xx;lr;r2=on;ftag=a30a720a;did=b75.65a1;nat=yes' is me
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: <core> [socket_info.c:583]: grep_sock_info(): grep_sock_info - checking if host==us: 11==11 && [xx.xxx.x.xx] == [xx.xxx.x.xx]
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: <core> [socket_info.c:587]: grep_sock_info(): grep_sock_info - checking if port 5060 (advertise 0) matches port 5061
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: <core> [socket_info.c:583]: grep_sock_info(): grep_sock_info - checking if host==us: 11==11 && [xx.xxx.x.xx] == [xx.xxx.x.xx]
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: <core> [socket_info.c:587]: grep_sock_info(): grep_sock_info - checking if port 5070 (advertise 0) matches port 5061
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: <core> [socket_info.c:583]: grep_sock_info(): grep_sock_info - checking if host==us: 11==11 && [xx.xxx.x.xx] == [xx.xxx.x.xx]
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: <core> [socket_info.c:587]: grep_sock_info(): grep_sock_info - checking if port 5090 (advertise 0) matches port 5061
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: <core> [socket_info.c:583]: grep_sock_info(): grep_sock_info - checking if host==us: 11==11 && [xx.xxx.x.xx] == [xx.xxx.x.xx]
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: <core> [socket_info.c:587]: grep_sock_info(): grep_sock_info - checking if port 5061 (advertise 0) matches port 5061
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: <core> [parser/msg_parser.c:106]: get_hdr_field(): found end of header
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: rr [loose.c:181]: find_next_route(): No next Route HF found
Jun 16 15:01:29 xxxxxxxxxxxxx /usr/sbin/kamailio[643]: DEBUG: rr [loose.c:847]: after_loose(): no next URI found

There is definitely another Route header immediately below the one found above, but find_next_route() doesn't find it

. I added my own debugging to loose.c and

   if ((_m->last_header->type!=HDR_ROUTE_T) || (_m->last_header==*_hdr)) {
      LM_DBG("No next Route HF found\n");
      LM_DBG("_m->last_header->type: %d\n", _m->last_header->type);
      return 1;
   }

logs find_next_route(): _m->last_header->type: 12 which is HDR_CONTENTLENGTH_T which is indeed the LAST header in the message. We have done very little work in the Kamailio source...just some database escaping in odbc for things to work properly with our database engine...but unless I'm missing something isn't it very wrong to be looking at the last header right here? I may attempt to figure out the message and/or hdr_field data structures and change it. It may also be that the issue doesn't occur when find_next_route is called with a valid _hdr which does seem to search for the "next" one vs going straight to the final header in the entire message.

If this is getting overly complicated for this mailing list please let me know...

Ryan

On Tue, Jun 16, 2015 at 11:40 AM, Ryan Kendrick <kendrick.ryan.c@gmail.com> wrote:

We are using Kamailio 4.2.5 as a registrar and proxy between many dispersed end-users of a soft phone app and our calling platform / switch.

Until now we have used udp exclusively but are trying to introduce tcp between end-users and Kamailio, leaving udp between Kam and our switch...while maintaining the ability for some end-users to use udp to Kam.

With some simple address checks I am able to always send to our switch over udp. If all end-users used tcp I could send everything else tcp, but I need to maintain udp support.

The specific problem I am having is on a reINVITE such as this one from our platform to the a-leg:

INVITE sip:xxxxxx@xxxxxxxxxxxxx:42679;user=phone SIP/2.0
Via: SIP/2.0/UDP xxxxxxxxxxxxx:5060;branch=z9hG4bK218cc8e12ll5035f67INV6a67885312aad
Max-Forwards: 35
Route: <sip:xxxxxxxxxxx;lr;r2=on;ftag=daba971c;did=b57.4872;nat=yes>
Route: <sip:xxxxxxxxxxx:5070;transport=tcp;lr;r2=on;ftag=daba971c;did=b57.4872;nat=yes>
Contact: <sip:xxxxxxxxxx@xxxxxxxxxxxxx:5060>
To: "xxxxxx"<sip:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxxx:5070>;tag=daba971c
From: <sip:xxxxxxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxxx:5070>;tag=6a678853-co76461-INS002
Call-ID: MDI4ZmFjNmZhN2Y1NWE2ZTViNTkyZGUwNWE2YzUzYmU
CSeq: 7646101 INVITE
Allow: INVITE,ACK,CANCEL,BYE,REFER,OPTIONS,NOTIFY,SUBSCRIBE,PRACK,INFO,UPDATE
Content-Type: application/sdp
Date: Mon, 15 Jun 2015 20:10:18 GMT
User-Agent: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Content-Length: 262

As you might notice, we have rr:enable_double_rr set:

There are some situations when the server needs to insert two Record-Route header fields instead of one. For example when using two disconnected networks or doing cross-protocol forwarding from UDP->TCP. This parameter enables inserting of 2 Record-Routes. The server will later remove both of them.

and I believe it is necessary to keep this way. Without it Kamailio doesn't even see the reINVITE...the switch probably tries tcp and that's not setup between the two.

The invite above is sent to the a-leg over udp but I would expect and need it to be tcp in this case. The reINVITE is part of an existing dialog. We call loose_route() followed by some simple bflag setting and flag checking, t_on_reply(), ... then t_relay().

I do have a functional workaround but would prefer to avoid such manual handling by utilizing built-in functionality properly.

   #
   # relay the message
   #
   if(route(TEST_TOGW)) {
      if (!t_relay_to_udp()) {
         sl_reply_error();
      }
   }
   else {
      if ($(hdr(Route)[-1]) =~ "tcp") {
         if(!t_relay_to_tcp()) {
            sl_reply_error();
         }
      }
      else if (!t_relay()) {
         sl_reply_error();
      }
   }

I'm not 100% sure how reliable or fast this will be, but it does work so far in my simple tests.

Is loose_route supposed to see and use the transport=tcp but isn't for some reason? It seems like the right thing to do to me. If not, is there anything else I can/should be doing in the tm and/or rr modules to make Kamailio realize it needs to send this message over TCP? If not in those two modules is there some recommended way perhaps via registrar or usrloc etc. to make Kamailio remember/store when a user is connected via TCP and be able to do a quick lookup before sending to them? Anything else I'm missing or not thinking of?

Please let me know if I can further explain and rest assured any assistance will be much appreciated!!!