[SR-Users] Help with rewriting headers for NAT manually

Shah Hussain Khattak shahhusayn at msn.com
Sun Jan 16 10:00:05 CET 2022 

Hi There,

If I look at the latest SIP trace you shared:

SIP/2.0 200 OK
Via: SIP/2.0/UDP 64.###.###.###:5060;branch=z9hG4bK26ab.5547ac15.0
Via: SIP/2.0/UDP 206.###.###.###:5060;rport=5060;received=206.###.###.###;branch=z9hG4bK6gj48a00dolcl3jm2gq0.1
Record-Route: <sip:10.###.###.254;r2=on;lr=on;ftag=as04035ef0>
Record-Route: <sip:209.###.###.###;r2=on;lr=on;ftag=as04035ef0>
Record-Route: <sip:64.###.###.###;lr;ftag=as04035ef0>
From: "Anonymous" <sip:anonymous at anonymous.invalid:5060>;tag=as04035ef0
To: <sip:928#######@64.###.###.###:5060>;tag=as7047ed05
Call-ID: 5ab1525b3712f34c2ab272ae55e649e5 at 10.44.###.###:5060
CSeq: 102 INVITE
Server: Asterisk PBX 16.18.0
Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH, MESSAGE
Supported: replaces, timer
Contact: <sip:928#######@10.###.###.104:5060>
Content-Type: application/sdp
Content-Length: 274

v=0
o=root 1911037741 1911037741 IN IP4 209.###.###.###
s=Asterisk PBX 16.18.0
c=IN IP4 209.###.###.###
t=0 0
m=audio 11384 RTP/AVP 0 101
a=rtpmap:0 PCMU/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=ptime:20
a=maxptime:150
a=sendrecv
a=nortpproxy:yes

ACK sip:928#######@209.###.###.###:5060 SIP/2.0
Via: SIP/2.0/UDP 64.###.###.###:5060;branch=z9hG4bK26ab.5547ac15.2
Via: SIP/2.0/UDP 206.###.###.###:5060;rport=5060;received=206.###.###.###;branch=z9hG4bK91l3it006gr9oiulcqn0.1
Max-Forwards: 67
From: "Anonymous" <sip:anonymous at anonymous.invalid:5060>;tag=as04035ef0
To: <sip:928#######@64.###.###.###:5060>;tag=as7047ed05
Contact: <sip:anonymous at 206.###.###.###:5060;transport=udp>
Call-ID: 5ab1525b3712f34c2ab272ae55e649e5 at 10.44.###.###:5060
CSeq: 102 ACK
User-Agent: packetrino
Content-Length: 0
Route: <sip:209.###.###.###;r2=on;lr=on;ftag=as04035ef0>
Route: <sip:10.###.###.254;r2=on;lr=on;ftag=as04035ef0>

The ACK is getting sent to the Kamailio with correct Route information:

Route: <sip:209.###.###.###;r2=on;lr=on;ftag=as04035ef0>
Route: <sip:10.###.###.254;r2=on;lr=on;ftag=as04035ef0>

The Kamailio server should strip the 1st and 2nd Route(s) header from the ACK and should relay it towards the next-hop as per the request URI. Please note that Kamailio is sending Double RR headers ( When a Proxy receives a request on one network interface and sends it onwards using a different interface e.g. WAN to LAN, this will normally require the addition of an extra Record-Route header. i.e. the Proxy must add two RR headers where you might normally expect it to add one.)

Record-Route: <sip:10.###.###.254;r2=on;lr=on;ftag=as04035ef0>
Record-Route: <sip:209.###.###.###;r2=on;lr=on;ftag=as04035ef0>

The problem is, the peer behavior is not compliant with the specs. It is sending the ACK with RURI set to:

ACK sip:928#######@209.###.###.###:5060 SIP/2.0

Ideally, it should have sent the ACK with the following Request-URI:

 ACK sip:928#######@10.###.###.104:5060 SIP/2.0

Once this ACK will be received on Kamailio, it will relay it towards the Asterisk IP, which is 10.###.###.104.

For further understanding of the ACK routing, you can refer to the following post:

https://lists.cs.columbia.edu/pipermail/sip-implementors/2019-February/031229.html

The peer is not copying the 200 OK Contact header URI into the ACK message and it is a problem.

Lastly, the trace might be showing only part of the puzzle, it is also suggested to get a capture on the remote peer end, because it is sending the 209.###.###.### IP in the ACK, which seems to be the public interface of the Kamailio server. I am not sure if there is some device in the path, that is changing the contact IP in the 200 OK to the Kamailio public IP?

Ovidiu also explained the same point, I just added a little bit more information for you to fight with your SIP provider. It is better to ask what RFC they are following on the basis of which they are writing the Kamailio public IP in the 200 OK ACK message?

Regards,
Shah Hussain

________________________________
From: sr-users <sr-users-bounces at lists.kamailio.org> on behalf of Chad <ccolumbu at hotmail.com>
Sent: Sunday, January 16, 2022 10:14 AM
To: Ovidiu Sas <osas at voipembedded.com>
Cc: Kamailio (SER) - Users Mailing List <sr-users at lists.kamailio.org>
Subject: Re: [SR-Users] Help with rewriting headers for NAT manually

Ok so in short I was not doing anything wrong (although I had some miss-configurations), but the carrier is (i.e. they
are a bad actor). When they said I was doing it wrong, they did not mean in the RFC sense they meant in the "to work
with us" sense. Now in order for me to get it to work with their SBC I have to mangle the contact on the way out an
unmangle it on the return in Kamailio somehow, as I originally purposed.
However I have no idea how to do that :)

Shouldn't we (the Kamailio community) assume there are lots of bad actors out there and possibly many Kamailio users
with this exact same issue (I personally know of at least 2 bad actor carriers right now) and create some kind of
template or snippet that we can publicly publish on the Kamailio docs or wiki for all of the Kamailio community to use
for this use case?

I have been fighting with carriers about this for years and they always said I was doing it wrong and I don't know the
SIP RFC well enough to fight back. So why not build a solution for everyone out there that has to deal with a bad actor?

--
^C


On 1/15/22 11:40 AM, Ovidiu Sas wrote:
> As expected, your carrier is bogus and "thinks" it knows better.
> Your carrier is treating your setup as a dumb endpoint and is
> re-writing the Contact header:
> You provide this contact header in 200 OK:
> Contact: <sip:928#######@10.###.###.104:5060>
> The carrier should set the RURI in ACK like this:
> ACK sip:928#######@10.###.###.104:5060 SIP/2.0
> Instead, your ACK is sent to you like this:
> ACK sip:928#######@209.###.###.###:5060 SIP/2.0
>
> The RURI in ACK should point to the private IP of the asterisk server,
> not to the public IP of the kamailio server.
> You need to ask the carrier to follow the SIP RFC and not treat your
> endpoints like dumb SIP endpoints.
>
> There's a high chance that they won't do it :)
> Your best chance is to manually mangle the URI in Contact in the 200
> OK in a way that when you receive the ACK with the mangled RURI, you
> can restore the original URI and let kamailio do the proper routing to
> the private IP of the asterisk serverr.
> You should be able to achieve this by using one of the following functions:
> https://kamailio.org/docs/modules/5.5.x/modules/mangler.html#mangler.f.encode_contact
> https://kamailio.org/docs/modules/5.5.x/modules/siputils.html#siputils.f.encode_contact
> https://kamailio.org/docs/modules/5.5.x/modules/siputils.html#siputils.f.contact_param_encode
>
> Regards,
> Ovidiu Sas
>
> On Sat, Jan 15, 2022 at 1:28 PM Chad <ccolumbu at hotmail.com> wrote:
>>
>> I changed the listen per your advice and here is the 200 and ACK.
>> I get no audio and the the call disconnects and I see this is the Asterisk log:
>> [Jan 15 10:17:13] WARNING[29953] chan_sip.c: Retransmission timeout reached on transmission
>> 5ab1525b3712f34c2ab272ae55e649e5 at 10.44.109.143:5060 for seqno 102 (Critical Response) -- See
>> https://wiki.asterisk.org/wiki/display/AST/SIP+Retransmissions
>> Packet timed out after 6401ms with no response
>> [Jan 15 10:17:13] WARNING[29953] chan_sip.c: Hanging up call 5ab1525b3712f34c2ab272ae55e649e5 at 10.44.109.143:5060 - no
>> reply to our critical packet (see https://wiki.asterisk.org/wik
>>
>> FYI 10.###.###.254 is the private virtual IP on the Kamailio server and 10.###.###.104 is the asterisk box.
>>
>> SIP/2.0 200 OK
>> Via: SIP/2.0/UDP 64.###.###.###:5060;branch=z9hG4bK26ab.5547ac15.0
>> Via: SIP/2.0/UDP 206.###.###.###:5060;rport=5060;received=206.###.###.###;branch=z9hG4bK6gj48a00dolcl3jm2gq0.1
>> Record-Route: <sip:10.###.###.254;r2=on;lr=on;ftag=as04035ef0>
>> Record-Route: <sip:209.###.###.###;r2=on;lr=on;ftag=as04035ef0>
>> Record-Route: <sip:64.###.###.###;lr;ftag=as04035ef0>
>> From: "Anonymous" <sip:anonymous at anonymous.invalid:5060>;tag=as04035ef0
>> To: <sip:928#######@64.###.###.###:5060>;tag=as7047ed05
>> Call-ID: 5ab1525b3712f34c2ab272ae55e649e5 at 10.44.###.###:5060
>> CSeq: 102 INVITE
>> Server: Asterisk PBX 16.18.0
>> Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH, MESSAGE
>> Supported: replaces, timer
>> Contact: <sip:928#######@10.###.###.104:5060>
>> Content-Type: application/sdp
>> Content-Length: 274
>>
>> v=0
>> o=root 1911037741 1911037741 IN IP4 209.###.###.###
>> s=Asterisk PBX 16.18.0
>> c=IN IP4 209.###.###.###
>> t=0 0
>> m=audio 11384 RTP/AVP 0 101
>> a=rtpmap:0 PCMU/8000
>> a=rtpmap:101 telephone-event/8000
>> a=fmtp:101 0-16
>> a=ptime:20
>> a=maxptime:150
>> a=sendrecv
>> a=nortpproxy:yes
>>
>> ACK sip:928#######@209.###.###.###:5060 SIP/2.0
>> Via: SIP/2.0/UDP 64.###.###.###:5060;branch=z9hG4bK26ab.5547ac15.2
>> Via: SIP/2.0/UDP 206.###.###.###:5060;rport=5060;received=206.###.###.###;branch=z9hG4bK91l3it006gr9oiulcqn0.1
>> Max-Forwards: 67
>> From: "Anonymous" <sip:anonymous at anonymous.invalid:5060>;tag=as04035ef0
>> To: <sip:928#######@64.###.###.###:5060>;tag=as7047ed05
>> Contact: <sip:anonymous at 206.###.###.###:5060;transport=udp>
>> Call-ID: 5ab1525b3712f34c2ab272ae55e649e5 at 10.44.###.###:5060
>> CSeq: 102 ACK
>> User-Agent: packetrino
>> Content-Length: 0
>> Route: <sip:209.###.###.###;r2=on;lr=on;ftag=as04035ef0>
>> Route: <sip:10.###.###.254;r2=on;lr=on;ftag=as04035ef0>
>>
>>
>> --
>> ^C
>>
>>
>> On 1/15/22 10:21 AM, Ovidiu Sas wrote:
>>> This is false. The IP in the Contact header must be routable by the
>>> SIP hop from the top Record-Route header in the reply.
>>> The carrier (and it seems that they have a PROXY also) must be able to
>>> route to their adjacent SIP hop, which is your public IP (the IP in
>>> the second Record-Route header).
>>> It seems that the carrier is not taking into account that they might
>>> interface with other proxies.
>>> Most likely, your carrier expects to interface with a simple SIP UA,
>>> not with another proxy. This is a pretty common setup for most of the
>>> carriers, although many new carrier implementations are taking care of
>>> the proxy to proxy calls.
>>>
>>> It would be helpful to see the ACK that is sent by the carrier in
>>> response to your 200ok (after you fix your config and you have your
>>> private IP listed in the Record-Route header).
>>>
>>> -ovidiu
>>>
>>> On Sat, Jan 15, 2022 at 12:33 PM Chad <ccolumbu at hotmail.com> wrote:
>>>>
>>>> Hmm, I don't think you are right that the Contact header can be a private IP even if the RR is correct.
>>>> I did some research on it and I found several places saying it must be a routable IP which is what the carrier also said.
>>>>
>>>> "The Contact header contains the SIP URI where the client wants to be contacted for subsequent requests. That means that
>>>> the host part of the URI must be globally reachable by anyone.
>>>> If your contact contains a private IP (behind a NAT?) then it is wrong, because other peers cannot reach you with that."
>>>>
>>>>
>>>> --
>>>> ^C
>>>>
>>>>
>>>> On 1/15/22 9:05 AM, Ovidiu Sas wrote:
>>>>> You have a different problem then.
>>>>> Having private IPs in Contact is fine. You need to lose route the
>>>>> calls (kamailio will add two Record-Route headers) and the origination
>>>>> server will set the RURI to the private IP from Contact, but it will
>>>>> send the in-dialog requests to the public IP of kamailio. This has
>>>>> nothing to do with virtual IPs.
>>>>> Maybe you have a buggy client that doesn't do proper loose routing.
>>>>>
>>>>> -ovidiu
>>>>>
>>>>> On Sat, Jan 15, 2022 at 11:50 AM Chad <ccolumbu at hotmail.com> wrote:
>>>>>>
>>>>>> Ovidiu,
>>>>>> Thank you again for your response.
>>>>>> One is public (an internet IP) and one is private (a 10.x ip).
>>>>>> Apparently this is a known problem with virtual IPs, it does not work.
>>>>>> When the asterisk server responds to the invite it sends a contact header with the private IP and Kamailio does not
>>>>>> rewrite it to the advertised public IP. So the originating server sees the private IP in the Contact header and tries to
>>>>>> send the traffic to the 10.x IP (which is non-routable) and the call dies.
>>>>>> I have been trying things for a long time to fix this (years) what you are saying will not fix it because of the virtual
>>>>>> IPs.
>>>>>> If it was a normal IP it would work fine. It has something to do with the routing table and how mhomed detects networks.
>>>>>>
>>>>>> --
>>>>>> ^C
>>>>>>
>>>>>>
>>>>>> On 1/15/22 8:36 AM, Ovidiu Sas wrote:
>>>>>>> Hello Chad,
>>>>>>>
>>>>>>> The floating IPs that you have, are they both private IPs or one
>>>>>>> private IP and the other one a public IP?
>>>>>>>
>>>>>>> If you have to two floating private IPs, then you need a config like this:
>>>>>>> listen=FLOATING_UDP_PRIVATE1 advertise PUBLIC_UDP_IP
>>>>>>> listen=FLOATING_UDP_PRIVATE2
>>>>>>>
>>>>>>> In the config, before relaying the initial INVITE you need to detect
>>>>>>> the direction of the call and set $fs accordingly:
>>>>>>> if (CAL_FROM_PRIVATE_TO_PUBLIC) {
>>>>>>>         $fs = udp:FLOATING_UDP_PRIVATE1
>>>>>>> }
>>>>>>> else {
>>>>>>>         $fs = udp:FLOATING_UDP_PRIVATE2
>>>>>>> }
>>>>>>>
>>>>>>> If you have a floating private IPs and a floating public IP, then you
>>>>>>> need a config like this:
>>>>>>> listen=FLOATING_UDP_PRIVATE
>>>>>>> listen=FLOATING_UDP_PUBLIC
>>>>>>>
>>>>>>> There should be no need to force the socket, but if you do, there's no
>>>>>>> harm (actually it's better and faster).
>>>>>>>
>>>>>>> Hope this clarifies things and helps,
>>>>>>> -ovidiu
>>>>>>>
>>>>>>> On Sat, Jan 15, 2022 at 9:48 AM Chad <ccolumbu at hotmail.com> wrote:
>>>>>>>>
>>>>>>>> Ovidiu,
>>>>>>>> Thank you for your response.
>>>>>>>>
>>>>>>>> I have done that, in addition to the linux ip_nonlocal_bind I have also set the Kamailio ip_free_bind=1 and it does not
>>>>>>>> work.
>>>>>>>> Here are my relevant config lines:
>>>>>>>> listen=LISTEN_UDP_PRIVATE advertise MY_PUBLIC_IP:5060
>>>>>>>> listen=LISTEN_UDP_PUBLIC
>>>>>>>>
>>>>>>>> mhomed=1
>>>>>>>> ip_free_bind=1
>>>>>>>>
>>>>>>>>
>>>>>>>> In my /etc/sysctl.conf I have (yes I applied it with sysctl -p, and I have been using it for a long time and have
>>>>>>>> rebooted as well):
>>>>>>>> net.ipv4.ip_nonlocal_bind=1
>>>>>>>> --
>>>>>>>> ^C
>>>>>>>>
>>>>>>>>
>>>>>>>> On 1/15/22 4:55 AM, Ovidiu Sas wrote:
>>>>>>>>> Hello Chad,
>>>>>>>>>
>>>>>>>>> You can add a listen directive to your config for the virtual IPs
>>>>>>>>> (both public and private) and then you don't need to manually modify
>>>>>>>>> any headers or use force_send_socket().
>>>>>>>>> You need to enable non local IP binding so kamailio can start on the
>>>>>>>>> server that doesn't have the virtual IP:
>>>>>>>>> echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind
>>>>>>>>> To make the change permanent, edit your sysctl.conf file and enable it there:
>>>>>>>>> net/ipv4/ip_nonlocal_bind = 1
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> Ovidiu Sas
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Jan 15, 2022 at 4:16 AM Chad <ccolumbu at hotmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> We are looking for some help (possibly a paid consultant) to help us with our Kamailio setup.
>>>>>>>>>> To keep this as short as possible: we use Kamailio as a NAT proxy to bridge our external IP and our private IP asterisk
>>>>>>>>>> servers (via dispatcher).
>>>>>>>>>> However both the external IP and the internal IP that the Kamailio server uses are virtual IPs created by keepalived.
>>>>>>>>>> Because of that neither mhomed nor fix_nated_contact work, and we use force_send_socket to direct the traffic.
>>>>>>>>>> We run linux Debian 10 for the OS.
>>>>>>>>>> Also we do not use a DB at all, everything is done with local config files.
>>>>>>>>>>
>>>>>>>>>> The problem is that when traffic goes out the Contact header has a private IP in it, like:
>>>>>>>>>> Contact: <sip:##########@10.10.10.###]:5060>
>>>>>>>>>>
>>>>>>>>>> There are 2 possible solutions to this:
>>>>>>>>>> 1. Make changes to linux, keepalived and/or Kamailio so that Kamailio recognize the virtual IPs so that mhomed and
>>>>>>>>>> fix_nated_contact work as usual.
>>>>>>>>>>
>>>>>>>>>> 2. Create a manual header rewrite system.
>>>>>>>>>>
>>>>>>>>>> If solution #2:
>>>>>>>>>> What we need to do is create a way to rewrite the contact header to the external IP on the way out, and on the way back
>>>>>>>>>> rewrite it back to the internal server that the call is already connected to.
>>>>>>>>>>
>>>>>>>>>> Not sure if we will need to store those paths on the server or if we can do some kind of cheat with another persistant
>>>>>>>>>> header like P-Preferred-Identity or P-Asserted-Identity (i.e. store the internal IP in the name field or something).
>>>>>>>>>>
>>>>>>>>>> If anyone out there know of a way to do this or wants to give it a try please reach out to me.
>>>>>>>>>>
>>>>>>>>>> Thank you all for your time.
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> ^C
>>>>>>>>>> Chad
>>>>>>>>>>
>>>>>>>>>> __________________________________________________________
>>>>>>>>>> Kamailio - Users Mailing List - Non Commercial Discussions
>>>>>>>>>>        * sr-users at lists.kamailio.org
>>>>>>>>>> Important: keep the mailing list in the recipients, do not reply only to the sender!
>>>>>>>>>> Edit mailing list options or unsubscribe:
>>>>>>>>>>        * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> VoIP Embedded, Inc.
>>>>>>>>> http://www.voipembedded.com
>>>>>>>>>
>>>>>>>>> __________________________________________________________
>>>>>>>>> Kamailio - Users Mailing List - Non Commercial Discussions
>>>>>>>>>        * sr-users at lists.kamailio.org
>>>>>>>>> Important: keep the mailing list in the recipients, do not reply only to the sender!
>>>>>>>>> Edit mailing list options or unsubscribe:
>>>>>>>>>        * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>
>
>

__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
  * sr-users at lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
  * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20220116/c2342bfd/attachment.htm>

More information about the sr-users mailing list