Setup:
One kamailio behind a NAT, one on public IP.
I register a client using the inside Kamailio as an outbound proxy. It adds a path header and forwards over TCP to the public server. This server saves the registration with the path header and send a response, which reaches the client fine.
Calling from the client to it's own number, the INVITE is sent over the inside proxy to the outside, it looks up the location database then drops the request. It doesn't realize that it has a TCP connection it can reuse open. The error message - a timeout - is sent back over the connection.
In my dream world, the server would reuse the open TCP connection to get back across the NAT. If we can get the response across, we should be able to get a new request back.
Now, I don't know the inner workings of TCP connection handling here. I guess I should test having the client open a connection directly to the outside proxy and see if that works.
Would it be possible with the current architecture to get Kamailio to reuse an existing TCP connection like this?
I started digging down the TCP code, but lost attention because of the complexity and fixed speling erors in other parts instead... Will try again.
/O
Hi Olle!
On 25.10.2012 17:00, Olle E. Johansson wrote:
Setup:
One kamailio behind a NAT, one on public IP.
I register a client using the inside Kamailio as an outbound proxy. It adds a path header and forwards over TCP to the public server. This server saves the registration with the path header and send a response, which reaches the client fine.
Calling from the client to it's own number, the INVITE is sent over the inside proxy to the outside, it looks up the location database then drops the request. It doesn't realize that it has a TCP connection it can reuse open. The error message - a timeout - is sent back over the connection.
In my dream world, the server would reuse the open TCP connection to get back across the NAT. If we can get the response across, we should be able to get a new request back.
Kamailio uses the next hop target (probably the URI in the Path header) and searches for open TCP connections to this target. I guess the Path header contains the private IP address of the outbound proxy, thus it does not match the open TCP connection. If there is not outboundproxy, the solution is simple: as always use fix_nated_register() on REGISTER. Then, after lookup() the proxy will search for a TCP connection to the "received" IP:port and find and uses the existing connection.
With the additional proxy, things get more complex.
I see 3 solutions:
1. The outbound proxy announces public IP:port address in the path header and the NAT has a static port forwarding of this port to the outbound proxy
2. The outbound proxy announces the ephemeral public IP:port address in the path header. Eg. open a TCP connection to the proxy, use STUN to get the public mapping and use it in the Path header (probably needs some code changes)
3. Use Klaus' pragmatic and secure approach: I guess in this setup the outbound proxy is in the user's domain, not in the proxy provider's domain. Thus, I (the proxy provider) does not any user provided data - the user could write anything into the Path header (eg. the IP address of my internal gateway). It just remember where the REGISTER came from (IP:port) and send INVITEs to this IP:port. (aka applying NAT traversal all the time).
If I read Path support in registrar module correct, on lookup the Path always overrides the "received" socket information with the loaded route-set. Thus, you have to either change registrar module to have this a config option (probably the implementation was a bit short sighted and did not expected the outbound proxy to be behind NAT) or to manually override $du after lookup, e.g:
for REGISTER: fix_nated_register();
for INVITE to user: lookup(...); reg_fetch_contacts("location", "$ou", "callee"); $du=$(ulc(callee=>received)); reg_free_contacts("callee");
Of course this is problematic with forking scenarios.
Generally one has to differ who operates the outbound proxy of the user. If it is the user, then the user is responsible for NAT traversal.
regards Klaus
PS: I just had an idea of a 4th workaround
on REGISTER add another PATH header on top of the existing ones with the URI pointing to sip:$si:$sp;transport=$pr
insert_hf("Path: sip:$si:$sp;transport=$pr\r\n", "Path"); msg_apply_changes(); save();
on lookup(), the $du is set to the public IP:port, but there is the fake Route target which should be removed. This is complex as there can be multiple Route headers.
I think the way to go is to update registrar module to not use the Path header target as $du but the received column (make it a module option).
... ops ... one more idea:
I think the easiest solution is to put another proxy in front of the registrar:
user <-> user-obp <-NAT-> ITSP-obp <-> registrar.
In this case you just have to use on the ITSP-obp: add_path_received() for REGISTER, and modparam("path", "use_received", 1)
I think it is not necessary to set modparam("registrar", "path_use_received", 1) on the registrar (actually I am not sure what this parameter does).
regards Klaus
Now, I don't know the inner workings of TCP connection handling here. I guess I should test having the client open a connection directly to the outside proxy and see if that works.
Would it be possible with the current architecture to get Kamailio to reuse an existing TCP connection like this?
I started digging down the TCP code, but lost attention because of the complexity and fixed speling erors in other parts instead... Will try again.
/O _______________________________________________ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Hi!
THanks, Klaus!
The solution was to use a trick by Peter Dunkley in an old mail. I could add another Path header, but could not run msg_apply_changes on that. Instead I forwarded the message to myself and then saved in location database. Now I have to path headers, with the top one pointing to the TCP connection with the received parameter, like you described.
Thanks for explaining the TCP connection matching and your open brainstorm that provided a solution!
The question is now how to get the proxy on the inside of the NAT - also Kamailio - to keep the TCP connection open for a very, very long time :-) But there are many TCP parameters for that. And NAT keepalives. /O
Indeed, just use a timer route to periodically send OPTIONS to the registrar.
Are you looping outgoing messages also through the registrar? Or does the request to the obp with has 2 Route headers?
K
Am 25.10.2012 um 21:57 schrieb "Olle E. Johansson" oej@edvina.net:
Hi!
THanks, Klaus!
The solution was to use a trick by Peter Dunkley in an old mail. I could add another Path header, but could not run msg_apply_changes on that. Instead I forwarded the message to myself and then saved in location database. Now I have to path headers, with the top one pointing to the TCP connection with the received parameter, like you described.
Thanks for explaining the TCP connection matching and your open brainstorm that provided a solution!
The question is now how to get the proxy on the inside of the NAT - also Kamailio - to keep the TCP connection open for a very, very long time :-) But there are many TCP parameters for that. And NAT keepalives. /O
25 okt 2012 kl. 22:07 skrev Klaus Darilion klaus.mailinglists@pernau.at:
Indeed, just use a timer route to periodically send OPTIONS to the registrar.
Are you looping outgoing messages also through the registrar? Or does the request to the obp with has 2 Route headers?
I have an ingres proxy behind NAT that adds a path header and sends the register to a proxy on a public IP. this one adds another PATH with received and loops back to itself before saving the location.
When calling, the INVITE will get two Route headers (based on the PATH headers), one to get back to the TCP connection and the other one back over that connection to the ingres proxy. Klaus-style, routing on received.
Of course, it's easy to add TLS in this scenario. I will soon test your Asterisk patch for Path (the Oolong-branch now) and see if it can play in this setup as well.
Good night.
/O
K
Am 25.10.2012 um 21:57 schrieb "Olle E. Johansson" oej@edvina.net:
Hi!
THanks, Klaus!
The solution was to use a trick by Peter Dunkley in an old mail. I could add another Path header, but could not run msg_apply_changes on that. Instead I forwarded the message to myself and then saved in location database. Now I have to path headers, with the top one pointing to the TCP connection with the received parameter, like you described.
Thanks for explaining the TCP connection matching and your open brainstorm that provided a solution!
The question is now how to get the proxy on the inside of the NAT - also Kamailio - to keep the TCP connection open for a very, very long time :-) But there are many TCP parameters for that. And NAT keepalives. /O
25 okt 2012 kl. 19:05 skrev Klaus Darilion klaus.mailinglists@pernau.at:
Kamailio uses the next hop target (probably the URI in the Path header) and searches for open TCP connections to this target. I guess the Path header contains the private IP address of the outbound proxy, thus it does not match the open TCP connection. If there is not outboundproxy, the solution is simple: as always use fix_nated_register() on REGISTER. Then, after lookup() the proxy will search for a TCP connection to the "received" IP:port and find and uses the existing connection.
Thinking about TLS - how do we match there?
/O
Am 26.10.2012 14:08, schrieb Olle E. Johansson:
25 okt 2012 kl. 19:05 skrev Klaus Darilion klaus.mailinglists@pernau.at:
Kamailio uses the next hop target (probably the URI in the Path header) and searches for open TCP connections to this target. I guess the Path header contains the private IP address of the outbound proxy, thus it does not match the open TCP connection. If there is not outboundproxy, the solution is simple: as always use fix_nated_register() on REGISTER. Then, after lookup() the proxy will search for a TCP connection to the "received" IP:port and find and uses the existing connection.
Thinking about TLS - how do we match there?
AFAIK there is no difference to TLS. If there is a TLS connection whose remote address matches the next hop, it will be used.
regards Klaus
26 okt 2012 kl. 21:08 skrev Klaus Darilion klaus.mailinglists@pernau.at:
Am 26.10.2012 14:08, schrieb Olle E. Johansson:
25 okt 2012 kl. 19:05 skrev Klaus Darilion klaus.mailinglists@pernau.at:
Kamailio uses the next hop target (probably the URI in the Path header) and searches for open TCP connections to this target. I guess the Path header contains the private IP address of the outbound proxy, thus it does not match the open TCP connection. If there is not outboundproxy, the solution is simple: as always use fix_nated_register() on REGISTER. Then, after lookup() the proxy will search for a TCP connection to the "received" IP:port and find and uses the existing connection.
Thinking about TLS - how do we match there?
AFAIK there is no difference to TLS. If there is a TLS connection whose remote address matches the next hop, it will be used.
That's bad. We need to check the domains in the certificate before re-using it. If they showed NO client cert, we should open a new one. If they showed a client, we should verify.
Will the on-send route give me the possibility or is it triggered before kamailio selects a tcp connection? I'm a bit unclear of the exact situation where the on-send route is called.
/O
On 26.10.2012 22:03, Olle E. Johansson wrote:
26 okt 2012 kl. 21:08 skrev Klaus Darilion klaus.mailinglists@pernau.at:
Am 26.10.2012 14:08, schrieb Olle E. Johansson:
25 okt 2012 kl. 19:05 skrev Klaus Darilion klaus.mailinglists@pernau.at:
Kamailio uses the next hop target (probably the URI in the Path header) and searches for open TCP connections to this target. I guess the Path header contains the private IP address of the outbound proxy, thus it does not match the open TCP connection. If there is not outboundproxy, the solution is simple: as always use fix_nated_register() on REGISTER. Then, after lookup() the proxy will search for a TCP connection to the "received" IP:port and find and uses the existing connection.
Thinking about TLS - how do we match there?
AFAIK there is no difference to TLS. If there is a TLS connection whose remote address matches the next hop, it will be used.
That's bad. We need to check the domains in the certificate before re-using it. If they showed NO client cert, we should open a new one. If they showed a client, we should verify.
Not necessarily. Usually SIP clients are behind NAT (not servers) and thus we need NAT traversal for these clients and these clients usually do not have a client certificate.
For servers, we usually want to have separate TLS connections for each direction. In your "untypical" scenario the NAT traversal should be fixed on the client side and then you could use 2 separate TLS connections.
Will the on-send route give me the possibility or is it triggered before kamailio selects a tcp connection? I'm a bit unclear of the exact situation where the on-send route is called.
[on-send] is just before the message is sent. AFAIK it is executed before the message is handled over to the transport layer - thus I think TLS paramters of the new connection are not available.
Generally, domain verification against the certificate name is broken. For example if you have to reopen a TLS connection for an in-dialog request, there is no proper URI (RURI, next hop Route URI) to compare against the certificate.
regards Klaus
/O
29 okt 2012 kl. 10:52 skrev Klaus Darilion klaus.mailinglists@pernau.at:
On 26.10.2012 22:03, Olle E. Johansson wrote:
26 okt 2012 kl. 21:08 skrev Klaus Darilion klaus.mailinglists@pernau.at:
Am 26.10.2012 14:08, schrieb Olle E. Johansson:
25 okt 2012 kl. 19:05 skrev Klaus Darilion klaus.mailinglists@pernau.at:
Kamailio uses the next hop target (probably the URI in the Path header) and searches for open TCP connections to this target. I guess the Path header contains the private IP address of the outbound proxy, thus it does not match the open TCP connection. If there is not outboundproxy, the solution is simple: as always use fix_nated_register() on REGISTER. Then, after lookup() the proxy will search for a TCP connection to the "received" IP:port and find and uses the existing connection.
Thinking about TLS - how do we match there?
AFAIK there is no difference to TLS. If there is a TLS connection whose remote address matches the next hop, it will be used.
That's bad. We need to check the domains in the certificate before re-using it. If they showed NO client cert, we should open a new one. If they showed a client, we should verify.
Not necessarily. Usually SIP clients are behind NAT (not servers) and thus we need NAT traversal for these clients and these clients usually do not have a client certificate.
For servers, we usually want to have separate TLS connections for each direction. In your "untypical" scenario the NAT traversal should be fixed on the client side and then you could use 2 separate TLS connections.
Will that happen with Kamailio today? Won't the TCP matching make us reuse the existing connection?
Will the on-send route give me the possibility or is it triggered before kamailio selects a tcp connection? I'm a bit unclear of the exact situation where the on-send route is called.
[on-send] is just before the message is sent. AFAIK it is executed before the message is handled over to the transport layer - thus I think TLS paramters of the new connection are not available.
Agree. Just wanted to check.
Generally, domain verification against the certificate name is broken. For example if you have to reopen a TLS connection for an in-dialog request, there is no proper URI (RURI, next hop Route URI) to compare against the certificate.
no, which is why we don't want IP addresses in the record-route headers for TLS.
/O
regards Klaus
/O
2012/10/29 Olle E. Johansson oej@edvina.net:
That's bad. We need to check the domains in the certificate before re-using it. If they showed NO client cert, we should open a new one. If they showed a client, we should verify.
Not necessarily. Usually SIP clients are behind NAT (not servers) and thus we need NAT traversal for these clients and these clients usually do not have a client certificate.
And that is 100% covered by Outbound RFC 5626.
For servers, we usually want to have separate TLS connections for each direction.
Unless RFC 5923 is implemented and requested. But after read this RFC I strongly think that it's very hard to implement. Even more, it requires inspecting the server certificate before connecting. Kamailio does not provides callbacks for the user to decide wheter to accept connecting to a TLS server or not onve the server's certificate is retrieved.
In your "untypical" scenario the NAT traversal should be fixed on the client side and then you could use 2 separate TLS connections.
Olle's scenario is very hard IMHO. If like a server asking for Outbound RFC 5626 usage, which is anti-RFC5626 (no proxies, but just UAs can do Outbound and just when connecting to their edge proxy).
Will that happen with Kamailio today? Won't the TCP matching make us reuse the existing connection?
You receive a TCP connection from 1.2.3.4:9999. And later you need to send a request to 1.2.3.4:5060. Why do you assume that you can reuse the connection from 1.2.3.4:9999? why if there are two SIP nodes listening into different ports (i.e. 5060 and 6060) within the same server with IP 1.2.3.4?
Will the on-send route give me the possibility or is it triggered before kamailio selects a tcp connection? I'm a bit unclear of the exact situation where the on-send route is called.
[on-send] is just before the message is sent. AFAIK it is executed before the message is handled over to the transport layer - thus I think TLS paramters of the new connection are not available.
Kamailio needs a callback for the user to inspect client's or server's TLS certificates and decide whether to continue or reject the TLS session.
Generally, domain verification against the certificate name is broken. For example if you have to reopen a TLS connection for an in-dialog request, there is no proper URI (RURI, next hop Route URI) to compare against the certificate.
The trick is:
- The proxy forwarding a request via TLS needs to set a domain in the Record-Route. - Such a domain should not have SRV records (otherwise the indialog request could arrive to other server).
This may require having a TLS certificate with varios AltSubjectName entries:
- mydomain.com - server1.mydomain.com - server2.mydomain.com - server3.mydomain.com
mydomain.com has NAPTR/SRV resouorces while the others are just A/AAAA resources. Clients are configured to use mydomain.com, but each proxy in the server infraestructure should set the proper serverN.mydomain.com in Record-Route when routing via TLS. This should fix the problem...
...but then you realize that lot of clients CANNOT resolve a domain in a Route header, so they cannot send in-dialog requests if the Record-Route contains a domain...
29 okt 2012 kl. 11:57 skrev Iñaki Baz Castillo ibc@aliax.net:
2012/10/29 Olle E. Johansson oej@edvina.net:
That's bad. We need to check the domains in the certificate before re-using it. If they showed NO client cert, we should open a new one. If they showed a client, we should verify.
Not necessarily. Usually SIP clients are behind NAT (not servers) and thus we need NAT traversal for these clients and these clients usually do not have a client certificate.
And that is 100% covered by Outbound RFC 5626.
Agree.
For servers, we usually want to have separate TLS connections for each direction.
Unless RFC 5923 is implemented and requested. But after read this RFC I strongly think that it's very hard to implement. Even more, it requires inspecting the server certificate before connecting. Kamailio does not provides callbacks for the user to decide wheter to accept connecting to a TLS server or not onve the server's certificate is retrieved.
Yes, the reuse has a big hole. How do I know if it's a client2server or a server2server connection that was opened to my server? If it's a server2server and I want to reuse it, I should require a TLS certificate. It fails to tell me how I separate them. Now, if I could reject the first request with a response code and reason that says "reopen with client cert" we could upgrade the connection.
In jabber, they are separate ports and SRv records...
In your "untypical" scenario the NAT traversal should be fixed on the client side and then you could use 2 separate TLS connections.
Olle's scenario is very hard IMHO. If like a server asking for Outbound RFC 5626 usage, which is anti-RFC5626 (no proxies, but just UAs can do Outbound and just when connecting to their edge proxy).
Will that happen with Kamailio today? Won't the TCP matching make us reuse the existing connection?
You receive a TCP connection from 1.2.3.4:9999. And later you need to send a request to 1.2.3.4:5060. Why do you assume that you can reuse the connection from 1.2.3.4:9999? why if there are two SIP nodes listening into different ports (i.e. 5060 and 6060) within the same server with IP 1.2.3.4?
Right. Thanks for that explanation. Missed that. But if the server for some reason opens FROM 5060 we're back in my issues.
Will the on-send route give me the possibility or is it triggered before kamailio selects a tcp connection? I'm a bit unclear of the exact situation where the on-send route is called.
[on-send] is just before the message is sent. AFAIK it is executed before the message is handled over to the transport layer - thus I think TLS paramters of the new connection are not available.
Kamailio needs a callback for the user to inspect client's or server's TLS certificates and decide whether to continue or reject the TLS session.
[event-route:TLS-connect-out] and [event-route:TLS-connect-in] are needed for this. One when we receive a connection and one when we connect somewhere else.
Generally, domain verification against the certificate name is broken. For example if you have to reopen a TLS connection for an in-dialog request, there is no proper URI (RURI, next hop Route URI) to compare against the certificate.
The trick is:
- The proxy forwarding a request via TLS needs to set a domain in the
Record-Route.
- Such a domain should not have SRV records (otherwise the indialog
request could arrive to other server).
This may require having a TLS certificate with varios AltSubjectName entries:
- mydomain.com
- server1.mydomain.com
- server2.mydomain.com
- server3.mydomain.com
Should be URI alt names with sip:server1.mydomain.com etc.
mydomain.com has NAPTR/SRV resouorces while the others are just A/AAAA resources. Clients are configured to use mydomain.com, but each proxy in the server infraestructure should set the proper serverN.mydomain.com in Record-Route when routing via TLS. This should fix the problem...
Right.
...but then you realize that lot of clients CANNOT resolve a domain in a Route header, so they cannot send in-dialog requests if the Record-Route contains a domain...
Should they do that with outbound? Just haul it down the open flow...
But yes, broken clients stop progress. I need to start testing this.
Thanks to both of you for valuable feedback! /O
-
Iñaki Baz Castillo -
Klaus Darilion -
Olle E. Johansson