I beg to disagree -- we should not create to much workarounds around imperfect clients. In particular, incomplete NAT traversal support is a serious shortcoming in a UA and I would discourage people from using such devices.

Other front to attack would be NATs -- there is an effort in IETF focusing on that, but that's obviously an activity which has no impact on currently installed base.

-jiri

At 01:57 AM 8/23/2004, Richard wrote:

Hi Jesus,

Changing UA is not always a viable solution due to pricing and other technical issues. Every UA has something broken in its implementation and it would be very costly to change it because one thing (in this case, NAT) is broken.

Thanks, Richard

-----Original Message----- From: Jesus Rodriguez [mailto:jesusr@voztele.com] Sent: Sunday, August 22, 2004 8:59 AM To: Richard Cc: serusers@lists.iptel.org Subject: Re: [Serusers] NAT ping and consumer router

Use an UA that supports it (Sipura or Cisco for example).

Saludos JesusR.

---

Jesus Rodriguez VozTelecom Sistemas, S.L. jesusr@voztele.com http://www.voztele.com Tel. 902360305

---

---

Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers

-- Jiri Kuthan http://iptel.org/~jiri/

Richard writes:

Going back to the real problem... In the short term, I think that the quickest solution is to find out a NAT device which allows inbound traffic to refresh NAT binding timer. I know that cisco PIX is one of them, but would like to find out if any lower end NAT firewall/router have it too...

any cisco ios router, e.g., low end dsl routers, have sip aware nat and you don't need any pinging. pix is cisco firewall and may be too expensive.

-- juha

The problem, as I see it from this discussion, is that some devices do not work correctly with the simple UDP packet sent to port 5060 of the remote UA, because there is no reply packet which is what keeps the NAT mapping of some NAT router/translators. I don't see this as a UA problem; if there is no NAT translation, then even the best-programmed UA can't receive an inbound INVITE.

The manner in which Asterisk handles this type of keepalive is somewhat simple but novel, and may be worth examination. Every X seconds, an OPTIONS request is made to the remote UA by the server. Even if the UA does not support the OPTIONS query, it typically hands back a SIP error, which serves the purpose of keeping the NAT translations open. If the device supports OPTIONS, then a "normal" SIP reply is sent, also serving the intended purpose.

Perhaps instead of a UDP packet with no content, a SIP OPTIONS request could be sent by SER. This could perhaps be an selective flag associated with the NAT support in SER, so that either the dummy packet or the OPTIONS packet could be transmitted by the module.

There are other solutions here, like reducing the interval of REGISTER requests to serve the same purpose of refreshing NAT table mappings. However, one could argue that this method has a much higher load than an OPTIONS packet, especially when scaling across thousands or tens of thousands of clients in an environment where external databases (i.e. Radius, SQL, etc) are used for authentication lookups.

Note that there have been numerous examples of such poorly-written SIP stacks on UA devices that they would crash on an OPTIONS request. Their repair is outside the scope of SER or this discussion.

JT

At 2:56 AM +0200 on 8/23/04, Jiri Kuthan wrote:

I beg to disagree -- we should not create to much workarounds around imperfect clients. In particular, incomplete NAT traversal support is a serious shortcoming in a UA and I would discourage people from using such devices.

Other front to attack would be NATs -- there is an effort in IETF focusing on that, but that's obviously an activity which has no impact on currently installed base.

-jiri

At 01:57 AM 8/23/2004, Richard wrote:

...

Hi Jesus,

Changing UA is not always a viable solution due to pricing and other technical issues. Every UA has something broken in its implementation and it would be very costly to change it because one thing (in this case, NAT) is broken.

Thanks, Richard

-----Original Message----- From: Jesus Rodriguez [mailto:jesusr@voztele.com] Sent: Sunday, August 22, 2004 8:59 AM To: Richard Cc: serusers@lists.iptel.org Subject: Re: [Serusers] NAT ping and consumer router

Use an UA that supports it (Sipura or Cisco for example).

Saludos JesusR.

---

Jesus Rodriguez VozTelecom Sistemas, S.L. jesusr@voztele.com http://www.voztele.com Tel. 902360305

---

-- Jiri Kuthan http://iptel.org/~jiri/

At 5:07 PM +0200 on 8/23/04, Jiri Kuthan wrote:

At 04:30 PM 8/23/2004, John Todd wrote:

...

The problem, as I see it from this discussion, is that some devices do not work correctly with the simple UDP packet sent to port 5060 of the remote UA, because there is no reply packet which is what keeps the NAT mapping of some NAT router/translators. I don't see this as a UA problem; if there is no NAT translation, then even the best-programmed UA can't receive an inbound INVITE.

It is a UA problem. NATs are based on client-server paradigm, have been there for a while, and applications desiring to receive packets out should first send some out. This is a differentiator for many products today. There are such which work (and have typically other shorctomings), there are broken implementations which don't. That's just normal early-technology shopping process.

There is a NAT problem too -- NATs feature quite undeterministic behaviour. That's something that needs to get fixed to and IETF behave effort is good for.

I agree. However, I don't see ICE or any of the other extensions becoming part of most UA implementations for at least another >1.5 years (assuming that approvals happen quickly at IETF) but those of us with customers have to come out with solutions faster than that in this quickly-solidifying market.

...

The manner in which Asterisk handles this type of keepalive is

somewhat simple but novel, and may be worth examination. Every X seconds, an OPTIONS request is made to the remote UA by the server. Even if the UA does not support the OPTIONS query, it typically hands back a SIP error, which serves the purpose of keeping the NAT translations open. If the device supports OPTIONS, then a "normal" SIP reply is sent, also serving the intended purpose.

Its great it mostly works but it is a hack. It introduces lot of brittlenes -- it will fail whenever NAT bindings change: if NAT reboots, it will fail if NAT is not too deterministic, it will fail if forcible IP address change occurs, etc. Getting it robust is simply hard without client support. (Which is BTW a simple application of the e2e principle.)

I agree that it is a hack, but so is any solution that tries to solve this problem from the "outside" of the NAT. Using OPTIONS is perhaps just a slightly different hack that may make more NAT boxes do the right thing.

(Side note: has anyone generated a list of NAT boxes/software which require outbound packets for translations to stay open? In other words: where, exactly, does SER's method NOT work?)

...

Perhaps instead of a UDP packet with no content, a SIP OPTIONS

request could be sent by SER. This could perhaps be an selective flag associated with the NAT support in SER, so that either the dummy packet or the OPTIONS packet could be transmitted by the module.

...

There are other solutions here, like reducing the interval of REGISTER requests to serve the same purpose of refreshing NAT table mappings. However, one could argue that this method has a much higher load than an OPTIONS packet, especially when scaling across thousands or tens of thousands of clients in an environment where external databases (i.e. Radius, SQL, etc) are used for authentication lookups.

I would like that better. We could perhaps mitigate the performance penalty by granting re-REGISTERs which don't change too much -- that could be possible done as authentication-less in-memory lookup.

I'm not familiar with what you reference here; are you talking about cached credentials, or some other method that isn't a full authentication lookup for REGISTER requests which "appear" to have the same characteristics as prior registrations? (danger! I've imagined what you might be talking about, and in my (perhaps incorrect) assumptions, I can see some security problems. With this method, it would probably be easy to "take over" a SIP UA's identity for inbound calls without password authentication if the attacker was behind the same NAT external address.)

...

Note that there have been numerous examples of such poorly-written

SIP stacks on UA devices that they would crash on an OPTIONS request. Their repair is outside the scope of SER or this discussion.

:) Well, broken end-devices are unfortunately a never-ending pain.

My flat forehead (from banging against a wall) is direct evidence of this issue.

So on the to-do-list, I think there is an effort to educate UA vendors how to get things right, and there is an option to force short re-registration interval and try to invent some way which will reduced the performance penalty.

-jiri

While I think that education of UA vendors is noble, I think that often these subtle and important issues fall outside of their comprehension in the headlong drive to get as many products out the door as possible, with minimally qualified VOIP programming staff. (/me points to flat forehead again)

The short re-registration interval technique would be great, if it can be shown to scale and remain secure.

I still like the method of using the OPTIONS queries as a short-term hack, even if I have to use (as one other poster did) the sipsak method to produce those queries. I actually kind of like that method, except that now I have to extract the list of "registered" users into yet another place for yet another script to run against them... Incorporating this into the nathelper module does not seem to be "overkill" - it's a module, not a core component of SER, so feature creep is (in my opinion) more acceptable in certain places which we already recognize as being "hacks."

JT

Hi all,

It was a good discussion. Although I didn't find out any consumer router/firewall working with NAT ping, I had some surprisingly good finding.

Before this whole nat ping and binding refreshing issue, I always think a better solution is SIP aware NAT, i.e. ALG. I didn't find any good device because most ALG router/firewalls were used to be high-end and relatively expensive. Also many ALG implementation, e.g. fixup protocol in cisco PIX have some serious flaws to be usable.

Fortunately I found one very good implementation of SIP ALG. It is linksys wireless router WRT54G with the latest firmware. This sub-$100 router changes the private IP address embedded in the SIP message, e.g. via and SDP addresses. The sip NAT binding has a 3600 seconds expiration timer. So as long as the phone register once an hour, it should be fine. The best part of this solution is that linksys provides source code which is based on linux 2.4 kernel. If there is ever a need to add a feature, we can produce our own firmware image.

To complete the story, this is not a panacea. It won't solve all problems. For example, it is still lacking deterministic port mapping. For example, if the router has a reboot, all the NAT bindings are lost. The new binding will mess up with existing ones saved in ser. Anyway, in the short term, this is probably the best we can find out.

Ironically a division of cisco can produce something much better than its flagship firewall products.

Thanks, Richard

-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@lists.iptel.org] On Behalf Of Jiri Kuthan Sent: Monday, August 23, 2004 9:15 AM To: John Todd Cc: serusers@lists.iptel.org Subject: RE: [Serusers] NAT ping and consumer router

At 08:50 PM 8/23/2004, John Todd wrote:

I agree. However, I don't see ICE or any of the other extensions becoming

part of most UA implementations for at least another >1.5 years (assuming that approvals happen quickly at IETF) but those of us with customers have to come out with solutions faster than that in this quickly-solidifying market.

agreed.

...

...

The manner in which Asterisk handles this type of keepalive is somewhat

simple but novel, and may be worth examination. Every X seconds, an OPTIONS request is made to the remote UA by the server. Even if the UA does not support the OPTIONS query, it typically hands back a SIP error, which serves the purpose of keeping the NAT translations open. If the device supports OPTIONS, then a "normal" SIP reply is sent, also serving the intended purpose.

...

Its great it mostly works but it is a hack. It introduces lot of

brittlenes -- it will

...

fail whenever NAT bindings change: if NAT reboots, it will fail if NAT is

not too

...

deterministic, it will fail if forcible IP address change occurs, etc.

Getting it

...

robust is simply hard without client support. (Which is BTW a simple

application of the e2e

...

principle.)

I agree that it is a hack, but so is any solution that tries to solve this

problem from the "outside" of the NAT. Using OPTIONS is perhaps just a slightly different hack that may make more NAT boxes do the right thing.

(Side note: has anyone generated a list of NAT boxes/software which require

outbound packets for translations to stay open? In other words: where, exactly, does SER's method NOT work?)

I'm unfortunately not aware of such.

...

I'm not familiar with what you reference here; are you talking about

cached credentials, or some other method that isn't a full authentication lookup for REGISTER requests which "appear" to have the same characteristics as prior registrations? (danger! I've imagined what you might be talking about, and in my (perhaps incorrect) assumptions, I can see some security problems. With this method, it would probably be easy to "take over" a SIP UA's identity for inbound calls without password authentication if the attacker was behind the same NAT external address.)

Well, the assumption would be that contacts (including port) have not changed. If that is the case, skipping authentication introduces rather slight security risks.

-jiri

_______________________________________________ Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers

Hi Jiri,

As I mentioned before, I was not a big fan of ALG before. I tried ALG on some devices, too many problems. However I was pleasantly surprised that linksys has a good implementation, i.e. all the problems I had with other implementation doesn't have it here.

Just like Nathelper and mediaproxy are temporary fix which introduce extra media hop, nothing is perfect. Also I experience too many problems of using cpl with nathelper, trying to figure out how to not mess up with SDP if two phones behind the same NAT. Being able to access and play with the source code of linksys is certainly a big plug considering that we don't have access to source code of any hard phone.

In lieu of a perfect UA, I think this is least evil solution, if you think this way...

Richard

-----Original Message----- From: Jiri Kuthan [mailto:jiri@iptel.org] Sent: Monday, August 23, 2004 11:02 PM To: Richard; serusers@lists.iptel.org Subject: RE: [Serusers] NAT ping and consumer router

Hi Richard,

I'm not entirely happy I am so frequent disagreement initiatior in this thread but I don't like ALGs too much either. They have a bunch of issues, primarily they don't work with security and secondly they have a high potential for misimplementing the application logic. This has turned out to be true in quite many cases in the past. (Nevertheless good to hear there is a working linksys product.)

-jiri

Hi Andres,

It is running 2.04.4 from linksys. It seems only available from the 2.x.x train from linksys. Even other models of linksys don't support it. If you want, you should be able to compile Sveasoft based on the 2.04.4.

Cheers, Richard

-----Original Message----- From: Andres [mailto:andres@telesip.net] Sent: Tuesday, August 24, 2004 5:41 PM To: Richard Cc: serusers@lists.iptel.org Subject: Re: [Serusers] NAT ping and consumer router

Fortunately I found one very good implementation of SIP ALG. It is linksys wireless router WRT54G with the latest firmware. This sub-$100 router

Hi Richard,

We have deployed quite a few WRT54G routers with firmware based on 2.07.1 (by Sveasoft) which as far as I can tell is the latest posted by Linksys. We use them because we can run traffic shaping on them which is absolutely necessary when we install our SIP devices in small businesses. However we have not seen any evidence of a SIP ALG on them. Can you tell me on which firmware you saw this?

Thanks,

At 04:13 PM 8/25/2004, Jesus Rodriguez wrote:

That's like known SBCs work. Before, they sent an OPTIONS packet but last software versions modify the Expire of the 200 OK from REGISTER so is the UA the one that sends a new REGISTER and generates traffic from the internal side every 30 seconds. These REGISTER are not forwarded to the proxy. The SBC keeps a table where saves the "real" REGISTER expire sent to the proxy. This way you don't flood the proxy with all those REGISTER.

Which appears like a questionnable benefit -- I mean it does not seem to make a big difference to me if the extra REGISTER load is put on SER PC or some other PC in front of it... (Not mentioning what other pain SBCs cause -- I just spent a good part of day/night reviewing a network setup in which such an SBC caused real harm.)

I objected several ideas on this mailing list, like ALGs, SBC, and other in-network NAT/SIP patches, so I should perhaps state what I consider a good idea. First it takes good NAT support in end-devices. Creating network-based patches to deal with NAT-incapable devices just creates a great amount of complexity which typically breaks other things. Make clients detect connectivity using STUN, perhaps ICE, let it send keep-alive, implement SIP stack correctly, be symmetric, etc. That's something which is achievable short-term, it is also buyer's responsibility to buy those that get it right.

Long-term removing indeterminism from NATs as in IETF/behave is another important acchitectural activity.

If people ask me how to make SIP/NAT stable right now, regardless of sanity, my preference is to find a good UAC and stick to it. (I have to admit I don't have a specific recommednations -- the capabilities change rapidly fimrware by firmware in a pace I can't follow.) Configure SER to help by reducing registration intervals, using a voice proxy, sending pings.

-jiri

At 07:45 PM 8/25/2004, Richard wrote:

Hi,

I hate to argue with a guru whose product benefits us a lot... :)

Anyway, if you can program your ALG and fix any problem one might have, why isn't it a better choice? Some routers give away source code. They are linux kernel 2.4 with netfilter. It tracks various protocols besides SIP. I checked their code, it is no different than the methods used in nathelper, mangle the ip address embedded in SIP message. I'd think that it is definitely better than reducing registration interval, using voice proxy and sending pings.

Security does not work -- SIP/TLS will fail.

Secondly, I don't share your optimism on that ALG vendors will get the application logic right. Field experience shows that my pesimistic attitude is quite realistic. There were even bizzar products that claimed support for SIP but actually mangled it in a way which broke all communication. (Till this firewall was removed, SIP was running at port 5070.)

Btw, I don't think that one can find out a lot consumer based router working with NAT ping. 80% of products in the market are based on linux kernel/netfilter which only refreshes binding with outbound traffic and the timer for binding in 30 seconds by default.

Thanks -- that's interesting information. Anyhow -- I think that's an argument for making end-devices to resend keep-alives frequently.

-jiri

On Wed, 25 Aug 2004, Jiri Kuthan wrote:

At 04:13 PM 8/25/2004, Jesus Rodriguez wrote:

...

That's like known SBCs work. Before, they sent an OPTIONS packet but last software versions modify the Expire of the 200 OK from REGISTER so is the UA the one that sends a new REGISTER and generates traffic from the internal side every 30 seconds. These REGISTER are not forwarded to the proxy. The SBC keeps a table where saves the "real" REGISTER expire sent to the proxy. This way you don't flood the proxy with all those REGISTER.

Which appears like a questionnable benefit -- I mean it does not seem to make a big difference to me if the extra REGISTER load is put on SER PC or some other PC in front of it... (Not mentioning what other pain SBCs cause -- I just spent a good part of day/night reviewing a network setup in which such an SBC caused real harm.)

I was just explaining the method used by this box and not supporting it ;)

I agree that SBC and ALGs are not a good thing and getting a good nat capable UA is a better and easier solution.

I objected several ideas on this mailing list, like ALGs, SBC, and other in-network NAT/SIP patches, so I should perhaps state what I consider a good idea. First it takes good NAT support in end-devices. Creating network-based patches to deal with NAT-incapable devices just creates a great amount of complexity which typically breaks other things. Make clients detect connectivity using STUN, perhaps ICE, let it send keep-alive, implement SIP stack correctly, be symmetric, etc. That's something which is achievable short-term, it is also buyer's responsibility to buy those that get it right.

Long-term removing indeterminism from NATs as in IETF/behave is another important acchitectural activity.

If people ask me how to make SIP/NAT stable right now, regardless of sanity, my preference is to find a good UAC and stick to it. (I have to admit I don't have a specific recommednations -- the capabilities change rapidly fimrware by firmware in a pace I can't follow.) Configure SER to help by reducing registration intervals, using a voice proxy, sending pings.

-jiri

Saludos JesusR.

------------------------------- Jesus Rodriguez VozTelecom Sistemas, S.L. jesusr@voztele.com http://www.voztele.com Tel. 902360305 -------------------------------