Op 31-12-09 09:42, Alex Balashov schreef:

No. It's a proxy, it can't originate requests.

I am sorry, maybe I do not understand enough about SIP and proxies. Wouldn't it be possible to fake re-invites? Tell each phone that it gets a reinvite coming from the other phone, and basically redirecting the sound stream to an rtpproxy for example? or a codec translator?

I'd like to switch from g711 to g729 when the link gets overloaded for example. Or record the phonecall.

I am trying to learn more about SIP. Aren't reinvites within a dialog common?

Maybe I could make changes to openser to enable this?

Antonio

On 12/31/2009 03:33 AM, Antonio Goméz Soto wrote:

...

Hi,

maybe I should go to the -dev list for this, but thought I'd ask here first. Is it possible to let openSER send a SIP reinvite in the middle of a call, to redirect the sound stream to some rtpproxy? Maybe using the MI?

Or should OpenSER be modified to do this?

Thanks, Antonio

---

Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users

Op 01-01-10 17:37, Alex Balashov schreef:

On 01/01/2010 11:32 AM, Antonio Goméz Soto wrote:

...

I am sorry, maybe I do not understand enough about SIP and proxies. Wouldn't it be possible to fake re-invites? Tell each phone that it gets a reinvite coming from the other phone, and basically redirecting the sound stream to an rtpproxy for example? or a codec translator?

I'd like to switch from g711 to g729 when the link gets overloaded for example. Or record the phonecall.

I am trying to learn more about SIP. Aren't reinvites within a dialog common?

Maybe I could make changes to openser to enable this?

Yes, reinvites are common, and you have correctly ascertained their purpose.

What I think you are not understanding is that a reinvite, like any other request, must be originated from an endpoint (a user agent). A proxy is not a user agent; it cannot generate replies. The only thing proxies do, as a matter of definition, is forward requests they receive, and forward replies to them in the opposite direction (when applicable).

Again, I draw your attention to the HTTP server vs. Squid analogy we made earlier. The question you are asking sounds equivalent to, "Can Squid send a GET request?" The answer is no, Squid cannot send any requests; it can only act on requests it receives and direct them somewhere.

Right. I see now. It can only send out requests which it has received, and not create new ones. But on the other hand, kamailio *does* keep dialog state, so might it be possible to create some add-on module, that injects fake messages into the input stream? Using info that is in the dialog state? Or.. come to think of it, would that mess up the dialog between the endpoint, because it would increment counters that are maintained by the endpoint?

Antonio

Op 01-01-10 17:54, Alex Balashov schreef: [ .. ]

The question you are asking is a technical one, but the answer is really philosophical and conceptual one (i.e. in terms of how RFC 3261 defines various types of SIP elements). Is it theoretically possible to add some hacks to a SIP proxy to make it behave as something other than a proxy by spoofing messages in a convincing way? Yes, it is - with enough ambition; in fact, Kamailio has a number of such hacks internally, such as the dialog module's timeout attribute which can spoof a BYE in both directions after a certain amount of time has passed. But that's what they are - hacks. Useful, perhaps, but they are departures from what a proxy is supposed to do that break proxy purity.

Ok, i see what you mean.

So, yes, you could hack the source to inject messages asynchronously. But when you do that, you no longer have a proxy, but something else, so it really comes down to (a) what you want and (b) what the product is intended to be. At the moment - several clever hacks notwithstanding, and some UAS functionality (e.g. registrar) notwithstanding as well - it's a proxy. As long as the intent of the product is to continue to be a proxy, it is largely limited to the sort of things a proxy can do, formally speaking.

And do you think the Kamailio maintainers oppose extending the functionality in this direction?

To do the kinds of things you're wanting to do in a way that is accommodated by the specifications and the protocol mechanics of SIP, what you're really looking for is something that can behave as a user agent in its own right.

Any suggestions (given the fact that it needs to accomodate a couple of ten thousands of phones?)

Antonio

1 jan 2010 kl. 18.24 skrev Antonio Goméz Soto:

Op 01-01-10 17:54, Alex Balashov schreef: [ .. ]

...

The question you are asking is a technical one, but the answer is really philosophical and conceptual one (i.e. in terms of how RFC 3261 defines various types of SIP elements). Is it theoretically possible to add some hacks to a SIP proxy to make it behave as something other than a proxy by spoofing messages in a convincing way? Yes, it is - with enough ambition; in fact, Kamailio has a number of such hacks internally, such as the dialog module's timeout attribute which can spoof a BYE in both directions after a certain amount of time has passed. But that's what they are - hacks. Useful, perhaps, but they are departures from what a proxy is supposed to do that break proxy purity.

Ok, i see what you mean.

...

So, yes, you could hack the source to inject messages asynchronously. But when you do that, you no longer have a proxy, but something else, so it really comes down to (a) what you want and (b) what the product is intended to be. At the moment - several clever hacks notwithstanding, and some UAS functionality (e.g. registrar) notwithstanding as well - it's a proxy. As long as the intent of the product is to continue to be a proxy, it is largely limited to the sort of things a proxy can do, formally speaking.

And do you think the Kamailio maintainers oppose extending the functionality in this direction?

...

To do the kinds of things you're wanting to do in a way that is accommodated by the specifications and the protocol mechanics of SIP, what you're really looking for is something that can behave as a user agent in its own right.

Any suggestions (given the fact that it needs to accomodate a couple of ten thousands of phones?)

Anything inserted by the proxy will mean that a middelbox adds +1 to the CSEQ. This means that the real UA will send another message with a duplicated CSEQ. If the server UA follows the RFC, this message should not be handled, as the CSEQ needs to be raised by at least one for every sent message from a UA. So the proxy will have to continue raising the CSEQ for all following messages by remembering a delta or you can just hope that the server UA ignores this issue.

The same issue occurs when a proxy does authentication (see the UA module in Kamailio for a warning exactly about this).

/O

On 01/02/2010 09:16 AM, Antonio Goméz Soto wrote:

couldn't you increment CSEQ by send bogus messages to the respective endpoints? A bogus SIP UPDATE or something?

No, because the sending UA uses certain pattern of CSeq values for requests it sends - there is a separate set of CSeq value states from the other endpoint UA, and they are not intrinsically related.

Secondly, I think you misunderstood what Daniel was saying; he was saying that to spoof subsequent requests the CSeq value would need to be increased relative to the "sender." But that assumes the proxy is a party to all of the UA's transactions, which cannot be guaranteed.

Once again, the problem here is fundamental - what you want is not what a proxy was designed to do, and Kamailio was not designed to venture outside of its principal character as a proxy to accommodate such needs. You really need something that can (a) asynchronously originate requests and (b) generate independent logical call legs without any requirement to know or conserve state integrity between two third parties.

Hi Antonio,

As I mentioned in one of my previous posts, and will emphasise again: large scale cannot happen magically just through the addition of a few small elements.

From your description of your environment, it sounds like the systems and features being used are primarily "host-wise" - e.g. tied to the runtime state/environment of a single instance of Asterisk on a single host. This is a limitation upon its scalability.

To provide services on a large scale, the system has to be composed of non-hostwise components; the architecture must be distributed. This is a defining property and an organising principle of its design from the very bottom, not a setting you can simply enable when your user base grows large enough. Building distributed VoIP service delivery platforms is a very different science than building non-distributed ones.

Once again, let me make a more general cluster analogy; if you have a single instance of an application on a different server, you cannot just add more some more servers, run the application in parallel on them, and expect it to work. Instead, either the whole application must be written in a way that anticipates its being deployed in parallel on multiple nodes, or single instances of it must be placed into some sort of harness that can implement a distributed/parallel abstraction layer for it while preserving for the application the illusion of single-instance runtime. Either way, multiple nodes executing the program must have a way of keeping shared logical state across the entire execution continuum (in a centralised or distributed way), passing messages amongst nodes asynchronously, synchronising storage access to prevent race conditions / mutual exclusion violations, avoid deadlocks, etc.

An application or service designed to run on multiple nodes to begin with will have these facilities baked into its architecture, while an application or service not designed to do that would probably have to be rewritten or, at least, very extensively modified in order to suit the new requirement. In more generic computation this probably means the use of something like the LAM/MPI libraries, or perhaps some sort of concept aspiring to Google's MapReduce and/or BigTable.

It's the same thing with VoIP and Asterisk. Much of what you've got now relies on particular Asterisk nodes performing particular functions, which just isn't how a distributed system works unless you are willing to settle for some sort of compromise involving node specialisation -- which might be okay: a dedicated conferencing server, dedicated ACD/queue server, etc. But this ultimately has scalability barriers too and represents an inefficiency.

I have mentioned one possible and common distributed Asterisk architecture before: a central FastAGI controller in which all application logic is implemented - and, in the case of things which already exist in Asterisk such as queues, often RE-implemented in a way compatible with distributed architecture - to which all calls are dispatched via N Asterisk servers. Such a backend could implement the necessary shared state for logical abstractions extended across N servers. The example I often give is one of Asterisk queues (in the sense of Queue()): a queue exists only in one Asterisk server, but you can reimplement the "user experience" aspect of a queue in FastAGI (estimated time to wait announcement, music, etc.), which would then allow you to extend one logical queue over multiple Asterisk servers and potentially support thousands of callers in "one" queue.

Even if you do not use such an architecture, you're going to have to think along these lines. Kamailio alone cannot make anything scale; the service delivery backend has to be built with a distributed architecture in mind.

-- Alex

Hello,

top posting is good :-) ...

just sharing my own way of scaling, let's say the SIP proxy way:

- I run all the services a proxy can do in the proxy (authentication, lcr, call filering, call forwarding, presence, user location) - keep asterisk for media related features (voicemail, announcement, recording)

I have the (big) advantage of usually designing from beginning the service and I convince customers that some features do not fit in a limited budget (cheap VoIP platform with cheap VoIP phones).

In your case you have to go with a more expensive server platform, meaning that you have to invest in several asterisk boxes and a routing mechanism between them, as you use very specific asterisk apps (like fop) and want to stick with that.

I would try to do: - partition the users per asterisk instance - as much as possible host the users talking to each other in same asterisk - use kamailio as central registry server - if the destination user is not on the same asterisk instance, forward it to kamailio and from there to the right asterisk instance

You will have all the features within groups, most of them across entire platform (e.g., things like call pickup will work only within the group).

Cheers, Daniel

On 1/4/10 1:46 PM, Antonio Goméz Soto wrote:

Alex,

thank you for your extensive answer. (sorry to top-post). It's just that I have been asked to scale up an existing system (which happens to be asterisk), and I'm now trying to get a grasp on how to do it.

Thanks, Antonio.

Op 02-01-10 17:24, Alex Balashov schreef:

...

Hi Antonio,

As I mentioned in one of my previous posts, and will emphasise again: large scale cannot happen magically just through the addition of a few small elements.

From your description of your environment, it sounds like the systems and features being used are primarily "host-wise" - e.g. tied to the runtime state/environment of a single instance of Asterisk on a single host. This is a limitation upon its scalability.

To provide services on a large scale, the system has to be composed of non-hostwise components; the architecture must be distributed. This is a defining property and an organising principle of its design from the very bottom, not a setting you can simply enable when your user base grows large enough. Building distributed VoIP service delivery platforms is a very different science than building non-distributed ones.

Once again, let me make a more general cluster analogy; if you have a single instance of an application on a different server, you cannot just add more some more servers, run the application in parallel on them, and expect it to work. Instead, either the whole application must be written in a way that anticipates its being deployed in parallel on multiple nodes, or single instances of it must be placed into some sort of harness that can implement a distributed/parallel abstraction layer for it while preserving for the application the illusion of single-instance runtime. Either way, multiple nodes executing the program must have a way of keeping shared logical state across the entire execution continuum (in a centralised or distributed way), passing messages amongst nodes asynchronously, synchronising storage access to prevent race conditions / mutual exclusion violations, avoid deadlocks, etc.

An application or service designed to run on multiple nodes to begin with will have these facilities baked into its architecture, while an application or service not designed to do that would probably have to be rewritten or, at least, very extensively modified in order to suit the new requirement. In more generic computation this probably means the use of something like the LAM/MPI libraries, or perhaps some sort of concept aspiring to Google's MapReduce and/or BigTable.

It's the same thing with VoIP and Asterisk. Much of what you've got now relies on particular Asterisk nodes performing particular functions, which just isn't how a distributed system works unless you are willing to settle for some sort of compromise involving node specialisation -- which might be okay: a dedicated conferencing server, dedicated ACD/queue server, etc. But this ultimately has scalability barriers too and represents an inefficiency.

I have mentioned one possible and common distributed Asterisk architecture before: a central FastAGI controller in which all application logic is implemented - and, in the case of things which already exist in Asterisk such as queues, often RE-implemented in a way compatible with distributed architecture - to which all calls are dispatched via N Asterisk servers. Such a backend could implement the necessary shared state for logical abstractions extended across N servers. The example I often give is one of Asterisk queues (in the sense of Queue()): a queue exists only in one Asterisk server, but you can reimplement the "user experience" aspect of a queue in FastAGI (estimated time to wait announcement, music, etc.), which would then allow you to extend one logical queue over multiple Asterisk servers and potentially support thousands of callers in "one" queue.

Even if you do not use such an architecture, you're going to have to think along these lines. Kamailio alone cannot make anything scale; the service delivery backend has to be built with a distributed architecture in mind.

-- Alex

---

Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users

Hello,

On 1/4/10 2:52 PM, Antonio Goméz Soto wrote:

Hi Daniel.

Op 04-01-10 14:11, Daniel-Constantin Mierla schreef:

...

I would try to do:

• partition the users per asterisk instance - as much as possible host

the users talking to each other in same asterisk

• use kamailio as central registry server - if the destination user is

not on the same asterisk instance, forward it to kamailio and from there to the right asterisk instance

It's not clear to me what this means. The phones register to Kamailio as their registrar server.

you can do this way, too, scaling better.

But I thought you want to preserve as much as possible current architecture. So the phones register to asterisk -- this to be done by configuring each phone to use the proper asterisk.

Once you add a new phone, you add a routing info in kamailio binding the phone id to asterisk ip (you can do that with alias_db, pdt, htable, and other modules, just a matter of taste).

In asterisk dialplan, if the dialed number is not a local phone, forward it to kamailio which will forward to right asterisk.

I can think of various options to optimize and auto-provision, like automatically updating kamailio's routing rules based on user registration.

Cheers, Daniel

And they will send INVITE's to their proxy (i.e. the asterisk server?). So only Kamailio will know the ip address of a phone (except if asterisk accesses the same database using realtime - is that what you mean?)

Antonio,

On 01/04/2010 07:46 AM, Antonio Goméz Soto wrote:

thank you for your extensive answer. (sorry to top-post). It's just that I have been asked to scale up an existing system (which happens to be asterisk), and I'm now trying to get a grasp on how to do it.

It may be necessary for you - and your superiors - to accept that not all problems can be solved with the constraints one is given, and this may be one of them.

As others have pointed out, you can reorganise Asterisk in various ways to take advantage of Kamailio by centralising and/or specialising some nodes. But while this may buy you more capacity, it is not a true distributed design, and a true distributed design doesn't seem impossible without reimplementing large aspects of your existing system.

-- Alex

On Saturday 02 January 2010 15:31:33 Antonio Goméz Soto wrote:

Hi all,

first I want to thank you for providing answers to my numerous questions. You have made perfectly clear that Kamailio is only a proxy, and is rather good at that.

Secondly, I now am rather pessimistic about scaling up our current asterisk system (which is only used by our own office) to be useful for the entire organisation.

Hi Antonio, I think that you are "blocked" with the limited vision that Asterisk gives you about VoIP world ... let me explain that ...

We actually use:

• flash operator panel to transfer calls

That could be done with xmlrpc calls if you use SEMS+Kamailio

• call recording (mid-call using *1)

SEMS have support for call-recording, we use it.

• call pickup

Well, this is a little bit complex ... but still doable.

• barge-in

... barge-in definition - telecom A feature of a key telephone system (KTS) or PBX, barge-in allows an authorized user from an authorized station to join, without invitation, an active call on a call in progress through the use of an authorization code the user enters via the telephone keypad. ...

Umm ... a type of chanspy I suppose ... well, this would be complicated ... but still doable.

• conferences

Supported by SEMS

• call forwarding

Supported by Kamailio with script routing.

• ACD functions + listening through ChanSpy

Umm ... maybe LB+Dispatcher

• and various other things

You will have to specify.

The data network people don't want all calls to flow through some central point, because of WAN network load, and think they can have phones talk to each other using g711 when they are one the same LAN, and g729 when they need to go through a WAN.

Trust me ... if you use VoIP phones (hardphones, no softphones), the will be not difference between G711 local calls and G729 local calls ... if phones are goog enought. No matter of what you do ... you could build a VoIP system with no "central point", it's just a matter of time,knowleadge and bugget

But as I see it now, it will be pretty much impossible to scale up using Kamailio if we still need operator panels in offices here and there, because there is no operator panel that can manipulate calls through Kamailio (because that's only a proxy), and there is no operator panel that can handle multiple parallel asterisk installations or SEMS or whatever.

You are wrong here, if you follow the "asterisk path" ... Asternic FOP 2.0 allow you to easly control more than one asterisk. On the other hand, you could also develop your own panels to control SEMS+Kamailo, you could build a VoIP infraestructure on witch you use SEMS as B2BUA withouth the need to forward all RTP traffic throught it, so you will get a very good and scalable Asterisk replacement.

Switching to different codecs depending on network location might be doable using an Kamailio extension, but that will require a lot of work, keeping a database, and manipulating the codec info in the SDP.

Functions to mangling the SDP are out there, but you will have to do the work, or hire someone to doit.

Mid-call recording poses it's own problems. If it must go undetected then ALL calls must go through a rtp proxy, which the network people will oppose, and even if not, Kamailio will not be the proper tool to enable it.

If you want it to be "undetectable" and don't want to use a RTP proxy, if you have the needed network hardware, you still could do it.

Also recording of G729 calls rules out SEMS and basically all other opensource solutions except asterisk.

You could use G729 with sems, we use it, you only need to write down some lines of C++ code to use the Intel G.729 ipp libraries, for example ... or you could develop a wrapper an use Asterisk .so files of G729 codecs, or a hardware card, it's a matter of time, knowleadge and bugget

Of course there also is the question of managing all this. We currently edit configfiles, but that doesn't scale all too well, so we'll probably need some management and reporting tools as well.

Yes, you have the tools, now it's time to beging to work, no pre-made systems, no just-click-and-run software.

And I currently have no clue on how to do attended transfer, or call pickup on such a system, both of which seem pretty basic to me in a corporate environment.

Eint ¿? ... an attended transfer it's no more than two calls A call B, B puts A on hold, B calls C, C accept the transfer, B transfer A to C ... it just work with the basic example .cfg of kamailio, because it's a work of the UA's involved on that calls, not a task of the proxy.

Call-pickup could also be done at proxy level, if your UA's support that supplementary services, if not .. a B2BUA must be used.

Scaling up our asterisk system while preserving all of its features seems to be a lot more involving than I initially thought...

Yes, it is ... a very big and very good challenge.

PS: And I'd like a one-to-one chat with some guys on the asterisk mailing list who simply point to openSER when someone asks for scalability ;-)

Could be used, it's a matter of knowing how to do it and that's the hard part ... ;-)

On 1/4/10 1:43 PM, Antonio Goméz Soto wrote:

[...]

...

...

And I currently have no clue on how to do attended transfer, or call pickup on such a system, both of which seem pretty basic to me in a corporate environment.

Eint ¿? ... an attended transfer it's no more than two calls A call B, B puts A on hold, B calls C, C accept the transfer, B transfer A to C ... it just work with the basic example .cfg of kamailio, because it's a work of the UA's involved on that calls, not a task of the proxy.

Call-pickup could also be done at proxy level, if your UA's support that supplementary services, if not .. a B2BUA must be used.

What is it exactly that a UA must support? Do regular VoIP phones support it?

for call transfer the phones should support REFER and most of them do.

For call pickup you need dialoginfo presence modules loaded on proxy and the phones to support the SIP extension rfc4235: http://kamailio.org/docs/modules/3.0.x/modules_k/presence_dialoginfo.html http://kamailio.org/docs/modules/3.0.x/modules_k/pua_dialoginfo.html

Not many phones support it now, but I guess more will do it in near future. Snom does it now pretty good.

Cheers, Daniel

Op 02-01-10 19:48, Olle E. Johansson schreef:

2 jan 2010 kl. 16.31 skrev Antonio Goméz Soto:

...

Hi all,

[snip]

Well, to scale you DO need a proxy. The issue here is what you use the proxy for and how you distribute the telephony services you need. You always have to start with the services you use. If Asterisk is a simple PSTN gateway for SIP, then it is very easy, since there's no state involved you are interested in sharing. If you are using Asterisk for PBX services and you do need blinking lamps, there is a state you are interested in.

Now, with a decent server, you can manage thousands of simulatenous calls on one Asterisk server. You can use OpenSER for failover between two Asterisk servers, but only use one in production in order to preserve states on one machine. That's is one way of making it easy for yourself.

If you really need to distribute states, there are now multiple ways to do that between multiple Asterisk servers, with custom device states as one example.

So the people who pointed you to OpenSER/kamailio was not wrong at all. If they said that it was easy to scale a PBX using OpenSER, they might have been wrong. It's not easy yet, but many of us are working on making it easier.

/O

Olle,

this sounds like decent advice. Looking at the asterisk source, I think I could safely say that with all lists converted to astobj2 (in 1.8) that asterisk can handle tens of thousands of phone registrations, and thousands of channels. I think asterisk is making great strides in scalability at the moment, don't you agree? It might even scale up to OpenSER/Kaimilio numbers.

It may not be able to handle the media for all those calls, but that is an entirely different matter.

OTOH, it might just be wishful thinking..

Thanks, Antonio