I agree that voicemail is the single biggest hurdle to carrier-grade services right now, and I don't know of any open source alternatives to sems and asterisk :-( If you're right, it should be possible to extend sems to become a real carrier-grade alternative. But it may require that those who have the needs actually get their hands dirty and contribute to the code base! However, I also see the lack of standards on provisioning of IP phones (tftp/http) and complex/non-standard implementations of SIP as problems. Yesterday, we discovered that a Cisco IP-PSTN gateway does not like SIP Ok messages from the latest build of X-Lite (1103, I believe). Cannot say whether it's Cisco or X-Lite, but incoming calls don't get through... Also, quality of service is a bit of problem when you sit with a mix of networks you control and networks you don't know anything about. I would love to see a generally adopted framework for monitoring and reporting QoS data, as well as a generally accepted adaptive approach to adjusting codec settings to maximize call quality for each conversation. Telchemy has an initiative here, so there are things moving, but...
As for a book, I don't really care whether it is hardcopy or on the web. I agree that the learning curve is a bit steep and that there are quite some aspects of ser that could be documented more. A repository of working config files for different applications would also be nice. Of course, the core developers have rarely time to do such a thing, unless one decides to make it a calling and steps out of the daily development work for a while... I'm sure if one central and knowledgable person would start a documentation initiative, there would be plenty of people willing to make drafts of individual sections/topics. I think the PHP HTML documentation on http://www.php.net where users can comment each section of the document is a very useful tool for increasing the base-level of understanding in the community.
Iptel is a source of paid consultants. But maybe they need to be more visible and communicate their offering a bit better? Of course, it's a matter of business model if you want to replicate the jboss model. There's certainly enough activity on the serusers list to warrant a more organized resource center of documentation, consultants for hire, etc. Somebody may turn ser into a major business opportunity as a core in others infrastructure, much like jboss. And RedHat turned Linux into something that enterprises could trust by adding stuff that enterprises like to hear is there and by offering support, without stopping the open source innovation. Is ser a big enough "application" to make the foundation for using that business model?
Well, Paul and I are both fairly "distant members" of the ser community, it would be interesting the hear the opinion of the people closer to ser?!
Regards, Greger
Java Rockx wrote:
Thanks for the reply. See my inline comments.
Regards, Paul
--- "Greger V. Teigre" greger@teigre.com wrote:
Thank you for sharing your views on using ser for carrier-grade services. I find that open source projects often lack information/sharing on scalability, redundancy, monitoring and other important things in a carrier-grade solution. Such information is often considered a competitive advantage with the result that the open source project looses a lot of the potential carrier-grade improvements.
I share with you the belief that ser can be put into productionfor carrier-grade services. My company has a focus on end-user centric service delivery and provide managed services, as well as turn-key solutions for service providers. Our focus so far has been on home/mobile office services with dial-up, VPN, (two-factor) authentication etc. Our platform has a powerful and flexible service authentication and authorization engine at the core, with RADIUS as one front-end (using LDAP backend). In addition, we have a web front-end for end-user self-service and service administration, as well as an XML interface for CRM integration. Finally, we have a Windows-based software that can automatically configure a user's PC, install components, keep them updated, and also be a "menu" or guide for the user in using the services (voicemail, forwarding etc). Our service providers often have multiple agents, channels, resellers. Each with branding requirements, as well as CRM system integration requirements. So, several virtual providers can be enabled on the same platform. Of course, providing billing records is a must...
We are currently adding VoIP as a service option in ourplatform. I am sure many people on this list are working on commercial projects each with a slightly different angle. :-)
So more directly to the perspectives in your email:
- Interesting that you have selected WhiteBox as the distro. Surely,
a RedHat subscription at $349 should not really add a lot to the costs? We have looked at both CentOs, another RHEL-based distro, and WhiteBox, but find that the $349 is pretty cheap for the "luxury" of not having to worry about distros dying or introducing problems in libraries etc.
It's funny you mention this because just this morning we were discussing a router update that RH released that is not in WhiteBox yet. So now we're considering going to to production with RHPW which is only $69 and comes with a free upgrade to REL 4 when available. RHPW is a bit of a backdoor loophole in the RedHat pricing structure.
- You have opted for the development version of ser. We have been
(conservatively) staying with the 0.8.14. Any particular reason for going for development? (Except for cool features; I think the AVP functionality is particularly interesting) What is the experienced stability? Will you put the development version into carrier production?!
Putting -dev17 into production is a possiblity. This really depends on when ser-0.9 stable gets released. If we can hold back the business until then we'll use stable 0.9 otherwise we're forced to use -dev17 (Which has been very stable in our testing). The reason for this is that we have dependencies on avpops and a few other modules that are not in 0.8.14
- SEMS is a bit immature and I have honestly been in doubt whether I
should spend any time on it at this stage. We have looked at Asterisk as an option (no time for religious wars even though there is some overlapping functionality). In fact, we also evaluate PSTN-based voicemail solutions for scalability and reliability. I must admit, it's a bit backwards, but...
Voicemail is a big problem right now. I know I'll make a few Asterisk advocates upset with my next comment - but our perception is our reality :-(. Asterisk is not ready for a carrier-grade implementation. Everyone we've seen and talked to that uses Asterisk has stability issues when you exceed a small to medium size company. We need to provide voicemail for as many as 500,000 users and Asterisk ain't going to do the trick.
I'm not suggesting that sems/sipums will work either, but I believe it has a better chance than Asterisk.
If sems/sipums ends up not working either then we will be force to purchase carrier-grade voicemail applicances that are SIP aware. :-(
- I'm surprised that you have set up serweb at all. Our experience
in delivering end-user services is that your Achilles heel is support. There is no limit to what a naive end-user can assume or believe and it will most definitely cause you a support call.... I look at serweb as an example application, and I think it would be very interesting to make a LGPL'ed library for ser functions that could be accessible through soap services. This way one can easily integrate ser functionality into the gazillion of CRM and customer front-end systems that you find in the telco world.
serweb is a total band-aide until we create similar functionality in our J2EE web farm. We've stripped out much of what is in serweb and only give users access to things like their personal phonebook, speed dialing, etc.
- You mention an enum directory for subscribers. I'm pretty sure
we're going there, but there quite some issues related to service provider roaming, how it is charged etc. Of course, existing telcos are keen to stop any all-IP, non-billable efforts, and those of us who deliver to telcos must (at least to a certain degree) adhere to their demands. Non-incumbents will lead the way here, but I believe that we need to establish a viable business model for exchanging traffic. The market (or the providers) are not ready for all-you-can-all-around-the-world options yet. As long as minutes are charged between IP and PSTN, there will be an incentive to move as much traffic as possible through PSTN (also for the non-incumbents with such business models).
I agree.
Well, talking about rambling... I think the point is: By showing our cards a bit, do we have some common interests that can benefit the ser community or create a new business relationship?
Possibly. Voicemail is still my single biggest problem. And the only solutions I know of are Asterisk and sems. Do you know of any others (open source)?
As for helping the ser community - I really think the single biggest way would be to product a real book - kind of like a FSF GDB or GCC book with detailed explainations of "if ser.cfg option does this" and "if you need to provide xxxx then you must have yyyy in ser.cfg" and expert hints and things to avoid. I'm talking about a real book here not an abbrievated cookbook or mini-howto's.
Also, a paid consultancy like the ones that Orion Server or JBoss have would be awesome. I've read in the serusers list of people offering to pay someone to fix their ser.cfg.
That all said - I'm not an author and I know I'm not an authoritative knowledgebase on ser so I'm sure I'm not the one to do this, although I'd love to help. The sad part is that I can only imagine how busy Daniel, Jan, Jiri, Andrei and company are so I'd say nobody should expect much involvement from them, other than editing and probably most ser users aren't saavy enough to read ser source code to find out exactly how the product works. And (documentation == success) of any product since lack of documentation will lead to fewer deployments.
Just my US$0.02 worth
g-)
Java Rockx wrote:
Greger,
I believe the small change to the onfailure_route[] was the thing that made the difference. Andrei pointed out a small error to me where I was calling fix_nated_contact() for more messages than I should have. (THANKS ANDREI!!!)
The other thing that made a **substantial** improvement was the fact that our PSTN gateway provider that uses Sonus equipment as added as an "alias" at the top of the ser.cfg. This caused ser to not include "Route:" headers on ACKs and this was causing the Sonus equipment to drop calls after two minutes.
It's funny how such small changes to ser.cfg can have dramatic effects.
I'd love to see a complete book published on ser. I'm sure, with the help of the maintainers and serusers participants this could become a reality. I mean this is the best damned SIP router I know of. Commercial solutions such as Sonus are not really a "solution" because they run as much as US$2 million per box and most small companies can't afford "one". And even if you could afford =>one<= you may not be able to afford others for scaling your VoIP platform
- so whats the point of attempting VoIP if you can't scale?
Our solution is modeled after the Google network whereby:
unlimited horizontal scalablity == unlimited cps (calls per second)
And this equates to:
successful VoIP Company == retired employee
If we can indeed do this for our implementation and do it on low end x86 or Opteron boxes (or blade servers), then the sky is the limit. Our engineer whom we acquired from RedHat a few months ago already has a full TCP/IP and UDP socket patch working so we can seperate ser and serweb and scale them independently of each other (over a VPN) since we don't need ser's existing FIFO.
SEMS is a whole other beast. We are now working on this using the ivr module with sipums for a carrier-grade voicemail solution. Early indications is that we still have plenty to do in order to get 99.999% reliablity with sems. Our goal is to not have sems run on the same box as ser as we already know this is bad. Voicemail and SIP proxy cannot be the same box if you expect to be a big VoIP provider, IMHO.
So our high-level VoIP platform uses the following as independent server farms:
MySQL 4.1.7 (farm 1) ser-0.8.99-dev17 (farm 2) serweb on Apache2 farm 3) sems/ivr/sipums (farm 4, still a work in progress) Third party PSTN gateway *** see comment below
All servers use Whitebox Linux Respin 1 and our network implementation allows us to completely administer all servers at any remote co-lo site --- even down to remote installation of the operating system. This includes make a change to the master ser.cfg and having it replicate to all ser proxies.
Monitoring is a big part of this network architecture and we've only got basic monitoring right now, so this will become a focus of our efforts soon.
We also use third parties for providing PSTN access. IMHO, PSTN gateways are much more trouble than their worth. We'd rather rely on multiple third parties for this than to try and do it ourselves. The cost of DSP boards is expensive, especially if you wanted to handle say 50,000 concurrent callers. For this very reason we use companies such as Level 3 and Global Crossing and all we have to do is simply monitor their QoS and route PSTN callers to other PSTN gateways as needed. And this is all at the network level, so ser is totally oblivious to this.
Another thing I've love to see is a database whereby VoIP service providers can lookup 10-digit DIDs in a common database (much like a root DNS server) and determine if a callee is a VoIP user and route the SIP call directly from our SIP proxy to their SIP proxy rather than making a SIP->PSTN->SIP call. We have the Chairman of the IPCC deeply invovled in our company and he could get the ball rolling on this, but we don't want to get distracted right now.
Anyhow, I've rambled long enough.
Regards, Paul Hazlett
We'll publish our
--- "Greger V. Teigre" greger@teigre.com wrote:
Paul, Thanks for posting your resulting config after so many hours of testing! I did a diff and can see that you have done several (smaller) changes. But what did the trick? (i.e. for Route problem) g-)
Java Rockx wrote:
Greger,
I finally got everything working without STUN and without an Outbound Proxy. I an only using rtpproxy/nathelper.
I've tested these settings with the following SIP Phones/ATAs
Sipura UTstarcom iAN-02EX Grandstream ATA486 Grandstream BT100 Cisco ATA186 Cisco 7960G WorldAccxx ATA X-Ten Pro X-Ten Lite
We are very happy with everything now. The only piece of the puzzle that I don't have working yet is sems/sipums for voicemail
- but I'm working on that.
Attached is my complete ser.cfg file that is working. Please note that I'm using ser-0.8.99-dev17 for this rather than ser-0.8.14
If anyone finds problems in this ser.cfg then please let me know - but like I said before, things seem very stable right now with -dev17.
=== message truncated ===
Do you Yahoo!? Meet the all-new My Yahoo! - Try it today! http://my.yahoo.com