Hello,

Let me clarify my yesterday's proposal for new features.

1) New modules: a) auth_ldap b) group_ldap c) uri_ldap d) avp_ldap

It would be desirable for these modules to re-use existing dbase abstraction layer in SER. So to treat Ldap as just one more backend. So far the Ldap support was created in an ad-hoc manner, without regard for global dbase approach.

Jiri Kuthan kas proposed to do "one module, which would act as a single LDAP driver for all DB-based modules". I think that this is something similar - having a common way (a driver) to access LDAP and then having all standard modules to just reuse this new ldap backend.

2) "soap_ipdr" module versus "sermanager" idea I proposed to embed IPDR reporting tool for billing in a new SER module. But after some discussion I think maybe the better option is to create this as an external C application. It would be a stand-alone application using gSOAP library, containing built-in simple Web server. Application could be called "sermanager" and could provide SOAP interfaces for user at the one side and connect to SER subsystem on the other. To connect to SER it would use sql, fifo, ldap or other technologies that are applicable for particular SER installation - it would be configurable by administrator. It would serve as a middlelayer (mediation software), providing vast set of functionalities for user,and freeing SER core from any additional work. SER will just serve for call routing/reporting.

I observed that the same idea happening in Asterisk. They will have external application called 'manager' that provides easy-to-use XML-RPC and SOAP interfaces to the world, at the same time speaking with Asterisk using AGI or simple file operations. Their reason was also to free Asterisk from additional work and create separate application entity to handle broad set of user-friendly functions.

"Sermanager" could contain functions divided in different areas of interest. They could be added as modules, so for example when user does not wish to use billing, he just unloads relevant module. I propose to split functionalities into following separate Web services (called "engines"). Of course every engine and every function will be accessible by different classes of users. Access information will be stored in small accompanying sql database or ldap backend, according to administrator wish.

a) "call_register" engine - call reporting (calls made, received, missed), without call costs. It will be in proprietary format, to be agreed. In addition calls made will be exported in standard IPDR format, VoIP profile.

b) "billing_management" engine - call costs reporting in proprietary format. This element will depend on external Web Services (BSS systems, SAP and others). On request from user, "sermanager" will send list of calls to this external service and retrieve calculated costs of calls. Then it will send this list to user.

c) "users_management" engine - adding, deleting, modifying users (either single users or batches of users)

d) "domain_management" engine - adding, deleting, modifying SIP domains. Optional - we have many domains on every single server, so we have tools to add dbase entries and eventually DNS entries. Not everybody will need that.

e) "registrar_management" - functions to export list of registered users per domain/per server.

f) "system_management" - reporting of SER statistics, export of SER log messages, current transactions, memory utilization, etc.

g) "terminal_management" - adding,deleting,modifying user agents, storage of device data and configuration files, etc.

h) "voicemail_management" - retrieving/deleting/modifying remote voicemail files on voicemail servers, such as Asterisk or SEMS. Voicemail messages will be returned in SOAP Body or as MIME attachment. Engine will require external Web service to run on Asterisk or SEMS machines. Unless we will have Web services from Asterisk that will provide access to voicemail functions - it may happen soon. Until then I can provide set of Web services to handle voicemail on Asterisk.

i) "phonebook_management" - adding/modifying/deleting phonebook entries in both: private and public phonebook, retrieving entries, searching for entries

j) "service_management" - initiating calls with REFER, initiating calls directly at User Agent (like Asterisk call-out), or other services, depending on needs. It may serve in future to add services like: automatic call-out campaigns, integration with SMS gateways and others

Other functions may include: Instant Messaging and Presence capabilities, like: sending Instant Message or SMS, getting list of messages, getting buddy status and/or location, etc. It is to be discussed as most of those functionalities are also provided by SIP itself.

Imagine the possibilities of having a SIP software phone with build-in SOAP client. Then you can easily integrate a server-stored public or private phonebook, create a buddy list that will reporting their online status and so on.

When the "sermanager" application is ready, I can to provide associated Web interface as well. It will be called "serwebmanager" and it can be placed on the same server as "sermanager" or remote dedicated Web server. It will speak with "sermanager" with SOAP and will generate viewable Web content using XSLT. It could be used by both single account users and system administrators. One advantage of such approach (apart for real simplicity of "sermanager" PHP scripts) is that the content may be quickly adopted to different classes of end devices such as: HTML browsers, WAP browsers or simplified XML browsers. Imagine the possibilities of having a SIP videophone with simplified HTML 4.0 Web browser build in and being able to login to your personal SER profile Web page and manage your calls, diverts and voicemail messages.

3) The "SIP-SOAP gateway" functionality. Ideology behind that proposal was to add new option to SER routing, to ask external application about what to do with incoming SIP request. So how to route it, in example: a) discard b) rewrite URI c) forward as is It is basically the same idea as in SIP CGI RFC3050, but done with SOAP instead of CGI interface.

I have chosen Parlay X Web Services 2.0, because they provide a complete model for Web service definition and message exchanges. We just need to create one or two functions in SER routing to be able to ask external Web service. And to perform some action upon receiving an answer.

Here, the concern may be a possible decrease of SER perfomance when connecting with SOAP or parsing XML. But I propose to use gSOAP library and as seen on its Web page http://gsoap2.sourceforge.net/ in Perfomance area, the well written gSOAP application is capable of handling approx.3000 roundtrip message exchanges per second. My rough estimate is that single system would not be saturated for system with up to 50k-60k subscribers. Of course provided that processing time of the actual application behind gSOAP is relatively small.

To facilitate discussion I attach the drawing of target architecture. Please tell me what you think.

-- Regards, Arek Bekiersz

----- Original Message ----- From: "Jiri Kuthan" jiri@iptel.org To: sip@perceval.net; "Serusers" serusers@lists.iptel.org Sent: Sunday, February 26, 2006 7:47 PM Subject: Re: [Serusers] Proposal for new features

I would actually like one module, which would act as a single LDAP driver

for all

DB-based modules, as opposed to rewrite every single module outthere using

LDAP.

May be a bit tricky with the LDAP hierarchical module, but still doable.

jiri

-- Jiri Kuthan http://iptel.org/~jiri/

Arek Bekiersz wrote:

Hi,

As my current work may be interesting for the rest, I wonder if you will find those new features useful. Every opinion is appreciated.

Some time ago I commited a small 'ldap' module, all-in-one. As there was some (small :-) ) feedback from people using this module, I propose to split it into following modules:

1. auth_ldap: set of authentication routines, working with standard or

user-defined Ldap schemas

2. group_ldap: group membership module based on Ldap backend

3. uri_ldap: uri mangling module, based on ldap backend

2-3 can be actually merged into 1). Latest SER code retrieves additional information during authentication and stores it in attributes, which are then accessible in the configuration script. So all you would need to do is to retrieve a set of name-value pairs during authentication and all operations like group or uri checking can be done in the script.

This has the advantage of being much more efficient, because you can get all the info about the caller from LDAP in a single LDAP query. Furthemore it is also easier to implement. RADIUS authentication, for example, works exactly this way.

4. avp_ldap: (new) module for operating on AV-pairs from Ldap, working

with both user-defined or standard dbase schemas

What is the difference between standard and user-defined dbase schemas ? I am interested in having good LDAP support, but before implementing anything I think we should try to find out whether we could perhaps live with single ldap module that would implement the database API, as mentioned earlier by Jiri. In the script you would do then:

modparam("auth_db", "db_url", "ldap://..")

And then you can use standard auth_db and other modules. Could you perhaps try to describe the structure/schema of the LDAP database, so that we can see whether something like this would be possible ?

Later, it is possible to write small SER 'soap' modules, using one of freely available C/C++ soap libraries. Those modules will perform various tasks:

5. soap_ipdr: module for reporting service usage information (billing)

in XML, to be used by external billing systems. Information will include all call attempts (also failed ones like CC,CAD,UCN,UCI and possibly CID completion codes from IPDR's VoIP service definition). It will return following XML documents (using SOAP as transport):

• IPDRDoc - usage for specific user (user@domain)
• IPDRSettlementDoc - usage for domain

NOTE: that using IPDRSettlementDoc for domain is based on model where our cooperants have one virtual SIP domain. it is then easiest way to make settlement using IPDRSettlementDoc for one domain.

I guess this would be best handled using a standalone daemon. There is a special module called flatstore which can be used to implement simple accounting spool. All accounting requests are stored on local filesystem in form of plain text files. You could then write a daemon which would take the files periodically and send them over IPDR/XML/whatever.

This way you can still keep SER fast, if there is something wrong with the SOAP/IPDR service then SER will just continue writing accounting requests to flatstore and the external daemon will catch up later once the SOAP/IPDR service recovers.

6. soap_parlayx: Such module would act as a gateway between SER and

external Parlay X Web Services software. When called, this module will analyze SIP request and forward it to external Parlay X software. That software will fulfill request and return ReturnValue, with specific Action. Then SER soap_parlayx module will perform desired action on SIP request.

I guess this would be similar to OSP (Open Settlement Protocol) module. Take a look at http://cvs.berlios.de/cgi-bin/viewcvs.cgi/ser/sip_router/modules/osp/

Jan.

More on Parlay X Web services v.2.0 can be found at: http://www.parlay.org/en/specifications/

NOTE: This behaviour mimics a hypothetic SIP-CGI gateway, but this time it is a SIP-SOAP gateway.

EXAMPLE: could be the HandleBusy operation from "Part 3: Call Notification" of Parlay X Web services:

1. SIP encounters "486 Busy" situation
2. SER Script writer calls parlayx_handle(HandleBusy) function from

soap_parlayx module 3. soap_parlayx module calls external software, sending handleBusyRequest, containing CallingParty, CalledParty 4. external software returns handleBusyResponse with Action structure, containing ActionToPerform (like 'route' on not route), RoutingAddress, Charging (charge for using of service) 5. SER soap_parlayx module modifies SIP request and forwards it to voicemail_SIP location. In addition it stores relevant info in IPDR data recorder for later use.

External Parlay X software will be basically a SOAP engine, it can be written in any language. Unlike a soap_parlayx module itself, that engine will not be a part of SER.

7. SIP SOAP initiative

Seems that IP world is slowly evolving in direction of service enabled architecture, usually based on SOAP/UDDI standards. For this reason maybe it is worth to initiate work on SIP SOAP draft standard?

It would basically mimic existing IETF SIP CGI draft standard, but with far greater opportunities for creating new services, that SOAP standard brings. It will beautifuly fulfil current work in progress in my company and an European project, codename VISP (Virtual Internet Service Provider), that we are coordinator of. Project is focusing on enabling smooth company-to-company interoperability in mutual creation of new services in fast changing IP world. Including VoIP. More information can be found on Web site: http://www.visp-project.org/

Jan Janak wrote:

...

1. auth_ldap
2. group_ldap
3. uri_ldap

2-3 can be actually merged into 1). Latest SER code retrieves additional information during authentication and stores it in attributes, which are then accessible in the configuration script. So all you would need to do is to retrieve a set of name-value pairs during authentication and all operations like group or uri checking can be done in the script.

This has the advantage of being much more efficient, because you can get all the info about the caller from LDAP in a single LDAP query. Furthemore it is also easier to implement. RADIUS authentication, for example, works exactly this way.

Yes. Sounds OK for me. If user wants to actually modify values of some attributes during request handling in ser script, he could use the 4 module, 'avp_ldap'.

What is the difference between standard and user-defined dbase schemas ? I am interested in having good LDAP support, but before implementing anything I think we should try to find out whether we could perhaps live with single ldap module that would implement the database API, as mentioned earlier by Jiri. In the script you would do then:

modparam("auth_db", "db_url", "ldap://..")

I mentioned that user should be allowed to provide the names of objectClasses or attributes that he uses in his database to ldap backend . Those names should be passed as module parameters in ser config script. I did it more or less this way in my ldap module.

So when user has some custom database schemas (that is: he defined his own objectClasses or attributes), he could use them with this module as well.

And then you can use standard auth_db and other modules. Could you perhaps try to describe the structure/schema of the LDAP database, so that we can see whether something like this would be possible ?

I tried in previous email. Bsically as you see this mail, I added objectClass to hold SIP user data. It is auxiliary object class, anc dontains elements like SIP URI (userinfo@domain), SIP password, givenName, surname, mail, service allow flag, SIP group, default Country Code prefix, default Local Area Code, preferredLanguage, list of SIP aliases (like real E.164 numbers), short textual description. It is more or less what people are putting in flat databases plus some goodies.

I guess this would be best handled using a standalone daemon. There is a special module called flatstore which can be used to implement simple accounting spool. All accounting requests are stored on local filesystem in form of plain text files. You could then write a daemon which would take the files periodically and send them over IPDR/XML/whatever.

This way you can still keep SER fast, if there is something wrong with the SOAP/IPDR service then SER will just continue writing accounting requests to flatstore and the external daemon will catch up later once the SOAP/IPDR service recovers.

Actually I use SER's native acc database. I then run external application to collect all necessary information and present it later in IPDR format. For me, having an external application to do it is OK. I described a concept of external "sermanager" the other day. I think such 'sermanager' could be a good place to do billing & reporting stuff.

I guess this would be similar to OSP (Open Settlement Protocol) module. Take a look at http://cvs.berlios.de/cgi-bin/viewcvs.cgi/ser/sip_router/modules/osp/

Yes, I know 'osp' module a little. Well, the proposed 'soap_parlayx' module could be similar, except is actually uses external Web services, not external OSP server as 'osp' module.

Idea was to provide yet another routing alternative to people, willing to write Parlay X compliant external routing engines and then just calling them from inside of config script, to tewll SER what to do. In simpliest form it can just rewrite Request-URI and retirn soem value (success,failure, no-op).