Thanks for a thorough write-up!  I'm "afraid" you are not very atypical when it comes to mixing your own data model into SER's.  In general, I miss an "official" opinion on how to extend the data model, which could help in migration. I know many people (including myself) use RADIUS to split the authentication, groups, and AVPs away from the internal run-time data of SER (primarily usrloc). This way the SER db can as a worst case be totally discarded on an upgrade (loosing current registrations). You also offload the DB lookups to another server. I have also written up a suggestion for a patch that would avoid user authentication against the RADIUS servers as long as a reREGISTER is just a refresh of the registration.
http://www.iptel.org/refresh_of_reregister_instead_of_new_authentication_in_certain_secure_situations

For some reason or another, many (most?) people opt for extending the SER database, which makes for painful migrations. How to extend and not extend would be a very useful guide, if it could help people avoid some migration headaches.

See inline for specific comments.

sip wrote:
Sure. I'll share what I can share without giving too much of the business away.

For the most part, we use SER's modules from the 0.9.x set. However, we do use the alias_db module from OpenSER as it's FAR easier to securely code a web-based system that only accesses the database than it is to have to rely on ANY access to the command line (using SER, because it places its aliases into memory, it would require us to add them via serctl or it wouldn't pick them up (or manually add them with fifo commands ourselves, which is out of the question. First rule of web security: thou shalt avoid ever writing to the filesystem from the web)). The alias_db module allows us to just add aliases during runtime to the DB. We take a tiny performance hit from that, but considering the other DB calls that have to be made for checks and what not for every incoming call, an extra call to check the alias is really insignificant.
Yes, makes sense.  As Jiri pointed out, this is improved a lot in Ottendorf. Also, with the new management interface, I assume the XMLRPC front-end would be your choice?

One of our programmers also backported the AVP functionality from the 0.1.x OpenSER uac module. We use this to modify the SIP comment portion of the From: header (which is allowed under the RFC) dynamically to ensure no spoofing.
;-) http://www.iptel.org/ser/contributions
makes it easy to submit a module or patch for various versions of SER. It is meant as a library of modules and patches that people can go to to extend functionality regardless of the version of SER they are using.

So if you consider "giving it back", feel free! :-)

As for why we modified the db model... well... it just wasn't right for our setup. We don't user SERWeb for anything. I'm sure it's a lovely product for basic administration, but we have our own site code, and we use a strict OO model for coding, so the use of any SERWeb code was out of the question.

We modified speeddial for our own purposes, so the speed_dial table had to be modified. We use the subscriber table for the website, which started off as a way to unify information, but then we switched to radius authentication and use the subscriber table strictly for the web (and for referencing other tables). Of course, that meant we needed to add things to it such as a signup IP address to keep track of user signups for statistics, we modified some of the fields here and there for our own use. We also added a lot of tables we need for other things, such as a call blocking table and a call return table.
:-) Yes, one thing leads to another...  What kind of resource would have helped you before you started?  Any ideas?

We went down the road of dynamic timers, so we needed a timer table for the users' timers. We completely redid the phonebook information, so we keep user phonebooks in a rather complex scheme.

Of course, we have our pricing tables and what not.

And that's just the SER side. It's constantly trading information back and forth with Asterisk, which has its own tables to handle our own billing code and proper billing for call forwards and such, as well as information about the expirations for users' DIDs and Voicemail.

:-) I see. You're starting to get some dependencies, yes.

We also have SEMS in the mix for basic voicemail and conference rooms, but we made some modifications here and there to handle our userbase.

We use SER primarily for SIP, but we've tied it in very closely with our web applications, so we've tied the DB all together. I'm not sure we're a normal case in that sense, but who's to say. We may well be.
As I said, my feeling is that you're the normal case.

And you, Greger, I KNOW, would absolutely CRINGE at our ser.cfg.  It's a massive, 40k file of madness. Amusingly, however, it's pretty well optimised.  It just has to take a LOT of different situations into account.
:-D You will probably like the new m4 build system I'm creating then... It splits up ser.cfg into logical pieces that can be maintained separately.
g-)

N.


On Mon, 04 Dec 2006 21:45:35 +0100, Greger V. Teigre wrote

> I'm trying to figure out how people use SER in order to get a picture of what the needs really are. My goal is to be able to group various usage patterns and create "best practice" areas at iptel.org, where people also can add comments and describe their experiences.
>
> Would you mind sharing a bit about the "mixture of SER and OpenSER modules" you are using, as well as why you opted to change the database model? 
> g-)
>
> sip wrote:
Indeed. No fears there. We've already got a dedicated server on order to test
it with. Won't likely be here until the beginning of the year, but we won't
have time to muck with it before then anyway with the usual end-of-year
madness going on. 

Very much looking forward to testing Ottendorf and trying to figure out the
best way to migrate. As is, it will be slightly complex as we've spent a good
portion of time redoing the SER database to our particular specifications and
needs... and many of the tables we use have been removed or redone for
Ottendorf, and many of the features we rely upon have been completely done
away with.  

Also, since we use a mixture of SER and OpenSER modules in our code, as well
as in-house modules to handle several things, migrating to Ottendorf is going
to be an exercise in frustration and pain. :) I'm not eager to start even
looking at that until after the holidays. 

N.

On Mon, 04 Dec 2006 10:39:25 +0100, Jiri Kuthan wrote
  
Just to make it clear, it is pretty much only possible with 
Ottendorf if you are having a reasonably big setup. The previous 
timer system was not accurate (you might have noticed, that 
retransmission timer fired in a range of +/- 1 sec which is kind of 
unexact given the timer length 0.5 sec), variable (all timers were 
pretty much supposed to be of the same length), high-resolution (it 
used to be 1 second).

I concur that migration is alway a painful exercise, but I can only recommend
folks to make themselves busy with it earlier than too late.

-jiri

At 23:05 01/12/2006, sip wrote:
    
Well... the TM bug may not be earth-shattering, but it's most decidedly
important. One of the functions we have that our users LOVE is the ability to
set their own timeout values for voicemail. We had had a default value that we
thought would make everyone happy, but soon discovered that many would
complain that it was too long, and many would complain that it was too short.

Having had many users who came along from asterisk-based services, where
setting individual timeout values is rather straightforward and easy.

One of the things we've been struggling with has been the timing values. While
it may not be important in the scheme that it's not ACTUALLY
service-affecting, I can assure you, if you offer a service, and it doesn't
work as prescribed, the consequences can be deadly. You shake user-confidence
-- and that sort of thing travels VERY quickly by word of mouth. It's so very
hard to gain users, and so very easy to lose them to a competitor -- even if
that competitor doesn't offer the same options. For what good are options that
only work sometimes?

SO... in short... yeah... I'm really happy that the timer stuff has been fixed
in Ottendorf. Of course, it'll be a LONG time before we can migrate the
production systems to it, but it gives me something to look forward to.

N.

On Thu, 30 Nov 2006 12:19:03 +0100, Andrei Pelinescu-Onciul wrote
      
On Nov 29, 2006 at 14:05, Kim Il <kim_il_s@yahoo.com> wrote:
        
this is getting more and more confusing. The one side says this is a bug,
          
the other says it is a feature. I will try to summarize my understanding
      
till now:
  
I believe we  better stop this line of discussion at least for now 
and instead concentrate on fixing the technical problems that were revealed
on both sides. I wouldn't want it to hinder possible future
collaboration.

        
*  openser claims that they improved something and the ser guys say this
          
fix is useless. Daniel, Bogdan could you plesase  explain what is really
behind this improvement (Sorry Weiter, but your explanation is far from being
sufficient).
      
I don't know exactly what are you referring to ("something"?), but I 
don't remember claiming that an openser fix was useless (at least I 
haven't, I might have implied that something, IMHO was not as 
important as claimed, but not that was useless).

        
* The ser guys show that openser is inferior to ser but Bogdan replies
          
this is on purpose (because otherwise a strange race situation can occur which
Andrei tells us is negligable).
      
Actually, the claim was along the lines that tm (the statefull part 
of the sip stack in ser) and/or the core performance was inferior.

The bug that Bogdan discovered is not negligible. I wanted to point out
that it was not as important as suggested.
This probably deserves a better explanation, so that people without
ser-internals knowledge can understand.
First of all variable timers in this context mean the possibility to
set different values for  the fr_timer and fr_inv_timer at runtime

(e.g. in function of some user preferences/avps or some special 
script logic) as oppossed to having some fixed (but configurable) values.
What really matters here is the fr_inv_timer. The fr_inv_timer is the
time that ser waits for a final response on an INVITE transaction
_after_ a provisional response was received (e.g. 180 Ringing). This 
 roughly corresponds to the Timer C from the sip rfc (rfc3261).
 According to the sip rfc both fr_tiemr and fr_inv_timer should be fixed
and _not_ configurable. However in practice it makes sense to change 
them
(especially the fr_inv_timer), to implement services such as 
redirect to voicemail after x seconds. The most common use of the 
variable timers is user (or group) configurable voicemail 
redirection. This means that the user is somehow able to configure 
the "ringing" interval after which a caller will be redirected to 
his voicemail. If variable timers are not used, the voicemail config 
option would include only on or off (the ringing time before 
voicemail redirection wouldn't be configurable, it would be the same 
for all the users). Now what the bug did, was that it could cause 
the extension of fr_inv_timer to the fr_inv_timer of another 
transaction, if _different_ variable timers were used and some race 
occured (note that the probability of hitting the race is quite high 
if lots of variable fr_inv_timers with different values are used and 
the system is very busy). This IMHO is not such a big problem since 
in the case when you are hit by it, it  really doesn't break 
anything important (I don't see sometimes redirecting later to 
voicemail as such a big problem, OTOH people might use it for other 
stuff).

        
* I will not get into the history discussion -which I have already touched
          
in previous mails, let alone those mails about respect and the other stuff I
could not really see how they relate to this discussion.
      
I would really appreciate if these points were clarified -I have the
          
impression the ball is now with Daniel and Bogdan. So plesse Bogdan do not
hold to your one mail policy and help clarifying this and provide a bit more
of details on the improvements of openser to core and tm.
      
I believe Andrei has made a good suggestion that it is in the interest of
          
the entire community to cooperate and divide the areas of interest. By
combining the great work the openser guys have done in the area of
documentation and featutres with the work of ser in the area of core we will
all get an even better ser/openser.
      
In fact even if we can't reach an agreement in dividing areas, small
cooperation like on fixing/improving code that is the same in both
versions would still be good.
I would say that even a friendly competing approach would be better then
nothing (with comparisons, benchmarks a.s.o), it points out problems 
in both versions and serves as a motivation to improve.

Andrei
[...]
_______________________________________________
Serusers mailing list
Serusers@lists.iptel.org
http://lists.iptel.org/mailman/listinfo/serusers
        
_______________________________________________
Serusers mailing list
Serusers@lists.iptel.org
http://lists.iptel.org/mailman/listinfo/serusers
      
--
Jiri Kuthan            http://iptel.org/~jiri/
    
_______________________________________________
Serusers mailing list
Serusers@lists.iptel.org
http://lists.iptel.org/mailman/listinfo/serusers