[Kamailio-Devel] [SR-Dev] [Serdev] AVPs

[Daniel-Constantin Mierla]

On 12/16/08 10:15, Andrei Pelinescu-Onciul wrote:
> On Dec 16, 2008 at 00:36, Daniel-Constantin Mierla <miconda at gmail.com> wrote:
>   
>> Hello,
>>
>> On 12/12/08 17:22, Jan Janak wrote:
>>     
>>> On 12-12 00:25, Daniel-Constantin Mierla wrote:
>>>   
>>>       
>>>> Hello,
>>>>
>>>> I think we need to decide about AVPs. Although not deeply investigated,  
>>>> ser seems to have couple of AVPs lists. In kamailio/openser it is one 
>>>> list.
>>>>
>>>> Can a ser developer describe a bit the AVP implementation as it is now  
>>>> there?
>>>>     
>>>>         
>>>   A long time ago we started using AVPs to store all kinds of configuration
>>>   information. We needed some sort of general configuration mechanism because
>>>   real-world scripts were getting too complex and at that time the AVPs were
>>>   readily available.
>>>
>>>   We started by loading attribute names and value from a database table into
>>>   AVPs. This mechanism was/is same as in k. We would always load a set of
>>>   attributes this way for a particular user (identified by username and domain
>>>   from his/her SIP URI). With this mechanism in place, we were able to load a
>>>   generic set of configuration options for a particular user of the server, I
>>>   guess you have the same.
>>>
>>>   To make the AVPs easily accessible from the configuration script, we added
>>>   support for AVP identifiers in the config script, so that you can access AVP
>>>   with name foo as $foo.
>>>
>>>   As we migrated more and more configuration options to AVPs, it became clear
>>>   that having just one set of AVPs was not enough. Although we could load all
>>>   the config options for the caller (From URI) from the database into a set of
>>>   AVPs, we could not do the same for the callee (Request-URI) at the same
>>>   time. We could not do it because they would conflict with the AVPs of the 
>>>   caller as both users could have AVPs with same names but different values.
>>>
>>>   To get around this issue we added another list of AVPs. The new AVP list
>>>   works the same way, it can contain AVPs with same names as AVPs in the
>>>   original list and they do not conflict. All functions that work with lists
>>>   of AVPs now take the list to work on as parameter.
>>>
>>>   To make both AVP lists available in the configuration script, we extended
>>>   the syntax of AVPs identifiers so the script write can choose the AVP list
>>>   to work with. SO instead of just $foo you can write either
>>>
>>>   $f.foo or $t.foo
>>>
>>>   $f refers to the original AVP list which is commonly associated with the
>>>   caller. The address/uid of the caller is taken from From header, hence the
>>>   'f' in the identifier. $t refers to the AVP list which contains
>>>   configuration settings of the callee. The address/uid of the calle can be
>>>   taken either from the Request-URI or To headers, hence the 't' in the
>>>   identifier.
>>>
>>>   The original syntax without any list identifier is still available, in other
>>>   words you can still write $foo, this is defined as a shorthand for
>>>   $f.foo. If you do not specify the AVP list to be used then you are referring
>>>   to the list containing AVPs of the caller (From).
>>>
>>>   It also turned out that in some cases we would be having too many attributes
>>>   in the database table storing AVPs. This can happen in bigger setups, having
>>>   tens or hundreds of thousands users or serving multiple domains. This can
>>>   slow the database down and makes SER load too much data. Based on my
>>>   observations it is common that a large number of users have AVPs with same
>>>   values. If ten thousands of users have attribute named 'foo' with value
>>>   'bar' then the attribute will be stored in the database ten thousand times.
>>>
>>>   As a remedy for this problem, we introduced the concept of AVP levels. The
>>>   AVPs described so far and stored in user_attrs table are called user-level
>>>   attributes and they store configuration specific to particular users. Then
>>>   we added another two AVP lists to store so called domain-level AVPs.
>>>
>>>   Domain-level AVPs are used to store configuration information that is shared
>>>   by a group of users. Domain-level AVPs are stored in a separate database
>>>   table, the name of the table is domain_attrs and its contents is cached in
>>>   memory by 'domain' SER module. This is an important difference, while
>>>   user-level AVPs need to be loaded every time a SIP request is processed,
>>>   domain-level AVPs are only loaded when the contents of domain_attrs table
>>>   has changed.
>>>
>>>   The domain-level AVPs are called 'domain' because they are tied to a
>>>   particular domain handled by the SIP server. That could be 'iptel.org',
>>>   'sip-router.org', and so on. This mainly useful for multi-domain
>>>   setups. There are two domain-level AVP lists because it is common that SER
>>>   works with two domains at the same time, the domain from From header
>>>   (caller's domain) and the domain from Request-URI or To (callee's domain).
>>>
>>>   Again, we extended the syntax of AVP identifiers in the configuration
>>>   file. So you can write:
>>>
>>>   $fu.foo -- this way you are asking for the value of the user-level foo AVP.
>>>   $fd.foo -- This will return the value of the domain-level foo AVP.
>>>
>>>   And similarly there is $tu and $td for callee's user-level and domain-level
>>>   AVPs. If you specify 'u' in the AVP identifiers then SER searches only the
>>>   list of user-level attributes. If you specify 'd' then SER searches only the
>>>   list of domain-level attributes.
>>>
>>>   This behavior changes if you do NOT specify the level to be searched. In
>>>   that case SER searches the user-level list first and if no match is found
>>>   then the domain-level list will be searched. Thus if you write:
>>>
>>>   $f.foo
>>>
>>>   then you are telling SER to search for the value of 'foo' AVP in the
>>>   user-level list and continue with the domain-level list if no match is
>>>   found. In other words, user-level AVPs have higher priority than
>>>   domain-level AVPs. With this system you can efficiently set a configuration
>>>   option for all users within a domain by creating a domain-level attribute
>>>   and yet you can override that option with user-level attributes for
>>>   particular users within that domain.
>>>
>>>   Then there are the global AVPs. Global AVPs can be used to store settings
>>>   applicable to the whole SIP server and all domains on that server. Global
>>>   AVPs are stored in global_attrs table and the contents of that table is
>>>   cached in memory by gflags module. There is only one list of global
>>>   AVPs. Global AVPs can be accessed with:
>>>
>>>   $g.foo
>>>
>>>   (note that there is no 'f' or 't' in the identifier). The list of global
>>>   AVPs is searched after the list of domain-level AVPs. So if you write:
>>>
>>>   $f.foo
>>>
>>>   Then SER searches user-level AVPs first, then domain-level and then the
>>>   global AVPs.
>>>
>>>   And finally there are also so called URI-level AVPs. They work just
>>>   like user-level AVPs, but a single user can have multiple sets of
>>>   uri-level AVPs. They are tied to SIP URIs of that user. Uri-level AVPs are
>>>   denoted by 'r' in the AVP identifier, for example:
>>>
>>>   $fr.foo
>>>   
>>>       
>> so, if I understood correctly, there are couple of combinations between 
>> the dot, with one or two letters, right?
>>
>> Like:
>>
>> $xy.name
>>
>> where x = f|t|g and y may be missing or u|d|r
>>     
>
> x = f|t ("track" in ser avp terminology
> y= r|u|d|g ("class")
>
> g is a special case (it cannot be combined with f or t).
>
>
> Here is the parse_avp_ident() usage comment:
>
> /** parse an avp indentifier.
>  *
>  * Parses the following avp indentifier forms:
>  *       - "i:<number>"  - old form, deprecated  (e.g. i:42)
>  *       - "s:<string>"  - old form, deprecated  (e.g. s:foo)
>  *       - "<track>.<name>"                      (e.g.: f.bar)
>  *       - "<track>.<name>[<index>]"             (e.g.: f.bar[1])
>  *       - "<track><class>.<name>"               (e.g:  tu.bar)
>  *       - "<track><class>.<name>[<index>]"      (e.g:  fd.bar[2])
>  *       - "<string>"                            (e.g.: foo)
>  * Where:
>  *          <string> = ascii string
>  *          <id>   = ascii string w/o '[', ']', '.' and '/'
>  *          <name> = <id> | '/' regex '/'
>  *                   (Note: regex use is deprecated)
>  *          <track> = 'f' | 't'
>  *                   (from or to)
>  *          <class> = 'r' | 'u' | 'd' | 'g'
>  *                    (uri, user, domain or global)
>  *          <index> = <number> | '-' <number> | ''
>  *                    (the avp index, if missing it means AVP_INDEX_ALL, but
>  *                     it's use is deprecated)
>  * More examples:
>  *       "fr.bar[1]"  - from track, uri class, avp "bar", the value 1.
>  *       "tu./^foo/"  - to track,  user class, all avps for which the name
>  *                      starts with foo (note RE in avp names are deprecated).
>  *        "t.did"     - to track, "did" avp
>  *
>  * @param name  - avp identifier
>  * @param *attr - the result will be stored here
>  * @return 0 on success, -1 on error
>  */
>
>   
>> Then we can get rid of overlapping in the namespace if we enforce usage 
>> of $f.foo for $foo . In K there are couple of PV that use dot in 
>> classname (a PV can be $classname or $classname(innername)), but they 
>> are proposed to be removed.
>>     
>
> For the time being in sip-router this is handled in the following way:
>
> $(...) -> pvar
> $foo(...) -> pvar
> $foo[...] -> avp
> $bar.foo -> avp (if it has a dot in it -> avp)
> $foo  (no dot, [] or ()) -> try first to resolve it as pvar and if it
> fails consider it an avp.
>
> For the current version I think this is the most compatible way.
yes, indeed, thanks!

>  I'll
> use the same approach for ser module fixups (so all ser module that use 
>  core fixup functions will be able to use also pvars).
> One could also force  kamailio or ser behaviour, by starting the
> script with #!KAMAILIO or #!SER.
>
> For future versions we should rethink how everything is accessed.
> For example I would use $foo for script variables and some other form
> for avps or the current pvar $foo. Also we would have to unify selects
> with the select like part of the pvars.
> (Note for ser users: pvars in kamailio are used to access script vars,
> avps, message headers and something which is kind of similar to ser
> selects.).
>   
Perhaps select implementations can be moved in modules, as well. 
Integration has to be done at some point anyhow.

Cheers,
Daniel

-- 
Daniel-Constantin Mierla
http://www.asipto.com