[Kamailio-Devel] [SR-Dev] [Serdev] AVPs

[Jan Janak]

On 16-12 10:52, Daniel-Constantin Mierla wrote:
> On 12/16/08 10:15, Andrei Pelinescu-Onciul wrote:
>> On Dec 16, 2008 at 00:36, Daniel-Constantin Mierla <miconda at gmail.com> wrote:
>>   
>>> Hello,
>>>
>>> On 12/12/08 17:22, Jan Janak wrote:
>>>     
>>>> On 12-12 00:25, Daniel-Constantin Mierla wrote:
>>>>         
>>>>> Hello,
>>>>>
>>>>> I think we need to decide about AVPs. Although not deeply 
>>>>> investigated,  ser seems to have couple of AVPs lists. In 
>>>>> kamailio/openser it is one list.
>>>>>
>>>>> Can a ser developer describe a bit the AVP implementation as it 
>>>>> is now  there?
>>>>>             
>>>>   A long time ago we started using AVPs to store all kinds of configuration
>>>>   information. We needed some sort of general configuration mechanism because
>>>>   real-world scripts were getting too complex and at that time the AVPs were
>>>>   readily available.
>>>>
>>>>   We started by loading attribute names and value from a database table into
>>>>   AVPs. This mechanism was/is same as in k. We would always load a set of
>>>>   attributes this way for a particular user (identified by username and domain
>>>>   from his/her SIP URI). With this mechanism in place, we were able to load a
>>>>   generic set of configuration options for a particular user of the server, I
>>>>   guess you have the same.
>>>>
>>>>   To make the AVPs easily accessible from the configuration script, we added
>>>>   support for AVP identifiers in the config script, so that you can access AVP
>>>>   with name foo as $foo.
>>>>
>>>>   As we migrated more and more configuration options to AVPs, it became clear
>>>>   that having just one set of AVPs was not enough. Although we could load all
>>>>   the config options for the caller (From URI) from the database into a set of
>>>>   AVPs, we could not do the same for the callee (Request-URI) at the same
>>>>   time. We could not do it because they would conflict with the 
>>>> AVPs of the   caller as both users could have AVPs with same names 
>>>> but different values.
>>>>
>>>>   To get around this issue we added another list of AVPs. The new AVP list
>>>>   works the same way, it can contain AVPs with same names as AVPs in the
>>>>   original list and they do not conflict. All functions that work with lists
>>>>   of AVPs now take the list to work on as parameter.
>>>>
>>>>   To make both AVP lists available in the configuration script, we extended
>>>>   the syntax of AVPs identifiers so the script write can choose the AVP list
>>>>   to work with. SO instead of just $foo you can write either
>>>>
>>>>   $f.foo or $t.foo
>>>>
>>>>   $f refers to the original AVP list which is commonly associated with the
>>>>   caller. The address/uid of the caller is taken from From header, hence the
>>>>   'f' in the identifier. $t refers to the AVP list which contains
>>>>   configuration settings of the callee. The address/uid of the calle can be
>>>>   taken either from the Request-URI or To headers, hence the 't' in the
>>>>   identifier.
>>>>
>>>>   The original syntax without any list identifier is still available, in other
>>>>   words you can still write $foo, this is defined as a shorthand for
>>>>   $f.foo. If you do not specify the AVP list to be used then you are referring
>>>>   to the list containing AVPs of the caller (From).
>>>>
>>>>   It also turned out that in some cases we would be having too many attributes
>>>>   in the database table storing AVPs. This can happen in bigger setups, having
>>>>   tens or hundreds of thousands users or serving multiple domains. This can
>>>>   slow the database down and makes SER load too much data. Based on my
>>>>   observations it is common that a large number of users have AVPs with same
>>>>   values. If ten thousands of users have attribute named 'foo' with value
>>>>   'bar' then the attribute will be stored in the database ten thousand times.
>>>>
>>>>   As a remedy for this problem, we introduced the concept of AVP levels. The
>>>>   AVPs described so far and stored in user_attrs table are called user-level
>>>>   attributes and they store configuration specific to particular users. Then
>>>>   we added another two AVP lists to store so called domain-level AVPs.
>>>>
>>>>   Domain-level AVPs are used to store configuration information that is shared
>>>>   by a group of users. Domain-level AVPs are stored in a separate database
>>>>   table, the name of the table is domain_attrs and its contents is cached in
>>>>   memory by 'domain' SER module. This is an important difference, while
>>>>   user-level AVPs need to be loaded every time a SIP request is processed,
>>>>   domain-level AVPs are only loaded when the contents of domain_attrs table
>>>>   has changed.
>>>>
>>>>   The domain-level AVPs are called 'domain' because they are tied to a
>>>>   particular domain handled by the SIP server. That could be 'iptel.org',
>>>>   'sip-router.org', and so on. This mainly useful for multi-domain
>>>>   setups. There are two domain-level AVP lists because it is common that SER
>>>>   works with two domains at the same time, the domain from From header
>>>>   (caller's domain) and the domain from Request-URI or To (callee's domain).
>>>>
>>>>   Again, we extended the syntax of AVP identifiers in the configuration
>>>>   file. So you can write:
>>>>
>>>>   $fu.foo -- this way you are asking for the value of the user-level foo AVP.
>>>>   $fd.foo -- This will return the value of the domain-level foo AVP.
>>>>
>>>>   And similarly there is $tu and $td for callee's user-level and domain-level
>>>>   AVPs. If you specify 'u' in the AVP identifiers then SER searches only the
>>>>   list of user-level attributes. If you specify 'd' then SER searches only the
>>>>   list of domain-level attributes.
>>>>
>>>>   This behavior changes if you do NOT specify the level to be searched. In
>>>>   that case SER searches the user-level list first and if no match is found
>>>>   then the domain-level list will be searched. Thus if you write:
>>>>
>>>>   $f.foo
>>>>
>>>>   then you are telling SER to search for the value of 'foo' AVP in the
>>>>   user-level list and continue with the domain-level list if no match is
>>>>   found. In other words, user-level AVPs have higher priority than
>>>>   domain-level AVPs. With this system you can efficiently set a configuration
>>>>   option for all users within a domain by creating a domain-level attribute
>>>>   and yet you can override that option with user-level attributes for
>>>>   particular users within that domain.
>>>>
>>>>   Then there are the global AVPs. Global AVPs can be used to store settings
>>>>   applicable to the whole SIP server and all domains on that server. Global
>>>>   AVPs are stored in global_attrs table and the contents of that table is
>>>>   cached in memory by gflags module. There is only one list of global
>>>>   AVPs. Global AVPs can be accessed with:
>>>>
>>>>   $g.foo
>>>>
>>>>   (note that there is no 'f' or 't' in the identifier). The list of global
>>>>   AVPs is searched after the list of domain-level AVPs. So if you write:
>>>>
>>>>   $f.foo
>>>>
>>>>   Then SER searches user-level AVPs first, then domain-level and then the
>>>>   global AVPs.
>>>>
>>>>   And finally there are also so called URI-level AVPs. They work just
>>>>   like user-level AVPs, but a single user can have multiple sets of
>>>>   uri-level AVPs. They are tied to SIP URIs of that user. Uri-level AVPs are
>>>>   denoted by 'r' in the AVP identifier, for example:
>>>>
>>>>   $fr.foo
>>>>         
>>> so, if I understood correctly, there are couple of combinations 
>>> between the dot, with one or two letters, right?
>>>
>>> Like:
>>>
>>> $xy.name
>>>
>>> where x = f|t|g and y may be missing or u|d|r
>>>     
>>
>> x = f|t ("track" in ser avp terminology
>> y= r|u|d|g ("class")
>>
>> g is a special case (it cannot be combined with f or t).
>>
>>
>> Here is the parse_avp_ident() usage comment:
>>
>> /** parse an avp indentifier.
>>  *
>>  * Parses the following avp indentifier forms:
>>  *       - "i:<number>"  - old form, deprecated  (e.g. i:42)
>>  *       - "s:<string>"  - old form, deprecated  (e.g. s:foo)
>>  *       - "<track>.<name>"                      (e.g.: f.bar)
>>  *       - "<track>.<name>[<index>]"             (e.g.: f.bar[1])
>>  *       - "<track><class>.<name>"               (e.g:  tu.bar)
>>  *       - "<track><class>.<name>[<index>]"      (e.g:  fd.bar[2])
>>  *       - "<string>"                            (e.g.: foo)
>>  * Where:
>>  *          <string> = ascii string
>>  *          <id>   = ascii string w/o '[', ']', '.' and '/'
>>  *          <name> = <id> | '/' regex '/'
>>  *                   (Note: regex use is deprecated)
>>  *          <track> = 'f' | 't'
>>  *                   (from or to)
>>  *          <class> = 'r' | 'u' | 'd' | 'g'
>>  *                    (uri, user, domain or global)
>>  *          <index> = <number> | '-' <number> | ''
>>  *                    (the avp index, if missing it means AVP_INDEX_ALL, but
>>  *                     it's use is deprecated)
>>  * More examples:
>>  *       "fr.bar[1]"  - from track, uri class, avp "bar", the value 1.
>>  *       "tu./^foo/"  - to track,  user class, all avps for which the name
>>  *                      starts with foo (note RE in avp names are deprecated).
>>  *        "t.did"     - to track, "did" avp
>>  *
>>  * @param name  - avp identifier
>>  * @param *attr - the result will be stored here
>>  * @return 0 on success, -1 on error
>>  */
>>
>>   
>>> Then we can get rid of overlapping in the namespace if we enforce 
>>> usage of $f.foo for $foo . In K there are couple of PV that use dot 
>>> in classname (a PV can be $classname or $classname(innername)), but 
>>> they are proposed to be removed.
>>>     
>>
>> For the time being in sip-router this is handled in the following way:
>>
>> $(...) -> pvar
>> $foo(...) -> pvar
>> $foo[...] -> avp
>> $bar.foo -> avp (if it has a dot in it -> avp)
>> $foo  (no dot, [] or ()) -> try first to resolve it as pvar and if it
>> fails consider it an avp.
>>
>> For the current version I think this is the most compatible way.
> yes, indeed, thanks!
>
>>  I'll
>> use the same approach for ser module fixups (so all ser module that use 
>>  core fixup functions will be able to use also pvars).
>> One could also force  kamailio or ser behaviour, by starting the
>> script with #!KAMAILIO or #!SER.
>>
>> For future versions we should rethink how everything is accessed.
>> For example I would use $foo for script variables and some other form
>> for avps or the current pvar $foo. Also we would have to unify selects
>> with the select like part of the pvars.
>> (Note for ser users: pvars in kamailio are used to access script vars,
>> avps, message headers and something which is kind of similar to ser
>> selects.).
>>   
> Perhaps select implementations can be moved in modules, as well.  
> Integration has to be done at some point anyhow.

  It already is, the core contains the core selects and shared parts.

    Jan.