[Kamailio-Devel] [Serdev] AVPs

[Jan Janak]

On 12-12 00:25, Daniel-Constantin Mierla wrote:
> Hello,
>
> I think we need to decide about AVPs. Although not deeply investigated,  
> ser seems to have couple of AVPs lists. In kamailio/openser it is one 
> list.
>
> Can a ser developer describe a bit the AVP implementation as it is now  
> there?

  A long time ago we started using AVPs to store all kinds of configuration
  information. We needed some sort of general configuration mechanism because
  real-world scripts were getting too complex and at that time the AVPs were
  readily available.

  We started by loading attribute names and value from a database table into
  AVPs. This mechanism was/is same as in k. We would always load a set of
  attributes this way for a particular user (identified by username and domain
  from his/her SIP URI). With this mechanism in place, we were able to load a
  generic set of configuration options for a particular user of the server, I
  guess you have the same.

  To make the AVPs easily accessible from the configuration script, we added
  support for AVP identifiers in the config script, so that you can access AVP
  with name foo as $foo.

  As we migrated more and more configuration options to AVPs, it became clear
  that having just one set of AVPs was not enough. Although we could load all
  the config options for the caller (From URI) from the database into a set of
  AVPs, we could not do the same for the callee (Request-URI) at the same
  time. We could not do it because they would conflict with the AVPs of the 
  caller as both users could have AVPs with same names but different values.

  To get around this issue we added another list of AVPs. The new AVP list
  works the same way, it can contain AVPs with same names as AVPs in the
  original list and they do not conflict. All functions that work with lists
  of AVPs now take the list to work on as parameter.

  To make both AVP lists available in the configuration script, we extended
  the syntax of AVPs identifiers so the script write can choose the AVP list
  to work with. SO instead of just $foo you can write either

  $f.foo or $t.foo

  $f refers to the original AVP list which is commonly associated with the
  caller. The address/uid of the caller is taken from From header, hence the
  'f' in the identifier. $t refers to the AVP list which contains
  configuration settings of the callee. The address/uid of the calle can be
  taken either from the Request-URI or To headers, hence the 't' in the
  identifier.

  The original syntax without any list identifier is still available, in other
  words you can still write $foo, this is defined as a shorthand for
  $f.foo. If you do not specify the AVP list to be used then you are referring
  to the list containing AVPs of the caller (From).

  It also turned out that in some cases we would be having too many attributes
  in the database table storing AVPs. This can happen in bigger setups, having
  tens or hundreds of thousands users or serving multiple domains. This can
  slow the database down and makes SER load too much data. Based on my
  observations it is common that a large number of users have AVPs with same
  values. If ten thousands of users have attribute named 'foo' with value
  'bar' then the attribute will be stored in the database ten thousand times.

  As a remedy for this problem, we introduced the concept of AVP levels. The
  AVPs described so far and stored in user_attrs table are called user-level
  attributes and they store configuration specific to particular users. Then
  we added another two AVP lists to store so called domain-level AVPs.

  Domain-level AVPs are used to store configuration information that is shared
  by a group of users. Domain-level AVPs are stored in a separate database
  table, the name of the table is domain_attrs and its contents is cached in
  memory by 'domain' SER module. This is an important difference, while
  user-level AVPs need to be loaded every time a SIP request is processed,
  domain-level AVPs are only loaded when the contents of domain_attrs table
  has changed.

  The domain-level AVPs are called 'domain' because they are tied to a
  particular domain handled by the SIP server. That could be 'iptel.org',
  'sip-router.org', and so on. This mainly useful for multi-domain
  setups. There are two domain-level AVP lists because it is common that SER
  works with two domains at the same time, the domain from From header
  (caller's domain) and the domain from Request-URI or To (callee's domain).

  Again, we extended the syntax of AVP identifiers in the configuration
  file. So you can write:

  $fu.foo -- this way you are asking for the value of the user-level foo AVP.
  $fd.foo -- This will return the value of the domain-level foo AVP.

  And similarly there is $tu and $td for callee's user-level and domain-level
  AVPs. If you specify 'u' in the AVP identifiers then SER searches only the
  list of user-level attributes. If you specify 'd' then SER searches only the
  list of domain-level attributes.

  This behavior changes if you do NOT specify the level to be searched. In
  that case SER searches the user-level list first and if no match is found
  then the domain-level list will be searched. Thus if you write:

  $f.foo

  then you are telling SER to search for the value of 'foo' AVP in the
  user-level list and continue with the domain-level list if no match is
  found. In other words, user-level AVPs have higher priority than
  domain-level AVPs. With this system you can efficiently set a configuration
  option for all users within a domain by creating a domain-level attribute
  and yet you can override that option with user-level attributes for
  particular users within that domain.

  Then there are the global AVPs. Global AVPs can be used to store settings
  applicable to the whole SIP server and all domains on that server. Global
  AVPs are stored in global_attrs table and the contents of that table is
  cached in memory by gflags module. There is only one list of global
  AVPs. Global AVPs can be accessed with:

  $g.foo

  (note that there is no 'f' or 't' in the identifier). The list of global
  AVPs is searched after the list of domain-level AVPs. So if you write:

  $f.foo

  Then SER searches user-level AVPs first, then domain-level and then the
  global AVPs.

  And finally there are also so called URI-level AVPs. They work just
  like user-level AVPs, but a single user can have multiple sets of
  uri-level AVPs. They are tied to SIP URIs of that user. Uri-level AVPs are
  denoted by 'r' in the AVP identifier, for example:

  $fr.foo

> In openser/kamailio we have more or less same architecture as for ser  
> 0.9.6 with couple extensions, but I will detail here so we can have full  
> picture:
> - avps are kept in shared memory

  In our case global and domain level AVPs are stored in shared memory.

> - they are bound to each sip message and moved to transaction if that  
> message create a transaction

  This is same.

> - avp can have two types of names
>    - integer id, referred as $avp(i:number)
>    - string id, referred as $avp(s:string)
> - there can be aliases for avp names, defined as "alias=[is]:id", so the  
> script writer can use $avp(alias)

  We've kind of dropped support for integer avps, we've been using string
  names pretty much everywhere, although the original code still exists.

> - a value of an avp can be either integer or string
> - not related to avp core part, but important to mention here -- tm,  
> controlled by a parameter, can make the avps from transaction (request)  
> available to onreply_route.

  In SER tm restores all AVPs stored in a transaction in all route blocks
  called from tm, such as onreply_route, failure_route. At least I hope I
  remember it correctly.

    Jan.