[Kamailio-Users] avpops OR sqlops

Daniel-Constantin Mierla miconda at gmail.com
Wed Dec 30 15:21:11 CET 2009


Hello,

On 12/29/09 3:20 PM, alex pappas wrote:
> Dear friends,
>
> Happy holidays :-)
>
> I'm trying to add the portability database in an htable hash table. 
> When I have 200.000 rows I can add these rows to the  hash table in 
> memory but when they exid that number then i usually get the error :
>
> ERROR:db_mysql:db_mysql_submit_query: driver error on query: Lost 
> connection to MySQL server

mysql module has option to auto-reconnect which is on by default. If the 
connection is lost then something happens with the mysql server, do you 
get something in the syslog for mysql daemon?

>
> From what i found till now the htable is siting in shared 
> memory.(corect me if I'm wrong). I'm starting kamailio with private 
> memory 256 and shared memory 2048.
>
> I also did and some fine tuning in mysql server(local):
>
> query_cache_type = 1
> query_cache_limit = 1024M
> query_cache_size = 1024M
> key_buffer = 64M
> table_cache  = 1024
> sort_buffer=4M
> read_buffer_size=1M
> max_allowed_packet      = 1536M
> thread_stack            = 128K
> thread_cache_size       = 8
> And i cached the table by running to the server the command SELECT 
> SQL_CACHE  * from htable;
>
> After this fine tuning I don't get any error when I run kamailio BUT 
> when I try to receive any vaule from the htable I get NULL back.(this 
> configuration work's when i have a few values only in the htable)
>
> Now the question is does private memory is involve in this? Is an 
> memory issue or something else?

Private memory is involved only when data is loaded from mysql, after 
that is moved in shared memory.

There is an MI command that can dump the content of a hash table (you 
can use kamctl for that) to check if there are some values in it.

Since you have a lot of records, make sure the size of htable is big (14 
is max and recommended).

Cheers,
Daniel


>
> Thank you for your help
> Best wishes
>
> Peace!
> Alex
>
>
>
>
> On Thu, Dec 24, 2009 at 7:31 PM, Daniel-Constantin Mierla 
> <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>
>     Hi Alex,
>
>
>     On 12/24/09 6:18 PM, alex pappas wrote:
>>     Daniel,
>>
>>     The reason that I was NOT understanding how to implement the
>>     htable is that even I read the all internet ;-) I did not see the
>>     already setup table Htable in my mysql server which was already
>>     made from the installation and  that's why the value type  and
>>     key types did not make sense!! now i fill a bit stupid :-(
>>
>>     I apologize for stilling you time.
>     no need for that! If I have no time I do not answer. Sometime even
>     obvious things are not seen, happens to me, this is life ...
>
>     Merry Christmas!
>     Daniel
>
>     PS. or do you celebrate Christmas in January, using the old
>     calendar date?
>
>
>>
>>     Cheers
>>     Alex
>>
>>
>>
>>     On Thu, Dec 24, 2009 at 4:36 PM, alex pappas
>>     <rebel.pappas at gmail.com <mailto:rebel.pappas at gmail.com>> wrote:
>>
>>         Thank you Daniel,
>>
>>         Cheers
>>         Alex
>>
>>
>>         On Thu, Dec 24, 2009 at 4:05 PM, Daniel-Constantin Mierla
>>         <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>>
>>             Hi Alex,
>>
>>             the default table structure for htable is defined by sql:
>>
>>             CREATE TABLE htable (
>>                 id INT(10) UNSIGNED AUTO_INCREMENT PRIMARY KEY NOT NULL,
>>                 key_name VARCHAR(64) DEFAULT '' NOT NULL,
>>                 key_type INT DEFAULT 0 NOT NULL,
>>                 value_type INT DEFAULT 0 NOT NULL,
>>                 key_value VARCHAR(128) DEFAULT '' NOT NULL
>>             ) ENGINE=MyISAM;
>>
>>             If you map over another table, then you must have at
>>             least four columns there (id is for generic purpose). See
>>             here descriptions of the columns:
>>             http://kamailio.org/docs/modules/3.0.x/modules_k/htable.html#id2529061
>>
>>             Cheers,
>>             Daniel
>>
>>
>>             On 12/23/09 6:13 PM, alex pappas wrote:
>>>             Hi Daniel,
>>>
>>>             My problem is how to define the hash table in
>>>             Kamailio.cfg. Especially I don't understand how to define :
>>>
>>>
>>>                   |key_name_column: here I understand that is the
>>>                   name of the key = fist col (phonenumber)
>>>                   |
>>>
>>>
>>>                   |key_type_column == string ?
>>>                   |
>>>
>>>
>>>                   |value_type_column == string ?
>>>                   |
>>>
>>>
>>>                   |key_value_column == second col(prefix)?
>>>                   |
>>>
>>>
>>>
>>>             I get the following errors:
>>>
>>>             ERROR:db_mysql:db_mysql_submit_query: driver error on
>>>             query: Unknown column 'key_type' in 'field list'
>>>             ERROR:core:db_do_query: error while submitting query
>>>             ERROR:htable:ht_db_load_table: Error while querying db
>>>             ERROR:core:init_mod: failed to initialize module htable
>>>             ERROR:core:main: error while initializing modules
>>>
>>>             Thank you
>>>
>>>             Alex
>>>
>>>
>>>
>>>             On Wed, Dec 23, 2009 at 6:43 PM, Daniel-Constantin
>>>             Mierla <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>>>
>>>                 Hello,
>>>
>>>
>>>                 On 12/23/09 4:28 PM, alex pappas wrote:
>>>>                 Hi all,
>>>>
>>>>                 It is possible to have some help in defining the
>>>>                 hash table?
>>>>
>>>>                 I'm giving some parameters but some of them i don't
>>>>                 really understand them. my configuration is:
>>>>
>>>>                 # --------- Htable ---------
>>>>                 modparam("htable", "htable", "a=>dbtable=ported1;")
>>>>                 modparam("htable", "db_url",
>>>>                 "mysql://pskoul:pskoul123@localhost/openser")
>>>>                 modparam("htable", "key_name_column", "phonenumber")
>>>>                 modparam("htable", "key_value", "prefix")
>>>>                 modparam("htable", "fetch_rows", 1000)
>>>>
>>>>                 The table I want to htable is a 2 cols table
>>>>                 (phonenumber,prefix) In mysql they are defined as
>>>>                 varchar(10). All the phonenumber values are unique
>>>>                 so the key in htable is phonenumber with value in
>>>>                 prefix.
>>>
>>>                 what don't you understand? I do not know what to
>>>                 explain. Is there any error you get? At least the
>>>                 parameter key_value is wrong, should be
>>>                 key_value_column.
>>>
>>>                 Cheers,
>>>                 Daniel
>>>
>>>
>>>>
>>>>                 Thanks All,
>>>>
>>>>                 Alex
>>>>
>>>>
>>>>
>>>>                 On Wed, Dec 23, 2009 at 1:04 PM, alex pappas
>>>>                 <rebel.pappas at gmail.com
>>>>                 <mailto:rebel.pappas at gmail.com>> wrote:
>>>>
>>>>                     Thank you Daniel,
>>>>
>>>>                     I will try first oracle db and then htable
>>>>                     which is also very interesting. When I'm done I
>>>>                     will get back with the results.
>>>>
>>>>                     Cheers
>>>>                     Alex
>>>>
>>>>
>>>>
>>>>
>>>>                     On Wed, Dec 23, 2009 at 11:44 AM,
>>>>                     Daniel-Constantin Mierla <miconda at gmail.com
>>>>                     <mailto:miconda at gmail.com>> wrote:
>>>>
>>>>                         Hello,
>>>>
>>>>
>>>>                         On 12/22/09 1:25 PM, alex pappas wrote:
>>>>>                         Hi all,
>>>>>
>>>>>                         Concerning portability again :-)
>>>>>
>>>>>                         Is there any way I can add in the memory
>>>>>                         all the numbers with portability?
>>>>>                         (2.000.000 rows)?
>>>>>                         And also reload these data a few times a
>>>>>                         day and access this data from the 
>>>>>                         Kamailio script?
>>>>
>>>>                         while I advise usage of sqlops/sql_query()
>>>>                         instead of avpops/avp_db_query(), if you
>>>>                         want to load such data in memory should be
>>>>                         in shared memory. For that look at htable
>>>>                         or maybe you can re-use pdt.
>>>>
>>>>                         Cheers,
>>>>                         Daniel
>>>>
>>>>>
>>>>>                          Cheers
>>>>>                         Alex
>>>>>
>>>>>
>>>>>
>>>>>                         On Tue, Dec 22, 2009 at 12:56 AM, Henning
>>>>>                         Westerholt <henning.westerholt at 1und1.de
>>>>>                         <mailto:henning.westerholt at 1und1.de>> wrote:
>>>>>
>>>>>                             On Mon, December 21, 2009 9:50 pm,
>>>>>                             Alex Balashov wrote:
>>>>>                             > [..]
>>>>>                             > Having said that, AVPs are still
>>>>>                             very useful and necessary because they
>>>>>                             > persist across the lifetime of a
>>>>>                             transaction, so you can access them in
>>>>>                             > subsequent reply and failure routes.
>>>>>                              This is not true of user
>>>>>                             > pseudovariables ($var(...)), nor, as
>>>>>                             far as I know, of any other class
>>>>>                             > of pseudovariable at this point
>>>>>                             except special transaction-persistent
>>>>>                             > ones exported by modules.
>>>>>                             >
>>>>>                             > As a result, it is often necessary
>>>>>                             for me to take data from $dbr
>>>>>                             > (sqlops) and load it into arrays of
>>>>>                             AVPs, such as for example when doing
>>>>>                             > custom fail-over in failure_route
>>>>>                             that uses logic for which dispatcher
>>>>>                             > is not suitable.  I store a set of
>>>>>                             gateways in an AVP array along with
>>>>>                             > another AVP value indicating the
>>>>>                             current array index and iterate over
>>>>>                             > them in failure_route, and so on.
>>>>>                             >
>>>>>                             > Daniel taught me how to use AVP
>>>>>                             arrays once, although I noticed this is
>>>>>                             > not mentioned anywhere in the avpops
>>>>>                             documentation.  It should be
>>>>>                             > published somewhere, as it is one of
>>>>>                             the most useful programmatic
>>>>>                             > features of Kamailio.
>>>>>                             >
>>>>>                             > Henning/Daniel, does Kamailio 3/SR
>>>>>                             offer any kind of better substitute
>>>>>                             > for what I am doing with AVPs as
>>>>>                             described above?  In other words, are
>>>>>                             > there any transaction-persistent
>>>>>                             pseudovariables and/or namespace
>>>>>                             > containers that are easier to deal
>>>>>                             with syntactically, and provide more
>>>>>                             > advanced data structure primitives
>>>>>                             like arrays?  I am aware of htable,
>>>>>                             > but I need something local to a
>>>>>                             transaction, not global.
>>>>>
>>>>>                             There are the $var PVs you already
>>>>>                             mentioned, and then also the $shv that
>>>>>                             are shared variables. But they are
>>>>>                             (process) global, and not
>>>>>                             transactional. Daniel proposed and
>>>>>                             developed the new xavp type for sr
>>>>>                             which are basically extended AVPs.
>>>>>                             More informations can be found here:
>>>>>                             http://sip-router.org/wiki/devel/xavp
>>>>>
>>>>>                             Regards,
>>>>>
>>>>>                             Henning
>>>>>
>>>>>
>>>>>
>>>>>                         _______________________________________________
>>>>>                         Kamailio (OpenSER) - Users mailing list
>>>>>                         Users at lists.kamailio.org
>>>>>                         <mailto:Users at lists.kamailio.org>
>>>>>                         http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
>>>>>                         http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
>>>>
>>>>                         -- 
>>>>                         Daniel-Constantin Mierla
>>>>                         *http://www.asipto.com/
>>>>                                
>>>>
>>>>
>>>>
>>>>
>>>>                 _______________________________________________
>>>>                 Kamailio (OpenSER) - Users mailing list
>>>>                 Users at lists.kamailio.org  <mailto:Users at lists.kamailio.org>
>>>>                 http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
>>>>                 http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
>>>
>>>                 -- 
>>>                 Daniel-Constantin Mierla
>>>                 *http://www.asipto.com/
>>>                      
>>>
>>>
>>>
>>>             _______________________________________________
>>>             Kamailio (OpenSER) - Users mailing list
>>>             Users at lists.kamailio.org  <mailto:Users at lists.kamailio.org>
>>>             http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
>>>             http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
>>
>>             -- 
>>             Daniel-Constantin Mierla
>>             *http://www.asipto.com/
>>                    
>>
>>
>>
>
>     -- 
>     Daniel-Constantin Mierla
>     *http://www.asipto.com/
>          
>
>
>
> _______________________________________________
> Kamailio (OpenSER) - Users mailing list
> Users at lists.kamailio.org
> http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
> http://lists.openser-project.org/cgi-bin/mailman/listinfo/users

-- 
Daniel-Constantin Mierla
* http://www.asipto.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/users/attachments/20091230/a1401913/attachment-0001.htm>


More information about the Users mailing list