On Thu, Dec 31, 2009 at 1:02 PM, alex pappas <rebel.pappas@gmail.com> wrote:

Hi,

>From the syslog when kamailio start ups I get the following:

Dec 31 12:35:45 YGOR /usr/local/sbin/kamailio[10281]: DBG:core:destroy_modules: shutting down module htable
Dec 31 12:35:46 YGOR kamailio: DBG:core:yyparse: loading module /usr/local/lib64/kamailio/modules/htable.so
Dec 31 12:35:46 YGOR kamailio: DBG:core:register_module: register_pv: htable
Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: htable matches module htable
Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: found <htable> in module htable [/usr/local/lib64/kamailio/modules/]
Dec 31 12:35:46 YGOR kamailio: DBG:htable:ht_table_spec: htable [a] - size [14]
Dec 31 12:35:46 YGOR kamailio: DBG:htable:ht_table_spec: htable [a] - dbtable [htable]
Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: htable matches module htable
Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: found <db_url> in module htable [/usr/local/lib64/kamailio/modules/]
Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: htable matches module htable
Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: found <fetch_rows> in module htable [/usr/local/lib64/kamailio/modules/]
Dec 31 12:35:46 YGOR kamailio: DBG:core:find_cmd_export_t: found <sht_print>(0) in module htable [/usr/local/lib64/kamailio/modules/]
Dec 31 12:35:52 YGOR /usr/local/sbin/kamailio[10371]: INFO:core:init_mod: initializing module htable
Dec 31 12:35:52 YGOR /usr/local/sbin/kamailio[10371]: DBG:htable:ht_db_open_con: database connection opened successfully
Dec 31 12:35:52 YGOR /usr/local/sbin/kamailio[10371]: DBG:htable:ht_db_load_tables: loading db table [htable] in ht [a]
Dec 31 12:35:52 YGOR /usr/local/sbin/kamailio[10371]: DBG:htable:ht_db_load_table: =============== loading hash table [a] from database [htable]
Dec 31 12:35:58 YGOR /usr/local/sbin/kamailio[10371]: DBG:htable:ht_db_load_table: Nothing to be loaded in hash table
Dec 31 12:35:58 YGOR /usr/local/sbin/kamailio[10371]: DBG:core:init_mod: register MI for htable
Dec 31 12:35:58 YGOR /usr/local/sbin/kamailio[10371]: DBG:htable:pv_parse_ht_name: htable [a] - key [$rU]
Dec 31 12:35:58 YGOR /usr/local/sbin/kamailio[10371]: DBG:htable:ht_get_table: htable found [a]

In the mysql htable I see :

mysql> select count(*) from openser10.htable;
+----------+
| count(*) |
+----------+
| 1893255 |
+----------+
1 row in set (0.00 sec)

And ofcourse when I run the sht_print(); I get many errors since htable in memory is empty:

Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1376] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1377] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1378] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1379] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1380] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1381] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1382] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1383] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1384] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1385] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1386] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1387] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1388] -- <0>
Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: htable[1389] -- <0>

Do you have any idea on why htable module does not load the data in the memory ?

Cheers
Alex
On Wed, Dec 30, 2009 at 6:01 PM, Daniel-Constantin Mierla <miconda@gmail.com> wrote:
Hello,

On 12/30/09 4:53 PM, alex pappas wrote:

Hi,

I don't see where the mysql is stacking but if I read correctly in http://kamailio.org/docs/modules/1.5.x/htable.html

The number of entries in the table is 2^size : 2^14=16384 rows data and i have 2.000.000 rows so maybe it is not possible to load them..But you know better of course.

no, that is the number of slots (buckets) in the hash table, not the number of items that can be loaded -- some more technical aspects of hash tables:

http://en.wikipedia.org/wiki/Hash_table

The number of items in htable is only a matter of shared memory. The number of slots controls access speed -- higher number gives less collisions.

You said at a moment that you could start, but htable was returning only null: put a sht_print() in config to see the content of the htable (you probably need high debug level: debug=5) with the first sip request.

Cheers,
Daniel
On Wed, Dec 30, 2009 at 5:32 PM, Daniel-Constantin Mierla <miconda@gmail.com> wrote:

Hello,

On 12/30/09 4:05 PM, alex pappas wrote:

Hi,

The error I get is the following:

ERROR:db_mysql:db_mysql_submit_query: driver error on query: Lost connection to MySQL server during query
ERROR:core:db_do_query: error while submitting query
ERROR:htable:ht_db_load_table: Error while querying db
ERROR:core:init_mod: failed to initialize module htable
ERROR:core:main: error while initializing modules

I'm running kamailio with 8M private memory per process and 2048M shared memory.

When kamailio is getting the data from mysql is passing it all the data chunk in private memory and then in the shared memory?

It is loaded in chunks of rows -- you can tune it with a module parameter:
http://kamailio.org/docs/modules/3.0.x/modules_k/htable.html#id2529075

Try to play with it to see if you get better results. You do not need more private memory, 8MB is ok.

Maybe the mysql server is closing due to lot of loaded data. Have you got any figure what is the number of rows when it starts losing the connection?

Cheers,
Daniel

because if this is the case then I need more than 1000M in private memory.

Cheers
Alex

P.S. i don't get any errors from mysql.

--
Daniel-Constantin Mierla
* http://www.asipto.com/
-- 
Daniel-Constantin Mierla
* http://www.asipto.com/
      