Hi Alex, I tried : if(!is_avp_set("$avp(s:lnp_prefix)")){ xlog("alx ------- The new rU $rU with prefix $avp(s:lnp_prefix) return code:$retcode ----------"); } and it looks like is doing to right thing BUT why I don't see the avp value? I need this value to added in the front of rU as portability prefix.. Any comments about the sqlops ? Thank Alex On Mon, Dec 21, 2009 at 6:37 PM, Alex Balashov <abalashov(a)evaristesys.com>wrote;wrote:

On 12/21/2009 02:51 PM, alex pappas wrote: Ah, yes, I have commonly run into this one as well. Yes, the database interface layers generally support only canonical scalar types - basic integer and varchar, as well as types that offer ready-made convertabiliy in a textual evaluation context (e.g. PostgreSQL boolean type, 't' or 'f'). No bigints unless you cast down to int. :) -- Alex Balashov - Principal Evariste Systems Web : http://www.evaristesys.com/ Tel : (+1) (678) 954-0670 Direct : (+1) (678) 954-0671

Alex, On 12/21/2009 03:00 PM, alex pappas wrote: I don't think it's really a question of speed as of flexibility, ease of programming, and conceptual propriety. The avpops module is a slightly antiquated facility, and its present-day use does not very closely reflect its original purpose as a database-backed "key-value" store of some sort. I get the impression that avp_db_query() was added to it as sort of a minor afterthought; I think that the assumption that persisted in the peak of avpops' reign was that the utility of *SER for most people was bound up in its modules (with their related database schemas). What happened afterward was that there was an explosion of usage that revolved around custom database integration and it became a more accepted fact of Kamailio's existence, and that is why the sqlops module was written to make it easier. The sqlops module operates in a manner that much more closely resembles the database APIs of other programmatic environments, e.g. the ability to determine number of rows returned, the ability to iterate through results without having to utilise crude and implicit AVP arrays, etc. I imagine that avp_db_query() and sql_query() are implemented very similarly underneath, and in any case, both utilise the same database interface layers (db_postgres, db_mysql, etc.), which is where the data interchange implementation that has the most relevance to speed actually resides. In short, I don't think sqlops is materially faster, but it is a more modern and sophisticated way to talk to your database and you should use it for that reason. -- Alex Balashov - Principal Evariste Systems Web : http://www.evaristesys.com/ Tel : (+1) (678) 954-0670 Direct : (+1) (678) 954-0671

On 12/21/2009 03:42 PM, Henning Westerholt wrote: Indeed. Having said that, AVPs are still very useful and necessary because they persist across the lifetime of a transaction, so you can access them in subsequent reply and failure routes. This is not true of user pseudovariables ($var(...)), nor, as far as I know, of any other class of pseudovariable at this point except special transaction-persistent ones exported by modules. As a result, it is often necessary for me to take data from $dbr (sqlops) and load it into arrays of AVPs, such as for example when doing custom fail-over in failure_route that uses logic for which dispatcher is not suitable. I store a set of gateways in an AVP array along with another AVP value indicating the current array index and iterate over them in failure_route, and so on. Daniel taught me how to use AVP arrays once, although I noticed this is not mentioned anywhere in the avpops documentation. It should be published somewhere, as it is one of the most useful programmatic features of Kamailio. Henning/Daniel, does Kamailio 3/SR offer any kind of better substitute for what I am doing with AVPs as described above? In other words, are there any transaction-persistent pseudovariables and/or namespace containers that are easier to deal with syntactically, and provide more advanced data structure primitives like arrays? I am aware of htable, but I need something local to a transaction, not global. -- Alex Balashov - Principal Evariste Systems Web : http://www.evaristesys.com/ Tel : (+1) (678) 954-0670 Direct : (+1) (678) 954-0671

Hi all, Concerning portability again :-) Is there any way I can add in the memory all the numbers with portability? (2.000.000 rows)? And also reload these data a few times a day and access this data from the Kamailio script? Cheers Alex On Tue, Dec 22, 2009 at 12:56 AM, Henning Westerholt < henning.westerholt(a)1und1.de&gt; wrote:

Hi, I use kamailio 1.5 and there is no such a module in that version Cheers Alex 2009/12/22 Raúl Alexis Betancor Santana &lt;rabs(a)dimension-virtual.com&gt;

On Tue, December 22, 2009 2:41 pm, Raúl Alexis Betancor Santana wrote: Indeed the module should work more or less unchanged in kamailio 1.5. The only thing that probably make problems are some of the sr specific defines, but you could probably just remove them. But if you use "only" 2.000.000 rows you should be able to use just a normal database, if the machine has enough RAM it should be also more then fast enough. The pdb module works with a dedicated pdb server, which supports advanced optimizations and is especially optimized for the portability lookup task, but its of course a bit more difficult to setup. Regards, Henning

On Tue, December 22, 2009 3:33 pm, alex pappas wrote: stats. > (sipp test) Hey Alex, well, 30 CPS are not that much. This looks more like a configuration problem or a slow network connection to the DB server. You could elevate this by using a local read-only replication copy on the localhost, which is easy to setup with mysql. If you want to query directly the oracle DB you could use the db_unixodbc module via ODBC, or giving the db_oracle module a try. Regards, Henning

On Wed, December 23, 2009 9:52 am, alex pappas wrote: Hey Alex, ok, this is indeed a really bad disk. :) Regarding the slow logging, you could check the syslog configuration if its configured to write asynchronously (there should be a "-" in front of the log statement in syslog.conf). And also check if you perhaps not use a to high log level in the kamailio cfg. Regards, Henning

hi, is a blade hp server which is works with external disk(they are connected with optical line.) BUT because for the blade I'm working in there is no space in that disk so I'm working in local disk which is pata technology!!! very old and slow IO. Does anyone knows hot to define a htable in kamailio.cfg. I included details in my previous email Thanks Alex On Thu, Dec 24, 2009 at 2:02 PM, Alex Balashov &lt;abalashov(a)evaristesys.com&gt;wrote;wrote:

Thank you Daniel, I will try first oracle db and then htable which is also very interesting. When I'm done I will get back with the results. Cheers Alex On Wed, Dec 23, 2009 at 11:44 AM, Daniel-Constantin Mierla < miconda(a)gmail.com&gt; wrote:

Hello, On 12/23/09 4:28 PM, alex pappas wrote: what don't you understand? I do not know what to explain. Is there any error you get? At least the parameter key_value is wrong, should be key_value_column. Cheers, Daniel -- Daniel-Constantin Mierla * http://www.asipto.com/

Hi Alex, the default table structure for htable is defined by sql: CREATE TABLE htable ( id INT(10) UNSIGNED AUTO_INCREMENT PRIMARY KEY NOT NULL, key_name VARCHAR(64) DEFAULT '' NOT NULL, key_type INT DEFAULT 0 NOT NULL, value_type INT DEFAULT 0 NOT NULL, key_value VARCHAR(128) DEFAULT '' NOT NULL ) ENGINE=MyISAM; If you map over another table, then you must have at least four columns there (id is for generic purpose). See here descriptions of the columns: http://kamailio.org/docs/modules/3.0.x/modules_k/htable.html#id2529061 Cheers, Daniel On 12/23/09 6:13 PM, alex pappas wrote: -- Daniel-Constantin Mierla * http://www.asipto.com/

Daniel, The reason that I was NOT understanding how to implement the htable is that even I read the all internet ;-) I did not see the already setup table Htable in my mysql server which was already made from the installation and that's why the value type and key types did not make sense!! now i fill a bit stupid :-( I apologize for stilling you time. Cheers Alex On Thu, Dec 24, 2009 at 4:36 PM, alex pappas &lt;rebel.pappas(a)gmail.com&gt; wrote:

Dear friends, Happy holidays :-) I'm trying to add the portability database in an htable hash table. When I have 200.000 rows I can add these rows to the hash table in memory but when they exid that number then i usually get the error : ERROR:db_mysql:db_mysql_submit_query: driver error on query: Lost connection to MySQL server if I'm wrong). I'm starting kamailio with private memory 256 and shared memory 2048. I also did and some fine tuning in mysql server(local): query_cache_type = 1 query_cache_limit = 1024M query_cache_size = 1024M key_buffer = 64M table_cache = 1024 sort_buffer=4M read_buffer_size=1M max_allowed_packet = 1536M thread_stack = 128K thread_cache_size = 8 And i cached the table by running to the server the command SELECT SQL_CACHE * from htable; After this fine tuning I don't get any error when I run kamailio BUT when I try to receive any vaule from the htable I get NULL back.(this configuration work's when i have a few values only in the htable) Now the question is does private memory is involve in this? Is an memory issue or something else? Thank you for your help Best wishes Peace! Alex On Thu, Dec 24, 2009 at 7:31 PM, Daniel-Constantin Mierla &lt;miconda(a)gmail.com > Hi Alex, > > > On 12/24/09 6:18 PM, alex pappas wrote: > > Daniel, > > The reason that I was NOT understanding how to implement the htable is that > even I read the all internet ;-) I did not see the already setup table > Htable in my mysql server which was already made from the installation and > that's why the value type and key types did not make sense!! now i fill a > bit stupid :-( > > I apologize for stilling you time. > > no need for that! If I have no time I do not answer. Sometime even obvious > things are not seen, happens to me, this is life ... > > Merry Christmas! > Daniel > > PS. or do you celebrate Christmas in January, using the old calendar date? > > > > Cheers > Alex > > > > On Thu, Dec 24, 2009 at 4:36 PM, alex pappas &lt;rebel.pappas(a)gmail.com&gt;wrote;wrote: > >> Thank you Daniel, >> >> Cheers >> Alex >> >> >> On Thu, Dec 24, 2009 at 4:05 PM, Daniel-Constantin Mierla < >> miconda(a)gmail.com>> >>> Hi Alex, >>> >>> the default table structure for htable is defined by sql: >>> >>> CREATE TABLE htable ( >>> id INT(10) UNSIGNED AUTO_INCREMENT PRIMARY KEY NOT NULL, >>> key_name VARCHAR(64) DEFAULT '' NOT NULL, >>> key_type INT DEFAULT 0 NOT NULL, >>> value_type INT DEFAULT 0 NOT NULL, >>> key_value VARCHAR(128) DEFAULT '' NOT NULL >>> ) ENGINE=MyISAM; >>> >>> If you map over another table, then you must have at least four columns >>> there (id is for generic purpose). See here descriptions of the columns: >>> http://kamailio.org/docs/modules/3.0.x/modules_k/htable.html#id2529061 >>> >>> Cheers, >>> Daniel >>> >>> >>> On 12/23/09 6:13 PM, alex pappas wrote: >>> >>> Hi Daniel, >>> >>> My problem is how to define the hash table in Kamailio.cfg. Especially I >>> don't understand how to define : >>> key_name_column: here I understand that is the name of the key = fist col >>> (phonenumber) >>> key_type_column == string ? >>> value_type_column == string ? >>> key_value_column == second col(prefix)? >>> >>> >>> I get the following errors: >>> >>> ERROR:db_mysql:db_mysql_submit_query: driver error on query: Unknown >>> column 'key_type' in 'field list' >>> ERROR:core:db_do_query: error while submitting query >>> ERROR:htable:ht_db_load_table: Error while querying db >>> ERROR:core:init_mod: failed to initialize module htable >>> ERROR:core:main: error while initializing modules >>> >>> Thank you >>> >>> Alex >>> >>> >>> >>> On Wed, Dec 23, 2009 at 6:43 PM, Daniel-Constantin Mierla < >>> miconda(a)gmail.com>>> >>>> Hello, >>>> >>>> >>>> On 12/23/09 4:28 PM, alex pappas wrote: >>>> >>>> Hi all, >>>> >>>> It is possible to have some help in defining the hash table? >>>> >>>> I'm giving some parameters but some of them i don't really understand >>>> them. my configuration is: >>>> >>>> # --------- Htable --------- >>>> modparam("htable", "htable", "a=>dbtable=ported1;") >>>> modparam("htable", "db_url", "mysql://pskoul:pskoul123@localhost >>>> /openser") >>>> modparam("htable", "key_name_column", "phonenumber") >>>> modparam("htable", "key_value", "prefix") >>>> modparam("htable", "fetch_rows", 1000) >>>> >>>> The table I want to htable is a 2 cols table (phonenumber,prefix) In >>>> mysql they are defined as varchar(10). All the phonenumber values are unique >>>> so the key in htable is phonenumber with value in prefix. >>>> >>>> >>>> what don't you understand? I do not know what to explain. Is there any >>>> error you get? At least the parameter key_value is wrong, should be >>>> key_value_column. >>>> >>>> Cheers, >>>> Daniel >>>> >>>> >>>> >>>> Thanks All, >>>> >>>> Alex >>>> >>>> >>>> >>>> On Wed, Dec 23, 2009 at 1:04 PM, alex pappas &lt;rebel.pappas(a)gmail.com&gt;wrote;wrote: >>>> >>>>> Thank you Daniel, >>>>> >>>>> I will try first oracle db and then htable which is also very >>>>> interesting. When I'm done I will get back with the results. >>>>> >>>>> Cheers >>>>> Alex >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Dec 23, 2009 at 11:44 AM, Daniel-Constantin Mierla < >>>>> miconda(a)gmail.com>>>>> >>>>>> Hello, >>>>>> >>>>>> >>>>>> On 12/22/09 1:25 PM, alex pappas wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> Concerning portability again :-) >>>>>> >>>>>> Is there any way I can add in the memory all the numbers with >>>>>> portability? (2.000.000 rows)? >>>>>> And also reload these data a few times a day and access this data from >>>>>> the Kamailio script? >>>>>> >>>>>> >>>>>> while I advise usage of sqlops/sql_query() instead of >>>>>> avpops/avp_db_query(), if you want to load such data in memory should be in >>>>>> shared memory. For that look at htable or maybe you can re-use pdt. >>>>>> >>>>>> Cheers, >>>>>> Daniel >>>>>> >>>>>> >>>>>> Cheers >>>>>> Alex >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Dec 22, 2009 at 12:56 AM, Henning Westerholt < >>>>>> henning.westerholt(a)1und1.de>>>>>> >>>>>>> On Mon, December 21, 2009 9:50 pm, Alex Balashov wrote: >>>>>>> > [..] >>>>>>> > Having said that, AVPs are still very useful and necessary because >>>>>>> they >>>>>>> > persist across the lifetime of a transaction, so you can access >>>>>>> them in >>>>>>> > subsequent reply and failure routes. This is not true of user >>>>>>> > pseudovariables ($var(...)), nor, as far as I know, of any other >>>>>>> class >>>>>>> > of pseudovariable at this point except special >>>>>>> transaction-persistent >>>>>>> > ones exported by modules. >>>>>>> > >>>>>>> > As a result, it is often necessary for me to take data from $dbr >>>>>>> > (sqlops) and load it into arrays of AVPs, such as for example when >>>>>>> doing >>>>>>> > custom fail-over in failure_route that uses logic for which >>>>>>> dispatcher >>>>>>> > is not suitable. I store a set of gateways in an AVP array along >>>>>>> with >>>>>>> > another AVP value indicating the current array index and iterate >>>>>>> over >>>>>>> > them in failure_route, and so on. >>>>>>> > >>>>>>> > Daniel taught me how to use AVP arrays once, although I noticed >>>>>>> this is >>>>>>> > not mentioned anywhere in the avpops documentation. It should be >>>>>>> > published somewhere, as it is one of the most useful programmatic >>>>>>> > features of Kamailio. >>>>>>> > >>>>>>> > Henning/Daniel, does Kamailio 3/SR offer any kind of better >>>>>>> substitute >>>>>>> > for what I am doing with AVPs as described above? In other words, >>>>>>> are >>>>>>> > there any transaction-persistent pseudovariables and/or namespace >>>>>>> > containers that are easier to deal with syntactically, and provide >>>>>>> more >>>>>>> > advanced data structure primitives like arrays? I am aware of >>>>>>> htable, >>>>>>> > but I need something local to a transaction, not global. >>>>>>> >>>>>>> There are the $var PVs you already mentioned, and then also the $shv >>>>>>> that >>>>>>> are shared variables. But they are (process) global, and not >>>>>>> transactional. Daniel proposed and developed the new xavp type for sr >>>>>>> which are basically extended AVPs. More informations can be found >>>>>>> here: >>>>>>> http://sip-router.org/wiki/devel/xavp >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Henning >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Kamailio (OpenSER) - Users mailing list >>>>>> Users@lists.kamailio.orghttp://lists.kamailio.org/cgi-bin/mailman/listinfo/usershttp://lists.openser-project.org/cgi-bin/mailman/listinfo/users >>>>>> >>>>>> >>>>>> -- >>>>>> Daniel-Constantin Mierla >>>>>> * http://www.asipto.com/ >>>>>> >>>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Kamailio (OpenSER) - Users mailing listUsers@lists.kamailio.orghttp://lists.kamailio.org/cgi-bin/mailman/listinfo/usershttp://lists.openser-project.org/cgi-bin/mailman/listinfo/users >>>> >>>> >>>> -- >>>> Daniel-Constantin Mierla >>>> * http://www.asipto.com/ >>>> >>>> >>> >>> _______________________________________________ >>> Kamailio (OpenSER) - Users mailing listUsers@lists.kamailio.orghttp://lists.kamailio.org/cgi-bin/mailman/listinfo/usershttp://lists.openser-project.org/cgi-bin/mailman/listinfo/users >>> >>> >>> -- >>> Daniel-Constantin Mierla >>> * http://www.asipto.com/ >>> >>> >> > > -- > Daniel-Constantin Mierla > * http://www.asipto.com/ > >

Hi, The error I get is the following: ERROR:db_mysql:db_mysql_submit_query: driver error on query: Lost connection to MySQL server during query ERROR:core:db_do_query: error while submitting query ERROR:htable:ht_db_load_table: Error while querying db ERROR:core:init_mod: failed to initialize module htable ERROR:core:main: error while initializing modules I'm running kamailio with 8M private memory per process and 2048M shared memory. When kamailio is getting the data from mysql is passing it all the data chunk in private memory and then in the shared memory? because if this is the case then I need more than 1000M in private memory. Cheers Alex P.S. i don't get any errors from mysql. On Wed, Dec 30, 2009 at 4:21 PM, Daniel-Constantin Mierla &lt;miconda(a)gmail.com > Hello, > > > On 12/29/09 3:20 PM, alex pappas wrote: > > Dear friends, > > Happy holidays :-) > > I'm trying to add the portability database in an htable hash table. When I > have 200.000 rows I can add these rows to the hash table in memory but when > they exid that number then i usually get the error : > > ERROR:db_mysql:db_mysql_submit_query: driver error on query: Lost > connection to MySQL server > > > mysql module has option to auto-reconnect which is on by default. If the > connection is lost then something happens with the mysql server, do you get > something in the syslog for mysql daemon? > > > > From what i found till now the htable is siting in shared memory.(corect me > if I'm wrong). I'm starting kamailio with private memory 256 and shared > memory 2048. > > I also did and some fine tuning in mysql server(local): > > query_cache_type = 1 > query_cache_limit = 1024M > query_cache_size = 1024M > key_buffer = 64M > table_cache = 1024 > sort_buffer=4M > read_buffer_size=1M > max_allowed_packet = 1536M > thread_stack = 128K > thread_cache_size = 8 > And i cached the table by running to the server the command SELECT > SQL_CACHE * from htable; > > After this fine tuning I don't get any error when I run kamailio BUT when I > try to receive any vaule from the htable I get NULL back.(this configuration > work's when i have a few values only in the htable) > > Now the question is does private memory is involve in this? Is an memory > issue or something else? > > > Private memory is involved only when data is loaded from mysql, after that > is moved in shared memory. > > There is an MI command that can dump the content of a hash table (you can > use kamctl for that) to check if there are some values in it. > > Since you have a lot of records, make sure the size of htable is big (14 is > max and recommended). > > Cheers, > Daniel > > > > > Thank you for your help > Best wishes > > Peace! > Alex > > > > > On Thu, Dec 24, 2009 at 7:31 PM, Daniel-Constantin Mierla < > miconda(a)gmail.com> >> Hi Alex, >> >> >> On 12/24/09 6:18 PM, alex pappas wrote: >> >> Daniel, >> >> The reason that I was NOT understanding how to implement the htable is >> that even I read the all internet ;-) I did not see the already setup table >> Htable in my mysql server which was already made from the installation and >> that's why the value type and key types did not make sense!! now i fill a >> bit stupid :-( >> >> I apologize for stilling you time. >> >> no need for that! If I have no time I do not answer. Sometime even >> obvious things are not seen, happens to me, this is life ... >> >> Merry Christmas! >> Daniel >> >> PS. or do you celebrate Christmas in January, using the old calendar date? >> >> >> >> >> Cheers >> Alex >> >> >> >> On Thu, Dec 24, 2009 at 4:36 PM, alex pappas &lt;rebel.pappas(a)gmail.com&gt;wrote;wrote: >> >>> Thank you Daniel, >>> >>> Cheers >>> Alex >>> >>> >>> On Thu, Dec 24, 2009 at 4:05 PM, Daniel-Constantin Mierla < >>> miconda(a)gmail.com>>> >>>> Hi Alex, >>>> >>>> the default table structure for htable is defined by sql: >>>> >>>> CREATE TABLE htable ( >>>> id INT(10) UNSIGNED AUTO_INCREMENT PRIMARY KEY NOT NULL, >>>> key_name VARCHAR(64) DEFAULT '' NOT NULL, >>>> key_type INT DEFAULT 0 NOT NULL, >>>> value_type INT DEFAULT 0 NOT NULL, >>>> key_value VARCHAR(128) DEFAULT '' NOT NULL >>>> ) ENGINE=MyISAM; >>>> >>>> If you map over another table, then you must have at least four columns >>>> there (id is for generic purpose). See here descriptions of the columns: >>>> http://kamailio.org/docs/modules/3.0.x/modules_k/htable.html#id2529061 >>>> >>>> Cheers, >>>> Daniel >>>> >>>> >>>> On 12/23/09 6:13 PM, alex pappas wrote: >>>> >>>> Hi Daniel, >>>> >>>> My problem is how to define the hash table in Kamailio.cfg. Especially I >>>> don't understand how to define : >>>> key_name_column: here I understand that is the name of the key = fist >>>> col (phonenumber) >>>> key_type_column == string ? >>>> value_type_column == string ? >>>> key_value_column == second col(prefix)? >>>> >>>> >>>> I get the following errors: >>>> >>>> ERROR:db_mysql:db_mysql_submit_query: driver error on query: Unknown >>>> column 'key_type' in 'field list' >>>> ERROR:core:db_do_query: error while submitting query >>>> ERROR:htable:ht_db_load_table: Error while querying db >>>> ERROR:core:init_mod: failed to initialize module htable >>>> ERROR:core:main: error while initializing modules >>>> >>>> Thank you >>>> >>>> Alex >>>> >>>> >>>> >>>> On Wed, Dec 23, 2009 at 6:43 PM, Daniel-Constantin Mierla < >>>> miconda(a)gmail.com>>>> >>>>> Hello, >>>>> >>>>> >>>>> On 12/23/09 4:28 PM, alex pappas wrote: >>>>> >>>>> Hi all, >>>>> >>>>> It is possible to have some help in defining the hash table? >>>>> >>>>> I'm giving some parameters but some of them i don't really understand >>>>> them. my configuration is: >>>>> >>>>> # --------- Htable --------- >>>>> modparam("htable", "htable", "a=>dbtable=ported1;") >>>>> modparam("htable", "db_url", "mysql://pskoul:pskoul123@localhost >>>>> /openser") >>>>> modparam("htable", "key_name_column", "phonenumber") >>>>> modparam("htable", "key_value", "prefix") >>>>> modparam("htable", "fetch_rows", 1000) >>>>> >>>>> The table I want to htable is a 2 cols table (phonenumber,prefix) In >>>>> mysql they are defined as varchar(10). All the phonenumber values are unique >>>>> so the key in htable is phonenumber with value in prefix. >>>>> >>>>> >>>>> what don't you understand? I do not know what to explain. Is there any >>>>> error you get? At least the parameter key_value is wrong, should be >>>>> key_value_column. >>>>> >>>>> Cheers, >>>>> Daniel >>>>> >>>>> >>>>> >>>>> Thanks All, >>>>> >>>>> Alex >>>>> >>>>> >>>>> >>>>> On Wed, Dec 23, 2009 at 1:04 PM, alex pappas &lt;rebel.pappas(a)gmail.com&gt;wrote;wrote: >>>>> >>>>>> Thank you Daniel, >>>>>> >>>>>> I will try first oracle db and then htable which is also very >>>>>> interesting. When I'm done I will get back with the results. >>>>>> >>>>>> Cheers >>>>>> Alex >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Dec 23, 2009 at 11:44 AM, Daniel-Constantin Mierla < >>>>>> miconda(a)gmail.com>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> >>>>>>> On 12/22/09 1:25 PM, alex pappas wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Concerning portability again :-) >>>>>>> >>>>>>> Is there any way I can add in the memory all the numbers with >>>>>>> portability? (2.000.000 rows)? >>>>>>> And also reload these data a few times a day and access this data >>>>>>> from the Kamailio script? >>>>>>> >>>>>>> >>>>>>> while I advise usage of sqlops/sql_query() instead of >>>>>>> avpops/avp_db_query(), if you want to load such data in memory should be in >>>>>>> shared memory. For that look at htable or maybe you can re-use pdt. >>>>>>> >>>>>>> Cheers, >>>>>>> Daniel >>>>>>> >>>>>>> >>>>>>> Cheers >>>>>>> Alex >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Dec 22, 2009 at 12:56 AM, Henning Westerholt < >>>>>>> henning.westerholt(a)1und1.de>>>>>>> >>>>>>>> On Mon, December 21, 2009 9:50 pm, Alex Balashov wrote: >>>>>>>> > [..] >>>>>>>> > Having said that, AVPs are still very useful and necessary because >>>>>>>> they >>>>>>>> > persist across the lifetime of a transaction, so you can access >>>>>>>> them in >>>>>>>> > subsequent reply and failure routes. This is not true of user >>>>>>>> > pseudovariables ($var(...)), nor, as far as I know, of any other >>>>>>>> class >>>>>>>> > of pseudovariable at this point except special >>>>>>>> transaction-persistent >>>>>>>> > ones exported by modules. >>>>>>>> > >>>>>>>> > As a result, it is often necessary for me to take data from $dbr >>>>>>>> > (sqlops) and load it into arrays of AVPs, such as for example when >>>>>>>> doing >>>>>>>> > custom fail-over in failure_route that uses logic for which >>>>>>>> dispatcher >>>>>>>> > is not suitable. I store a set of gateways in an AVP array along >>>>>>>> with >>>>>>>> > another AVP value indicating the current array index and iterate >>>>>>>> over >>>>>>>> > them in failure_route, and so on. >>>>>>>> > >>>>>>>> > Daniel taught me how to use AVP arrays once, although I noticed >>>>>>>> this is >>>>>>>> > not mentioned anywhere in the avpops documentation. It should be >>>>>>>> > published somewhere, as it is one of the most useful programmatic >>>>>>>> > features of Kamailio. >>>>>>>> > >>>>>>>> > Henning/Daniel, does Kamailio 3/SR offer any kind of better >>>>>>>> substitute >>>>>>>> > for what I am doing with AVPs as described above? In other words, >>>>>>>> are >>>>>>>> > there any transaction-persistent pseudovariables and/or namespace >>>>>>>> > containers that are easier to deal with syntactically, and provide >>>>>>>> more >>>>>>>> > advanced data structure primitives like arrays? I am aware of >>>>>>>> htable, >>>>>>>> > but I need something local to a transaction, not global. >>>>>>>> >>>>>>>> There are the $var PVs you already mentioned, and then also the >>>>>>>> $shv that >>>>>>>> are shared variables. But they are (process) global, and not >>>>>>>> transactional. Daniel proposed and developed the new xavp type for >>>>>>>> sr >>>>>>>> which are basically extended AVPs. More informations can be found >>>>>>>> here: >>>>>>>> http://sip-router.org/wiki/devel/xavp >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Henning >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Kamailio (OpenSER) - Users mailing list >>>>>>> Users@lists.kamailio.orghttp://lists.kamailio.org/cgi-bin/mailman/listinfo/usershttp://lists.openser-project.org/cgi-bin/mailman/listinfo/users >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Daniel-Constantin Mierla >>>>>>> * http://www.asipto.com/ >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Kamailio (OpenSER) - Users mailing listUsers@lists.kamailio.orghttp://lists.kamailio.org/cgi-bin/mailman/listinfo/usershttp://lists.openser-project.org/cgi-bin/mailman/listinfo/users >>>>> >>>>> >>>>> -- >>>>> Daniel-Constantin Mierla >>>>> * http://www.asipto.com/ >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Kamailio (OpenSER) - Users mailing listUsers@lists.kamailio.orghttp://lists.kamailio.org/cgi-bin/mailman/listinfo/usershttp://lists.openser-project.org/cgi-bin/mailman/listinfo/users >>>> >>>> >>>> -- >>>> Daniel-Constantin Mierla >>>> * http://www.asipto.com/ >>>> >>>> >>> >> >> -- >> Daniel-Constantin Mierla >> * http://www.asipto.com/ >> >> > > _______________________________________________ > Kamailio (OpenSER) - Users mailing listUsers@lists.kamailio.orghttp://lists.kamailio.org/cgi-bin/mailman/listinfo/usershttp://lists.openser-project.org/cgi-bin/mailman/listinfo/users > > > -- > Daniel-Constantin Mierla > * http://www.asipto.com/ > >

Hi, I don't see where the mysql is stacking but if I read correctly in http://kamailio.org/docs/modules/1.5.x/htable.html *The number of entries in the table is 2^size* : 2^14=16384 rows data and i have 2.000.000 rows so maybe it is not possible to load them..But you know better of course. Cheers Alex On Wed, Dec 30, 2009 at 5:32 PM, Daniel-Constantin Mierla &lt;miconda(a)gmail.com

Hello, On 12/30/09 4:53 PM, alex pappas wrote: no, that is the number of slots (buckets) in the hash table, not the number of items that can be loaded -- some more technical aspects of hash tables: http://en.wikipedia.org/wiki/Hash_table The number of items in htable is only a matter of shared memory. The number of slots controls access speed -- higher number gives less collisions. You said at a moment that you could start, but htable was returning only null: put a sht_print() in config to see the content of the htable (you probably need high debug level: debug=5) with the first sip request. Cheers, Daniel -- Daniel-Constantin Mierla * http://www.asipto.com/

Hi, When I have less rows in the table then : Dec 31 13:11:26 YGOR kamailio: DBG:core:find_cmd_export_t: found <sht_print>(0) in module htable [/usr/local/lib64/kamailio/modules/] Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: INFO:core:init_mod: initializing module htable Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: DBG:htable:ht_db_open_con: database connection opened successfully Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: DBG:htable:ht_db_load_tables: loading db table [htable] in ht [a] Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: DBG:htable:ht_db_load_table: =============== loading hash table [a] from database [htable] Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: DBG:htable:*ht_db_load_table: loaded 67 values in hash table* Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: DBG:core:init_mod: register MI for htable Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: DBG:htable:pv_parse_ht_name: htable [a] - key [$rU] Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: DBG:htable:ht_get_table: htable found [a] Cheers Alex On Thu, Dec 31, 2009 at 1:02 PM, alex pappas &lt;rebel.pappas(a)gmail.com&gt; wrote:

Thank you Daniel, Let's celebrate a SUPER 2010 ! Peace Alex On Thu, Dec 31, 2009 at 6:33 PM, Daniel-Constantin Mierla &lt;miconda(a)gmail.com > Hello, > > > On 12/31/09 12:14 PM, alex pappas wrote: > > Hi, > > When I have less rows in the table then : > > Dec 31 13:11:26 YGOR kamailio: DBG:core:find_cmd_export_t: found > <sht_print>(0) in module htable [/usr/local/lib64/kamailio/modules/] > Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: INFO:core:init_mod: > initializing module htable > Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: > DBG:htable:ht_db_open_con: database connection opened successfully > Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: > DBG:htable:ht_db_load_tables: loading db table [htable] in ht [a] > Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: > DBG:htable:ht_db_load_table: =============== loading hash table [a] from > database [htable] > Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: DBG:htable:*ht_db_load_table: > loaded 67 values in hash table* > > > this time is ok, indeed. As said in previous email, I will try too look > deeper at it in the next days. > > Thanks for troubleshooting and happy new year! > > Daniel > > > Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: DBG:core:init_mod: > register MI for htable > Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: > DBG:htable:pv_parse_ht_name: htable [a] - key [$rU] > Dec 31 13:11:33 YGOR /usr/local/sbin/kamailio[10843]: > DBG:htable:ht_get_table: htable found [a] > > > > Cheers > Alex > > > On Thu, Dec 31, 2009 at 1:02 PM, alex pappas &lt;rebel.pappas(a)gmail.com&gt;wrote;wrote: > >> Hi, >> >> From the syslog when kamailio start ups I get the following: >> >> >> Dec 31 12:35:45 YGOR /usr/local/sbin/kamailio[10281]: >> DBG:core:destroy_modules: shutting down module htable >> Dec 31 12:35:46 YGOR kamailio: DBG:core:yyparse: loading module >> /usr/local/lib64/kamailio/modules/htable.so >> Dec 31 12:35:46 YGOR kamailio: DBG:core:register_module: register_pv: >> htable >> Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: htable >> matches module htable >> Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: found >> <htable> in module htable [/usr/local/lib64/kamailio/modules/] >> Dec 31 12:35:46 YGOR kamailio: DBG:htable:ht_table_spec: htable [a] - size >> [14] >> Dec 31 12:35:46 YGOR kamailio: DBG:htable:ht_table_spec: htable [a] - >> dbtable [htable] >> Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: htable >> matches module htable >> Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: found >> <db_url> in module htable [/usr/local/lib64/kamailio/modules/] >> Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: htable >> matches module htable >> Dec 31 12:35:46 YGOR kamailio: DBG:core:set_mod_param_regex: found >> <fetch_rows> in module htable [/usr/local/lib64/kamailio/modules/] >> Dec 31 12:35:46 YGOR kamailio: DBG:core:find_cmd_export_t: found >> <sht_print>(0) in module htable [/usr/local/lib64/kamailio/modules/] >> Dec 31 12:35:52 YGOR /usr/local/sbin/kamailio[10371]: INFO:core:init_mod: >> initializing module htable >> Dec 31 12:35:52 YGOR /usr/local/sbin/kamailio[10371]: >> DBG:htable:ht_db_open_con: database connection opened successfully >> Dec 31 12:35:52 YGOR /usr/local/sbin/kamailio[10371]: >> DBG:htable:ht_db_load_tables: loading db table [htable] in ht [a] >> Dec 31 12:35:52 YGOR /usr/local/sbin/kamailio[10371]: >> DBG:htable:ht_db_load_table: =============== loading hash table [a] from >> database [htable] >> Dec 31 12:35:58 YGOR /usr/local/sbin/kamailio[10371]: DBG:htable*:ht_db_load_table: >> Nothing to be loaded in hash table* >> Dec 31 12:35:58 YGOR /usr/local/sbin/kamailio[10371]: DBG:core:init_mod: >> register MI for htable >> Dec 31 12:35:58 YGOR /usr/local/sbin/kamailio[10371]: >> DBG:htable:pv_parse_ht_name: htable [a] - key [$rU] >> Dec 31 12:35:58 YGOR /usr/local/sbin/kamailio[10371]: >> DBG:htable:ht_get_table: htable found [a] >> >> In the mysql htable I see : >> >> >> mysql> select count(*) from openser10.htable; >> +----------+ >> | count(*) | >> +----------+ >> | 1893255 | >> +----------+ >> 1 row in set (0.00 sec) >> >> And ofcourse when I run the sht_print(); I get many errors since htable in >> memory is empty: >> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1376] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1377] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1378] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1379] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1380] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1381] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1382] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1383] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1384] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1385] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1386] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1387] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1388] -- <0> >> Dec 31 12:52:48 YGOR /usr/local/sbin/kamailio[10375]: ERROR:htable:ht_dbg: >> htable[1389] -- <0> >> >> Do you have any idea on why htable module does not load the data in the >> memory ? >> >> Cheers >> Alex >> >> >> >> >> >> >> >> On Wed, Dec 30, 2009 at 6:01 PM, Daniel-Constantin Mierla < >> miconda(a)gmail.com>> >>> Hello, >>> >>> >>> On 12/30/09 4:53 PM, alex pappas wrote: >>> >>> Hi, >>> >>> I don't see where the mysql is stacking but if I read correctly in >>> http://kamailio.org/docs/modules/1.5.x/htable.html >>> >>> *The number of entries in the table is 2^size* : 2^14=16384 rows data >>> and i have 2.000.000 rows so maybe it is not possible to load them..But you >>> know better of course. >>> >>> no, that is the number of slots (buckets) in the hash table, not the >>> number of items that can be loaded -- some more technical aspects of hash >>> tables: >>> >>> http://en.wikipedia.org/wiki/Hash_table >>> >>> The number of items in htable is only a matter of shared memory. The >>> number of slots controls access speed -- higher number gives less >>> collisions. >>> >>> You said at a moment that you could start, but htable was returning only >>> null: put a sht_print() in config to see the content of the htable (you >>> probably need high debug level: debug=5) with the first sip request. >>> >>> Cheers, >>> Daniel >>> >>> >>> >>> >>> >>> >>> On Wed, Dec 30, 2009 at 5:32 PM, Daniel-Constantin Mierla < >>> miconda(a)gmail.com>>> >>>> Hello, >>>> >>>> On 12/30/09 4:05 PM, alex pappas wrote: >>>> >>>>> Hi, >>>>> >>>>> The error I get is the following: >>>>> >>>>> ERROR:db_mysql:db_mysql_submit_query: driver error on query: Lost >>>>> connection to MySQL server during query >>>>> ERROR:core:db_do_query: error while submitting query >>>>> ERROR:htable:ht_db_load_table: Error while querying db >>>>> ERROR:core:init_mod: failed to initialize module htable >>>>> ERROR:core:main: error while initializing modules >>>>> >>>>> I'm running kamailio with 8M private memory per process and 2048M >>>>> shared memory. >>>>> >>>>> When kamailio is getting the data from mysql is passing it all the data >>>>> chunk in private memory and then in the shared memory? >>>>> >>>> It is loaded in chunks of rows -- you can tune it with a module >>>> parameter: >>>> http://kamailio.org/docs/modules/3.0.x/modules_k/htable.html#id2529075 >>>> >>>> Try to play with it to see if you get better results. You do not need >>>> more private memory, 8MB is ok. >>>> >>>> Maybe the mysql server is closing due to lot of loaded data. Have you >>>> got any figure what is the number of rows when it starts losing the >>>> connection? >>>> >>>> Cheers, >>>> Daniel >>>> >>>> >>>> >>>>> because if this is the case then I need more than 1000M in private >>>>> memory. >>>>> >>>>> Cheers >>>>> Alex >>>>> >>>>> P.S. i don't get any errors from mysql. >>>>> >>>>> >>>>> >>>>> >>>> -- >>>> Daniel-Constantin Mierla >>>> * http://www.asipto.com/ >>>> >>>> >>> >>> -- >>> Daniel-Constantin Mierla >>> * http://www.asipto.com/ >>> >>> >> > > -- > Daniel-Constantin Mierla > * http://www.asipto.com/ > >

Hello, On 12/21/09 11:56 PM, Henning Westerholt wrote: as Henning said, xavp try to become a bit more friendly to config admins, but it is not adopted in any other module, you can use them just in config. Otherwise, the AVPs are the only one that help you for scenarios like failover. Note that with Kamailio 3.0 there are several lists of AVPs, inherited from SER, more here: http://sip-router.org/wiki/devel/avps-ser Kamailio AVPs map in the From list. Cheers, Daniel -- Daniel-Constantin Mierla * http://www.asipto.com/

On 12/21/2009 03:50 PM, Henning Westerholt wrote: I thought that would evaluate the numerical expression value of the AVP, not whether it is null? -- Alex Balashov - Principal Evariste Systems Web : http://www.evaristesys.com/ Tel : (+1) (678) 954-0670 Direct : (+1) (678) 954-0671

On Mon, December 21, 2009 9:51 pm, Alex Balashov wrote: not whether it is null? If you use the ' ! ' operator, it will evaluate the integer expression, which of course fail if you have a string avp content: $avp(s:test) = "foobar"; if (! $avp(s:test)) { xlog(" ! "); } if (! defined $avp(s:test)) { xlog("! defined"); } if (strlen($avp(s:test)) == 0) { xlog(" strlen "); } Dec 21 23:01:45 laptop ../../ser[18709]: WARNING: <core> [rvalue.c:987]: automatic string to int conversion for "foobar" failed Dec 21 23:01:45 laptop ../../ser[18709]: WARNING: <core> [rvalue.c:1839]: rval expression conversion to int failed (54,8-54,19) Dec 21 23:01:45 laptop ../../ser[18709]: ERROR: <script>: ! If the AVP is not defined, i get this output: Dec 21 23:14:30 laptop ../../ser[20028]: ERROR: <script>: ! Dec 21 23:14:30 laptop ../../ser[20028]: ERROR: <script>: ! defined Dec 21 23:14:30 laptop ../../ser[20028]: ERROR: <script>: strlen If I set the AVP to the empty string, i get this output: Dec 21 23:15:54 laptop ../../ser[20152]: ERROR: <script>: ! Dec 21 23:15:54 laptop ../../ser[20152]: ERROR: <script>: strlen I did also a test on kamailio, it seems that the 'defined' operator is not equivalent to the ' ! ' operator here, as the latter evaluate to true also if the $avp is the empty string. If also tried with to test for a not existing header with $hdr() on sip-router, this is equivalent to the not defined AVP. Regards, Henning