kamailio 4.2.3 Segmentation fault when using db_cluster module - sr-users

Olle E. Johansson

9 Mar 9 Mar

5:41 p.m.

New subject: kamailio 4.2.3 Segmentation fault when using db_cluster module

On 09 Mar 2015, at 17:16, Jan Hazenberg <jenus(a)cyberchaos.nl> wrote:

Hello, I have a issue on kamailio 4.2.3 when using the db_cluster module. I have the following config: #!define DBURL "cluster://cls1" # ----- db_cluster params ----- modparam("db_cluster", "connection", "con1=>mysql://kamailio:kamailiorw@10.121.0.120/kamailio") modparam("db_cluster", "connection", "con2=>mysql://kamailio:kamailiorw@10.121.0.121/kamailio") modparam("db_cluster", "connection", "con3=>mysql://kamailio:kamailiorw@10.121.0.122/kamailio") modparam("db_cluster", "cluster", "cls1=>con1=9r8r;con2=9r8r;con3=9r8r") modparam("db_cluster", "inactive_interval", 180) modparam("db_cluster", "max_query_length", 5) # ----- sqlops params ----- modparam("sqlops","sqlcon","ca=>cluster://cls1") When i start kamailio it fails with: Mar 9 17:11:49 localhost kamailio[31063]: INFO: rr [../outbound/api.h:54]: ob_load_api(): Failed to import bind_ob Mar 9 17:11:49 localhost kamailio[31063]: INFO: rr [rr_mod.c:160]: mod_init(): outbound module not available Mar 9 17:11:49 localhost kamailio[31063]: INFO: usrloc [hslot.c:53]: ul_init_locks(): locks array size 1024 Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions [parse_config.c:251]: parse_config_file(): file not found: /usr/local/etc/kamailio/permissions.allow Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions [permissions.c:608]: mod_init(): default allow file (/usr/local/etc/kamailio/permissions.allow) not found => empty rule set Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions [parse_config.c:251]: parse_config_file(): file not found: /usr/local/etc/kamailio/permissions.deny Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions [permissions.c:617]: mod_init(): default deny file (/usr/local/etc/kamailio/permissions.deny) not found => empty rule set Mar 9 17:11:49 localhost kernel: kamailio[31063]: segfault at 80 ip 00007f7f4f9ffddc sp 00007fffde2f88a0 error 4 in db_cluster.so[7f7f4f9ec000+28000] Mar 9 17:11:50 localhost kamailio: ERROR: <core> [daemonize.c:315]: daemonize(): Main process exited before writing to pipe Could this be a bug or is this a configuration issue? I tested the dbnodes and they seem to respond fine to any query's i send.

Configurations should never create segmentation faults, so it's clearly a bug. Can you find the core dump file and produce a backtrace? /O

Reply

Jan Hazenberg

8:21 p.m.

Olle, Here is the gdb debug trace (gdb) backtrace #0 0x00007ff76e13bddc in db_cluster_delete (_h=0x7ff7716f2428, _k=0x0, _o=0x0, _v=0x0, _n=0) at dbcl_api.c:390 #1 0x00007ff769d0193c in load_dialog_info_from_db (dlg_hash_size=4096, fetch_num_rows=200) at dlg_db_handler.c:462 #2 0x00007ff769cfcb9a in init_dlg_db (db_url=0x7ff769f58a90 <db_url>, dlg_hash_size=4096, db_update_period=60, fetch_num_rows=200) at dlg_db_handler.c:167 #3 0x00007ff769d1f15c in mod_init () at dialog.c:696 #4 0x000000000053a68d in init_mod (m=0x7ff7715af7b0) at sr_module.c:966 #5 0x000000000053a99a in init_modules () at sr_module.c:995 #6 0x0000000000531151 in main (argc=1, argv=0x7fffd2fad8f8) at main.c:2519 (gdb) Jan Olle E. Johansson schreef op 2015-03-09 17:41:

...

On 09 Mar 2015, at 17:16, Jan Hazenberg <jenus(a)cyberchaos.nl> wrote:

Hello, I have a issue on kamailio 4.2.3 when using the db_cluster module. I have the following config: #!define DBURL "cluster://cls1" # ----- db_cluster params ----- modparam("db_cluster", "connection", "con1=>mysql://kamailio:kamailiorw@10.121.0.120/kamailio") modparam("db_cluster", "connection", "con2=>mysql://kamailio:kamailiorw@10.121.0.121/kamailio") modparam("db_cluster", "connection", "con3=>mysql://kamailio:kamailiorw@10.121.0.122/kamailio") modparam("db_cluster", "cluster", "cls1=>con1=9r8r;con2=9r8r;con3=9r8r") modparam("db_cluster", "inactive_interval", 180) modparam("db_cluster", "max_query_length", 5) # ----- sqlops params ----- modparam("sqlops","sqlcon","ca=>cluster://cls1") When i start kamailio it fails with: Mar 9 17:11:49 localhost kamailio[31063]: INFO: rr [../outbound/api.h:54]: ob_load_api(): Failed to import bind_ob Mar 9 17:11:49 localhost kamailio[31063]: INFO: rr [rr_mod.c:160]: mod_init(): outbound module not available Mar 9 17:11:49 localhost kamailio[31063]: INFO: usrloc [hslot.c:53]: ul_init_locks(): locks array size 1024 Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions [parse_config.c:251]: parse_config_file(): file not found: /usr/local/etc/kamailio/permissions.allow Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions [permissions.c:608]: mod_init(): default allow file (/usr/local/etc/kamailio/permissions.allow) not found => empty rule set Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions [parse_config.c:251]: parse_config_file(): file not found: /usr/local/etc/kamailio/permissions.deny Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions [permissions.c:617]: mod_init(): default deny file (/usr/local/etc/kamailio/permissions.deny) not found => empty rule set Mar 9 17:11:49 localhost kernel: kamailio[31063]: segfault at 80 ip 00007f7f4f9ffddc sp 00007fffde2f88a0 error 4 in db_cluster.so[7f7f4f9ec000+28000] Mar 9 17:11:50 localhost kamailio: ERROR: <core> [daemonize.c:315]: daemonize(): Main process exited before writing to pipe Could this be a bug or is this a configuration issue? I tested the dbnodes and they seem to respond fine to any query's i send.

Configurations should never create segmentation faults, so it's clearly a bug. Can you find the core dump file and produce a backtrace? /O _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

Reply

Jan Hazenberg

10:46 a.m.

Daniel, Here the gdb output: (gdb) p cls $1 = (dbcl_cls_t *) 0x7ff7715b7390 (gdb) p *cls $2 = {name = {s = 0x7ff7715b72d8 "cls1=>con1=9r8r;con2=9r8r;con3=9r8r", len = 4}, clsid = 1863538307, ref = 1, rlist = {{clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x7ff7715b7030, 0x7ff7715b6cd0, 0x7ff7715b6970, 0x0, 0x0}, clen = 3, prio = 9, mode = 114, crt = 2}}, wlist = {{clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, { clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}, {clist = {0x7ff7715b7030, 0x7ff7715b6cd0, 0x7ff7715b6970, 0x0, 0x0}, clen = 3, prio = 8, mode = 114, crt = 0}, {clist = {0x0, 0x0, 0x0, 0x0, 0x0}, clen = 0, prio = 0, mode = 0, crt = 0}}, usedcon = 0x7ff7715b6cd0, next = 0x0} (gdb) p i $3 = 8 (gdb) bt full #0 0x00007ff76e13bddc in db_cluster_delete (_h=0x7ff7716f2428, _k=0x0, _o=0x0, _v=0x0, _n=0) at dbcl_api.c:390 ret = -1 i = 8 k = 0 sec = 13502139 rc = 0 rok = 0 j = 0 dbh = 0x7ff7716f4ad0 cls = 0x7ff7715b7390 __FUNCTION__ = "db_cluster_delete" #1 0x00007ff769d0193c in load_dialog_info_from_db (dlg_hash_size=4096, fetch_num_rows=200) at dlg_db_handler.c:462 res = 0x7ff7716f4c40 values = 0x1d2fad4b0 rows = 0x0 i = 0 nr_rows = 0 dlg = 0x7fffd2fad8f0 callid = {s = 0x7fffd2fad4b0 " \325\372\322\377\177", len = 1875525114} from_uri = {s = 0x0, len = 0} to_uri = {s = 0x73cdb8 "DEBUG", len = -755312400} from_tag = {s = 0x7ff7715b5d68 "cluster://cls1", len = 1775544688} to_tag = {s = 0x9c8b50 <version_table> "\364\266q", len = 1903119320} req_uri = {s = 0x7fffd2fad460 "\260\324\372\322\001", len = 1846750789} cseq1 = {s = 0xe <Address 0xe out of bounds>, len = 1901813096} cseq2 = {s = 0x7fffd2fad350 "\205P\313o\367\177", len = 1901584760} contact1 = {s = 0x7fffd2fad340 "\220P\313o\367\177", len = 0} contact2 = {s = 0x7ff769d4450b "dialog_vars", len = 11} rroute1 = {s = 0x4 <Address 0x4 out of bounds>, len = 1846854459} rroute2 = {s = 0x7ff7715b68b8 "con1=>mysql://kamailio:kamailiorw@10.121.0.120/kamailio", len = 1903119424} toroute_name = {s = 0x7ff76fcb5085 "table_name", len = 10} xdata = {s = 0x7ff76fcb5090 "table_version", len = 13} next_id = 32759 jdoc = {root = 0x7fffd2fad3c0, flags = 6405665, buf = {s = 0x7fffd2fad340 "\220P\313o\367\177", len = 1777699264}, malloc_fn = 0x7ff7716f2428, free_fn = 0x7ff769f59c00 <dialog_dbf>} __FUNCTION__ = "load_dialog_info_from_db" #2 0x00007ff769cfcb9a in init_dlg_db (db_url=0x7ff769f58a90 <db_url>, dlg_hash_size=4096, db_update_period=60, fetch_num_rows=200) at dlg_db_handler.c:167 __FUNCTION__ = "init_dlg_db" #3 0x00007ff769d1f15c in mod_init () at dialog.c:696 n = 12 __FUNCTION__ = "mod_init" #4 0x000000000053a68d in init_mod (m=0x7ff7715af7b0) at sr_module.c:966 __FUNCTION__ = "init_mod" #5 0x000000000053a99a in init_modules () at sr_module.c:995 t = 0x7ff77213bdc0 <result> i = 200 __FUNCTION__ = "init_modules" #6 0x0000000000531151 in main (argc=1, argv=0x7fffd2fad8f8) at main.c:2519 cfg_stream = 0x1f9c010 c = -1 r = 11538501 tmp = 0x1 <Address 0x1 out of bounds> tmp_len = 0 port = 0 proto = 32767 options = 0x71ea48 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:" ret = -1 seed = 84040262 rfd = 4 debug_save = 0 debug_flag = 0 dont_fork_cnt = 0 n_lst = 0xf63d4e2e p = 0x7ff772774000 "" ---Type <return> to continue, or q <return> to quit--- __FUNCTION__ = "main" (gdb) Jan Daniel-Constantin Mierla schreef op 2015-03-10 09:34:

...

Hello, can you give the output from gdb for: p cls p *cls p i bt full Cheers, Daniel On 09/03/15 20:21, Jan Hazenberg wrote: > Olle, > > Here is the gdb debug trace > > (gdb) backtrace > #0 0x00007ff76e13bddc in db_cluster_delete (_h=0x7ff7716f2428, > _k=0x0, _o=0x0, _v=0x0, _n=0) at dbcl_api.c:390 > #1 0x00007ff769d0193c in load_dialog_info_from_db > (dlg_hash_size=4096, fetch_num_rows=200) at dlg_db_handler.c:462 > #2 0x00007ff769cfcb9a in init_dlg_db (db_url=0x7ff769f58a90 <db_url>, > dlg_hash_size=4096, db_update_period=60, fetch_num_rows=200) at > dlg_db_handler.c:167 > #3 0x00007ff769d1f15c in mod_init () at dialog.c:696 > #4 0x000000000053a68d in init_mod (m=0x7ff7715af7b0) at > sr_module.c:966 > #5 0x000000000053a99a in init_modules () at sr_module.c:995 > #6 0x0000000000531151 in main (argc=1, argv=0x7fffd2fad8f8) at > main.c:2519 > (gdb) > > > Jan > > > Olle E. Johansson schreef op 2015-03-09 17:41: >> On 09 Mar 2015, at 17:16, Jan Hazenberg <jenus(a)cyberchaos.nl> wrote: >> >>> Hello, >>> >>> I have a issue on kamailio 4.2.3 when using the db_cluster module. I >>> have the following config: >>> >>> #!define DBURL "cluster://cls1" >>> >>> # ----- db_cluster params ----- >>> modparam("db_cluster", "connection", >>> >>> "con1=>mysql://kamailio:kamailiorw@10.121.0.120/kamailio") >>> modparam("db_cluster", "connection", >>> >>> "con2=>mysql://kamailio:kamailiorw@10.121.0.121/kamailio") >>> modparam("db_cluster", "connection", >>> >>> "con3=>mysql://kamailio:kamailiorw@10.121.0.122/kamailio") >>> modparam("db_cluster", "cluster", >>> "cls1=>con1=9r8r;con2=9r8r;con3=9r8r") >>> modparam("db_cluster", "inactive_interval", 180) >>> modparam("db_cluster", "max_query_length", 5) >>> >>> # ----- sqlops params ----- >>> modparam("sqlops","sqlcon","ca=>cluster://cls1") >>> >>> >>> When i start kamailio it fails with: >>> >>> Mar 9 17:11:49 localhost kamailio[31063]: INFO: rr >>> [../outbound/api.h:54]: ob_load_api(): Failed to import bind_ob >>> Mar 9 17:11:49 localhost kamailio[31063]: INFO: rr [rr_mod.c:160]: >>> mod_init(): outbound module not available >>> Mar 9 17:11:49 localhost kamailio[31063]: INFO: usrloc >>> [hslot.c:53]: ul_init_locks(): locks array size 1024 >>> Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions >>> [parse_config.c:251]: parse_config_file(): file not found: >>> /usr/local/etc/kamailio/permissions.allow >>> Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions >>> [permissions.c:608]: mod_init(): default allow file >>> (/usr/local/etc/kamailio/permissions.allow) not found => empty rule >>> set >>> Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions >>> [parse_config.c:251]: parse_config_file(): file not found: >>> /usr/local/etc/kamailio/permissions.deny >>> Mar 9 17:11:49 localhost kamailio[31063]: INFO: permissions >>> [permissions.c:617]: mod_init(): default deny file >>> (/usr/local/etc/kamailio/permissions.deny) not found => empty rule >>> set >>> Mar 9 17:11:49 localhost kernel: kamailio[31063]: segfault at 80 ip >>> 00007f7f4f9ffddc sp 00007fffde2f88a0 error 4 in >>> db_cluster.so[7f7f4f9ec000+28000] >>> Mar 9 17:11:50 localhost kamailio: ERROR: <core> [daemonize.c:315]: >>> daemonize(): Main process exited before writing to pipe >>> >>> >>> Could this be a bug or is this a configuration issue? I tested the >>> dbnodes and they seem to respond fine to any query's i send. >> >> Configurations should never create segmentation faults, so it's >> clearly a bug. Can you find the core dump file and produce a >> backtrace? >> >> /O >> _______________________________________________ >> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >> list >> sr-users(a)lists.sip-router.org >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users > > _______________________________________________ > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing > list > sr-users(a)lists.sip-router.org > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

Reply

Jan Hazenberg

3:01 p.m.

Daniel, Yes thanks, that seems to solve the issue. Jan Daniel-Constantin Mierla schreef op 2015-03-10 13:57:

...

Thanks, it helped to spot the issue. Can you try with the latest version from branch 4.2? Cheers, Daniel On 10/03/15 10:46, Jan Hazenberg wrote: > Daniel, > > > Here the gdb output: > > ...

Reply