[sr-dev] [kamailio] [db_mysql] Kamailio fails during start if cannot connect to MySQL server (#192)

Andy Ericksen notifications at github.com
Thu Jun 4 16:40:58 CEST 2015 

I'm testing failure scenarios before going to production.

This test has the MySQL server offline to observe the Proxy's behavior when it cannot reach the DB server.

In this case we are using the DB for checking a global black list, we are not using the DB for auth or location.

If the DB server goes offline after Kamailio has started there does not seem to be an issue.  However if the DB server is offline when Kamailio is starting, Kamailio fails to start correctly.  

Kamailio log (hostnames and IP addresses changed or redacted):

```
Jun  4 10:21:24 robo-proxy kamailio: INFO: <core> [tcp_main.c:4748]: init_tcp(): using epoll_lt as the io watch method (auto detected)
Jun  4 10:21:24 robo-proxy kamailio: WARNING: <core> [daemonize.c:360]: daemonize(): pid file contains old pid, replacing pid
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: rr [../outbound/api.h:54]: ob_load_api(): Failed to import bind_ob
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: rr [rr_mod.c:160]: mod_init(): outbound module not available
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: pike [pike.c:115]: pike_init(): PIKE - initializing
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: pike [ip_tree.c:88]: init_lock_set(): probing 256 set size
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [udp_server.c:175]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [udp_server.c:225]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [udp_server.c:175]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [udp_server.c:225]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 1, pid 4635, "udp receiver child=0 sock=127.0.0.1:5060"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 3, pid 4638, "udp receiver child=2 sock=127.0.0.1:5060"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 2, pid 4636, "udp receiver child=1 sock=127.0.0.1:5060"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 4, pid 4640, "udp receiver child=3 sock=127.0.0.1:5060"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 8, pid 4647, "udp receiver child=3 sock=1.2.3.4:5060"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 7, pid 4645, "udp receiver child=2 sock=1.2.3.4:5060"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 5, pid 4642, "udp receiver child=0 sock=1.2.3.4:5060"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 10, pid 4654, "timer"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4635 exited normally, status=255
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4636 exited normally, status=255
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4638 exited normally, status=255
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4640 exited normally, status=255
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4642 exited normally, status=255
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4645 exited normally, status=255
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4647 exited normally, status=255
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [main.c:799]: handle_sigs(): terminating due to SIGCHLD
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4653]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4660]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4644]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4661]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 12
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 13
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 14
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 15
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 16
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 17
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4658]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 18
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 19
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 20
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 21
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 26
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 24
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 23
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: <core> [pt.c:479]: fork_tcp_process(): init_child failed for process 13, pid 4659, "tcp receiver (generic) child=0"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: <core> [tcp_main.c:4871]: tcp_init_children(): fork failed: Connection refused
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con]
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so)
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: <core> [pt.c:479]: fork_tcp_process(): init_child failed for process 16, pid 4664, "tcp receiver (generic) child=3"
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: <core> [tcp_main.c:4871]: tcp_init_children(): fork failed: Connection refused
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 5
Jun  4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 28
```

---
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/192
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-dev/attachments/20150604/ef303ef8/attachment-0001.html>

More information about the sr-dev mailing list