Hi All I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is that Kamailio at times fails to start up for no obvious reasons, I.e. in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug log "level 2 " shows the following :
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4) 27(24762) ERROR: <core> [db.c:303]: could not add connection to the pool27(24762) ERROR: group [group.c:211]: unable to connect to the database 27(24762) ERROR: <core> [sr_module.c:889]: init_mod_child(): Error while initializing module group (/usr/local/kamailio/lib/kamailio/modules_k/group.so) 27(24762) ERROR: <core> [pt.c:481]: ERROR: fork_tcp_process(): init_child failed for process 27, pid 24762, "tcp receiver child=1"
and then
30(24765) : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 0(24735) ALERT: <core> [main.c:748]: child process 24762 exited normally, status=255 0(24735) INFO: <core> [main.c:766]: INFO: terminating due to SIGCHLD
The problem happens on 4 different servers, I did try to compile more than once, the servers are 32 Bit CentOS 5.6. The connection to the database is monitored and there are no issues with connecting to the DB "it works at times and times not for Kamailio so a user/pwd issue is not likely to cause this, manual cnnect worked", config file errors are not found.
I did also check max no of connections on MySQL server which is set to 500 on the server and only 100 cons are used. The group module uses the same connection string as other modules which is predefined and there are other modules that access the db before the group module and those do not have a problem.
Please advice.
Regards
Hi, forget the whole list of errors and just resolve the DB connectivity between the Kamailio server and DB server first .
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't
connect to MySQL server on 'xxxxxxxxx' (4)
try connecting to the remote DB server 'xxxxxxxxx' from linux shell i.e
# mysql -u<user> -p<passwd> -h 'xxxxxxxxx'
Regards, Sammy
On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawad ali.jawad@splendor.net wrote:
Hi All I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is that Kamailio at times fails to start up for no obvious reasons, I.e. in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug log "level 2 " shows the following :
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4) 27(24762) ERROR: <core> [db.c:303]: could not add connection to the pool27(24762) ERROR: group [group.c:211]: unable to connect to the database 27(24762) ERROR: <core> [sr_module.c:889]: init_mod_child(): Error while initializing module group (/usr/local/kamailio/lib/kamailio/modules_k/group.so) 27(24762) ERROR: <core> [pt.c:481]: ERROR: fork_tcp_process(): init_child failed for process 27, pid 24762, "tcp receiver child=1"
and then
30(24765) : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 0(24735) ALERT: <core> [main.c:748]: child process 24762 exited normally, status=255 0(24735) INFO: <core> [main.c:766]: INFO: terminating due to SIGCHLD
The problem happens on 4 different servers, I did try to compile more than once, the servers are 32 Bit CentOS 5.6. The connection to the database is monitored and there are no issues with connecting to the DB "it works at times and times not for Kamailio so a user/pwd issue is not likely to cause this, manual cnnect worked", config file errors are not found.
I did also check max no of connections on MySQL server which is set to 500 on the server and only 100 cons are used. The group module uses the same connection string as other modules which is predefined and there are other modules that access the db before the group module and those do not have a problem.
Please advice.
Regards
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards
On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govind govoiper@gmail.com wrote:
Hi, forget the whole list of errors and just resolve the DB connectivity between the Kamailio server and DB server first .
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4)
try connecting to the remote DB server 'xxxxxxxxx' from linux shell i.e
# mysql -u<user> -p<passwd> -h 'xxxxxxxxx'
Regards, Sammy
On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawad ali.jawad@splendor.net wrote:
Hi All I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is that Kamailio at times fails to start up for no obvious reasons, I.e. in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug log "level 2 " shows the following :
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4) 27(24762) ERROR: <core> [db.c:303]: could not add connection to the pool27(24762) ERROR: group [group.c:211]: unable to connect to the database 27(24762) ERROR: <core> [sr_module.c:889]: init_mod_child(): Error while initializing module group (/usr/local/kamailio/lib/kamailio/modules_k/group.so) 27(24762) ERROR: <core> [pt.c:481]: ERROR: fork_tcp_process(): init_child failed for process 27, pid 24762, "tcp receiver child=1"
and then
30(24765) : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 0(24735) ALERT: <core> [main.c:748]: child process 24762 exited normally, status=255 0(24735) INFO: <core> [main.c:766]: INFO: terminating due to SIGCHLD
The problem happens on 4 different servers, I did try to compile more than once, the servers are 32 Bit CentOS 5.6. The connection to the database is monitored and there are no issues with connecting to the DB "it works at times and times not for Kamailio so a user/pwd issue is not likely to cause this, manual cnnect worked", config file errors are not found.
I did also check max no of connections on MySQL server which is set to 500 on the server and only 100 cons are used. The group module uses the same connection string as other modules which is predefined and there are other modules that access the db before the group module and those do not have a problem.
Please advice.
Regards
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawad ali.jawad@splendor.net wrote:
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards
On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govind govoiper@gmail.com wrote:
Hi, forget the whole list of errors and just resolve the DB connectivity
between
the Kamailio server and DB server first .
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4)
try connecting to the remote DB server 'xxxxxxxxx' from linux shell i.e
# mysql -u<user> -p<passwd> -h 'xxxxxxxxx'
Regards, Sammy
On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawad ali.jawad@splendor.net
wrote:
Hi All I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is that Kamailio at times fails to start up for no obvious reasons, I.e. in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug log "level 2 " shows the following :
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4) 27(24762) ERROR: <core> [db.c:303]: could not add connection to the pool27(24762) ERROR: group [group.c:211]: unable to connect to the database 27(24762) ERROR: <core> [sr_module.c:889]: init_mod_child(): Error while initializing module group (/usr/local/kamailio/lib/kamailio/modules_k/group.so) 27(24762) ERROR: <core> [pt.c:481]: ERROR: fork_tcp_process(): init_child failed for process 27, pid 24762, "tcp receiver child=1"
and then
30(24765) : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 0(24735) ALERT: <core> [main.c:748]: child process 24762 exited normally, status=255 0(24735) INFO: <core> [main.c:766]: INFO: terminating due to SIGCHLD
The problem happens on 4 different servers, I did try to compile more than once, the servers are 32 Bit CentOS 5.6. The connection to the database is monitored and there are no issues with connecting to the DB "it works at times and times not for Kamailio so a user/pwd issue is not likely to cause this, manual cnnect worked", config file errors are not found.
I did also check max no of connections on MySQL server which is set to 500 on the server and only 100 cons are used. The group module uses the same connection string as other modules which is predefined and there are other modules that access the db before the group module and those do not have a problem.
Please advice.
Regards
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net) Beirut, Lebanon Phone: +9611373725/ext 116 FAX: +9611375554
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hello,
check also for limits in the system/firewall, maybe there are limits on new connections per second. Try with children=1 and see if it happens again, if not, then it is something that blocks many new connections in short time.
Cheers, Daniel
On 1/2/12 5:31 AM, Sammy Govind wrote:
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawad <ali.jawad@splendor.net mailto:ali.jawad@splendor.net> wrote:
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govind <govoiper@gmail.com <mailto:govoiper@gmail.com>> wrote: > Hi, > forget the whole list of errors and just resolve the DB connectivity between > the Kamailio server and DB server first . > >> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >> connect to MySQL server on 'xxxxxxxxx' (4) > > > try connecting to the remote DB server 'xxxxxxxxx' from linux shell i.e > > # mysql -u<user> -p<passwd> -h 'xxxxxxxxx' > > Regards, > Sammy > > > On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawad <ali.jawad@splendor.net <mailto:ali.jawad@splendor.net>> wrote: >> >> Hi All >> I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is >> that Kamailio at times fails to start up for no obvious reasons, I.e. >> in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug >> log "level 2 " shows the following : >> >> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >> connect to MySQL server on 'xxxxxxxxx' (4) >> 27(24762) ERROR: <core> [db.c:303]: could not add connection to the >> pool27(24762) ERROR: group [group.c:211]: unable to connect to the >> database >> 27(24762) ERROR: <core> [sr_module.c:889]: init_mod_child(): Error >> while initializing module group >> (/usr/local/kamailio/lib/kamailio/modules_k/group.so) >> 27(24762) ERROR: <core> [pt.c:481]: ERROR: fork_tcp_process(): >> init_child failed for process 27, pid 24762, "tcp receiver child=1" >> >> and then >> >> >> 30(24765) : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 >> 0(24735) ALERT: <core> [main.c:748]: child process 24762 exited >> normally, status=255 >> 0(24735) INFO: <core> [main.c:766]: INFO: terminating due to SIGCHLD >> >> The problem happens on 4 different servers, I did try to compile more >> than once, the servers are 32 Bit CentOS 5.6. The connection to the >> database is monitored and there are no issues with connecting to the >> DB "it works at times and times not for Kamailio so a user/pwd issue >> is not likely to cause this, manual cnnect worked", config file errors >> are not found. >> >> I did also check max no of connections on MySQL server which is set to >> 500 on the server and only 100 cons are used. The group module uses >> the same connection string as other modules which is predefined and >> there are other modules that access the db before the group module and >> those do not have a problem. >> >> Please advice. >> >> Regards >> >> _______________________________________________ >> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list >> sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users > > > > _______________________________________________ > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list > sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users > -- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net <http://www.splendor.net>) Beirut, Lebanon Phone: +9611373725 <tel:%2B9611373725>/ext 116 FAX: +9611375554 <tel:%2B9611375554> _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi Daniel Running on a LAN with no firewalls enabled, will try you suggestion though first thing tomorrow morning, what are the implications of running children=1 ? Thanks
On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
check also for limits in the system/firewall, maybe there are limits on new connections per second. Try with children=1 and see if it happens again, if not, then it is something that blocks many new connections in short time.
Cheers, Daniel
On 1/2/12 5:31 AM, Sammy Govind wrote:
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawad ali.jawad@splendor.net wrote:
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards
On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govind govoiper@gmail.com wrote:
Hi, forget the whole list of errors and just resolve the DB connectivity between the Kamailio server and DB server first .
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4)
try connecting to the remote DB server 'xxxxxxxxx' from linux shell i.e
# mysql -u<user> -p<passwd> -h 'xxxxxxxxx'
Regards, Sammy
On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawad ali.jawad@splendor.net wrote:
Hi All I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is that Kamailio at times fails to start up for no obvious reasons, I.e. in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug log "level 2 " shows the following :
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4) 27(24762) ERROR: <core> [db.c:303]: could not add connection to the pool27(24762) ERROR: group [group.c:211]: unable to connect to the database 27(24762) ERROR: <core> [sr_module.c:889]: init_mod_child(): Error while initializing module group (/usr/local/kamailio/lib/kamailio/modules_k/group.so) 27(24762) ERROR: <core> [pt.c:481]: ERROR: fork_tcp_process(): init_child failed for process 27, pid 24762, "tcp receiver child=1"
and then
30(24765) : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 0(24735) ALERT: <core> [main.c:748]: child process 24762 exited normally, status=255 0(24735) INFO: <core> [main.c:766]: INFO: terminating due to SIGCHLD
The problem happens on 4 different servers, I did try to compile more than once, the servers are 32 Bit CentOS 5.6. The connection to the database is monitored and there are no issues with connecting to the DB "it works at times and times not for Kamailio so a user/pwd issue is not likely to cause this, manual cnnect worked", config file errors are not found.
I did also check max no of connections on MySQL server which is set to 500 on the server and only 100 cons are used. The group module uses the same connection string as other modules which is predefined and there are other modules that access the db before the group module and those do not have a problem.
Please advice.
Regards
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net) Beirut, Lebanon Phone: +9611373725/ext 116 FAX: +9611375554
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hello,
On 1/2/12 12:24 PM, Ali Jawad wrote:
Hi Daniel Running on a LAN with no firewalls enabled,
limits can be from kernel, selinix, mysql server a.s.o. not only from firewalls.
will try you suggestion though first thing tomorrow morning, what are the implications of running children=1 ?
This will create only one worker process (not advisable for production in long term, just for testing) -- in this way there should be very few mysql connections opened, revealing if there is a problem or not with the number of connections and the short period to create them.
Cheers, Daniel
Thanks
On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
check also for limits in the system/firewall, maybe there are limits on new connections per second. Try with children=1 and see if it happens again, if not, then it is something that blocks many new connections in short time.
Cheers, Daniel
On 1/2/12 5:31 AM, Sammy Govind wrote:
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawadali.jawad@splendor.net wrote:
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards
On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govindgovoiper@gmail.com wrote:
Hi, forget the whole list of errors and just resolve the DB connectivity between the Kamailio server and DB server first .
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4)
try connecting to the remote DB server 'xxxxxxxxx' from linux shell i.e
# mysql -u<user> -p<passwd> -h 'xxxxxxxxx'
Regards, Sammy
On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawadali.jawad@splendor.net wrote:
Hi All I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is that Kamailio at times fails to start up for no obvious reasons, I.e. in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug log "level 2 " shows the following :
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4) 27(24762) ERROR:<core> [db.c:303]: could not add connection to the pool27(24762) ERROR: group [group.c:211]: unable to connect to the database 27(24762) ERROR:<core> [sr_module.c:889]: init_mod_child(): Error while initializing module group (/usr/local/kamailio/lib/kamailio/modules_k/group.so) 27(24762) ERROR:<core> [pt.c:481]: ERROR: fork_tcp_process(): init_child failed for process 27, pid 24762, "tcp receiver child=1"
and then
30(24765) :<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 0(24735) ALERT:<core> [main.c:748]: child process 24762 exited normally, status=255 0(24735) INFO:<core> [main.c:766]: INFO: terminating due to SIGCHLD
The problem happens on 4 different servers, I did try to compile more than once, the servers are 32 Bit CentOS 5.6. The connection to the database is monitored and there are no issues with connecting to the DB "it works at times and times not for Kamailio so a user/pwd issue is not likely to cause this, manual cnnect worked", config file errors are not found.
I did also check max no of connections on MySQL server which is set to 500 on the server and only 100 cons are used. The group module uses the same connection string as other modules which is predefined and there are other modules that access the db before the group module and those do not have a problem.
Please advice.
Regards
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net) Beirut, Lebanon Phone: +9611373725/ext 116 FAX: +9611375554
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi Daniel
SeLinux is disabled, as I have played around with Children and got a stable startup at CHILDREN=3, at 4 and up I get the problem described earlier right away, so it seems that MySQL is throttling Kamailio ?
1- What is the effect of running 3 children only ? 2- I know this is not a Kamailio thing, but any idea about what needs to be changed on MySQL server to accommodate more children ? No of connections is 500 conns out of which max 120 are utilized
Regards
On Mon, Jan 2, 2012 at 1:31 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/2/12 12:24 PM, Ali Jawad wrote:
Hi Daniel Running on a LAN with no firewalls enabled,
limits can be from kernel, selinix, mysql server a.s.o. not only from firewalls.
will try you suggestion though first thing tomorrow morning, what are the implications of running children=1 ?
This will create only one worker process (not advisable for production in long term, just for testing) -- in this way there should be very few mysql connections opened, revealing if there is a problem or not with the number of connections and the short period to create them.
Cheers, Daniel
Thanks
On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
check also for limits in the system/firewall, maybe there are limits on new connections per second. Try with children=1 and see if it happens again, if not, then it is something that blocks many new connections in short time.
Cheers, Daniel
On 1/2/12 5:31 AM, Sammy Govind wrote:
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawadali.jawad@splendor.net wrote:
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards
On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govindgovoiper@gmail.com wrote:
Hi, forget the whole list of errors and just resolve the DB connectivity between the Kamailio server and DB server first .
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4)
try connecting to the remote DB server 'xxxxxxxxx' from linux shell i.e
# mysql -u<user> -p<passwd> -h 'xxxxxxxxx'
Regards, Sammy
On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawadali.jawad@splendor.net wrote:
Hi All I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is that Kamailio at times fails to start up for no obvious reasons, I.e. in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug log "level 2 " shows the following :
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4) 27(24762) ERROR:<core> [db.c:303]: could not add connection to the pool27(24762) ERROR: group [group.c:211]: unable to connect to the database 27(24762) ERROR:<core> [sr_module.c:889]: init_mod_child(): Error while initializing module group (/usr/local/kamailio/lib/kamailio/modules_k/group.so) 27(24762) ERROR:<core> [pt.c:481]: ERROR: fork_tcp_process(): init_child failed for process 27, pid 24762, "tcp receiver child=1"
and then
30(24765) :<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 0(24735) ALERT:<core> [main.c:748]: child process 24762 exited normally, status=255 0(24735) INFO:<core> [main.c:766]: INFO: terminating due to SIGCHLD
The problem happens on 4 different servers, I did try to compile more than once, the servers are 32 Bit CentOS 5.6. The connection to the database is monitored and there are no issues with connecting to the DB "it works at times and times not for Kamailio so a user/pwd issue is not likely to cause this, manual cnnect worked", config file errors are not found.
I did also check max no of connections on MySQL server which is set to 500 on the server and only 100 cons are used. The group module uses the same connection string as other modules which is predefined and there are other modules that access the db before the group module and those do not have a problem.
Please advice.
Regards
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net) Beirut, Lebanon Phone: +9611373725/ext 116 FAX: +9611375554
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
Hello,
On 1/3/12 9:14 AM, Ali Jawad wrote:
Hi Daniel
SeLinux is disabled, as I have played around with Children and got a stable startup at CHILDREN=3, at 4 and up I get the problem described earlier right away, so it seems that MySQL is throttling Kamailio ?
1- What is the effect of running 3 children only ?
the capacity of handling sip traffic is lowered. You should still be able to handle couple of hundred messages per second if you don't have slow sql/dns operations. For production, it is recommended to have at least 8 to be in safe side.
2- I know this is not a Kamailio thing, but any idea about what needs to be changed on MySQL server to accommodate more children ? No of connections is 500 conns out of which max 120 are utilized
So mysql is on a different server, right? Are you sure there is no other switch/router doing such throttling? Can you watch on both kamailio and mysql server to see if the connections are initiated from kamailio and reaching mysql? Maybe you can discover if mysql server get the tcp connection but refuses it -- not sure if mysql server can run in debug/verbose mode to get some log messages with hints about what is the problem.
If the tcp connection does not get to mysql server, then it is a throttling in mysql client library or on a device in between the servers.
If you do 'kamctl ps' when having children=3, how many kamailio processes are there (extra should be timers, ...)? Each should have one mysql connection, so it helps to see at which number the limit starts.
Cheers, Daniel
Regards
On Mon, Jan 2, 2012 at 1:31 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/2/12 12:24 PM, Ali Jawad wrote:
Hi Daniel Running on a LAN with no firewalls enabled,
limits can be from kernel, selinix, mysql server a.s.o. not only from firewalls.
will try you suggestion though first thing tomorrow morning, what are the implications of running children=1 ?
This will create only one worker process (not advisable for production in long term, just for testing) -- in this way there should be very few mysql connections opened, revealing if there is a problem or not with the number of connections and the short period to create them.
Cheers, Daniel
Thanks
On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
check also for limits in the system/firewall, maybe there are limits on new connections per second. Try with children=1 and see if it happens again, if not, then it is something that blocks many new connections in short time.
Cheers, Daniel
On 1/2/12 5:31 AM, Sammy Govind wrote:
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawadali.jawad@splendor.net wrote:
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards
On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govindgovoiper@gmail.com wrote:
Hi, forget the whole list of errors and just resolve the DB connectivity between the Kamailio server and DB server first .
> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't > connect to MySQL server on 'xxxxxxxxx' (4)
try connecting to the remote DB server 'xxxxxxxxx' from linux shell i.e
# mysql -u<user> -p<passwd> -h 'xxxxxxxxx'
Regards, Sammy
On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawadali.jawad@splendor.net wrote: > Hi All > I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is > that Kamailio at times fails to start up for no obvious reasons, I.e. > in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug > log "level 2 " shows the following : > > 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't > connect to MySQL server on 'xxxxxxxxx' (4) > 27(24762) ERROR:<core> [db.c:303]: could not add connection to the > pool27(24762) ERROR: group [group.c:211]: unable to connect to the > database > 27(24762) ERROR:<core> [sr_module.c:889]: init_mod_child(): Error > while initializing module group > (/usr/local/kamailio/lib/kamailio/modules_k/group.so) > 27(24762) ERROR:<core> [pt.c:481]: ERROR: fork_tcp_process(): > init_child failed for process 27, pid 24762, "tcp receiver child=1" > > and then > > > 30(24765) :<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 > 0(24735) ALERT:<core> [main.c:748]: child process 24762 exited > normally, status=255 > 0(24735) INFO:<core> [main.c:766]: INFO: terminating due to SIGCHLD > > The problem happens on 4 different servers, I did try to compile more > than once, the servers are 32 Bit CentOS 5.6. The connection to the > database is monitored and there are no issues with connecting to the > DB "it works at times and times not for Kamailio so a user/pwd issue > is not likely to cause this, manual cnnect worked", config file errors > are not found. > > I did also check max no of connections on MySQL server which is set to > 500 on the server and only 100 cons are used. The group module uses > the same connection string as other modules which is predefined and > there are other modules that access the db before the group module and > those do not have a problem. > > Please advice. > > Regards > > _______________________________________________ > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing > list > sr-users@lists.sip-router.org > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net) Beirut, Lebanon Phone: +9611373725/ext 116 FAX: +9611375554
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
Hi I dont think a few hundred per second is enough I need to find out what is wrong and bring up no of children to 8. The MySQL server is at another host on the same LAN, as for the command kamctl ps see below
Process:: ID=0 PID=13949 Type=attendant Process:: ID=1 PID=13950 Type=udp receiver child=0 sock=xx.xx.xx.xx:5060 Process:: ID=2 PID=13951 Type=udp receiver child=1 sock=xx.xx.xx.xx:5060 Process:: ID=3 PID=13952 Type=udp receiver child=2 sock=xx.xx.xx.xx:5060 Process:: ID=4 PID=13953 Type=udp receiver child=0 sock=xx.xx.xx.xx:5061 Process:: ID=5 PID=13954 Type=udp receiver child=1 sock=xx.xx.xx.xx:5061 Process:: ID=6 PID=13955 Type=udp receiver child=2 sock=xx.xx.xx.xx:5061 Process:: ID=7 PID=13956 Type=udp receiver child=0 sock=xx.xx.xx.xx:443 Process:: ID=8 PID=13957 Type=udp receiver child=1 sock=xx.xx.xx.xx:443 Process:: ID=9 PID=13958 Type=udp receiver child=2 sock=xx.xx.xx.xx:443 Process:: ID=10 PID=13959 Type=udp receiver child=0 sock=xx.xx.xx.xx:8080 Process:: ID=11 PID=13960 Type=udp receiver child=1 sock=xx.xx.xx.xx:8080 Process:: ID=12 PID=13961 Type=udp receiver child=2 sock=xx.xx.xx.xx:8080 Process:: ID=13 PID=13962 Type=udp receiver child=0 sock=xx.xx.xx.xx:123 Process:: ID=14 PID=13963 Type=udp receiver child=1 sock=xx.xx.xx.xx:123 Process:: ID=15 PID=13964 Type=udp receiver child=2 sock=xx.xx.xx.xx:123 Process:: ID=16 PID=13965 Type=slow timer Process:: ID=17 PID=13966 Type=timer Process:: ID=18 PID=13967 Type=MI FIFO Process:: ID=19 PID=13968 Type=ctl handler Process:: ID=20 PID=13969 Type=TIMER NH Process:: ID=21 PID=13970 Type=tcp receiver child=0 Process:: ID=22 PID=13971 Type=tcp receiver child=1 Process:: ID=23 PID=13972 Type=tcp receiver child=2 Process:: ID=24 PID=13973 Type=tcp main process
I use a few different ports as you can see.
All fine with that output at 3 ?
Regards
On Tue, Jan 3, 2012 at 11:29 AM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/3/12 9:14 AM, Ali Jawad wrote:
Hi Daniel
SeLinux is disabled, as I have played around with Children and got a stable startup at CHILDREN=3, at 4 and up I get the problem described earlier right away, so it seems that MySQL is throttling Kamailio ?
1- What is the effect of running 3 children only ?
the capacity of handling sip traffic is lowered. You should still be able to handle couple of hundred messages per second if you don't have slow sql/dns operations. For production, it is recommended to have at least 8 to be in safe side.
2- I know this is not a Kamailio thing, but any idea about what needs to be changed on MySQL server to accommodate more children ? No of connections is 500 conns out of which max 120 are utilized
So mysql is on a different server, right? Are you sure there is no other switch/router doing such throttling? Can you watch on both kamailio and mysql server to see if the connections are initiated from kamailio and reaching mysql? Maybe you can discover if mysql server get the tcp connection but refuses it -- not sure if mysql server can run in debug/verbose mode to get some log messages with hints about what is the problem.
If the tcp connection does not get to mysql server, then it is a throttling in mysql client library or on a device in between the servers.
If you do 'kamctl ps' when having children=3, how many kamailio processes are there (extra should be timers, ...)? Each should have one mysql connection, so it helps to see at which number the limit starts.
Cheers, Daniel
Regards
On Mon, Jan 2, 2012 at 1:31 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/2/12 12:24 PM, Ali Jawad wrote:
Hi Daniel Running on a LAN with no firewalls enabled,
limits can be from kernel, selinix, mysql server a.s.o. not only from firewalls.
will try you suggestion though first thing tomorrow morning, what are the implications of running children=1 ?
This will create only one worker process (not advisable for production in long term, just for testing) -- in this way there should be very few mysql connections opened, revealing if there is a problem or not with the number of connections and the short period to create them.
Cheers, Daniel
Thanks
On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
check also for limits in the system/firewall, maybe there are limits on new connections per second. Try with children=1 and see if it happens again, if not, then it is something that blocks many new connections in short time.
Cheers, Daniel
On 1/2/12 5:31 AM, Sammy Govind wrote:
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawadali.jawad@splendor.net wrote:
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards
On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govindgovoiper@gmail.com wrote: > > Hi, > forget the whole list of errors and just resolve the DB connectivity > between > the Kamailio server and DB server first . > >> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >> connect to MySQL server on 'xxxxxxxxx' (4) > > > try connecting to the remote DB server 'xxxxxxxxx' from linux shell > i.e > > # mysql -u<user> -p<passwd> -h 'xxxxxxxxx' > > Regards, > Sammy > > > On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawadali.jawad@splendor.net > wrote: >> >> Hi All >> I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is >> that Kamailio at times fails to start up for no obvious reasons, >> I.e. >> in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug >> log "level 2 " shows the following : >> >> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >> connect to MySQL server on 'xxxxxxxxx' (4) >> 27(24762) ERROR:<core> [db.c:303]: could not add connection to >> the >> pool27(24762) ERROR: group [group.c:211]: unable to connect to the >> database >> 27(24762) ERROR:<core> [sr_module.c:889]: init_mod_child(): Error >> while initializing module group >> (/usr/local/kamailio/lib/kamailio/modules_k/group.so) >> 27(24762) ERROR:<core> [pt.c:481]: ERROR: fork_tcp_process(): >> init_child failed for process 27, pid 24762, "tcp receiver child=1" >> >> and then >> >> >> 30(24765) :<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 >> 0(24735) ALERT:<core> [main.c:748]: child process 24762 exited >> normally, status=255 >> 0(24735) INFO:<core> [main.c:766]: INFO: terminating due to >> SIGCHLD >> >> The problem happens on 4 different servers, I did try to compile >> more >> than once, the servers are 32 Bit CentOS 5.6. The connection to the >> database is monitored and there are no issues with connecting to the >> DB "it works at times and times not for Kamailio so a user/pwd issue >> is not likely to cause this, manual cnnect worked", config file >> errors >> are not found. >> >> I did also check max no of connections on MySQL server which is set >> to >> 500 on the server and only 100 cons are used. The group module uses >> the same connection string as other modules which is predefined >> and >> there are other modules that access the db before the group module >> and >> those do not have a problem. >> >> Please advice. >> >> Regards >> >> _______________________________________________ >> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >> list >> sr-users@lists.sip-router.org >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users > > > > _______________________________________________ > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing > list > sr-users@lists.sip-router.org > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >
-- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net) Beirut, Lebanon Phone: +9611373725/ext 116 FAX: +9611375554
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
Hello,
all it is fine with the output. So it looks like the limit is somewhere above 25 connections.
As a try of making a workaround in kamailio side, could be a patch to add a global parameter to introduce some latency before forking a worker, so these processes won't be created all at a time, but one after the other.
However, would be good to find the real reason for such behavior. If the tcp connection gets to mysql server, maybe the question should be addressed to mysql forums/mailing lists.
Cheers, Daniel
On 1/3/12 10:48 AM, Ali Jawad wrote:
Hi I dont think a few hundred per second is enough I need to find out what is wrong and bring up no of children to 8. The MySQL server is at another host on the same LAN, as for the command kamctl ps see below
Process:: ID=0 PID=13949 Type=attendant Process:: ID=1 PID=13950 Type=udp receiver child=0 sock=xx.xx.xx.xx:5060 Process:: ID=2 PID=13951 Type=udp receiver child=1 sock=xx.xx.xx.xx:5060 Process:: ID=3 PID=13952 Type=udp receiver child=2 sock=xx.xx.xx.xx:5060 Process:: ID=4 PID=13953 Type=udp receiver child=0 sock=xx.xx.xx.xx:5061 Process:: ID=5 PID=13954 Type=udp receiver child=1 sock=xx.xx.xx.xx:5061 Process:: ID=6 PID=13955 Type=udp receiver child=2 sock=xx.xx.xx.xx:5061 Process:: ID=7 PID=13956 Type=udp receiver child=0 sock=xx.xx.xx.xx:443 Process:: ID=8 PID=13957 Type=udp receiver child=1 sock=xx.xx.xx.xx:443 Process:: ID=9 PID=13958 Type=udp receiver child=2 sock=xx.xx.xx.xx:443 Process:: ID=10 PID=13959 Type=udp receiver child=0 sock=xx.xx.xx.xx:8080 Process:: ID=11 PID=13960 Type=udp receiver child=1 sock=xx.xx.xx.xx:8080 Process:: ID=12 PID=13961 Type=udp receiver child=2 sock=xx.xx.xx.xx:8080 Process:: ID=13 PID=13962 Type=udp receiver child=0 sock=xx.xx.xx.xx:123 Process:: ID=14 PID=13963 Type=udp receiver child=1 sock=xx.xx.xx.xx:123 Process:: ID=15 PID=13964 Type=udp receiver child=2 sock=xx.xx.xx.xx:123 Process:: ID=16 PID=13965 Type=slow timer Process:: ID=17 PID=13966 Type=timer Process:: ID=18 PID=13967 Type=MI FIFO Process:: ID=19 PID=13968 Type=ctl handler Process:: ID=20 PID=13969 Type=TIMER NH Process:: ID=21 PID=13970 Type=tcp receiver child=0 Process:: ID=22 PID=13971 Type=tcp receiver child=1 Process:: ID=23 PID=13972 Type=tcp receiver child=2 Process:: ID=24 PID=13973 Type=tcp main process
I use a few different ports as you can see.
All fine with that output at 3 ?
Regards
On Tue, Jan 3, 2012 at 11:29 AM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/3/12 9:14 AM, Ali Jawad wrote:
Hi Daniel
SeLinux is disabled, as I have played around with Children and got a stable startup at CHILDREN=3, at 4 and up I get the problem described earlier right away, so it seems that MySQL is throttling Kamailio ?
1- What is the effect of running 3 children only ?
the capacity of handling sip traffic is lowered. You should still be able to handle couple of hundred messages per second if you don't have slow sql/dns operations. For production, it is recommended to have at least 8 to be in safe side.
2- I know this is not a Kamailio thing, but any idea about what needs to be changed on MySQL server to accommodate more children ? No of connections is 500 conns out of which max 120 are utilized
So mysql is on a different server, right? Are you sure there is no other switch/router doing such throttling? Can you watch on both kamailio and mysql server to see if the connections are initiated from kamailio and reaching mysql? Maybe you can discover if mysql server get the tcp connection but refuses it -- not sure if mysql server can run in debug/verbose mode to get some log messages with hints about what is the problem.
If the tcp connection does not get to mysql server, then it is a throttling in mysql client library or on a device in between the servers.
If you do 'kamctl ps' when having children=3, how many kamailio processes are there (extra should be timers, ...)? Each should have one mysql connection, so it helps to see at which number the limit starts.
Cheers, Daniel
Regards
On Mon, Jan 2, 2012 at 1:31 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/2/12 12:24 PM, Ali Jawad wrote:
Hi Daniel Running on a LAN with no firewalls enabled,
limits can be from kernel, selinix, mysql server a.s.o. not only from firewalls.
will try you suggestion though first thing tomorrow morning, what are the implications of running children=1 ?
This will create only one worker process (not advisable for production in long term, just for testing) -- in this way there should be very few mysql connections opened, revealing if there is a problem or not with the number of connections and the short period to create them.
Cheers, Daniel
Thanks
On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
check also for limits in the system/firewall, maybe there are limits on new connections per second. Try with children=1 and see if it happens again, if not, then it is something that blocks many new connections in short time.
Cheers, Daniel
On 1/2/12 5:31 AM, Sammy Govind wrote:
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawadali.jawad@splendor.net wrote: > Hi > Thanks for your input, but as said this works at times yes and times > now so I am not sure it is a definite DB issue, the servers are in the > same LAN, and there are other modules that use the DB string that work > when launched before this one while this one does not work. > Regards > > On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govindgovoiper@gmail.com > wrote: >> Hi, >> forget the whole list of errors and just resolve the DB connectivity >> between >> the Kamailio server and DB server first . >> >>> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >>> connect to MySQL server on 'xxxxxxxxx' (4) >> >> try connecting to the remote DB server 'xxxxxxxxx' from linux shell >> i.e >> >> # mysql -u<user> -p<passwd> -h 'xxxxxxxxx' >> >> Regards, >> Sammy >> >> >> On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawadali.jawad@splendor.net >> wrote: >>> Hi All >>> I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is >>> that Kamailio at times fails to start up for no obvious reasons, >>> I.e. >>> in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug >>> log "level 2 " shows the following : >>> >>> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >>> connect to MySQL server on 'xxxxxxxxx' (4) >>> 27(24762) ERROR:<core> [db.c:303]: could not add connection to >>> the >>> pool27(24762) ERROR: group [group.c:211]: unable to connect to the >>> database >>> 27(24762) ERROR:<core> [sr_module.c:889]: init_mod_child(): Error >>> while initializing module group >>> (/usr/local/kamailio/lib/kamailio/modules_k/group.so) >>> 27(24762) ERROR:<core> [pt.c:481]: ERROR: fork_tcp_process(): >>> init_child failed for process 27, pid 24762, "tcp receiver child=1" >>> >>> and then >>> >>> >>> 30(24765) :<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 >>> 0(24735) ALERT:<core> [main.c:748]: child process 24762 exited >>> normally, status=255 >>> 0(24735) INFO:<core> [main.c:766]: INFO: terminating due to >>> SIGCHLD >>> >>> The problem happens on 4 different servers, I did try to compile >>> more >>> than once, the servers are 32 Bit CentOS 5.6. The connection to the >>> database is monitored and there are no issues with connecting to the >>> DB "it works at times and times not for Kamailio so a user/pwd issue >>> is not likely to cause this, manual cnnect worked", config file >>> errors >>> are not found. >>> >>> I did also check max no of connections on MySQL server which is set >>> to >>> 500 on the server and only 100 cons are used. The group module uses >>> the same connection string as other modules which is predefined >>> and >>> there are other modules that access the db before the group module >>> and >>> those do not have a problem. >>> >>> Please advice. >>> >>> Regards >>> >>> _______________________________________________ >>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >>> list >>> sr-users@lists.sip-router.org >>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >> >> >> _______________________________________________ >> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >> list >> sr-users@lists.sip-router.org >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >> > -- > Ali Jawad > Information Systems Manager > Splendor Telecom (www.splendor.net) > Beirut, Lebanon > Phone: +9611373725/ext 116 > FAX: +9611375554 > > _______________________________________________ > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing > list > sr-users@lists.sip-router.org > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
Hello,
just another idea to try, not sure if it is convenient -- can you install and use different version of mysql server and client library? This should reveal whether is an issue specific to mysql version.
Cheers, Daniel
On 1/3/12 11:35 AM, Daniel-Constantin Mierla wrote:
Hello,
all it is fine with the output. So it looks like the limit is somewhere above 25 connections.
As a try of making a workaround in kamailio side, could be a patch to add a global parameter to introduce some latency before forking a worker, so these processes won't be created all at a time, but one after the other.
However, would be good to find the real reason for such behavior. If the tcp connection gets to mysql server, maybe the question should be addressed to mysql forums/mailing lists.
Cheers, Daniel
On 1/3/12 10:48 AM, Ali Jawad wrote:
Hi I dont think a few hundred per second is enough I need to find out what is wrong and bring up no of children to 8. The MySQL server is at another host on the same LAN, as for the command kamctl ps see below
Process:: ID=0 PID=13949 Type=attendant Process:: ID=1 PID=13950 Type=udp receiver child=0 sock=xx.xx.xx.xx:5060 Process:: ID=2 PID=13951 Type=udp receiver child=1 sock=xx.xx.xx.xx:5060 Process:: ID=3 PID=13952 Type=udp receiver child=2 sock=xx.xx.xx.xx:5060 Process:: ID=4 PID=13953 Type=udp receiver child=0 sock=xx.xx.xx.xx:5061 Process:: ID=5 PID=13954 Type=udp receiver child=1 sock=xx.xx.xx.xx:5061 Process:: ID=6 PID=13955 Type=udp receiver child=2 sock=xx.xx.xx.xx:5061 Process:: ID=7 PID=13956 Type=udp receiver child=0 sock=xx.xx.xx.xx:443 Process:: ID=8 PID=13957 Type=udp receiver child=1 sock=xx.xx.xx.xx:443 Process:: ID=9 PID=13958 Type=udp receiver child=2 sock=xx.xx.xx.xx:443 Process:: ID=10 PID=13959 Type=udp receiver child=0 sock=xx.xx.xx.xx:8080 Process:: ID=11 PID=13960 Type=udp receiver child=1 sock=xx.xx.xx.xx:8080 Process:: ID=12 PID=13961 Type=udp receiver child=2 sock=xx.xx.xx.xx:8080 Process:: ID=13 PID=13962 Type=udp receiver child=0 sock=xx.xx.xx.xx:123 Process:: ID=14 PID=13963 Type=udp receiver child=1 sock=xx.xx.xx.xx:123 Process:: ID=15 PID=13964 Type=udp receiver child=2 sock=xx.xx.xx.xx:123 Process:: ID=16 PID=13965 Type=slow timer Process:: ID=17 PID=13966 Type=timer Process:: ID=18 PID=13967 Type=MI FIFO Process:: ID=19 PID=13968 Type=ctl handler Process:: ID=20 PID=13969 Type=TIMER NH Process:: ID=21 PID=13970 Type=tcp receiver child=0 Process:: ID=22 PID=13971 Type=tcp receiver child=1 Process:: ID=23 PID=13972 Type=tcp receiver child=2 Process:: ID=24 PID=13973 Type=tcp main process
I use a few different ports as you can see.
All fine with that output at 3 ?
Regards
On Tue, Jan 3, 2012 at 11:29 AM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/3/12 9:14 AM, Ali Jawad wrote:
Hi Daniel
SeLinux is disabled, as I have played around with Children and got a stable startup at CHILDREN=3, at 4 and up I get the problem described earlier right away, so it seems that MySQL is throttling Kamailio ?
1- What is the effect of running 3 children only ?
the capacity of handling sip traffic is lowered. You should still be able to handle couple of hundred messages per second if you don't have slow sql/dns operations. For production, it is recommended to have at least 8 to be in safe side.
2- I know this is not a Kamailio thing, but any idea about what needs to be changed on MySQL server to accommodate more children ? No of connections is 500 conns out of which max 120 are utilized
So mysql is on a different server, right? Are you sure there is no other switch/router doing such throttling? Can you watch on both kamailio and mysql server to see if the connections are initiated from kamailio and reaching mysql? Maybe you can discover if mysql server get the tcp connection but refuses it -- not sure if mysql server can run in debug/verbose mode to get some log messages with hints about what is the problem.
If the tcp connection does not get to mysql server, then it is a throttling in mysql client library or on a device in between the servers.
If you do 'kamctl ps' when having children=3, how many kamailio processes are there (extra should be timers, ...)? Each should have one mysql connection, so it helps to see at which number the limit starts.
Cheers, Daniel
Regards
On Mon, Jan 2, 2012 at 1:31 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/2/12 12:24 PM, Ali Jawad wrote:
Hi Daniel Running on a LAN with no firewalls enabled,
limits can be from kernel, selinix, mysql server a.s.o. not only from firewalls.
will try you suggestion though first thing tomorrow morning, what are the implications of running children=1 ?
This will create only one worker process (not advisable for production in long term, just for testing) -- in this way there should be very few mysql connections opened, revealing if there is a problem or not with the number of connections and the short period to create them.
Cheers, Daniel
Thanks
On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla miconda@gmail.com wrote: > Hello, > > check also for limits in the system/firewall, maybe there are > limits on > new > connections per second. Try with children=1 and see if it happens > again, > if > not, then it is something that blocks many new connections in short > time. > > Cheers, > Daniel > > > On 1/2/12 5:31 AM, Sammy Govind wrote: > > Hi, > Its all upto you then, that's an intermittent issue and > information is > not > complete to trace the issue. Did you see the same Errors every > time it > fails > !? > > -- > Regards > Sammy > > On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawadali.jawad@splendor.net > wrote: >> Hi >> Thanks for your input, but as said this works at times yes and >> times >> now so I am not sure it is a definite DB issue, the servers are >> in the >> same LAN, and there are other modules that use the DB string >> that work >> when launched before this one while this one does not work. >> Regards >> >> On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govindgovoiper@gmail.com >> wrote: >>> Hi, >>> forget the whole list of errors and just resolve the DB >>> connectivity >>> between >>> the Kamailio server and DB server first . >>> >>>> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >>>> connect to MySQL server on 'xxxxxxxxx' (4) >>> >>> try connecting to the remote DB server 'xxxxxxxxx' from linux >>> shell >>> i.e >>> >>> # mysql -u<user> -p<passwd> -h 'xxxxxxxxx' >>> >>> Regards, >>> Sammy >>> >>> >>> On Fri, Dec 30, 2011 at 3:33 PM, Ali >>> Jawadali.jawad@splendor.net >>> wrote: >>>> Hi All >>>> I have tried the below with Kamailio 3.1.0 and 3.2.1. the >>>> problem is >>>> that Kamailio at times fails to start up for no obvious reasons, >>>> I.e. >>>> in 10 restarts Kamailio might fail to start 3,0 or 5 times. >>>> Te debug >>>> log "level 2 " shows the following : >>>> >>>> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >>>> connect to MySQL server on 'xxxxxxxxx' (4) >>>> 27(24762) ERROR:<core> [db.c:303]: could not add >>>> connection to >>>> the >>>> pool27(24762) ERROR: group [group.c:211]: unable to connect >>>> to the >>>> database >>>> 27(24762) ERROR:<core> [sr_module.c:889]: >>>> init_mod_child(): Error >>>> while initializing module group >>>> (/usr/local/kamailio/lib/kamailio/modules_k/group.so) >>>> 27(24762) ERROR:<core> [pt.c:481]: ERROR: >>>> fork_tcp_process(): >>>> init_child failed for process 27, pid 24762, "tcp receiver >>>> child=1" >>>> >>>> and then >>>> >>>> >>>> 30(24765) :<core> [pass_fd.c:293]: ERROR: receive_fd: >>>> EOF on 45 >>>> 0(24735) ALERT:<core> [main.c:748]: child process >>>> 24762 exited >>>> normally, status=255 >>>> 0(24735) INFO:<core> [main.c:766]: INFO: terminating >>>> due to >>>> SIGCHLD >>>> >>>> The problem happens on 4 different servers, I did try to compile >>>> more >>>> than once, the servers are 32 Bit CentOS 5.6. The connection >>>> to the >>>> database is monitored and there are no issues with connecting >>>> to the >>>> DB "it works at times and times not for Kamailio so a >>>> user/pwd issue >>>> is not likely to cause this, manual cnnect worked", config file >>>> errors >>>> are not found. >>>> >>>> I did also check max no of connections on MySQL server which >>>> is set >>>> to >>>> 500 on the server and only 100 cons are used. The group >>>> module uses >>>> the same connection string as other modules which is predefined >>>> and >>>> there are other modules that access the db before the group >>>> module >>>> and >>>> those do not have a problem. >>>> >>>> Please advice. >>>> >>>> Regards >>>> >>>> _______________________________________________ >>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users >>>> mailing >>>> list >>>> sr-users@lists.sip-router.org >>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>> >>> >>> _______________________________________________ >>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users >>> mailing >>> list >>> sr-users@lists.sip-router.org >>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>> >> -- >> Ali Jawad >> Information Systems Manager >> Splendor Telecom (www.splendor.net) >> Beirut, Lebanon >> Phone: +9611373725/ext 116 >> FAX: +9611375554 >> >> _______________________________________________ >> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >> list >> sr-users@lists.sip-router.org >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users > > > > _______________________________________________ > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users > mailing list > sr-users@lists.sip-router.org > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users > > > -- > Daniel-Constantin Mierla -- http://www.asipto.com > http://linkedin.com/in/miconda -- http://twitter.com/miconda > > > _______________________________________________ > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users > mailing list > sr-users@lists.sip-router.org > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
Same client library different server version working fine ! Any chance the patch can be released anytime soon ? I will migrate to the newer MySQL server in the next few mos though, I can wait if needed. Regards
On Tue, Jan 3, 2012 at 12:44 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
just another idea to try, not sure if it is convenient -- can you install and use different version of mysql server and client library? This should reveal whether is an issue specific to mysql version.
Cheers, Daniel
On 1/3/12 11:35 AM, Daniel-Constantin Mierla wrote:
Hello,
all it is fine with the output. So it looks like the limit is somewhere above 25 connections.
As a try of making a workaround in kamailio side, could be a patch to add a global parameter to introduce some latency before forking a worker, so these processes won't be created all at a time, but one after the other.
However, would be good to find the real reason for such behavior. If the tcp connection gets to mysql server, maybe the question should be addressed to mysql forums/mailing lists.
Cheers, Daniel
On 1/3/12 10:48 AM, Ali Jawad wrote:
Hi I dont think a few hundred per second is enough I need to find out what is wrong and bring up no of children to 8. The MySQL server is at another host on the same LAN, as for the command kamctl ps see below
Process:: ID=0 PID=13949 Type=attendant Process:: ID=1 PID=13950 Type=udp receiver child=0 sock=xx.xx.xx.xx:5060 Process:: ID=2 PID=13951 Type=udp receiver child=1 sock=xx.xx.xx.xx:5060 Process:: ID=3 PID=13952 Type=udp receiver child=2 sock=xx.xx.xx.xx:5060 Process:: ID=4 PID=13953 Type=udp receiver child=0 sock=xx.xx.xx.xx:5061 Process:: ID=5 PID=13954 Type=udp receiver child=1 sock=xx.xx.xx.xx:5061 Process:: ID=6 PID=13955 Type=udp receiver child=2 sock=xx.xx.xx.xx:5061 Process:: ID=7 PID=13956 Type=udp receiver child=0 sock=xx.xx.xx.xx:443 Process:: ID=8 PID=13957 Type=udp receiver child=1 sock=xx.xx.xx.xx:443 Process:: ID=9 PID=13958 Type=udp receiver child=2 sock=xx.xx.xx.xx:443 Process:: ID=10 PID=13959 Type=udp receiver child=0 sock=xx.xx.xx.xx:8080 Process:: ID=11 PID=13960 Type=udp receiver child=1 sock=xx.xx.xx.xx:8080 Process:: ID=12 PID=13961 Type=udp receiver child=2 sock=xx.xx.xx.xx:8080 Process:: ID=13 PID=13962 Type=udp receiver child=0 sock=xx.xx.xx.xx:123 Process:: ID=14 PID=13963 Type=udp receiver child=1 sock=xx.xx.xx.xx:123 Process:: ID=15 PID=13964 Type=udp receiver child=2 sock=xx.xx.xx.xx:123 Process:: ID=16 PID=13965 Type=slow timer Process:: ID=17 PID=13966 Type=timer Process:: ID=18 PID=13967 Type=MI FIFO Process:: ID=19 PID=13968 Type=ctl handler Process:: ID=20 PID=13969 Type=TIMER NH Process:: ID=21 PID=13970 Type=tcp receiver child=0 Process:: ID=22 PID=13971 Type=tcp receiver child=1 Process:: ID=23 PID=13972 Type=tcp receiver child=2 Process:: ID=24 PID=13973 Type=tcp main process
I use a few different ports as you can see.
All fine with that output at 3 ?
Regards
On Tue, Jan 3, 2012 at 11:29 AM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/3/12 9:14 AM, Ali Jawad wrote:
Hi Daniel
SeLinux is disabled, as I have played around with Children and got a stable startup at CHILDREN=3, at 4 and up I get the problem described earlier right away, so it seems that MySQL is throttling Kamailio ?
1- What is the effect of running 3 children only ?
the capacity of handling sip traffic is lowered. You should still be able to handle couple of hundred messages per second if you don't have slow sql/dns operations. For production, it is recommended to have at least 8 to be in safe side.
2- I know this is not a Kamailio thing, but any idea about what needs to be changed on MySQL server to accommodate more children ? No of connections is 500 conns out of which max 120 are utilized
So mysql is on a different server, right? Are you sure there is no other switch/router doing such throttling? Can you watch on both kamailio and mysql server to see if the connections are initiated from kamailio and reaching mysql? Maybe you can discover if mysql server get the tcp connection but refuses it -- not sure if mysql server can run in debug/verbose mode to get some log messages with hints about what is the problem.
If the tcp connection does not get to mysql server, then it is a throttling in mysql client library or on a device in between the servers.
If you do 'kamctl ps' when having children=3, how many kamailio processes are there (extra should be timers, ...)? Each should have one mysql connection, so it helps to see at which number the limit starts.
Cheers, Daniel
Regards
On Mon, Jan 2, 2012 at 1:31 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/2/12 12:24 PM, Ali Jawad wrote: > > Hi Daniel > Running on a LAN with no firewalls enabled,
limits can be from kernel, selinix, mysql server a.s.o. not only from firewalls.
> will try you suggestion > though first thing tomorrow morning, what are the implications of > running children=1 ?
This will create only one worker process (not advisable for production in long term, just for testing) -- in this way there should be very few mysql connections opened, revealing if there is a problem or not with the number of connections and the short period to create them.
Cheers, Daniel
> Thanks > > On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla > miconda@gmail.com wrote: >> >> Hello, >> >> check also for limits in the system/firewall, maybe there are limits >> on >> new >> connections per second. Try with children=1 and see if it happens >> again, >> if >> not, then it is something that blocks many new connections in short >> time. >> >> Cheers, >> Daniel >> >> >> On 1/2/12 5:31 AM, Sammy Govind wrote: >> >> Hi, >> Its all upto you then, that's an intermittent issue and information >> is >> not >> complete to trace the issue. Did you see the same Errors every time >> it >> fails >> !? >> >> -- >> Regards >> Sammy >> >> On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawadali.jawad@splendor.net >> wrote: >>> >>> Hi >>> Thanks for your input, but as said this works at times yes and >>> times >>> now so I am not sure it is a definite DB issue, the servers are in >>> the >>> same LAN, and there are other modules that use the DB string that >>> work >>> when launched before this one while this one does not work. >>> Regards >>> >>> On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govindgovoiper@gmail.com >>> wrote: >>>> >>>> Hi, >>>> forget the whole list of errors and just resolve the DB >>>> connectivity >>>> between >>>> the Kamailio server and DB server first . >>>> >>>>> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >>>>> connect to MySQL server on 'xxxxxxxxx' (4) >>>> >>>> >>>> try connecting to the remote DB server 'xxxxxxxxx' from linux >>>> shell >>>> i.e >>>> >>>> # mysql -u<user> -p<passwd> -h 'xxxxxxxxx' >>>> >>>> Regards, >>>> Sammy >>>> >>>> >>>> On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawadali.jawad@splendor.net >>>> wrote: >>>>> >>>>> Hi All >>>>> I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem >>>>> is >>>>> that Kamailio at times fails to start up for no obvious reasons, >>>>> I.e. >>>>> in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te >>>>> debug >>>>> log "level 2 " shows the following : >>>>> >>>>> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >>>>> connect to MySQL server on 'xxxxxxxxx' (4) >>>>> 27(24762) ERROR:<core> [db.c:303]: could not add connection >>>>> to >>>>> the >>>>> pool27(24762) ERROR: group [group.c:211]: unable to connect to >>>>> the >>>>> database >>>>> 27(24762) ERROR:<core> [sr_module.c:889]: init_mod_child(): >>>>> Error >>>>> while initializing module group >>>>> (/usr/local/kamailio/lib/kamailio/modules_k/group.so) >>>>> 27(24762) ERROR:<core> [pt.c:481]: ERROR: >>>>> fork_tcp_process(): >>>>> init_child failed for process 27, pid 24762, "tcp receiver >>>>> child=1" >>>>> >>>>> and then >>>>> >>>>> >>>>> 30(24765) :<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on >>>>> 45 >>>>> 0(24735) ALERT:<core> [main.c:748]: child process 24762 >>>>> exited >>>>> normally, status=255 >>>>> 0(24735) INFO:<core> [main.c:766]: INFO: terminating due to >>>>> SIGCHLD >>>>> >>>>> The problem happens on 4 different servers, I did try to compile >>>>> more >>>>> than once, the servers are 32 Bit CentOS 5.6. The connection to >>>>> the >>>>> database is monitored and there are no issues with connecting to >>>>> the >>>>> DB "it works at times and times not for Kamailio so a user/pwd >>>>> issue >>>>> is not likely to cause this, manual cnnect worked", config file >>>>> errors >>>>> are not found. >>>>> >>>>> I did also check max no of connections on MySQL server which is >>>>> set >>>>> to >>>>> 500 on the server and only 100 cons are used. The group module >>>>> uses >>>>> the same connection string as other modules which is predefined >>>>> and >>>>> there are other modules that access the db before the group >>>>> module >>>>> and >>>>> those do not have a problem. >>>>> >>>>> Please advice. >>>>> >>>>> Regards >>>>> >>>>> _______________________________________________ >>>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users >>>>> mailing >>>>> list >>>>> sr-users@lists.sip-router.org >>>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>>> >>>> >>>> >>>> _______________________________________________ >>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >>>> list >>>> sr-users@lists.sip-router.org >>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>>> >>> -- >>> Ali Jawad >>> Information Systems Manager >>> Splendor Telecom (www.splendor.net) >>> Beirut, Lebanon >>> Phone: +9611373725/ext 116 >>> FAX: +9611375554 >>> >>> _______________________________________________ >>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >>> list >>> sr-users@lists.sip-router.org >>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >> >> >> >> >> _______________________________________________ >> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >> list >> sr-users@lists.sip-router.org >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >> >> >> -- >> Daniel-Constantin Mierla -- http://www.asipto.com >> http://linkedin.com/in/miconda -- http://twitter.com/miconda >> >> >> _______________________________________________ >> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >> list >> sr-users@lists.sip-router.org >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>
Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
Hello,
On 1/3/12 11:58 AM, Ali Jawad wrote:
Same client library different server version working fine !
ok, good to know. What was the version of mysql sever giving troubles?
Any chance the patch can be released anytime soon ?
That was an idea, but it is not sure it will fix, it has to be tried. I will see when I have time for it and notify you to try it once is committed.
Cheers, Daniel
I will migrate to the newer MySQL server in the next few mos though, I can wait if needed. Regards
On Tue, Jan 3, 2012 at 12:44 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
just another idea to try, not sure if it is convenient -- can you install and use different version of mysql server and client library? This should reveal whether is an issue specific to mysql version.
Cheers, Daniel
On 1/3/12 11:35 AM, Daniel-Constantin Mierla wrote:
Hello,
all it is fine with the output. So it looks like the limit is somewhere above 25 connections.
As a try of making a workaround in kamailio side, could be a patch to add a global parameter to introduce some latency before forking a worker, so these processes won't be created all at a time, but one after the other.
However, would be good to find the real reason for such behavior. If the tcp connection gets to mysql server, maybe the question should be addressed to mysql forums/mailing lists.
Cheers, Daniel
On 1/3/12 10:48 AM, Ali Jawad wrote:
Hi I dont think a few hundred per second is enough I need to find out what is wrong and bring up no of children to 8. The MySQL server is at another host on the same LAN, as for the command kamctl ps see below
Process:: ID=0 PID=13949 Type=attendant Process:: ID=1 PID=13950 Type=udp receiver child=0 sock=xx.xx.xx.xx:5060 Process:: ID=2 PID=13951 Type=udp receiver child=1 sock=xx.xx.xx.xx:5060 Process:: ID=3 PID=13952 Type=udp receiver child=2 sock=xx.xx.xx.xx:5060 Process:: ID=4 PID=13953 Type=udp receiver child=0 sock=xx.xx.xx.xx:5061 Process:: ID=5 PID=13954 Type=udp receiver child=1 sock=xx.xx.xx.xx:5061 Process:: ID=6 PID=13955 Type=udp receiver child=2 sock=xx.xx.xx.xx:5061 Process:: ID=7 PID=13956 Type=udp receiver child=0 sock=xx.xx.xx.xx:443 Process:: ID=8 PID=13957 Type=udp receiver child=1 sock=xx.xx.xx.xx:443 Process:: ID=9 PID=13958 Type=udp receiver child=2 sock=xx.xx.xx.xx:443 Process:: ID=10 PID=13959 Type=udp receiver child=0 sock=xx.xx.xx.xx:8080 Process:: ID=11 PID=13960 Type=udp receiver child=1 sock=xx.xx.xx.xx:8080 Process:: ID=12 PID=13961 Type=udp receiver child=2 sock=xx.xx.xx.xx:8080 Process:: ID=13 PID=13962 Type=udp receiver child=0 sock=xx.xx.xx.xx:123 Process:: ID=14 PID=13963 Type=udp receiver child=1 sock=xx.xx.xx.xx:123 Process:: ID=15 PID=13964 Type=udp receiver child=2 sock=xx.xx.xx.xx:123 Process:: ID=16 PID=13965 Type=slow timer Process:: ID=17 PID=13966 Type=timer Process:: ID=18 PID=13967 Type=MI FIFO Process:: ID=19 PID=13968 Type=ctl handler Process:: ID=20 PID=13969 Type=TIMER NH Process:: ID=21 PID=13970 Type=tcp receiver child=0 Process:: ID=22 PID=13971 Type=tcp receiver child=1 Process:: ID=23 PID=13972 Type=tcp receiver child=2 Process:: ID=24 PID=13973 Type=tcp main process
I use a few different ports as you can see.
All fine with that output at 3 ?
Regards
On Tue, Jan 3, 2012 at 11:29 AM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/3/12 9:14 AM, Ali Jawad wrote:
Hi Daniel
SeLinux is disabled, as I have played around with Children and got a stable startup at CHILDREN=3, at 4 and up I get the problem described earlier right away, so it seems that MySQL is throttling Kamailio ?
1- What is the effect of running 3 children only ?
the capacity of handling sip traffic is lowered. You should still be able to handle couple of hundred messages per second if you don't have slow sql/dns operations. For production, it is recommended to have at least 8 to be in safe side.
2- I know this is not a Kamailio thing, but any idea about what needs to be changed on MySQL server to accommodate more children ? No of connections is 500 conns out of which max 120 are utilized
So mysql is on a different server, right? Are you sure there is no other switch/router doing such throttling? Can you watch on both kamailio and mysql server to see if the connections are initiated from kamailio and reaching mysql? Maybe you can discover if mysql server get the tcp connection but refuses it -- not sure if mysql server can run in debug/verbose mode to get some log messages with hints about what is the problem.
If the tcp connection does not get to mysql server, then it is a throttling in mysql client library or on a device in between the servers.
If you do 'kamctl ps' when having children=3, how many kamailio processes are there (extra should be timers, ...)? Each should have one mysql connection, so it helps to see at which number the limit starts.
Cheers, Daniel
Regards
On Mon, Jan 2, 2012 at 1:31 PM, Daniel-Constantin Mierla miconda@gmail.com wrote: > Hello, > > > On 1/2/12 12:24 PM, Ali Jawad wrote: >> Hi Daniel >> Running on a LAN with no firewalls enabled, > > limits can be from kernel, selinix, mysql server a.s.o. not only from > firewalls. > > >> will try you suggestion >> though first thing tomorrow morning, what are the implications of >> running children=1 ? > > This will create only one worker process (not advisable for production > in > long term, just for testing) -- in this way there should be very few > mysql > connections opened, revealing if there is a problem or not with the > number > of connections and the short period to create them. > > Cheers, > Daniel > >> Thanks >> >> On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla >> miconda@gmail.com wrote: >>> Hello, >>> >>> check also for limits in the system/firewall, maybe there are limits >>> on >>> new >>> connections per second. Try with children=1 and see if it happens >>> again, >>> if >>> not, then it is something that blocks many new connections in short >>> time. >>> >>> Cheers, >>> Daniel >>> >>> >>> On 1/2/12 5:31 AM, Sammy Govind wrote: >>> >>> Hi, >>> Its all upto you then, that's an intermittent issue and information >>> is >>> not >>> complete to trace the issue. Did you see the same Errors every time >>> it >>> fails >>> !? >>> >>> -- >>> Regards >>> Sammy >>> >>> On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawadali.jawad@splendor.net >>> wrote: >>>> Hi >>>> Thanks for your input, but as said this works at times yes and >>>> times >>>> now so I am not sure it is a definite DB issue, the servers are in >>>> the >>>> same LAN, and there are other modules that use the DB string that >>>> work >>>> when launched before this one while this one does not work. >>>> Regards >>>> >>>> On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govindgovoiper@gmail.com >>>> wrote: >>>>> Hi, >>>>> forget the whole list of errors and just resolve the DB >>>>> connectivity >>>>> between >>>>> the Kamailio server and DB server first . >>>>> >>>>>> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >>>>>> connect to MySQL server on 'xxxxxxxxx' (4) >>>>> >>>>> try connecting to the remote DB server 'xxxxxxxxx' from linux >>>>> shell >>>>> i.e >>>>> >>>>> # mysql -u<user> -p<passwd> -h 'xxxxxxxxx' >>>>> >>>>> Regards, >>>>> Sammy >>>>> >>>>> >>>>> On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawadali.jawad@splendor.net >>>>> wrote: >>>>>> Hi All >>>>>> I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem >>>>>> is >>>>>> that Kamailio at times fails to start up for no obvious reasons, >>>>>> I.e. >>>>>> in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te >>>>>> debug >>>>>> log "level 2 " shows the following : >>>>>> >>>>>> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't >>>>>> connect to MySQL server on 'xxxxxxxxx' (4) >>>>>> 27(24762) ERROR:<core> [db.c:303]: could not add connection >>>>>> to >>>>>> the >>>>>> pool27(24762) ERROR: group [group.c:211]: unable to connect to >>>>>> the >>>>>> database >>>>>> 27(24762) ERROR:<core> [sr_module.c:889]: init_mod_child(): >>>>>> Error >>>>>> while initializing module group >>>>>> (/usr/local/kamailio/lib/kamailio/modules_k/group.so) >>>>>> 27(24762) ERROR:<core> [pt.c:481]: ERROR: >>>>>> fork_tcp_process(): >>>>>> init_child failed for process 27, pid 24762, "tcp receiver >>>>>> child=1" >>>>>> >>>>>> and then >>>>>> >>>>>> >>>>>> 30(24765) :<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on >>>>>> 45 >>>>>> 0(24735) ALERT:<core> [main.c:748]: child process 24762 >>>>>> exited >>>>>> normally, status=255 >>>>>> 0(24735) INFO:<core> [main.c:766]: INFO: terminating due to >>>>>> SIGCHLD >>>>>> >>>>>> The problem happens on 4 different servers, I did try to compile >>>>>> more >>>>>> than once, the servers are 32 Bit CentOS 5.6. The connection to >>>>>> the >>>>>> database is monitored and there are no issues with connecting to >>>>>> the >>>>>> DB "it works at times and times not for Kamailio so a user/pwd >>>>>> issue >>>>>> is not likely to cause this, manual cnnect worked", config file >>>>>> errors >>>>>> are not found. >>>>>> >>>>>> I did also check max no of connections on MySQL server which is >>>>>> set >>>>>> to >>>>>> 500 on the server and only 100 cons are used. The group module >>>>>> uses >>>>>> the same connection string as other modules which is predefined >>>>>> and >>>>>> there are other modules that access the db before the group >>>>>> module >>>>>> and >>>>>> those do not have a problem. >>>>>> >>>>>> Please advice. >>>>>> >>>>>> Regards >>>>>> >>>>>> _______________________________________________ >>>>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users >>>>>> mailing >>>>>> list >>>>>> sr-users@lists.sip-router.org >>>>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>>>> >>>>> >>>>> _______________________________________________ >>>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >>>>> list >>>>> sr-users@lists.sip-router.org >>>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>>>> >>>> -- >>>> Ali Jawad >>>> Information Systems Manager >>>> Splendor Telecom (www.splendor.net) >>>> Beirut, Lebanon >>>> Phone: +9611373725/ext 116 >>>> FAX: +9611375554 >>>> >>>> _______________________________________________ >>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >>>> list >>>> sr-users@lists.sip-router.org >>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>> >>> >>> >>> _______________________________________________ >>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >>> list >>> sr-users@lists.sip-router.org >>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>> >>> >>> -- >>> Daniel-Constantin Mierla -- http://www.asipto.com >>> http://linkedin.com/in/miconda -- http://twitter.com/miconda >>> >>> >>> _______________________________________________ >>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >>> list >>> sr-users@lists.sip-router.org >>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>> > -- > Daniel-Constantin Mierla -- http://www.asipto.com > http://linkedin.com/in/miconda -- http://twitter.com/miconda >
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
You can try to run ngrep -qW byline port 3306 at kamailio server and see if there are any errors, you should also use tcpdump and see if there are any rejects on IP level.
On Tue, Jan 3, 2012 at 3:14 AM, Ali Jawad ali.jawad@splendor.net wrote:
Hi Daniel
SeLinux is disabled, as I have played around with Children and got a stable startup at CHILDREN=3, at 4 and up I get the problem described earlier right away, so it seems that MySQL is throttling Kamailio ?
1- What is the effect of running 3 children only ? 2- I know this is not a Kamailio thing, but any idea about what needs to be changed on MySQL server to accommodate more children ? No of connections is 500 conns out of which max 120 are utilized
Regards
On Mon, Jan 2, 2012 at 1:31 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
On 1/2/12 12:24 PM, Ali Jawad wrote:
Hi Daniel Running on a LAN with no firewalls enabled,
limits can be from kernel, selinix, mysql server a.s.o. not only from firewalls.
will try you suggestion though first thing tomorrow morning, what are the implications of running children=1 ?
This will create only one worker process (not advisable for production in long term, just for testing) -- in this way there should be very few
mysql
connections opened, revealing if there is a problem or not with the
number
of connections and the short period to create them.
Cheers, Daniel
Thanks
On Mon, Jan 2, 2012 at 2:03 PM, Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
check also for limits in the system/firewall, maybe there are limits on new connections per second. Try with children=1 and see if it happens
again,
if not, then it is something that blocks many new connections in short
time.
Cheers, Daniel
On 1/2/12 5:31 AM, Sammy Govind wrote:
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawadali.jawad@splendor.net wrote:
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards
On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govindgovoiper@gmail.com
wrote:
Hi, forget the whole list of errors and just resolve the DB connectivity between the Kamailio server and DB server first .
> 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't > connect to MySQL server on 'xxxxxxxxx' (4)
try connecting to the remote DB server 'xxxxxxxxx' from linux shell
i.e
# mysql -u<user> -p<passwd> -h 'xxxxxxxxx'
Regards, Sammy
On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawadali.jawad@splendor.net wrote: > > Hi All > I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is > that Kamailio at times fails to start up for no obvious reasons,
I.e.
> in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug > log "level 2 " shows the following : > > 27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't > connect to MySQL server on 'xxxxxxxxx' (4) > 27(24762) ERROR:<core> [db.c:303]: could not add connection to the > pool27(24762) ERROR: group [group.c:211]: unable to connect to the > database > 27(24762) ERROR:<core> [sr_module.c:889]: init_mod_child(): Error > while initializing module group > (/usr/local/kamailio/lib/kamailio/modules_k/group.so) > 27(24762) ERROR:<core> [pt.c:481]: ERROR: fork_tcp_process(): > init_child failed for process 27, pid 24762, "tcp receiver child=1" > > and then > > > 30(24765) :<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 > 0(24735) ALERT:<core> [main.c:748]: child process 24762 exited > normally, status=255 > 0(24735) INFO:<core> [main.c:766]: INFO: terminating due to
SIGCHLD
> > The problem happens on 4 different servers, I did try to compile
more
> than once, the servers are 32 Bit CentOS 5.6. The connection to the > database is monitored and there are no issues with connecting to the > DB "it works at times and times not for Kamailio so a user/pwd issue > is not likely to cause this, manual cnnect worked", config file
errors
> are not found. > > I did also check max no of connections on MySQL server which is set
to
> 500 on the server and only 100 cons are used. The group module uses > the same connection string as other modules which is predefined
and
> there are other modules that access the db before the group module
and
> those do not have a problem. > > Please advice. > > Regards > > _______________________________________________ > SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing > list > sr-users@lists.sip-router.org > http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing
list
sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net) Beirut, Lebanon Phone: +9611373725/ext 116 FAX: +9611375554
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing
list
sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com http://linkedin.com/in/miconda -- http://twitter.com/miconda
-- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net) Beirut, Lebanon Phone: +9611373725/ext 116 FAX: +9611375554
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
When it fails I see the same errors yes. Thanks !
On Mon, Jan 2, 2012 at 7:31 AM, Sammy Govind govoiper@gmail.com wrote:
Hi, Its all upto you then, that's an intermittent issue and information is not complete to trace the issue. Did you see the same Errors every time it fails !?
-- Regards Sammy
On Sun, Jan 1, 2012 at 10:06 PM, Ali Jawad ali.jawad@splendor.net wrote:
Hi Thanks for your input, but as said this works at times yes and times now so I am not sure it is a definite DB issue, the servers are in the same LAN, and there are other modules that use the DB string that work when launched before this one while this one does not work. Regards
On Sun, Jan 1, 2012 at 4:13 PM, Sammy Govind govoiper@gmail.com wrote:
Hi, forget the whole list of errors and just resolve the DB connectivity between the Kamailio server and DB server first .
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4)
try connecting to the remote DB server 'xxxxxxxxx' from linux shell i.e
# mysql -u<user> -p<passwd> -h 'xxxxxxxxx'
Regards, Sammy
On Fri, Dec 30, 2011 at 3:33 PM, Ali Jawad ali.jawad@splendor.net wrote:
Hi All I have tried the below with Kamailio 3.1.0 and 3.2.1. the problem is that Kamailio at times fails to start up for no obvious reasons, I.e. in 10 restarts Kamailio might fail to start 3,0 or 5 times. Te debug log "level 2 " shows the following :
27(24762) ERROR: db_mysql [km_my_con.c:109]: driver error: Can't connect to MySQL server on 'xxxxxxxxx' (4) 27(24762) ERROR: <core> [db.c:303]: could not add connection to the pool27(24762) ERROR: group [group.c:211]: unable to connect to the database 27(24762) ERROR: <core> [sr_module.c:889]: init_mod_child(): Error while initializing module group (/usr/local/kamailio/lib/kamailio/modules_k/group.so) 27(24762) ERROR: <core> [pt.c:481]: ERROR: fork_tcp_process(): init_child failed for process 27, pid 24762, "tcp receiver child=1"
and then
30(24765) : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 45 0(24735) ALERT: <core> [main.c:748]: child process 24762 exited normally, status=255 0(24735) INFO: <core> [main.c:766]: INFO: terminating due to SIGCHLD
The problem happens on 4 different servers, I did try to compile more than once, the servers are 32 Bit CentOS 5.6. The connection to the database is monitored and there are no issues with connecting to the DB "it works at times and times not for Kamailio so a user/pwd issue is not likely to cause this, manual cnnect worked", config file errors are not found.
I did also check max no of connections on MySQL server which is set to 500 on the server and only 100 cons are used. The group module uses the same connection string as other modules which is predefined and there are other modules that access the db before the group module and those do not have a problem.
Please advice.
Regards
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Ali Jawad Information Systems Manager Splendor Telecom (www.splendor.net) Beirut, Lebanon Phone: +9611373725/ext 116 FAX: +9611375554
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-
Ali Jawad -
Daniel-Constantin Mierla -
Sammy Govind -
Vitaly Nikolaev