20 Jan
2015
20 Jan
'15
11:52 p.m.
with latest master, i tried to add [group] to mysql db_urls and sqlops
urls, eg.
modparam("mtree", "db_url",
"mysql://user:pass@[sip_proxy]127.0.0.1/sip_proxy")
and in my.cfg the group was defined like this:
[sip_proxy]
ssl-ca=/etc/mysql/ca-cert.pem
ssl-cert=/etc/mysql/client-cert.pem
ssl-key=/etc/mysql/client-key.pem
the result was that when i start kamailio, it starts to run out of mysql
connections and dumps core no matter how high i set max_connections in
my.cfg:
Jan 21 00:47:33 siika /usr/bin/sip-proxy[30612]: ERROR: db_mysql [km_my_con.c:114]:
db_mysql_new_connection(): driver error: Too many connections
Jan 21 00:47:33 siika /usr/bin/sip-proxy[30612]: ERROR: <core> [db.c:314]:
db_do_init2(): could not add connection to the pool
Jan 21 00:47:33 siika /usr/bin/sip-proxy[30612]: ERROR: mtree [mtree_mod.c:346]:
mi_child_init(): failed to connect to database
Jan 21 00:47:33 siika /usr/bin/sip-proxy[30612]: ERROR: <core> [sr_module.c:900]:
init_mod_child(): Error while initializing module mtree
(/usr/lib/sip-proxy/modules/mtree.so)
without the [group] kamailio starts fine and does not complain about too
many connections.
is this a bug in group handling or something else?
-- juha
21 Jan
21 Jan
6:02 a.m.
when [group] thing didn't work, i added
ssl-ca=/etc/mysql/cacert.pem
to [client] section of my.cfg that kamailio according to db_mysql/README
is reading.
after that, kamailio started ok, but didn't use ssl for mysql queries.
what is it that i'm missing? has anyone succeeded in making kamailio to
query mysql server over ssl?
-- juha
9:52 p.m.
Juha Heinanen writes:
when [group] thing didn't work, i added
ssl-ca=/etc/mysql/cacert.pem
to [client] section of my.cfg that kamailio according to db_mysql/README
is reading.
after that, kamailio started ok, but didn't use ssl for mysql queries.
what is it that i'm missing? has anyone succeeded in making kamailio to
query mysql server over ssl?
based on zero responses, i guess the answer is "no". if so, that pretty
much prevents using kamailio in an environment where mysql service is
provided by a cloud service, such as amazon ec2.
should i put a note in db_mysql module README telling that we don't
currently know, which [client] params of my.cfg the module supports?
-- juha
10:25 p.m.
On 21/01/15 21:52, Juha Heinanen wrote:
Juha Heinanen writes:
This doesn't have to do with kamailio at all, it is mysql client
configuration in my.cfg.
Kamailio itself uses mysql_options() to set the group to be used:
- http://dev.mysql.com/doc/refman/5.5/en/mysql-options.html
It is nothing else that kamailio can do. So whatever mysql supports for
groups in my.cnf it is the mysql client library that deals with. Looking
at above documentation and kamailio code, the order is right, so all
seems fine in the code. If you cannot find something in mysql docs, it
is a lack on their side.
Therefore I don't think the kamailio docs need to be updated with your
remark -- again, that is about what mysql client library supports.
I would try to test with incremental number of children to see where the
limits is reached, then try to identify where that values is set. Also,
running mysql server in debug mode might help.
Cheers,
Daniel
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
when [group] thing didn't work, i added
ssl-ca=/etc/mysql/cacert.pem
to [client] section of my.cfg that kamailio according to db_mysql/README
is reading.
after that, kamailio started ok, but didn't use ssl for mysql queries.
what is it that i'm missing? has anyone succeeded in making kamailio to
query mysql server over ssl?
based on zero responses, i guess the answer is
"no". if so, that pretty
much prevents using kamailio in an environment where mysql service is
provided by a cloud service, such as amazon ec2.
should i put a note in db_mysql module README telling that we don't
currently know, which [client] params of my.cfg the module supports?
11:21 p.m.
Daniel-Constantin Mierla writes:
This doesn't have to do with kamailio at all, it
is mysql client
configuration in my.cfg.
Kamailio itself uses mysql_options() to set the group to be used:
- http://dev.mysql.com/doc/refman/5.5/en/mysql-options.html
It is nothing else that kamailio can do. So whatever mysql supports for
groups in my.cnf it is the mysql client library that deals with. Looking
at above documentation and kamailio code, the order is right, so all
seems fine in the code. If you cannot find something in mysql docs, it
is a lack on their side.
Therefore I don't think the kamailio docs need to be updated with your
remark -- again, that is about what mysql client library supports.
if i add line:
ssl-ca=/etc/mysql/ca-cert.pem
to [client] section of my.cfg and connect to the server with mysql cli
command
mysql -u root -p
i get:
mysql> show status like 'ssl_cipher';
+---------------+--------------------+
| Variable_name | Value |
+---------------+--------------------+
| Ssl_cipher | DHE-RSA-AES256-SHA |
+---------------+--------------------+
1 row in set (0.00 sec)
showing that ssl is used. without the ssl-ca line Value is empty
showing that ssl is not used.
when i execute in my kamailio config:
sql_query("sip_proxy", "show status like 'ssl_cipher'",
"result");
xlog("L_INFO", "**** got result <variable=$dbr(result=>[0,
0])/value=$dbr(result=>[0, 1])>\n");
sql_result_free("result");
i get:
Jan 22 00:05:53 siika /usr/bin/sip-proxy[6735]: INFO: **** got result
<variable=Ssl_cipher/value=>
showing that ssl is not used even when ssl-ca line is in [client]
section of my.cfg.
according to db_mysql README:
In addition to the given group, also the [client] section is read, in
the order given in my.cnf.
if that is true, why is ssl not used for queries as specified in my.cfg?
-- juha
11:43 p.m.
On 21/01/15 23:21, Juha Heinanen wrote:
Daniel-Constantin Mierla writes:
A book
indexed by google says that [client] section is also read along
with the custom group. The example is shown with Perl, but I guess the
same client library is behind. Same seems to be suggested in the perl docs:
-
http://search.cpan.org/~capttofu/DBD-mysql-4.029/lib/DBD/mysql.pm#Class_Met…
Now, those can be old, maybe newer versions of mysql client library
behave differently.
Have you tried to put all wanted options to the group?
Daniel
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
This doesn't have to do with kamailio at all,
it is mysql client
configuration in my.cfg.
Kamailio itself uses mysql_options() to set the group to be used:
- http://dev.mysql.com/doc/refman/5.5/en/mysql-options.html
It is nothing else that kamailio can do. So whatever mysql supports for
groups in my.cnf it is the mysql client library that deals with. Looking
at above documentation and kamailio code, the order is right, so all
seems fine in the code. If you cannot find something in mysql docs, it
is a lack on their side.
Therefore I don't think the kamailio docs need to be updated with your
remark -- again, that is about what mysql client library supports.
if i add line:
ssl-ca=/etc/mysql/ca-cert.pem
to [client] section of my.cfg and connect to the server with mysql cli
command
mysql -u root -p
i get:
mysql> show status like 'ssl_cipher';
+---------------+--------------------+
| Variable_name | Value |
+---------------+--------------------+
| Ssl_cipher | DHE-RSA-AES256-SHA |
+---------------+--------------------+
1 row in set (0.00 sec)
showing that ssl is used. without the ssl-ca line Value is empty
showing that ssl is not used.
when i execute in my kamailio config:
sql_query("sip_proxy", "show status like 'ssl_cipher'",
"result");
xlog("L_INFO", "**** got result <variable=$dbr(result=>[0,
0])/value=$dbr(result=>[0, 1])>\n");
sql_result_free("result");
i get:
Jan 22 00:05:53 siika /usr/bin/sip-proxy[6735]: INFO: **** got result
<variable=Ssl_cipher/value=>
showing that ssl is not used even when ssl-ca line is in [client]
section of my.cfg.
according to db_mysql README:
In addition to the given group, also the [client] section is read, in
the order given in my.cnf.
if that is true, why is ssl not used for queries as specified in my.cfg?
22 Jan
22 Jan
3:32 a.m.
Daniel-Constantin Mierla writes:
A book indexed by google says that [client] section is
also read along
with the custom group. The example is shown with Perl, but I guess the
same client library is behind. Same seems to be suggested in the perl
docs:
i gave up on using [group] because of kamailio didn't start. these
tests are thus done just using the default [client] group.
Now, those can be old, maybe newer versions of mysql
client library
behave differently.
i would expect mysql cli client from debian wheezy mysql-client-5.5
package to use libmysqlclient18 package, i.e., the same one my kamailio
is using.
Have you tried to put all wanted options to the group?
i have all wanted options in [client] group:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
ssl-ca=/etc/mysql/cacert.pem
according to mysql 5.5 ref man, ssl-ca option implies ssl option and
those are enough for mysql cli client to use ssl.
-- juha
11:12 p.m.
On 22/01/15 03:32, Juha Heinanen wrote:
What page did you read? That could be for mysql client cli tool, but
appears it doesn't apply to libmysql client -- at the bottom of the next
link, it says that a specific function has to be used in c code to set
the certificates and enable connecting via tls:
- http://dev.mysql.com/doc/refman/5.5/en/using-ssl-connections.html
Daniel
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Daniel-Constantin Mierla writes:
I checked the dependencies of packages on wheezy and libmysqlclient is
not listed for mysql-client, so the later is not using the lib.
A book indexed by google says that [client]
section is also read along
with the custom group. The example is shown with Perl, but I guess the
same client library is behind. Same seems to be suggested in the perl
docs:
i gave up on using [group] because of kamailio didn't start. these
tests are thus done just using the default [client] group.
Now, those can be old, maybe newer versions of
mysql client library
behave differently.
i would expect mysql cli client from debian wheezy
mysql-client-5.5
package to use libmysqlclient18 package, i.e., the same one my kamailio
is using. Have you tried to put all wanted options to the
group?
i have all wanted options in [client] group:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
ssl-ca=/etc/mysql/cacert.pem
according to mysql 5.5 ref man, ssl-ca option implies ssl option and
those are enough for mysql cli client to use ssl.
23 Jan
23 Jan
1:10 a.m.
Daniel-Constantin Mierla writes:
i don't know about page number but here is the text:
--ssl-ca=file_name
The path to a file in PEM format that contains a list of trusted SSL
certificate authorities. This option implies --ssl.
I checked the dependencies of packages on wheezy and
libmysqlclient is
not listed for mysql-client, so the later is not using the lib.
ok.
i have all
wanted options in [client] group:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
ssl-ca=/etc/mysql/cacert.pem
according to mysql 5.5 ref man, ssl-ca option implies ssl option and
those are enough for mysql cli client to use ssl.
What page did you read? That could be for mysql client cli tool, but
appears it doesn't apply to libmysql client -- at the bottom of the next
link, it says that a specific function has to be used in c code to set
the certificates and enable connecting via tls:
- http://dev.mysql.com/doc/refman/5.5/en/using-ssl-connections.html
here is the text:
The C API enables application programs to use SSL:
To establish a secure connection, use the mysql_ssl_set() C API
function to set the appropriate certificate options before calling
mysql_real_connect(). See Section 23.8.7.67, “mysql_ssl_set()”.
so if [client] has ssl-ca option, mysql_ssl_set() should be called?
-- juha
3:58 a.m.
i tried again with [group] and my sip proxy started with ssl stuff in
the group when i increase my.cfg max_connections from 200 to 400. i
then checked and the number of connections to mysql server was 362!
after that also queries were done over ssl.
now two problems remain:
- why ssl usage requires huge increase in number of db connections?
- why ssl is not used when i add ssl options to [client] instead of
[group]?
-- juha
25 Jan
25 Jan
10:31 a.m.
On 23 Jan 2015, at 03:58, Juha Heinanen <jh(a)tutpro.com> wrote:
- why ssl is not used when i add ssl options to
[client] instead of
[group]?
That is indeed strange.
I've started labbing a bit with this. I did not get the client to connect
with SSL unless I added ssl-cipher to the client like this
/usr/local/mysql/bin/mysql -h 127.0.0.1 -P 63306 --ssl
--ssl-cipher=DHE-RSA-AES256-SHA:AES128-SHA << EOF
SHOW STATUS LIKE 'Ssl_cipher';
EOF
The default for Mysql - even if you have the --ssl option is no ssl.
There's no way to really test the SSL configuration with common tools, since Mysql
adds a few packets
in front of the SSL negotiation. The server indicates SSL support and then the client
starts SSL.
http://dev.mysql.com/doc/internals/en/connection-phase-packets.html#packet-…
This opens up for a downgrade attack, much like STARTSSL in SMTP.
There's also no setting for version of SSL/TLS you want to use, only a CIPHER
setting.
Mysql and/or MariaDB really needs to take this seriously and change their way of handling
TLS.
The current implementation indicates that they needed a check-box option, but it was a
long
time ago and haven't been updated since.
There is an overhead, which is propably why many people don't use it (apart from the
weird
configuration). This presentation has some numbers.
http://www.yassl.com/files/yassl_securing_mysql.pdf
We can move the MySQL TLS settings into the db_mysql module, it seems like there are API
calls for
it. If so, the db_mysql module could perform the query above and verify that SSL is
active.
Right now, I would put a query using SQLOPS like you showed in an earlier mail in
htable::mod-init event route
and disable the server if I got the wrong answer.
/O
11:15 p.m.
Olle E. Johansson writes:
I've started labbing a bit with this. I did not
get the client to connect
with SSL unless I added ssl-cipher to the client like this
/usr/local/mysql/bin/mysql -h 127.0.0.1 -P 63306 --ssl
--ssl-cipher=DHE-RSA-AES256-SHA:AES128-SHA << EOF
SHOW STATUS LIKE 'Ssl_cipher';
EOF
i can connect to mysql server without any mysql cli options if i have
ssl-ca=/etc/mysql/ca-cert.pem
in [client] section of my.cfg:
# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1462
Server version: 5.5.40-0+wheezy1 (Debian)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current
input statement.
mysql> SHOW STATUS LIKE 'Ssl_cipher';
+---------------+--------------------+
| Variable_name | Value |
+---------------+--------------------+
| Ssl_cipher | DHE-RSA-AES256-SHA |
+---------------+--------------------+
1 row in set (0.00 sec)
did you try to connect to the server with ssl from kamailio using
[group]?
-- juha
22 Jan
22 Jan
9:47 a.m.
On 21 Jan 2015, at 21:52, Juha Heinanen <jh(a)tutpro.com> wrote:
Juha Heinanen writes:
We've seen reports of issues with Postgresql with TLS too, I don't know
what happened, but I think we need to focus on both and fix this.
There is a known geneal problem with libraries using OpenSSL - I don't know if
this has been looked at in Kamailio, but we did a fix in Asterisk a while ago.
If you have modules using libraries that use OpenSSL - like we have in
Curl, Mysql, Postgres and possibly other modules - as well as our own use in
the TLS module - there's a risk that OpenSSL gets initialized too many
times and bad things happen. ("Bad things" need to be defined here).
I think Kevin did a library trick with the linker so that Asterisk
catch these initialization calls first and use just one. Asterisk is
multithreaded and Kamailio is multiprocess, so I don't know how this
affects Kamailio or if we can get some inspiration by this fix.
Rambling a bit, but trying to point in some sort of general direction. :-)
I will put on my list to set up a lab with Mysql TLS connections and try.
/O
when [group] thing didn't work, i added
ssl-ca=/etc/mysql/cacert.pem
to [client] section of my.cfg that kamailio according to db_mysql/README
is reading.
after that, kamailio started ok, but didn't use ssl for mysql queries.
what is it that i'm missing? has anyone succeeded in making kamailio to
query mysql server over ssl?
based on zero responses, i guess the answer is "no". if so, that pretty
much prevents using kamailio in an environment where mysql service is
provided by a cloud service, such as amazon ec2.
should i put a note in db_mysql module README telling that we don't
currently know, which [client] params of my.cfg the module supports? 
4:51 p.m.
On Thu, Jan 22, 2015 at 2:47 AM, Olle E. Johansson <oej(a)edvina.net> wrote:
On 21 Jan 2015, at 21:52, Juha Heinanen <jh(a)tutpro.com> wrote:
Just chiming in to point out the magic module Olle is referring to:
http://svn.asterisk.org/svn/asterisk/trunk/main/libasteriskssl.c
For context, the peer review for the patch that fixed this issue is here:
https://reviewboard.asterisk.org/r/1006/
Although due to some issues in review board, part of the patch doesn't show
up (hence the link to the actual source).
Matt
--
Matthew Jordan
Digium, Inc. | Engineering Manager
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com & http://asterisk.org
Juha Heinanen writes:
We've seen reports of issues with Postgresql with TLS too, I don't know
what happened, but I think we need to focus on both and fix this.
There is a known geneal problem with libraries using OpenSSL - I don't
know if
this has been looked at in Kamailio, but we did a fix in Asterisk a while
ago.
If you have modules using libraries that use OpenSSL - like we have in
Curl, Mysql, Postgres and possibly other modules - as well as our own use
in
the TLS module - there's a risk that OpenSSL gets initialized too many
times and bad things happen. ("Bad things" need to be defined here).
I think Kevin did a library trick with the linker so that Asterisk
catch these initialization calls first and use just one. Asterisk is
multithreaded and Kamailio is multiprocess, so I don't know how this
affects Kamailio or if we can get some inspiration by this fix.
Rambling a bit, but trying to point in some sort of general direction. :-)
I will put on my list to set up a lab with Mysql TLS connections and try.
when [group] thing didn't work, i added
ssl-ca=/etc/mysql/cacert.pem
to [client] section of my.cfg that kamailio according to db_mysql/README
is reading.
after that, kamailio started ok, but didn't use ssl for mysql queries.
what is it that i'm missing? has anyone succeeded in making kamailio to
query mysql server over ssl?
based on zero responses, i guess the answer is "no". if so, that pretty
much prevents using kamailio in an environment where mysql service is
provided by a cloud service, such as amazon ec2.
should i put a note in db_mysql module README telling that we don't
currently know, which [client] params of my.cfg the module supports?
28 Jan
28 Jan
2:13 p.m.
On 22/01/15 16:51, Matthew Jordan wrote:
On Thu, Jan 22, 2015 at 2:47 AM, Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>> wrote:
On 21 Jan 2015, at 21:52, Juha Heinanen <jh(a)tutpro.com
<mailto:jh@tutpro.com>> wrote:
that
pretty
Interesting approach by temporarily replacing the libssl functions and
call the original ones later.
In this case it wouldn't have worked because the postgress module is
used to connect to database (e.g., some modules load records from
database during module initialization), so libssl must be properly
initialized to the phase of opening a tls connection.
I broke the initialization process in few phases that allows to get
libssl initialized before any other module and seems to work, based on
the feedback of who reported the issue.
MySQL issue was something different, related more to mysql client config.
Cheers,
Daniel
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Juha Heinanen writes:
> when [group] thing didn't work, i added
>
> ssl-ca=/etc/mysql/cacert.pem
>
> to [client] section of my.cfg that kamailio according to
db_mysql/README
> is reading.
>
> after that, kamailio started ok, but didn't use ssl for mysql
queries.
>
> what is it that i'm missing? has anyone succeeded in making
kamailio
to
query
mysql server over ssl?
based on zero responses, i guess the answer is "no". if so, much prevents using kamailio in an environment
where mysql
service is
provided by a cloud service, such as amazon ec2.
should i put a note in db_mysql module README telling that we don't
currently know, which [client] params of my.cfg the module supports?
We've seen reports of issues with Postgresql with TLS too, I don't
know
what happened, but I think we need to focus on both and fix this.
There is a known geneal problem with libraries using OpenSSL - I
don't know if
this has been looked at in Kamailio, but we did a fix in Asterisk
a while ago.
If you have modules using libraries that use OpenSSL - like we have in
Curl, Mysql, Postgres and possibly other modules - as well as our
own use in
the TLS module - there's a risk that OpenSSL gets initialized too many
times and bad things happen. ("Bad things" need to be defined here).
I think Kevin did a library trick with the linker so that Asterisk
catch these initialization calls first and use just one. Asterisk is
multithreaded and Kamailio is multiprocess, so I don't know how this
affects Kamailio or if we can get some inspiration by this fix.
Rambling a bit, but trying to point in some sort of general
direction. :-)
I will put on my list to set up a lab with Mysql TLS connections
and try.
Just chiming in to point out the magic module Olle is referring to:
http://svn.asterisk.org/svn/asterisk/trunk/main/libasteriskssl.c
For context, the peer review for the patch that fixed this issue is here:
https://reviewboard.asterisk.org/r/1006/
Although due to some issues in review board, part of the patch doesn't
show up (hence the link to the actual source).
21 Jan
21 Jan
8:42 a.m.
I haven't used the group functionality at all, saw it that Jan added
very long time ago (iirc), but didn't spend time to understand what implies.
However, if you get a core dump, we can look at the backtrace and see
where it crashes.
Cheers,
Daniel
On 20/01/15 23:52, Juha Heinanen wrote:
with latest master, i tried to add [group] to mysql
db_urls and sqlops
urls, eg.
modparam("mtree", "db_url",
"mysql://user:pass@[sip_proxy]127.0.0.1/sip_proxy")
and in my.cfg the group was defined like this:
[sip_proxy]
ssl-ca=/etc/mysql/ca-cert.pem
ssl-cert=/etc/mysql/client-cert.pem
ssl-key=/etc/mysql/client-key.pem
the result was that when i start kamailio, it starts to run out of mysql
connections and dumps core no matter how high i set max_connections in
my.cfg:
Jan 21 00:47:33 siika /usr/bin/sip-proxy[30612]: ERROR: db_mysql [km_my_con.c:114]:
db_mysql_new_connection(): driver error: Too many connections
Jan 21 00:47:33 siika /usr/bin/sip-proxy[30612]: ERROR: <core> [db.c:314]:
db_do_init2(): could not add connection to the pool
Jan 21 00:47:33 siika /usr/bin/sip-proxy[30612]: ERROR: mtree [mtree_mod.c:346]:
mi_child_init(): failed to connect to database
Jan 21 00:47:33 siika /usr/bin/sip-proxy[30612]: ERROR: <core> [sr_module.c:900]:
init_mod_child(): Error while initializing module mtree
(/usr/lib/sip-proxy/modules/mtree.so)
without the [group] kamailio starts fine and does not complain about too
many connections.
is this a bug in group handling or something else?
-- juha
_______________________________________________
sr-dev mailing list
sr-dev(a)lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
9:24 p.m.
Daniel-Constantin Mierla writes:
I haven't used the group functionality at all, saw
it that Jan added
very long time ago (iirc), but didn't spend time to understand what implies.
However, if you get a core dump, we can look at the backtrace and see
where it crashes.
i don't think the crash is relevant here, because before the crash i got
error messages about running out of connections.
-- juha
3400
days inactive
3408
days old
16 comments
4 participants
participants (4)
-
Daniel-Constantin Mierla -
Juha Heinanen -
Matthew Jordan -
Olle E. Johansson