On Thu, 11 Oct 2012 12:37:23 +0200 Pedro Antonio Vico Solano pvsolano@amper.es wrote:
We are developing a solution based on the SQLite module for Kamailio and we've found some bugs/errors regarding it. We use kamailio 3.2.4 and SQLite 3.7.14.1. The errors are about this:
Thanks for the config examples.
To speed up things.
- Memory leaks on querys via sqlops. Kamailio main process leaks 4kB
each 3 queries (aprox.) and that memory is never freed. [Attached basic example]
Could you provide valgrind leak report on this?
- Segmentation fault when doing a SELECT COUNT(*) via sqlops.
[Attached basic example 2]
And gdb backtrace on this?
Thanks, Timo
Hello,
fyi, 3.2.x is packaged with memory debug on, so if you set memdbg and memlog paramters to a value equal or higher than debug, then you see all the malloc/free operations in the syslog. That could help to identify what is allocated/from where and it is not freed.
Also, you can get a summary at shut down if you set this parameter: https://www.kamailio.org/wiki/cookbooks/3.2.x/core#mem_summary
Cheers, Daniel
On 10/12/12 8:46 AM, Timo Teras wrote:
On Thu, 11 Oct 2012 12:37:23 +0200 Pedro Antonio Vico Solano pvsolano@amper.es wrote:
We are developing a solution based on the SQLite module for Kamailio and we've found some bugs/errors regarding it. We use kamailio 3.2.4 and SQLite 3.7.14.1. The errors are about this:
Thanks for the config examples.
To speed up things.
- Memory leaks on querys via sqlops. Kamailio main process leaks 4kB
each 3 queries (aprox.) and that memory is never freed. [Attached basic example]
Could you provide valgrind leak report on this?
- Segmentation fault when doing a SELECT COUNT(*) via sqlops.
[Attached basic example 2]
And gdb backtrace on this?
Thanks, Timo
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi,
just some more observations regarding the memory issue: Not the pkg-/ or shmem is leaking, but the process consumes more memory each time (as shown in a "ps" command). Maybe in some situations a sqlite3_free is missing?
Carsten
2012/10/12 Daniel-Constantin Mierla miconda@gmail.com:
Hello,
fyi, 3.2.x is packaged with memory debug on, so if you set memdbg and memlog paramters to a value equal or higher than debug, then you see all the malloc/free operations in the syslog. That could help to identify what is allocated/from where and it is not freed.
Also, you can get a summary at shut down if you set this parameter: https://www.kamailio.org/wiki/cookbooks/3.2.x/core#mem_summary
Cheers, Daniel
On 10/12/12 8:46 AM, Timo Teras wrote:
On Thu, 11 Oct 2012 12:37:23 +0200 Pedro Antonio Vico Solano pvsolano@amper.es wrote:
We are developing a solution based on the SQLite module for Kamailio and we've found some bugs/errors regarding it. We use kamailio 3.2.4 and SQLite 3.7.14.1. The errors are about this:
Thanks for the config examples.
To speed up things.
- Memory leaks on querys via sqlops. Kamailio main process leaks 4kB
each 3 queries (aprox.) and that memory is never freed. [Attached basic example]
Could you provide valgrind leak report on this?
- Segmentation fault when doing a SELECT COUNT(*) via sqlops.
[Attached basic example 2]
And gdb backtrace on this?
Thanks, Timo
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio Advanced Training, Berlin, Nov 5-8, 2012 - http://asipto.com/u/kat Kamailio Advanced Training, Miami, USA, Nov 12-14, 2012 - http://asipto.com/u/katu
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
On Thu, 11 Oct 2012 12:37:23 +0200 Pedro Antonio Vico Solano pvsolano@amper.es wrote:
Hello everyone,
We are developing a solution based on the SQLite module for Kamailio and we've found some bugs/errors regarding it. We use kamailio 3.2.4 and SQLite 3.7.14.1. The errors are about this:
- Memory leaks on querys via sqlops. Kamailio main process leaks 4kB
each 3 queries (aprox.) and that memory is never freed. [Attached basic example]
- Segmentation fault when doing a SELECT COUNT(*) via sqlops.
[Attached basic example 2]
Can you help us?
Does this help for the memory leak?
diff --git a/modules_k/db_sqlite/dbase.c b/modules_k/db_sqlite/dbase.c index 0b32e07..c1d3b71 100644 --- a/modules_k/db_sqlite/dbase.c +++ b/modules_k/db_sqlite/dbase.c @@ -543,7 +543,12 @@ int db_sqlite_update(const db1_con_t* _h, const db_key_t* _k, const db_op_t* _o,
int db_sqlite_raw_query(const db1_con_t* _h, const str* _s, db1_res_t** _r) { - return db_do_raw_query(_h, _s, _r, + int rc; + + rc = db_do_raw_query(_h, _s, _r, db_sqlite_submit_query, db_sqlite_store_result); + db_sqlite_cleanup_query(_h); + + return rc; }
Seems that all other database driver postpone the resource release to free_results - but there's no reason why we couldn't do this right away in sqlite. Since in sqlite free_results we just free cached result set.
Will take a look at the other bug later when I get the time to reproduce it.
-Timo
Hello,
Have you found the way to run SELECT COUNT(*) ? I use kamailio 3.2.4 with sqlite 3.7.5-1 and it crashes on such queries too.
As a workaround I use $dbr(res=>rows) with "SELECT some_field FROM table" query, but the bug is really exists.
Hello Timo,
We've done a basic test and it seems to solve the memory leak problem. We'll do exhaustive tests in a few days.
We'll keep you informed.
Thanks a lot, Pedro
De: Timo Teras timo.teras@iki.fi Para: "SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List" sr-users@lists.sip-router.org cc: pvsolano@amper.es Fecha: 15/10/2012 16:13 Asunto: Re: [SR-Users] db_sqlite bugs Enviado por: Timo Teräs timo.teras@gmail.com
On Thu, 11 Oct 2012 12:37:23 +0200 Pedro Antonio Vico Solano pvsolano@amper.es wrote:
Hello everyone,
We are developing a solution based on the SQLite module for Kamailio and we've found some bugs/errors regarding it. We use kamailio 3.2.4 and SQLite 3.7.14.1. The errors are about this:
- Memory leaks on querys via sqlops. Kamailio main process leaks 4kB
each 3 queries (aprox.) and that memory is never freed. [Attached basic example]
- Segmentation fault when doing a SELECT COUNT(*) via sqlops.
[Attached basic example 2]
Can you help us?
Does this help for the memory leak?
diff --git a/modules_k/db_sqlite/dbase.c b/modules_k/db_sqlite/dbase.c index 0b32e07..c1d3b71 100644 --- a/modules_k/db_sqlite/dbase.c +++ b/modules_k/db_sqlite/dbase.c @@ -543,7 +543,12 @@ int db_sqlite_update(const db1_con_t* _h, const db_key_t* _k, const db_op_t* _o,
int db_sqlite_raw_query(const db1_con_t* _h, const str* _s, db1_res_t** _r) {
return db_do_raw_query(_h, _s, _r,
int rc;rc = db_do_raw_query(_h, _s, _r, db_sqlite_submit_query, db_sqlite_store_result);db_sqlite_cleanup_query(_h);return rc;}
Seems that all other database driver postpone the resource release to free_results - but there's no reason why we couldn't do this right away in sqlite. Since in sqlite free_results we just free cached result set.
Will take a look at the other bug later when I get the time to reproduce it.
-Timo
ADVERTENCIA
Este mensaje y/o sus anexos, pueden contener información personal y confidencial cuyo uso, reproducción o distribución no autorizados están legalmente prohibidos. Por lo tanto, si Vd. no fuera su destinatario y, erróneamente, lo hubiera recibido, le rogamos que informe al remitente y lo borre de inmediato.
En cumplimiento de la Ley Orgánica 15/1999, de Protección de Datos de Carácter Personal le informamos de que su dirección de correo electrónico, así como sus datos personales y de empresa pasarán a formar parte de nuestro fichero de Gestión, y serán tratados con la única finalidad de mantenimiento de la relación adquirida con usted. Los datos personales que existen en nuestro poder están protegidos por nuestra Política de Seguridad, y no serán compartidos con ninguna otra empresa. Usted puede ejercitar los derechos de acceso, rectificación, cancelación y oposición dirigiéndose por escrito a la dirección arriba indicada.
This e-mail and its attachments may include confidential personal information which may be protected by any legal rules and cannot be used, copied, distributed or disclosed to any person without authorisation. If you are not the intended recipient and have received this e-mail by mistake, please advise the sender and erase it.
In compliance with the Spanish Organic Act 15/1999 on Personal Data Protection, we hereby inform you that your email address, as well as your personal and business information, will be included in our Management files and used solely for purposes corresponding to our commercial relationship. All personal data in our possession is protected by our Data Safety Policy and thus shall not be released to any other third party whatsoever. You may exercise your right to access, rectify, cancel and contest by writing to the address provided above.
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi,
On Mon, 29 Oct 2012 13:01:57 +0200 Vitaliy Aleksandrov vitalik.voip@gmail.com wrote:
Have you found the way to run SELECT COUNT(*) ? I use kamailio 3.2.4 with sqlite 3.7.5-1 and it crashes on such queries too.
As a workaround I use $dbr(res=>rows) with "SELECT some_field FROM table" query, but the bug is really exists.
I think I know why the crash happens. And it's easy to "workaround". However, I'm not sure what would be the proper fix for this. It seems that sqlite does not support reporting the column type for computed fields.
This implies that the db_sqlite driver cannot deduce the proper return value type either (I will likely just hard code it to INT now). However, this means that other kind of expressions - e.g. string manipulations - will not work then.
I guess we should first try decltype if available and fall back to first row column type. However, this will fail too if the first row has NULL value for the column.
The fundamental problem is that Kamailio DB API expects the driver to return stored type - which on sqlite is not fixed.
Will try to get something sketched up soonish.
-Timo
On Mon, 29 Oct 2012 16:07:56 +0200 Timo Teras timo.teras@iki.fi wrote:
I guess we should first try decltype if available and fall back to first row column type. However, this will fail too if the first row has NULL value for the column.
The fundamental problem is that Kamailio DB API expects the driver to return stored type - which on sqlite is not fixed.
Will try to get something sketched up soonish.
Can you try the attached patch if it fixes it? I compile tested it only, but I believe this is the proper thing to do.
Thanks, Timo
On Thu, 15 Nov 2012 15:00:24 +0100 Pedro Antonio Vico Solano pvsolano@amper.es wrote:
We've done more tests with both of your patches and it seems to work fine.
You can upload it to GIT.
Done.
Commit ecf95eb Wed Oct 17 09:00:14 fixes the mem leak. Commit 0920586 Thu Nov 15 16:11:41 fixes the computed field issue.
Miconda, what is the current policy for pushing fixes to the stable branches? Should I do it now (both are reported tested)? Or someone else? Which branches are currently maintained and need to be updated?
Thanks, Timo
Hello,
On 11/15/12 3:18 PM, Timo Teras wrote:
On Thu, 15 Nov 2012 15:00:24 +0100 Pedro Antonio Vico Solano pvsolano@amper.es wrote:
We've done more tests with both of your patches and it seems to work fine.
You can upload it to GIT.
Done.
Commit ecf95eb Wed Oct 17 09:00:14 fixes the mem leak. Commit 0920586 Thu Nov 15 16:11:41 fixes the computed field issue.
Miconda, what is the current policy for pushing fixes to the stable branches? Should I do it now (both are reported tested)? Or someone else? Which branches are currently maintained and need to be updated?
branches 3.3 and 3.2 must be updated - use git cherry-pick to get the commits from master branch, some guidelines that can help if needed, at:
- http://www.kamailio.org/wiki/devel/backporting-to-3.2.x#backporting_to_branc...
Cheers, Daniel
-
Carsten Bock -
Daniel-Constantin Mierla -
Pedro Antonio Vico Solano -
Timo Teras -
Vitaliy Aleksandrov