Hello Daniel,
I followed your recommendation and upgraded to kamailio-5.3.3 but the
problem remains with this version.
So I applied this patch:
and
I run the tests again.
After applying the patch, I was no longer able to replicate the problem :)
Thank you for your great support.
Best Regards,
Virgílio Cunha
Em qua., 18 de mar. de 2020 às 18:22, Virgílio Cunha <
virgilio18vsc(a)gmail.com> escreveu:
Hi,
Sorry for using this topic for this question, but where can I find
Kamailio's release policy?
I needed to know how long the kamailio 5.3.x branch will maintain its
support.
Best Regards,
Virgílio Cunha
Em ter., 17 de mar. de 2020 às 11:03, Daniel-Constantin Mierla <
miconda(a)gmail.com> escreveu:
> Hello,
>
> if it is a runtime issue, then it is better to upgrade to a supported
> release series and try to reproduce it.
>
> Looking at startup/shutdown code is easier than the rest used during
> runtime by a module, but it is not the case here. So we may waste a lot of
> time to dig into it and discover it was fixed or the code in an older
> release was refactored not matching any longer what is now in stable
> releases.
>
> Cheers,
> Daniel
> On 17.03.20 11:35, Virgílio Cunha wrote:
>
> Hi,
>
> The problem does not just occur when kamailio is restarted. The restart
> was the way I found to reproduce the problem, because when a proxy is
> restarted, there is more traffic from DMQ messages, so the problem is more
> likely to occur.
>
> The full backtrace is following:
>
> GNU gdb (GDB) Red Hat Enterprise Linux (7.2-92.el6)
> Copyright (C) 2010 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
>
http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from
> /opt/ptin/kamailio-ksipadp/bin/ksipadp-proxy...Reading symbols from
> /usr/lib/debug/usr/sbin/kamailio.debug...done.
> done.
> [New Thread 506]
> Reading symbols from /lib64/libdl.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libdl.so.2
> Reading symbols from /lib64/libresolv.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libresolv.so.2
> Reading symbols from /lib64/libc.so.6...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libc.so.6
> Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/ld-linux-x86-64.so.2
> Reading symbols from /usr/lib64/kamailio/modules/db_postgres.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/db_postgres.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/db_postgres.so
> Reading symbols from /usr/lib64/libpq.so.5...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libpq.so.5
> Reading symbols from /usr/lib64/kamailio/libsrdb2.so.1...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/libsrdb2.so.1.0.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/libsrdb2.so.1
> Reading symbols from /usr/lib64/kamailio/libsrdb1.so.1...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/libsrdb1.so.1.0.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/libsrdb1.so.1
> Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libssl.so.10
> Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libcrypto.so.10
> Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libgssapi_krb5.so.2
> Reading symbols from /lib64/libldap_r-2.4.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libldap_r-2.4.so.2
> Reading symbols from /lib64/libpthread.so.0...(no debugging symbols
> found)...done.
> [Thread debugging using libthread_db enabled]
> Loaded symbols for /lib64/libpthread.so.0
> Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libkrb5.so.3
> Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libcom_err.so.2
> Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libk5crypto.so.3
> Reading symbols from /lib64/libz.so.1...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libz.so.1
> Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libkrb5support.so.0
> Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libkeyutils.so.1
> Reading symbols from /lib64/liblber-2.4.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/liblber-2.4.so.2
> Reading symbols from /usr/lib64/libsasl2.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libsasl2.so.2
> Reading symbols from /usr/lib64/libssl3.so...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libssl3.so
> Reading symbols from /usr/lib64/libsmime3.so...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libsmime3.so
> Reading symbols from /usr/lib64/libnss3.so...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libnss3.so
> Reading symbols from /usr/lib64/libnssutil3.so...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libnssutil3.so
> Reading symbols from /lib64/libplds4.so...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libplds4.so
> Reading symbols from /lib64/libplc4.so...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libplc4.so
> Reading symbols from /lib64/libnspr4.so...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libnspr4.so
> Reading symbols from /lib64/libselinux.so.1...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libselinux.so.1
> Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libcrypt.so.1
> Reading symbols from /lib64/librt.so.1...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/librt.so.1
> Reading symbols from /lib64/libfreebl3.so...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libfreebl3.so
> Reading symbols from /usr/lib64/kamailio/modules/ctl.so...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/modules/ctl.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/ctl.so
> Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libnss_files.so.2
> Reading symbols from /usr/lib64/kamailio/modules/kex.so...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/modules/kex.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/kex.so
> Reading symbols from /usr/lib64/kamailio/libsrutils.so.1...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/libsrutils.so.1.0.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/libsrutils.so.1
> Reading symbols from /lib64/libm.so.6...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libm.so.6
> Reading symbols from /usr/lib64/kamailio/modules/tm.so...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/modules/tm.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/tm.so
> Reading symbols from /usr/lib64/kamailio/modules/tmx.so...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/modules/tmx.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/tmx.so
> Reading symbols from /usr/lib64/kamailio/modules/corex.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/corex.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/corex.so
> Reading symbols from /usr/lib64/kamailio/modules/cfg_rpc.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/cfg_rpc.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/cfg_rpc.so
> Reading symbols from /usr/lib64/kamailio/modules/sl.so...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/modules/sl.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/sl.so
> Reading symbols from /usr/lib64/kamailio/modules/rr.so...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/modules/rr.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/rr.so
> Reading symbols from /usr/lib64/kamailio/modules/maxfwd.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/maxfwd.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/maxfwd.so
> Reading symbols from /usr/lib64/kamailio/modules/siputils.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/siputils.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/siputils.so
> Reading symbols from /usr/lib64/kamailio/modules/sanity.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/sanity.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/sanity.so
> Reading symbols from /usr/lib64/kamailio/modules/textops.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/textops.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/textops.so
> Reading symbols from /usr/lib64/kamailio/modules/textopsx.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/textopsx.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/textopsx.so
> Reading symbols from /usr/lib64/kamailio/modules/dmq.so...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/modules/dmq.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/dmq.so
> Reading symbols from /usr/lib64/kamailio/modules/htable.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/htable.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/htable.so
> Reading symbols from /usr/lib64/kamailio/modules/pv.so...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/modules/pv.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/pv.so
> Reading symbols from /usr/lib64/kamailio/modules/xlog.so...Reading
> symbols from /usr/lib/debug/usr/lib64/kamailio/modules/xlog.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/xlog.so
> Reading symbols from /usr/lib64/kamailio/modules/jsonrpcs.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/jsonrpcs.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/jsonrpcs.so
> Reading symbols from /usr/lib64/kamailio/modules/uac.so...Reading symbols
> from /usr/lib/debug/usr/lib64/kamailio/modules/uac.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/uac.so
> Reading symbols from
> /usr/lib64/kamailio/modules/uac_redirect.so...Reading symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/uac_redirect.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/uac_redirect.so
> Reading symbols from /usr/lib64/kamailio/modules/db_text.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/db_text.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/db_text.so
> Reading symbols from /usr/lib64/kamailio/modules/dispatcher.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/dispatcher.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/dispatcher.so
> Reading symbols from /usr/lib64/kamailio/modules/sqlops.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/sqlops.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/sqlops.so
> Reading symbols from /usr/lib64/kamailio/modules/regex.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/regex.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/regex.so
> Reading symbols from /lib64/libpcre.so.0...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libpcre.so.0
> Reading symbols from /usr/lib64/kamailio/modules/permissions.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/permissions.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/permissions.so
> Reading symbols from /usr/lib64/kamailio/modules/dialplan.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/dialplan.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/dialplan.so
> Reading symbols from /usr/lib64/kamailio/modules/mtree.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/mtree.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/mtree.so
> Reading symbols from /usr/lib64/kamailio/modules/cfgutils.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/cfgutils.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/cfgutils.so
> Reading symbols from /usr/lib64/kamailio/modules/statsd.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/statsd.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/statsd.so
> Reading symbols from /usr/lib64/kamailio/modules/app_lua.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/app_lua.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/app_lua.so
> Reading symbols from /usr/lib64/liblua-5.1.so...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/liblua-5.1.so
> Reading symbols from /usr/lib64/kamailio/modules/rtimer.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/rtimer.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/rtimer.so
> Reading symbols from /usr/lib64/kamailio/modules/mqueue.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/mqueue.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/mqueue.so
> Reading symbols from /usr/lib64/kamailio/modules/auth.so...Reading
> symbols from /usr/lib/debug/usr/lib64/kamailio/modules/auth.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/auth.so
> Reading symbols from /usr/lib64/kamailio/modules/usrloc.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/usrloc.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/usrloc.so
> Reading symbols from /usr/lib64/kamailio/modules/registrar.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/registrar.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/registrar.so
> Reading symbols from /usr/lib64/kamailio/modules/dmq_usrloc.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/dmq_usrloc.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/dmq_usrloc.so
> Reading symbols from /usr/lib64/kamailio/modules/xhttp.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/xhttp.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/xhttp.so
> Reading symbols from /usr/lib64/kamailio/modules/exec.so...Reading
> symbols from /usr/lib/debug/usr/lib64/kamailio/modules/exec.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/exec.so
> Reading symbols from /usr/lib64/kamailio/modules/http_client.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/http_client.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/http_client.so
> Reading symbols from /usr/lib64/libxml2.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libxml2.so.2
> Reading symbols from /usr/lib64/libcurl.so.4...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libcurl.so.4
> Reading symbols from /lib64/libidn.so.11...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libidn.so.11
> Reading symbols from /lib64/libldap-2.4.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libldap-2.4.so.2
> Reading symbols from /usr/lib64/libssh2.so.1...(no debugging symbols
> found)...done.
> Loaded symbols for /usr/lib64/libssh2.so.1
> Reading symbols from /usr/lib64/kamailio/modules/json.so...Reading
> symbols from /usr/lib/debug/usr/lib64/kamailio/modules/json.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/json.so
> Reading symbols from /lib64/libjson-c.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libjson-c.so.2
> Reading symbols from /usr/lib64/kamailio/modules/ipops.so...Reading
> symbols from
> /usr/lib/debug/usr/lib64/kamailio/modules/ipops.so.debug...done.
> done.
> Loaded symbols for /usr/lib64/kamailio/modules/ipops.so
> Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols
> found)...done.
> Loaded symbols for /lib64/libnss_dns.so.2
> Core was generated by `/opt/ptin/kamailio-ksipadp/bin/ksipadp-proxy -f
> /opt/ptin/kamailio-ksipadp/prox'.
> Program terminated with signal 11, Segmentation fault.
> #0 0x00007fefe58ea7c4 in atomic_get (v=0x0) at
> ../../core/atomic/atomic_common.h:66
> 66 return atomic_get_int(&(v->val));
> Missing separate debuginfos, use: debuginfo-install
> kamailio-ksipadp-2.8.0-1.ptin.el6.x86_64
> (gdb) bt full
> #0 0x00007fefe58ea7c4 in atomic_get (v=0x0) at
> ../../core/atomic/atomic_common.h:66
> No locals.
> #1 0x00007fefe58ee246 in job_queue_size (queue=0x0) at worker.c:254
> No locals.
> #2 0x00007fefe58ed920 in add_dmq_job (msg=0x7ff028b453f8,
> peer=0x7fefebcc4df8) at worker.c:184
> i = 1
> found_available = 0
> new_job = {f = 0x7fefe2613797 <usrloc_dmq_handle_msg>, msg =
> 0x7fefebce67c8, orig_peer = 0x7fefebcc4df8, next = 0x0, prev = 0x0}
> worker = 0x7fefe7a1d890
> cloned_msg = 0x7fefebce67c8
> cloned_msg_len = 3888
> __FUNCTION__ = "add_dmq_job"
> #3 0x00007fefe58dfec1 in dmq_handle_message (msg=0x7ff028b453f8,
> str1=0x0, str2=0x0) at message.c:66
> peer = 0x7fefebcc4df8
> __FUNCTION__ = "dmq_handle_message"
> #4 0x0000000000433729 in do_action (h=0x7ffc89e35670, a=0x7ff0285a9340,
> msg=0x7ff028b453f8) at core/action.c:1054
> ret = -5
> v = 1
> dst = {send_sock = 0x7ff0285a8810, to = {s = {sa_family = 21376,
> sa_data = "\343\211\374\177\000\000\370S\264(\360\177\000"}, sin = {
> sin_family = 21376, sin_port = 35299, sin_addr = {s_addr =
> 32764}, sin_zero = "\370S\264(\360\177\000"}, sin6 = {sin6_family = 21376,
> sin6_port = 35299, sin6_flowinfo = 32764, sin6_addr =
> {__in6_u = {__u6_addr8 =
"\370S\264(\360\177\000\000pV\343\211\374\177\000",
> __u6_addr16 = {21496, 10420, 32752, 0, 22128, 35299,
> 32764, 0}, __u6_addr32 = {682906616, 32752, 2313377392, 32764}}},
> sin6_scope_id = 1}}, id = 32764, proto = 0 '\000',
> send_flags = {f = 0, blst_imask = 0}}
> tmp = 0x389c422190 ""
> new_uri = 0x7c00000000 <Address 0x7c00000000 out of bounds>
> end = 0x6e00000077 <Address 0x6e00000077 out of bounds>
> crt = 0x389c98e120 ""
> cmd = 0x7ff0283219b0
> len = 0
> user = 0
> uri = {user = {s = 0x7ffc89e34a10 "@K\343\211\374\177", len =
> 4459394}, passwd = {s = 0x7ffc89e34e80 "\356Q\343\211\374\177",
> len = 682906616}, host = {s = 0x7ff0285a9020 "\177\001", len
> = -1981589904}, port = {s = 0x0, len = 1}, params = {
> s = 0x7ff028b453f8 "\017", len = 295169284}, sip_params = {s
> = 0x32 <Address 0x32 out of bounds>, len = 1073741824}, headers = {
> s = 0x2000000 <Address 0x2000000 out of bounds>, len =
> 4289104}, port_no = 60676, proto = 5047, type = 2312043322, flags =
> 2168712452,
> transport = {s = 0x0, len = -1675583518}, ttl = {s = 0x89e34e90
> <Address 0x89e34e90 out of bounds>, len = 35715777}, user_param = {
> s = 0xffffffff0000003c <Address 0xffffffff0000003c out of
> bounds>, len = -1671419584}, maddr = {s = 0x0, len = -1981592704}, method =
> {
> s = 0x389c603c48 "", len = -1671391960}, lr = {s = 0x0, len =
> -1981592544}, r2 = {s = 0x1dde9a0 " 9\336\001", len = 0}, gr = {
> s = 0x4 <Address 0x4 out of bounds>, len = 2479},
> transport_val = {s = 0x389c422190 "", len = 1}, ttl_val = {
> s = 0x7ffc89e34b40 "pM\343\211\374\177", len = 5683491},
> user_param_val = {s = 0x40e7e8 "\370\377\377\377\377\377\377\377@ݘ
> \234\070",
> len = -1671362728}, maddr_val = {s = 0x404130 "", len = 0},
> method_val = {s = 0x1000003b4 <Address 0x1000003b4 out of bounds>, len = 0},
> lr_val = {s = 0x7fefe7374b9a "%x%c%x%c", len = -1673386776},
> r2_val = {s = 0x7ffc89e34bd0 "", len = -1981592584}, gr_val = {
> ---Type <return> to continue, or q <return> to quit---
> s = 0x389c422190 "", len = 0}}
> next_hop = {user = {s = 0x1 <Address 0x1 out of bounds>, len =
> 40}, passwd = {s = 0x0, len = -1407107317}, host = {
> s = 0x2000000 <Address 0x2000000 out of bounds>, len =
> 31398688}, port = {s = 0x1dde9a0 " 9\336\001", len = 31320480}, params = {
> s = 0x7ffc89e34c20 "", len = -1981593152}, sip_params = {s =
> 0x7ffc89e34910 "\200N\343\211\374\177", len = 682906616}, headers = {
> s = 0x32 <Address 0x32 out of bounds>, len = 1073741824},
> port_no = 0, proto = 512, type = ERROR_URI_T, flags = 4289104, transport = {
> s = 0x7ffc89e34900 "\020J\343\211\374\177", len = 4457737},
> ttl = {s = 0x7ffc89e348f0 "", len = -1981593520}, user_param = {
> s = 0x7ffc89e34858 "\r", len = -1981593264}, maddr = {s =
> 0x7ffc89e34870 "", len = -1981593456}, method = {s = 0x7ffc89e348b0
"\r",
> len = -1981593392}, lr = {s = 0x8cd2d0 <Address 0x8cd2d0 out
> of bounds>, len = 13}, r2 = {
> s = 0x10100000002 <Address 0x10100000002 out of bounds>, len
> = -1}, gr = {s = 0x0, len = 0}, transport_val = {s = 0x0, len = 682906616},
> ttl_val = {s = 0x7ff0285a9020 "\177\001", len = -1981593296},
> user_param_val = {s = 0x7ffc89e34fa0 "\001\200\255\373\374\177", len = 0},
> maddr_val = {s = 0x10000000d <Address 0x10000000d out of
> bounds>, len = -1981591296}, method_val = {
> s = 0x7fef00000000 <Address 0x7fef00000000 out of bounds>,
> len = -1671151657}, lr_val = {s = 0x7fefe7374ba2 "", len = 682906616},
> r2_val = {s = 0x32 <Address 0x32 out of bounds>, len =
> 1073741824}, gr_val = {s = 0x2000000 <Address 0x2000000 out of bounds>,
> len = 4289104}}
> u = 0x389c4224e8
> port = 0
> dst_host = 0x89e34bd0
> i = 32764
> flags = 52
> avp = 0x189e35040
> st = {flags = 2313374752, id = 32764, name = {n = -1981593512, s
> = {s = 0x7ffc89e34858 "\r", len = -1981593152}, re = 0x7ffc89e34858},
> avp = 0x7ffc89e347a0}
> sct = 0x0
> sjt = 0x7ff028b453f8
> rve = 0x32
> mct = 0x40000000
> rv = 0x7ffc89e34c20
> rv1 = 0x2000000
> c1 = {cache_type = 2313374144, val_type = 32764, c = {avp_val =
> {n = 1, s = {s = 0x1 <Address 0x1 out of bounds>, len = 1}, re = 0x1}, pval
> = {
> rs = {s = 0x1 <Address 0x1 out of bounds>, len = 1}, ri =
> -1407115050, flags = 52}},
> i2s = "\000\000\000\000\000\000\000\000\274\224
> \254\064\000\000\000\240G\343\211\374\177"}
> s = {s = 0x1 <Address 0x1 out of bounds>, len = -1981593512}
> srevp = {0x7ffc89e34960, 0x34ac208a97}
> mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0,
> str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {
> type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s =
> 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u
> = {
> number = 0, string = 0x0, str = {s = 0x0, len = 0}, data =
> 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0,
> string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr =
> 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str
> = {
> s = 0x0, len = 0}, data = 0x0, attr = 0x0, select =
> 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0,
> len = 0},
> data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE,
> u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0,
> attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number
> = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0,
> select = 0x0}}}
> ---Type <return> to continue, or q <return> to quit---
> __FUNCTION__ = "do_action"
> #5 0x00000000004404b8 in run_actions (h=0x7ffc89e35670,
> a=0x7ff0285a9340, msg=0x7ff028b453f8) at core/action.c:1552
> t = 0x7ff0285a9340
> ret = -1
> mod = 0x1dde9a0
> ms = 176570657
> __FUNCTION__ = "run_actions"
> #6 0x0000000000433692 in do_action (h=0x7ffc89e35670, a=0x7ff0285a9668,
> msg=0x7ff028b453f8) at core/action.c:1045
> ret = 1
> v = 1
> dst = {send_sock = 0x7ffc89e35380, to = {s = {sa_family = 61042,
> sa_data = "I", '\000' <repeats 12 times>}, sin = {sin_family
= 61042,
> sin_port = 73, sin_addr = {s_addr = 0}, sin_zero =
> "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 61042, sin6_port =
> 73,
> sin6_flowinfo = 0, sin6_addr = {__in6_u = {__u6_addr8 =
> "\000\000\000\000\000\000\000\000\020\000\000\000\005\000\000",
__u6_addr16
> = {
> 0, 0, 0, 0, 16, 0, 5, 0}, __u6_addr32 = {0, 0, 16,
> 5}}}, sin6_scope_id = 673678288}}, id = 32752, proto = 0 '\000', send_flags
> = {
> f = 0, blst_imask = 32752}}
> tmp = 0x66307b "H\213E\270H\203\300\070\351O\003"
> new_uri = 0x7ff028a93048 ""
> end = 0x7ff028a93038 "\303l\256"
> crt = 0x2000000 <Address 0x2000000 out of bounds>
> cmd = 0x1773508
> len = 6791248
> user = 0
> uri = {user = {s = 0xae6ecd "", len = 11430897}, passwd = {s =
> 0x7c95f8 "core: core/parser/msg_parser.c", len = 682507016}, host = {
> s = 0xae6c43 "To: <sip:usrloc@10.112.103.64:5080>\r\nFrom:
<
>
sip:usrloc@10.112.101.128:5060>;tag=8a3534eb92149bbe647f599d354f2511-090e\r\nCSeq:
> 10 KDMQ\r\nCall-ID: 409f4ac5788a8cc4-24333(a)10.112.101.128\r\nContent-Length:
> 39"..., len = 11430924}, port = {s = 0x0, len = 682507072}, params = {
> s = 0x7ff028ae3c90 <incomplete sequence \350>, len = 0},
> sip_params = {s = 0x7ffc89e35330 "\246", len = 6697083}, headers = {
> s = 0x7c9170 "core", len = -1981590832}, port_no = 8, proto =
> 0, type = ERROR_URI_T, flags = 30415600, transport = {
> s = 0xa8428197 <Address 0xa8428197 out of bounds>, len =
> -1675583518}, ttl = {s = 0x3000000030 <Address 0x3000000030 out of bounds>,
> len = 44108294}, user_param = {s = 0x400000017 <Address
> 0x400000017 out of bounds>, len = 8176032}, maddr = {s = 0x0, len =
> -1981590448},
> method = {s = 0x389c603c48 "", len = 11431629}, lr = {s =
> 0x7ffc89e35450 "\220T\343\211\374\177", len = 6869803}, r2 = {
> s = 0x7cc190 "get_hdr_field", len = 6730416}, gr = {s =
> 0x7ffc89e35300 "", len = 673677328}, transport_val = {s = 0x389c422190
"",
> len = 673677328}, ttl_val = {s = 0x7ffc89e35350
> "\200S\343\211\374\177", len = -413652166}, user_param_val = {s = 0x40e7f9
> "", len = 0},
> maddr_val = {s = 0x2000000 <Address 0x2000000 out of bounds>,
> len = 24588552}, method_val = {
> s = 0x67a450
> "\030A\211\321A)\301D\211ȉ\302H\213\205(\377\377\377\211P \276\004", len =
> 8965880}, lr_val = {
> s = 0x8cd2d0 <Address 0x8cd2d0 out of bounds>, len = 8},
> r2_val = {s = 0xa6 <Address 0xa6 out of bounds>, len = 8136007}, gr_val = {
> s = 0x7c95f8 "core: core/parser/msg_parser.c", len =
> 682176488}}
> next_hop = {user = {s = 0x1e7374b9a <Address 0x1e7374b9a out of
> bounds>, len = 674472240}, passwd = {
> s = 0xffffffff <Address 0xffffffff out of bounds>, len =
> 4289104}, host = {s = 0x4e2cb1825 <Address 0x4e2cb1825 out of bounds>,
> len = -1667705792}, port = {s = 0x0, len = 1073741824},
> params = {s = 0x2000000 <Address 0x2000000 out of bounds>, len = 4289104},
> sip_params = {s = 0x7ffc89e35180 "\360Q\343\211\374\177", len =
> 6031673}, headers = {s = 0x7fefe2cb183f "", len = -1981591296},
> port_no = 23013, proto = 58022, type = 32751, flags =
> 2313376240, transport = {s = 0x7fefe7374b9a "%x%c%x%c", len =
-1981590944},
> ttl = {
> ---Type <return> to continue, or q <return> to quit---
> s = 0x1a <Address 0x1a out of bounds>, len = 4289104},
> user_param = {s = 0x7fefe2a659e5 "5e6a5a14-1fa-", len = -1670974014},
maddr
> = {
> s = 0x30fbad8001 <Address 0x30fbad8001 out of bounds>, len =
> -492414491}, method = {s = 0x7fefe2a659e5 "5e6a5a14-1fa-", len =
> -492414491},
> lr = {s = 0x7fefe2a659e5 "5e6a5a14-1fa-", len = -492414478}, r2
> = {s = 0x7fefe2a659ff "", len = -492414491}, gr = {s = 0x7fefe2a659ff
"",
> len = 0}, transport_val = {s = 0x0, len = 6730416}, ttl_val =
> {s = 0x7ffc89e351a0 "", len = 673677328}, user_param_val = {
> s = 0x2000000 <Address 0x2000000 out of bounds>, len =
> 673677328}, maddr_val = {s = 0x7ffc89e351f0 "Cl\256", len = -413652166},
> method_val = {s = 0x7ff028278010 "\001", len = 0}, lr_val = {s
> = 0x2000000 <Address 0x2000000 out of bounds>, len = 24589344}, r2_val = {
> s = 0x67a138 "", len = 8965088}, gr_val = {s = 0x8cd2d0
> <Address 0x8cd2d0 out of bounds>, len = 682507072}}
> u = 0x66273d
> port = 0
> dst_host = 0x40000000
> i = 28
> flags = 0
> avp = 0xae6cc7
> st = {flags = 31129600, id = 0, name = {n = 0, s = {s = 0x0, len
> = 93}, re = 0x0}, avp = 0x7ffc89e35080}
> sct = 0x7ffc89e35490
> sjt = 0x6943d3
> rve = 0x7ff0285a8808
> mct = 0x6a283b
> rv = 0x0
> rv1 = 0x7c9170
> c1 = {cache_type = 24, val_type = 48, c = {avp_val = {n =
> -1981590720, s = {s = 0x7ffc89e35340 "\370\225|", len = -1981590912},
> re = 0x7ffc89e35340}, pval = {rs = {s = 0x7ffc89e35340
> "\370\225|", len = -1981590912}, ri = 0, flags = 0}},
> i2s = '\000' <repeats 21 times>}
> s = {s = 0x7fefe2cb183f "", len = -490006491}
> srevp = {0x7fefe2cb1825, 0x7fefe2cb1832}
> mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0,
> str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {
> type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s =
> 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u
> = {
> number = 0, string = 0x0, str = {s = 0x0, len = 0}, data =
> 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0,
> string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr =
> 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str
> = {
> s = 0x0, len = 0}, data = 0x0, attr = 0x0, select =
> 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0,
> len = 0},
> data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE,
> u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0,
> attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number
> = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0,
> select = 0x0}}}
> __FUNCTION__ = "do_action"
> #7 0x00000000004404b8 in run_actions (h=0x7ffc89e35670,
> a=0x7ff0285a85d0, msg=0x7ff028b453f8) at core/action.c:1552
> t = 0x7ff0285a9668
> ret = 1
> mod = 0x417250
> ms = 176570657
> __FUNCTION__ = "run_actions"
> #8 0x0000000000440c4a in run_top_route (a=0x7ff0285a85d0,
> msg=0x7ff028b453f8, c=0x0) at core/action.c:1641
> ---Type <return> to continue, or q <return> to quit---
> ctx = {rec_lev = 2, run_flags = 0, last_retcode = 1, jmp_env =
> {{__jmpbuf = {140669451785720, -8516593617994257148, 50, 1073741824,
> 33554432,
> 4289104, -8516593618092823292, 8518532807277145348},
> __mask_was_saved = 0, __saved_mask = {__val = {140668355198191, 4302791168,
> 48,
> 48, 18446744073709551615, 140722621863712,
> 140668355198919, 50, 0, 4611686022755909632, 140669451785208,
> 140722621863776, 5814046,
> 6608344928, 140669451785208, 0}}}}}
> p = 0x7ffc89e35670
> ret = 682907128
> sfbk = 0
> #9 0x000000000052e6b9 in receive_msg (
> buf=0xae6bc0 "KDMQ sip:usrloc@10.112.103.64:5080 SIP/2.0\r\nVia:
> SIP/2.0/UDP 10.112.101.128;branch=z9hG4bK8791.0c350dd3", '0' <repeats
24
> times>, ".0\r\nTo: <sip:usrloc@10.112.103.64:5080>\r\nFrom: <
> sip:usrloc@10.112.101.128"..., len=781, rcv_info=0x7ffc89e35a70) at
> core/receive.c:264
> msg = 0x7ff028b453f8
> ctx = {rec_lev = -1981589208, run_flags = 32764, last_retcode =
> 8, jmp_env = {{__jmpbuf = {163, 4253893, 8100093, 140669451683568,
> 140722621864176, 4845170, 0, 21474836480},
> __mask_was_saved = 673677840, __saved_mask = {__val = {140668768878592,
> 140722621864448,
> 6697083, 8094080, 5501958, 243141751568,
> 140669476122624, 140722621864896, 4294967295, 88212724672, 1,
> 140668357584160, 0, 33554432,
> 1024, 6608345520}}}}}
> bctx = 0x7ffc89e359b0
> ret = 0
> stats_on = 0
> tvb = {tv_sec = 140722621864208, tv_usec = 8961768}
> tve = {tv_sec = 0, tv_usec = 243139749096}
> tz = {tz_minuteswest = 50, tz_dsttime = 0}
> diff = 0
> inb = {
> s = 0xae6bc0 "KDMQ sip:usrloc@10.112.103.64:5080
> SIP/2.0\r\nVia: SIP/2.0/UDP 10.112.101.128;branch=z9hG4bK8791.0c350dd3",
> '0' <repeats 24 times>, ".0\r\nTo:
<sip:usrloc@10.112.103.64:5080>\r\nFrom:
> <sip:usrloc@10.112.101.128"..., len = 781}
> netinfo = {data = {s = 0x7ffc00000001 <Address 0x7ffc00000001 out
> of bounds>, len = 0}, dst = 0x1, rcv = 0x389c422190}
> keng = 0x0
> __FUNCTION__ = "receive_msg"
> #10 0x00000000006456a3 in udp_rcv_loop () at core/udp_server.c:515
> len = 781
> buf = "KDMQ sip:usrloc@10.112.103.64:5080 SIP/2.0\r\nVia:
> SIP/2.0/UDP 10.112.101.128;branch=z9hG4bK8791.0c350dd3", '0' <repeats
24
> times>, ".0\r\nTo: <sip:usrloc@10.112.103.64:5080>\r\nFrom: <
> sip:usrloc@10.112.101.128:5060>;tag=8a3"...
> tmp = 0x100000000 <Address 0x100000000 out of bounds>
> from = 0x7ff028b2c728
> fromlen = 16
> ri = {src_ip = {af = 2, len = 4, u = {addrl = {140722462617610,
> 140722621864592}, addr32 = {2154131466, 32764, 2313378448, 32764}, addr16 =
> {
> 28682, 32869, 32764, 0, 23184, 35299, 32764, 0}, addr =
> "\npe\200\374\177\000\000\220Z\343\211\374\177\000"}}, dst_ip = {af = 2,
> len = 4, u = {addrl = {1080520714, 0}, addr32 = {1080520714,
> 0, 0, 0}, addr16 = {28682, 16487, 0, 0, 0, 0, 0, 0},
> addr = "\npg@", '\000' <repeats 11 times>}},
src_port =
> 5060, dst_port = 5080, proto_reserved1 = 0, proto_reserved2 = 0, src_su =
> {s = {
> sa_family = 2, sa_data =
> "\023\304\npe\200\000\000\000\000\000\000\000"}, sin = {sin_family = 2,
> sin_port = 50195, sin_addr = {
> s_addr = 2154131466}, sin_zero =
> "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2, sin6_port =
50195,
> ---Type <return> to continue, or q <return> to quit---
> sin6_flowinfo = 2154131466, sin6_addr = {__in6_u =
> {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0,
0,
> 0, 0},
> __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}},
> bind_address = 0x7ff02830fff0, proto = 1 '\001'}
> __FUNCTION__ = "udp_rcv_loop"
> #11 0x0000000000423c7d in main_loop () at main.c:1623
> i = 0
> pid = 0
> si = 0x7ff02830fff0
> si_desc = "udp receiver child=0 sock=10.112.103.64:5080\000\000\000\000
> _\343\211\374\177\000\000\062\000\000\000\000\000\000\000\000\000\000@
>
\000\000\000\000\000\000\000\002\000\000\000\000PrA\000\000\000\000\000PSn\234\070\000\000\000\060\000\000\000\060\000\000\000\020]\343\211\374\177\000\000@
> \\\343\211\374\177\000\000\000px\347\357\177\000"
> nrprocs = 4
> woneinit = 0
> __FUNCTION__ = "main_loop"
> #12 0x000000000042b0ce in main (argc=13, argv=0x7ffc89e36008) at
> main.c:2646
> cfg_stream = 0x1bb5010
> c = -1
> r = 0
> tmp = 0x7ffc89e36ef7 ""
> tmp_len = 32764
> port = -1981587768
> proto = 56
> options = 0x7434f8
> ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
> ret = -1
> seed = 4012862799
> rfd = 4
> debug_save = 0
> debug_flag = 0
> dont_fork_cnt = 0
> n_lst = 0x7ffc89e35f08
> p = 0x100000000 <Address 0x100000000 out of bounds>
> st = {st_dev = 64769, st_ino = 148592, st_nlink = 2, st_mode =
> 16877, st_uid = 498, st_gid = 498, __pad0 = 0, st_rdev = 0, st_size = 4096,
> st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec =
> 1576522467, tv_nsec = 972000117}, st_mtim = {tv_sec = 1519927576,
> tv_nsec = 668342155}, st_ctim = {tv_sec = 1584026268, tv_nsec
> = 165523404}, __unused = {0, 0, 0}}
> __FUNCTION__ = "main"
> (gdb)
>
> Em ter., 17 de mar. de 2020 às 09:52, Daniel-Constantin Mierla <
> miconda(a)gmail.com> escreveu:
>
>> Hello,
>>
>> does it happen only when doing restart of kamailio? Can you send the
>> full backtrace from gdb?
>>
>> Also, 5.0.x is no longer a supported release for quite some time, try to
>> upgrade to 5.2.x or even better to the latest one 5.3.x and see if you can
>> reproduce it with one of the supported releases, maybe it is something that
>> was already fixed.
>>
>> Cheers,
>> Daniel
>> On 17.03.20 10:32, Virgílio Cunha wrote:
>>
>> Hello,
>>
>> I have in production environment with 4 proxies using DMQ to sync
>> terminal registers and the DMQ configuration parameters is following:
>> loadmodule "dmq.so"
>> #modparam("dmq", "server_address",
"sip:MY_IP_ADDRESS:MY_PORT_ADDRESS")
>> modparam("dmq", "notification_address",
"DMQ_HOSTS")
>> modparam("dmq", "multi_notify", 1)
>> modparam("dmq", "num_workers", 4)
>>
>>
>> In the lab environment with the same configuration that in prod, I can
>> simulate this problem with 12000 terminal REGISTERS and, when doing several
>> restarts on a proxy, one of them is generated a coredump.
>> Analyzing the core, I can see that the problem is concurrency in the dmq
>> synchronization.
>>
>> Job_queue_size (...) is called with a valid *queue *and, but inside
>> this function, the *queue *is null.
>>
>>
>> (gdb) up
>> #1 0x00007fefe58ee246 in job_queue_size (queue=0x0) at worker.c:254
>> 254 return atomic_get(&queue->count);
>> (gdb) print queue
>> $1 = (job_queue_t *) 0x0
>> (gdb) up
>> #2 0x00007fefe58ed920 in add_dmq_job (msg=0x7ff028b453f8,
>> peer=0x7fefebcc4df8) at worker.c:184
>> 184 if(job_queue_size(workers[i].queue) == 0) {
>> (gdb) print i
>> $2 = 1
>> (gdb) print workers[i].queue
>> $3 = (job_queue_t *) 0x7fefebce49b0
>>
>> If I configure DMQ with num_workers=1 I can't reproduce this problem.
>> I'm using the kamailio 5.0.8 release.
>>
>> Is this problem known to you? What is the right way to solve it?
>>
>> Best Regards,
>> Virgílio Cunha
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing
Listsr-users@lists.kamailio.orghttps://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>> --
>> Daniel-Constantin Mierla --
www.asipto.comwww.twitter.com/miconda --
www.linkedin.com/in/miconda
>>
>> --
> Daniel-Constantin Mierla --
www.asipto.comwww.twitter.com/miconda --
www.linkedin.com/in/miconda
>
>