[SR-Users] Segmentation fault in DMQ module

Virgílio Cunha virgilio18vsc at gmail.com
Thu Mar 26 13:20:05 CET 2020


Hello Daniel,

I followed your recommendation and upgraded to kamailio-5.3.3 but the
problem remains with this version.
So I applied this patch:
https://github.com/kamailio/kamailio/commit/21bdbff049571ca22f1205a041c8b1162005d319
and
I run the tests again.
After applying the patch, I was no longer able to replicate the problem :)

Thank you for your great support.

Best Regards,
Virgílio Cunha

Em qua., 18 de mar. de 2020 às 18:22, Virgílio Cunha <
virgilio18vsc at gmail.com> escreveu:

> Hi,
>
> Sorry for using this topic for this question, but where can I find
> Kamailio's release policy?
> I needed to know how long the kamailio 5.3.x branch will maintain its
> support.
>
> Best Regards,
> Virgílio Cunha
>
> Em ter., 17 de mar. de 2020 às 11:03, Daniel-Constantin Mierla <
> miconda at gmail.com> escreveu:
>
>> Hello,
>>
>> if it is a runtime issue, then it is better to upgrade to a supported
>> release series and try to reproduce it.
>>
>> Looking at startup/shutdown code is easier than the rest used during
>> runtime by a module, but it is not the case here. So we may waste a lot of
>> time to dig into it and discover it was fixed or the code in an older
>> release was refactored not matching any longer what is now in stable
>> releases.
>>
>> Cheers,
>> Daniel
>> On 17.03.20 11:35, Virgílio Cunha wrote:
>>
>> Hi,
>>
>> The problem does not just occur when kamailio is restarted. The restart
>> was the way I found to reproduce the problem, because when a proxy is
>> restarted, there is more traffic from DMQ messages, so the problem is more
>> likely to occur.
>>
>> The full backtrace is following:
>>
>> GNU gdb (GDB) Red Hat Enterprise Linux (7.2-92.el6)
>> Copyright (C) 2010 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later <
>> http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "x86_64-redhat-linux-gnu".
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>...
>> Reading symbols from
>> /opt/ptin/kamailio-ksipadp/bin/ksipadp-proxy...Reading symbols from
>> /usr/lib/debug/usr/sbin/kamailio.debug...done.
>> done.
>> [New Thread 506]
>> Reading symbols from /lib64/libdl.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libdl.so.2
>> Reading symbols from /lib64/libresolv.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libresolv.so.2
>> Reading symbols from /lib64/libc.so.6...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libc.so.6
>> Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/ld-linux-x86-64.so.2
>> Reading symbols from /usr/lib64/kamailio/modules/db_postgres.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/db_postgres.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/db_postgres.so
>> Reading symbols from /usr/lib64/libpq.so.5...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libpq.so.5
>> Reading symbols from /usr/lib64/kamailio/libsrdb2.so.1...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/libsrdb2.so.1.0.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/libsrdb2.so.1
>> Reading symbols from /usr/lib64/kamailio/libsrdb1.so.1...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/libsrdb1.so.1.0.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/libsrdb1.so.1
>> Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libssl.so.10
>> Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libcrypto.so.10
>> Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libgssapi_krb5.so.2
>> Reading symbols from /lib64/libldap_r-2.4.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libldap_r-2.4.so.2
>> Reading symbols from /lib64/libpthread.so.0...(no debugging symbols
>> found)...done.
>> [Thread debugging using libthread_db enabled]
>> Loaded symbols for /lib64/libpthread.so.0
>> Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libkrb5.so.3
>> Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libcom_err.so.2
>> Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libk5crypto.so.3
>> Reading symbols from /lib64/libz.so.1...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libz.so.1
>> Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libkrb5support.so.0
>> Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libkeyutils.so.1
>> Reading symbols from /lib64/liblber-2.4.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/liblber-2.4.so.2
>> Reading symbols from /usr/lib64/libsasl2.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libsasl2.so.2
>> Reading symbols from /usr/lib64/libssl3.so...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libssl3.so
>> Reading symbols from /usr/lib64/libsmime3.so...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libsmime3.so
>> Reading symbols from /usr/lib64/libnss3.so...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libnss3.so
>> Reading symbols from /usr/lib64/libnssutil3.so...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libnssutil3.so
>> Reading symbols from /lib64/libplds4.so...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libplds4.so
>> Reading symbols from /lib64/libplc4.so...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libplc4.so
>> Reading symbols from /lib64/libnspr4.so...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libnspr4.so
>> Reading symbols from /lib64/libselinux.so.1...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libselinux.so.1
>> Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libcrypt.so.1
>> Reading symbols from /lib64/librt.so.1...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/librt.so.1
>> Reading symbols from /lib64/libfreebl3.so...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libfreebl3.so
>> Reading symbols from /usr/lib64/kamailio/modules/ctl.so...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/modules/ctl.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/ctl.so
>> Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libnss_files.so.2
>> Reading symbols from /usr/lib64/kamailio/modules/kex.so...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/modules/kex.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/kex.so
>> Reading symbols from /usr/lib64/kamailio/libsrutils.so.1...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/libsrutils.so.1.0.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/libsrutils.so.1
>> Reading symbols from /lib64/libm.so.6...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libm.so.6
>> Reading symbols from /usr/lib64/kamailio/modules/tm.so...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/modules/tm.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/tm.so
>> Reading symbols from /usr/lib64/kamailio/modules/tmx.so...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/modules/tmx.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/tmx.so
>> Reading symbols from /usr/lib64/kamailio/modules/corex.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/corex.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/corex.so
>> Reading symbols from /usr/lib64/kamailio/modules/cfg_rpc.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/cfg_rpc.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/cfg_rpc.so
>> Reading symbols from /usr/lib64/kamailio/modules/sl.so...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/modules/sl.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/sl.so
>> Reading symbols from /usr/lib64/kamailio/modules/rr.so...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/modules/rr.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/rr.so
>> Reading symbols from /usr/lib64/kamailio/modules/maxfwd.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/maxfwd.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/maxfwd.so
>> Reading symbols from /usr/lib64/kamailio/modules/siputils.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/siputils.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/siputils.so
>> Reading symbols from /usr/lib64/kamailio/modules/sanity.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/sanity.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/sanity.so
>> Reading symbols from /usr/lib64/kamailio/modules/textops.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/textops.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/textops.so
>> Reading symbols from /usr/lib64/kamailio/modules/textopsx.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/textopsx.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/textopsx.so
>> Reading symbols from /usr/lib64/kamailio/modules/dmq.so...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/modules/dmq.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/dmq.so
>> Reading symbols from /usr/lib64/kamailio/modules/htable.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/htable.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/htable.so
>> Reading symbols from /usr/lib64/kamailio/modules/pv.so...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/modules/pv.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/pv.so
>> Reading symbols from /usr/lib64/kamailio/modules/xlog.so...Reading
>> symbols from /usr/lib/debug/usr/lib64/kamailio/modules/xlog.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/xlog.so
>> Reading symbols from /usr/lib64/kamailio/modules/jsonrpcs.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/jsonrpcs.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/jsonrpcs.so
>> Reading symbols from /usr/lib64/kamailio/modules/uac.so...Reading symbols
>> from /usr/lib/debug/usr/lib64/kamailio/modules/uac.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/uac.so
>> Reading symbols from
>> /usr/lib64/kamailio/modules/uac_redirect.so...Reading symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/uac_redirect.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/uac_redirect.so
>> Reading symbols from /usr/lib64/kamailio/modules/db_text.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/db_text.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/db_text.so
>> Reading symbols from /usr/lib64/kamailio/modules/dispatcher.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/dispatcher.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/dispatcher.so
>> Reading symbols from /usr/lib64/kamailio/modules/sqlops.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/sqlops.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/sqlops.so
>> Reading symbols from /usr/lib64/kamailio/modules/regex.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/regex.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/regex.so
>> Reading symbols from /lib64/libpcre.so.0...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libpcre.so.0
>> Reading symbols from /usr/lib64/kamailio/modules/permissions.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/permissions.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/permissions.so
>> Reading symbols from /usr/lib64/kamailio/modules/dialplan.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/dialplan.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/dialplan.so
>> Reading symbols from /usr/lib64/kamailio/modules/mtree.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/mtree.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/mtree.so
>> Reading symbols from /usr/lib64/kamailio/modules/cfgutils.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/cfgutils.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/cfgutils.so
>> Reading symbols from /usr/lib64/kamailio/modules/statsd.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/statsd.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/statsd.so
>> Reading symbols from /usr/lib64/kamailio/modules/app_lua.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/app_lua.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/app_lua.so
>> Reading symbols from /usr/lib64/liblua-5.1.so...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/liblua-5.1.so
>> Reading symbols from /usr/lib64/kamailio/modules/rtimer.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/rtimer.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/rtimer.so
>> Reading symbols from /usr/lib64/kamailio/modules/mqueue.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/mqueue.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/mqueue.so
>> Reading symbols from /usr/lib64/kamailio/modules/auth.so...Reading
>> symbols from /usr/lib/debug/usr/lib64/kamailio/modules/auth.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/auth.so
>> Reading symbols from /usr/lib64/kamailio/modules/usrloc.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/usrloc.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/usrloc.so
>> Reading symbols from /usr/lib64/kamailio/modules/registrar.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/registrar.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/registrar.so
>> Reading symbols from /usr/lib64/kamailio/modules/dmq_usrloc.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/dmq_usrloc.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/dmq_usrloc.so
>> Reading symbols from /usr/lib64/kamailio/modules/xhttp.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/xhttp.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/xhttp.so
>> Reading symbols from /usr/lib64/kamailio/modules/exec.so...Reading
>> symbols from /usr/lib/debug/usr/lib64/kamailio/modules/exec.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/exec.so
>> Reading symbols from /usr/lib64/kamailio/modules/http_client.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/http_client.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/http_client.so
>> Reading symbols from /usr/lib64/libxml2.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libxml2.so.2
>> Reading symbols from /usr/lib64/libcurl.so.4...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libcurl.so.4
>> Reading symbols from /lib64/libidn.so.11...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libidn.so.11
>> Reading symbols from /lib64/libldap-2.4.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libldap-2.4.so.2
>> Reading symbols from /usr/lib64/libssh2.so.1...(no debugging symbols
>> found)...done.
>> Loaded symbols for /usr/lib64/libssh2.so.1
>> Reading symbols from /usr/lib64/kamailio/modules/json.so...Reading
>> symbols from /usr/lib/debug/usr/lib64/kamailio/modules/json.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/json.so
>> Reading symbols from /lib64/libjson-c.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libjson-c.so.2
>> Reading symbols from /usr/lib64/kamailio/modules/ipops.so...Reading
>> symbols from
>> /usr/lib/debug/usr/lib64/kamailio/modules/ipops.so.debug...done.
>> done.
>> Loaded symbols for /usr/lib64/kamailio/modules/ipops.so
>> Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols
>> found)...done.
>> Loaded symbols for /lib64/libnss_dns.so.2
>> Core was generated by `/opt/ptin/kamailio-ksipadp/bin/ksipadp-proxy -f
>> /opt/ptin/kamailio-ksipadp/prox'.
>> Program terminated with signal 11, Segmentation fault.
>> #0  0x00007fefe58ea7c4 in atomic_get (v=0x0) at
>> ../../core/atomic/atomic_common.h:66
>> 66              return atomic_get_int(&(v->val));
>> Missing separate debuginfos, use: debuginfo-install
>> kamailio-ksipadp-2.8.0-1.ptin.el6.x86_64
>> (gdb) bt full
>> #0  0x00007fefe58ea7c4 in atomic_get (v=0x0) at
>> ../../core/atomic/atomic_common.h:66
>> No locals.
>> #1  0x00007fefe58ee246 in job_queue_size (queue=0x0) at worker.c:254
>> No locals.
>> #2  0x00007fefe58ed920 in add_dmq_job (msg=0x7ff028b453f8,
>> peer=0x7fefebcc4df8) at worker.c:184
>>         i = 1
>>         found_available = 0
>>         new_job = {f = 0x7fefe2613797 <usrloc_dmq_handle_msg>, msg =
>> 0x7fefebce67c8, orig_peer = 0x7fefebcc4df8, next = 0x0, prev = 0x0}
>>         worker = 0x7fefe7a1d890
>>         cloned_msg = 0x7fefebce67c8
>>         cloned_msg_len = 3888
>>         __FUNCTION__ = "add_dmq_job"
>> #3  0x00007fefe58dfec1 in dmq_handle_message (msg=0x7ff028b453f8,
>> str1=0x0, str2=0x0) at message.c:66
>>         peer = 0x7fefebcc4df8
>>         __FUNCTION__ = "dmq_handle_message"
>> #4  0x0000000000433729 in do_action (h=0x7ffc89e35670, a=0x7ff0285a9340,
>> msg=0x7ff028b453f8) at core/action.c:1054
>>         ret = -5
>>         v = 1
>>         dst = {send_sock = 0x7ff0285a8810, to = {s = {sa_family = 21376,
>> sa_data = "\343\211\374\177\000\000\370S\264(\360\177\000"}, sin = {
>>               sin_family = 21376, sin_port = 35299, sin_addr = {s_addr =
>> 32764}, sin_zero = "\370S\264(\360\177\000"}, sin6 = {sin6_family = 21376,
>>               sin6_port = 35299, sin6_flowinfo = 32764, sin6_addr =
>> {__in6_u = {__u6_addr8 = "\370S\264(\360\177\000\000pV\343\211\374\177\000",
>>                   __u6_addr16 = {21496, 10420, 32752, 0, 22128, 35299,
>> 32764, 0}, __u6_addr32 = {682906616, 32752, 2313377392, 32764}}},
>>               sin6_scope_id = 1}}, id = 32764, proto = 0 '\000',
>> send_flags = {f = 0, blst_imask = 0}}
>>         tmp = 0x389c422190 ""
>>         new_uri = 0x7c00000000 <Address 0x7c00000000 out of bounds>
>>         end = 0x6e00000077 <Address 0x6e00000077 out of bounds>
>>         crt = 0x389c98e120 ""
>>         cmd = 0x7ff0283219b0
>>         len = 0
>>         user = 0
>>         uri = {user = {s = 0x7ffc89e34a10 "@K\343\211\374\177", len =
>> 4459394}, passwd = {s = 0x7ffc89e34e80 "\356Q\343\211\374\177",
>>             len = 682906616}, host = {s = 0x7ff0285a9020 "\177\001", len
>> = -1981589904}, port = {s = 0x0, len = 1}, params = {
>>             s = 0x7ff028b453f8 "\017", len = 295169284}, sip_params = {s
>> = 0x32 <Address 0x32 out of bounds>, len = 1073741824}, headers = {
>>             s = 0x2000000 <Address 0x2000000 out of bounds>, len =
>> 4289104}, port_no = 60676, proto = 5047, type = 2312043322, flags =
>> 2168712452,
>>           transport = {s = 0x0, len = -1675583518}, ttl = {s = 0x89e34e90
>> <Address 0x89e34e90 out of bounds>, len = 35715777}, user_param = {
>>             s = 0xffffffff0000003c <Address 0xffffffff0000003c out of
>> bounds>, len = -1671419584}, maddr = {s = 0x0, len = -1981592704}, method =
>> {
>>             s = 0x389c603c48 "", len = -1671391960}, lr = {s = 0x0, len =
>> -1981592544}, r2 = {s = 0x1dde9a0 " 9\336\001", len = 0}, gr = {
>>             s = 0x4 <Address 0x4 out of bounds>, len = 2479},
>> transport_val = {s = 0x389c422190 "", len = 1}, ttl_val = {
>>             s = 0x7ffc89e34b40 "pM\343\211\374\177", len = 5683491},
>> user_param_val = {s = 0x40e7e8 "\370\377\377\377\377\377\377\377@ݘ
>> \234\070",
>>             len = -1671362728}, maddr_val = {s = 0x404130 "", len = 0},
>> method_val = {s = 0x1000003b4 <Address 0x1000003b4 out of bounds>, len = 0},
>>           lr_val = {s = 0x7fefe7374b9a "%x%c%x%c", len = -1673386776},
>> r2_val = {s = 0x7ffc89e34bd0 "", len = -1981592584}, gr_val = {
>> ---Type <return> to continue, or q <return> to quit---
>>             s = 0x389c422190 "", len = 0}}
>>         next_hop = {user = {s = 0x1 <Address 0x1 out of bounds>, len =
>> 40}, passwd = {s = 0x0, len = -1407107317}, host = {
>>             s = 0x2000000 <Address 0x2000000 out of bounds>, len =
>> 31398688}, port = {s = 0x1dde9a0 " 9\336\001", len = 31320480}, params = {
>>             s = 0x7ffc89e34c20 "", len = -1981593152}, sip_params = {s =
>> 0x7ffc89e34910 "\200N\343\211\374\177", len = 682906616}, headers = {
>>             s = 0x32 <Address 0x32 out of bounds>, len = 1073741824},
>> port_no = 0, proto = 512, type = ERROR_URI_T, flags = 4289104, transport = {
>>             s = 0x7ffc89e34900 "\020J\343\211\374\177", len = 4457737},
>> ttl = {s = 0x7ffc89e348f0 "", len = -1981593520}, user_param = {
>>             s = 0x7ffc89e34858 "\r", len = -1981593264}, maddr = {s =
>> 0x7ffc89e34870 "", len = -1981593456}, method = {s = 0x7ffc89e348b0 "\r",
>>             len = -1981593392}, lr = {s = 0x8cd2d0 <Address 0x8cd2d0 out
>> of bounds>, len = 13}, r2 = {
>>             s = 0x10100000002 <Address 0x10100000002 out of bounds>, len
>> = -1}, gr = {s = 0x0, len = 0}, transport_val = {s = 0x0, len = 682906616},
>>           ttl_val = {s = 0x7ff0285a9020 "\177\001", len = -1981593296},
>> user_param_val = {s = 0x7ffc89e34fa0 "\001\200\255\373\374\177", len = 0},
>>           maddr_val = {s = 0x10000000d <Address 0x10000000d out of
>> bounds>, len = -1981591296}, method_val = {
>>             s = 0x7fef00000000 <Address 0x7fef00000000 out of bounds>,
>> len = -1671151657}, lr_val = {s = 0x7fefe7374ba2 "", len = 682906616},
>>           r2_val = {s = 0x32 <Address 0x32 out of bounds>, len =
>> 1073741824}, gr_val = {s = 0x2000000 <Address 0x2000000 out of bounds>,
>>             len = 4289104}}
>>         u = 0x389c4224e8
>>         port = 0
>>         dst_host = 0x89e34bd0
>>         i = 32764
>>         flags = 52
>>         avp = 0x189e35040
>>         st = {flags = 2313374752, id = 32764, name = {n = -1981593512, s
>> = {s = 0x7ffc89e34858 "\r", len = -1981593152}, re = 0x7ffc89e34858},
>>           avp = 0x7ffc89e347a0}
>>         sct = 0x0
>>         sjt = 0x7ff028b453f8
>>         rve = 0x32
>>         mct = 0x40000000
>>         rv = 0x7ffc89e34c20
>>         rv1 = 0x2000000
>>         c1 = {cache_type = 2313374144, val_type = 32764, c = {avp_val =
>> {n = 1, s = {s = 0x1 <Address 0x1 out of bounds>, len = 1}, re = 0x1}, pval
>> = {
>>               rs = {s = 0x1 <Address 0x1 out of bounds>, len = 1}, ri =
>> -1407115050, flags = 52}},
>>           i2s = "\000\000\000\000\000\000\000\000\274\224
>> \254\064\000\000\000\240G\343\211\374\177"}
>>         s = {s = 0x1 <Address 0x1 out of bounds>, len = -1981593512}
>>         srevp = {0x7ffc89e34960, 0x34ac208a97}
>>         mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0,
>> str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {
>>             type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s =
>> 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u
>> = {
>>               number = 0, string = 0x0, str = {s = 0x0, len = 0}, data =
>> 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0,
>>               string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr =
>> 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str
>> = {
>>                 s = 0x0, len = 0}, data = 0x0, attr = 0x0, select =
>> 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0,
>> len = 0},
>>               data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE,
>> u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0,
>>               attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number
>> = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0,
>>               select = 0x0}}}
>> ---Type <return> to continue, or q <return> to quit---
>>         __FUNCTION__ = "do_action"
>> #5  0x00000000004404b8 in run_actions (h=0x7ffc89e35670,
>> a=0x7ff0285a9340, msg=0x7ff028b453f8) at core/action.c:1552
>>         t = 0x7ff0285a9340
>>         ret = -1
>>         mod = 0x1dde9a0
>>         ms = 176570657
>>         __FUNCTION__ = "run_actions"
>> #6  0x0000000000433692 in do_action (h=0x7ffc89e35670, a=0x7ff0285a9668,
>> msg=0x7ff028b453f8) at core/action.c:1045
>>         ret = 1
>>         v = 1
>>         dst = {send_sock = 0x7ffc89e35380, to = {s = {sa_family = 61042,
>> sa_data = "I", '\000' <repeats 12 times>}, sin = {sin_family = 61042,
>>               sin_port = 73, sin_addr = {s_addr = 0}, sin_zero =
>> "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 61042, sin6_port =
>> 73,
>>               sin6_flowinfo = 0, sin6_addr = {__in6_u = {__u6_addr8 =
>> "\000\000\000\000\000\000\000\000\020\000\000\000\005\000\000", __u6_addr16
>> = {
>>                     0, 0, 0, 0, 16, 0, 5, 0}, __u6_addr32 = {0, 0, 16,
>> 5}}}, sin6_scope_id = 673678288}}, id = 32752, proto = 0 '\000', send_flags
>> = {
>>             f = 0, blst_imask = 32752}}
>>         tmp = 0x66307b "H\213E\270H\203\300\070\351O\003"
>>         new_uri = 0x7ff028a93048 ""
>>         end = 0x7ff028a93038 "\303l\256"
>>         crt = 0x2000000 <Address 0x2000000 out of bounds>
>>         cmd = 0x1773508
>>         len = 6791248
>>         user = 0
>>         uri = {user = {s = 0xae6ecd "", len = 11430897}, passwd = {s =
>> 0x7c95f8 "core: core/parser/msg_parser.c", len = 682507016}, host = {
>>             s = 0xae6c43 "To: <sip:usrloc at 10.112.103.64:5080>\r\nFrom: <
>> sip:usrloc at 10.112.101.128:5060>;tag=8a3534eb92149bbe647f599d354f2511-090e\r\nCSeq:
>> 10 KDMQ\r\nCall-ID: 409f4ac5788a8cc4-24333 at 10.112.101.128\r\nContent-Length:
>> 39"..., len = 11430924}, port = {s = 0x0, len = 682507072}, params = {
>>             s = 0x7ff028ae3c90  <incomplete sequence \350>, len = 0},
>> sip_params = {s = 0x7ffc89e35330 "\246", len = 6697083}, headers = {
>>             s = 0x7c9170 "core", len = -1981590832}, port_no = 8, proto =
>> 0, type = ERROR_URI_T, flags = 30415600, transport = {
>>             s = 0xa8428197 <Address 0xa8428197 out of bounds>, len =
>> -1675583518}, ttl = {s = 0x3000000030 <Address 0x3000000030 out of bounds>,
>>             len = 44108294}, user_param = {s = 0x400000017 <Address
>> 0x400000017 out of bounds>, len = 8176032}, maddr = {s = 0x0, len =
>> -1981590448},
>>           method = {s = 0x389c603c48 "", len = 11431629}, lr = {s =
>> 0x7ffc89e35450 "\220T\343\211\374\177", len = 6869803}, r2 = {
>>             s = 0x7cc190 "get_hdr_field", len = 6730416}, gr = {s =
>> 0x7ffc89e35300 "", len = 673677328}, transport_val = {s = 0x389c422190 "",
>>             len = 673677328}, ttl_val = {s = 0x7ffc89e35350
>> "\200S\343\211\374\177", len = -413652166}, user_param_val = {s = 0x40e7f9
>> "", len = 0},
>>           maddr_val = {s = 0x2000000 <Address 0x2000000 out of bounds>,
>> len = 24588552}, method_val = {
>>             s = 0x67a450
>> "\030A\211\321A)\301D\211ȉ\302H\213\205(\377\377\377\211P \276\004", len =
>> 8965880}, lr_val = {
>>             s = 0x8cd2d0 <Address 0x8cd2d0 out of bounds>, len = 8},
>> r2_val = {s = 0xa6 <Address 0xa6 out of bounds>, len = 8136007}, gr_val = {
>>             s = 0x7c95f8 "core: core/parser/msg_parser.c", len =
>> 682176488}}
>>         next_hop = {user = {s = 0x1e7374b9a <Address 0x1e7374b9a out of
>> bounds>, len = 674472240}, passwd = {
>>             s = 0xffffffff <Address 0xffffffff out of bounds>, len =
>> 4289104}, host = {s = 0x4e2cb1825 <Address 0x4e2cb1825 out of bounds>,
>>             len = -1667705792}, port = {s = 0x0, len = 1073741824},
>> params = {s = 0x2000000 <Address 0x2000000 out of bounds>, len = 4289104},
>>           sip_params = {s = 0x7ffc89e35180 "\360Q\343\211\374\177", len =
>> 6031673}, headers = {s = 0x7fefe2cb183f "", len = -1981591296},
>>           port_no = 23013, proto = 58022, type = 32751, flags =
>> 2313376240, transport = {s = 0x7fefe7374b9a "%x%c%x%c", len = -1981590944},
>> ttl = {
>> ---Type <return> to continue, or q <return> to quit---
>>             s = 0x1a <Address 0x1a out of bounds>, len = 4289104},
>> user_param = {s = 0x7fefe2a659e5 "5e6a5a14-1fa-", len = -1670974014}, maddr
>> = {
>>             s = 0x30fbad8001 <Address 0x30fbad8001 out of bounds>, len =
>> -492414491}, method = {s = 0x7fefe2a659e5 "5e6a5a14-1fa-", len =
>> -492414491},
>>           lr = {s = 0x7fefe2a659e5 "5e6a5a14-1fa-", len = -492414478}, r2
>> = {s = 0x7fefe2a659ff "", len = -492414491}, gr = {s = 0x7fefe2a659ff "",
>>             len = 0}, transport_val = {s = 0x0, len = 6730416}, ttl_val =
>> {s = 0x7ffc89e351a0 "", len = 673677328}, user_param_val = {
>>             s = 0x2000000 <Address 0x2000000 out of bounds>, len =
>> 673677328}, maddr_val = {s = 0x7ffc89e351f0 "Cl\256", len = -413652166},
>>           method_val = {s = 0x7ff028278010 "\001", len = 0}, lr_val = {s
>> = 0x2000000 <Address 0x2000000 out of bounds>, len = 24589344}, r2_val = {
>>             s = 0x67a138 "", len = 8965088}, gr_val = {s = 0x8cd2d0
>> <Address 0x8cd2d0 out of bounds>, len = 682507072}}
>>         u = 0x66273d
>>         port = 0
>>         dst_host = 0x40000000
>>         i = 28
>>         flags = 0
>>         avp = 0xae6cc7
>>         st = {flags = 31129600, id = 0, name = {n = 0, s = {s = 0x0, len
>> = 93}, re = 0x0}, avp = 0x7ffc89e35080}
>>         sct = 0x7ffc89e35490
>>         sjt = 0x6943d3
>>         rve = 0x7ff0285a8808
>>         mct = 0x6a283b
>>         rv = 0x0
>>         rv1 = 0x7c9170
>>         c1 = {cache_type = 24, val_type = 48, c = {avp_val = {n =
>> -1981590720, s = {s = 0x7ffc89e35340 "\370\225|", len = -1981590912},
>>               re = 0x7ffc89e35340}, pval = {rs = {s = 0x7ffc89e35340
>> "\370\225|", len = -1981590912}, ri = 0, flags = 0}},
>>           i2s = '\000' <repeats 21 times>}
>>         s = {s = 0x7fefe2cb183f "", len = -490006491}
>>         srevp = {0x7fefe2cb1825, 0x7fefe2cb1832}
>>         mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0,
>> str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {
>>             type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s =
>> 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u
>> = {
>>               number = 0, string = 0x0, str = {s = 0x0, len = 0}, data =
>> 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0,
>>               string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr =
>> 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str
>> = {
>>                 s = 0x0, len = 0}, data = 0x0, attr = 0x0, select =
>> 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0,
>> len = 0},
>>               data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE,
>> u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0,
>>               attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number
>> = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0,
>>               select = 0x0}}}
>>         __FUNCTION__ = "do_action"
>> #7  0x00000000004404b8 in run_actions (h=0x7ffc89e35670,
>> a=0x7ff0285a85d0, msg=0x7ff028b453f8) at core/action.c:1552
>>         t = 0x7ff0285a9668
>>         ret = 1
>>         mod = 0x417250
>>         ms = 176570657
>>         __FUNCTION__ = "run_actions"
>> #8  0x0000000000440c4a in run_top_route (a=0x7ff0285a85d0,
>> msg=0x7ff028b453f8, c=0x0) at core/action.c:1641
>> ---Type <return> to continue, or q <return> to quit---
>>         ctx = {rec_lev = 2, run_flags = 0, last_retcode = 1, jmp_env =
>> {{__jmpbuf = {140669451785720, -8516593617994257148, 50, 1073741824,
>> 33554432,
>>                 4289104, -8516593618092823292, 8518532807277145348},
>> __mask_was_saved = 0, __saved_mask = {__val = {140668355198191, 4302791168,
>> 48,
>>                   48, 18446744073709551615, 140722621863712,
>> 140668355198919, 50, 0, 4611686022755909632, 140669451785208,
>> 140722621863776, 5814046,
>>                   6608344928, 140669451785208, 0}}}}}
>>         p = 0x7ffc89e35670
>>         ret = 682907128
>>         sfbk = 0
>> #9  0x000000000052e6b9 in receive_msg (
>>     buf=0xae6bc0 "KDMQ sip:usrloc at 10.112.103.64:5080 SIP/2.0\r\nVia:
>> SIP/2.0/UDP 10.112.101.128;branch=z9hG4bK8791.0c350dd3", '0' <repeats 24
>> times>, ".0\r\nTo: <sip:usrloc at 10.112.103.64:5080>\r\nFrom: <
>> sip:usrloc at 10.112.101.128"..., len=781, rcv_info=0x7ffc89e35a70) at
>> core/receive.c:264
>>         msg = 0x7ff028b453f8
>>         ctx = {rec_lev = -1981589208, run_flags = 32764, last_retcode =
>> 8, jmp_env = {{__jmpbuf = {163, 4253893, 8100093, 140669451683568,
>>                 140722621864176, 4845170, 0, 21474836480},
>> __mask_was_saved = 673677840, __saved_mask = {__val = {140668768878592,
>> 140722621864448,
>>                   6697083, 8094080, 5501958, 243141751568,
>> 140669476122624, 140722621864896, 4294967295, 88212724672, 1,
>> 140668357584160, 0, 33554432,
>>                   1024, 6608345520}}}}}
>>         bctx = 0x7ffc89e359b0
>>         ret = 0
>>         stats_on = 0
>>         tvb = {tv_sec = 140722621864208, tv_usec = 8961768}
>>         tve = {tv_sec = 0, tv_usec = 243139749096}
>>         tz = {tz_minuteswest = 50, tz_dsttime = 0}
>>         diff = 0
>>         inb = {
>>           s = 0xae6bc0 "KDMQ sip:usrloc at 10.112.103.64:5080
>> SIP/2.0\r\nVia: SIP/2.0/UDP 10.112.101.128;branch=z9hG4bK8791.0c350dd3",
>> '0' <repeats 24 times>, ".0\r\nTo: <sip:usrloc at 10.112.103.64:5080>\r\nFrom:
>> <sip:usrloc at 10.112.101.128"..., len = 781}
>>         netinfo = {data = {s = 0x7ffc00000001 <Address 0x7ffc00000001 out
>> of bounds>, len = 0}, dst = 0x1, rcv = 0x389c422190}
>>         keng = 0x0
>>         __FUNCTION__ = "receive_msg"
>> #10 0x00000000006456a3 in udp_rcv_loop () at core/udp_server.c:515
>>         len = 781
>>         buf = "KDMQ sip:usrloc at 10.112.103.64:5080 SIP/2.0\r\nVia:
>> SIP/2.0/UDP 10.112.101.128;branch=z9hG4bK8791.0c350dd3", '0' <repeats 24
>> times>, ".0\r\nTo: <sip:usrloc at 10.112.103.64:5080>\r\nFrom: <
>> sip:usrloc at 10.112.101.128:5060>;tag=8a3"...
>>         tmp = 0x100000000 <Address 0x100000000 out of bounds>
>>         from = 0x7ff028b2c728
>>         fromlen = 16
>>         ri = {src_ip = {af = 2, len = 4, u = {addrl = {140722462617610,
>> 140722621864592}, addr32 = {2154131466, 32764, 2313378448, 32764}, addr16 =
>> {
>>                 28682, 32869, 32764, 0, 23184, 35299, 32764, 0}, addr =
>> "\npe\200\374\177\000\000\220Z\343\211\374\177\000"}}, dst_ip = {af = 2,
>>             len = 4, u = {addrl = {1080520714, 0}, addr32 = {1080520714,
>> 0, 0, 0}, addr16 = {28682, 16487, 0, 0, 0, 0, 0, 0},
>>               addr = "\npg@", '\000' <repeats 11 times>}}, src_port =
>> 5060, dst_port = 5080, proto_reserved1 = 0, proto_reserved2 = 0, src_su =
>> {s = {
>>               sa_family = 2, sa_data =
>> "\023\304\npe\200\000\000\000\000\000\000\000"}, sin = {sin_family = 2,
>> sin_port = 50195, sin_addr = {
>>                 s_addr = 2154131466}, sin_zero =
>> "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2, sin6_port = 50195,
>> ---Type <return> to continue, or q <return> to quit---
>>               sin6_flowinfo = 2154131466, sin6_addr = {__in6_u =
>> {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0,
>> 0, 0},
>>                   __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}},
>> bind_address = 0x7ff02830fff0, proto = 1 '\001'}
>>         __FUNCTION__ = "udp_rcv_loop"
>> #11 0x0000000000423c7d in main_loop () at main.c:1623
>>         i = 0
>>         pid = 0
>>         si = 0x7ff02830fff0
>>         si_desc = "udp receiver child=0 sock=10.112.103.64:5080\000\000\000\000
>> _\343\211\374\177\000\000\062\000\000\000\000\000\000\000\000\000\000@
>> \000\000\000\000\000\000\000\002\000\000\000\000PrA\000\000\000\000\000PSn\234\070\000\000\000\060\000\000\000\060\000\000\000\020]\343\211\374\177\000\000@
>> \\\343\211\374\177\000\000\000px\347\357\177\000"
>>         nrprocs = 4
>>         woneinit = 0
>>         __FUNCTION__ = "main_loop"
>> #12 0x000000000042b0ce in main (argc=13, argv=0x7ffc89e36008) at
>> main.c:2646
>>         cfg_stream = 0x1bb5010
>>         c = -1
>>         r = 0
>>         tmp = 0x7ffc89e36ef7 ""
>>         tmp_len = 32764
>>         port = -1981587768
>>         proto = 56
>>         options = 0x7434f8
>> ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
>>         ret = -1
>>         seed = 4012862799
>>         rfd = 4
>>         debug_save = 0
>>         debug_flag = 0
>>         dont_fork_cnt = 0
>>         n_lst = 0x7ffc89e35f08
>>         p = 0x100000000 <Address 0x100000000 out of bounds>
>>         st = {st_dev = 64769, st_ino = 148592, st_nlink = 2, st_mode =
>> 16877, st_uid = 498, st_gid = 498, __pad0 = 0, st_rdev = 0, st_size = 4096,
>>           st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec =
>> 1576522467, tv_nsec = 972000117}, st_mtim = {tv_sec = 1519927576,
>>             tv_nsec = 668342155}, st_ctim = {tv_sec = 1584026268, tv_nsec
>> = 165523404}, __unused = {0, 0, 0}}
>>         __FUNCTION__ = "main"
>> (gdb)
>>
>> Em ter., 17 de mar. de 2020 às 09:52, Daniel-Constantin Mierla <
>> miconda at gmail.com> escreveu:
>>
>>> Hello,
>>>
>>> does it happen only when doing restart of kamailio? Can you send the
>>> full backtrace from gdb?
>>>
>>> Also, 5.0.x is no longer a supported release for quite some time, try to
>>> upgrade to 5.2.x or even better to the latest one 5.3.x and see if you can
>>> reproduce it with one of the supported releases, maybe it is something that
>>> was already fixed.
>>>
>>> Cheers,
>>> Daniel
>>> On 17.03.20 10:32, Virgílio Cunha wrote:
>>>
>>> Hello,
>>>
>>> I have in production environment with 4 proxies using DMQ to sync
>>> terminal registers and the DMQ configuration parameters is following:
>>> loadmodule "dmq.so"
>>> #modparam("dmq", "server_address", "sip:MY_IP_ADDRESS:MY_PORT_ADDRESS")
>>> modparam("dmq", "notification_address", "DMQ_HOSTS")
>>> modparam("dmq", "multi_notify", 1)
>>> modparam("dmq", "num_workers", 4)
>>>
>>>
>>> In the lab environment with the same configuration that in prod, I can
>>> simulate this problem with 12000 terminal REGISTERS and, when doing several
>>> restarts on a proxy, one of them is generated a coredump.
>>> Analyzing the core, I can see that the problem is concurrency in the dmq
>>> synchronization.
>>>
>>> Job_queue_size (...) is called with a valid *queue *and, but inside
>>> this function, the *queue *is null.
>>>
>>>
>>> (gdb) up
>>> #1  0x00007fefe58ee246 in job_queue_size (queue=0x0) at worker.c:254
>>> 254             return atomic_get(&queue->count);
>>> (gdb) print queue
>>> $1 = (job_queue_t *) 0x0
>>> (gdb) up
>>> #2  0x00007fefe58ed920 in add_dmq_job (msg=0x7ff028b453f8,
>>> peer=0x7fefebcc4df8) at worker.c:184
>>> 184                     if(job_queue_size(workers[i].queue) == 0) {
>>> (gdb) print i
>>> $2 = 1
>>> (gdb) print workers[i].queue
>>> $3 = (job_queue_t *) 0x7fefebce49b0
>>>
>>> If I configure DMQ with  num_workers=1 I can't reproduce this problem.
>>> I'm using the kamailio 5.0.8 release.
>>>
>>> Is this problem known to you? What is the right way to solve it?
>>>
>>> Best Regards,
>>> Virgílio Cunha
>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing Listsr-users at lists.kamailio.orghttps://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>> --
>>> Daniel-Constantin Mierla -- www.asipto.comwww.twitter.com/miconda -- www.linkedin.com/in/miconda
>>>
>>> --
>> Daniel-Constantin Mierla -- www.asipto.comwww.twitter.com/miconda -- www.linkedin.com/in/miconda
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20200326/5e33ce20/attachment.html>


More information about the sr-users mailing list