<div dir="ltr">Yes,<br>I did read all(past 3+ years) his replies specific to DMQ and DMQ USRLOC and only one matched exact description and there has no resolution to it. <div>Github open+closed issues for DMQ didn't have anything similar either. Could it be something I'm doing wrong !? <div><br>Additional info:
One of the server is direct on Public IP and Other one is behind NAT. Another test setup where it consistently reproducible is two server behind NAT(AWS)
</div><div>Here are the mod params. Only usrloc sync is done via DMQ and no other module is using DMQ. <br><font face="monospace"><br></font></div><div><font face="monospace">listen=udp:LocalIP:5060 advertise PublicIP:5060</font><br></div><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace">modparam("dmq","server_</span><span style="font-family:monospace">address", DMQ_LOCAL_SERVER)</span><br></div><div><font face="monospace">modparam("dmq", "notification_address", DMQ_REMOTE_SERVER)<br>modparam("dmq", "multi_notify", 0) //1 for DNS SRV<br>modparam("dmq", "num_workers", 10)<br>modparam("dmq", "ping_interval", 60)<br><br>modparam("dmq_usrloc", "enable", 1)<br>modparam("dmq_usrloc", "sync", 1)<br>modparam("dmq_usrloc", "batch_size", 4000)<br>modparam("dmq_usrloc", "batch_usleep", 1000)<br>modparam("dmq_usrloc", "usrloc_domain", "location")</font><br></div><div><br></div><div>Where:
<font face="monospace">DMQ_REMOTE_SERVER = sip:PublicIP2:5060
</font><br><br></div><div>GDB info as requested:</div><div><font face="monospace"><br></font></div><div><font face="monospace">Core was generated by `/usr/local/sbin/kamailio -w /tmp/kamailio -P /var/run/kamailio/kamailio.pid -f'.<br>Program terminated with signal SIGSEGV, Segmentation fault.<br>#0 0x00007f248c4cef15 in send_reply (msg=0x7f2469f88d40, code=0, reason=0x7ffd775e3ab8) at sl.c:276<br>276 if(reason->s[reason->len-1]=='\0') {<br>(gdb)<br>(gdb)<br>(gdb) frame 0<br>#0 0x00007f248c4cef15 in send_reply (msg=0x7f2469f88d40, code=0, reason=0x7ffd775e3ab8) at sl.c:276<br>276 if(reason->s[reason->len-1]=='\0') {<br>(gdb) p *reason<br>$1 = {s = 0x0, len = 0}<br>(gdb)<br>(gdb) frame 1<br>#1 0x00007f24656c6549 in worker_loop (id=2) at worker.c:129<br>129 if(slb.freply(current_job->msg, peer_response.resp_code,<br>(gdb) p *worker<br>$3 = {queue = 0x7f2469f240a8, jobs_processed = 5, lock = {val = 2}, pid = 935}<br>(gdb)<br>(gdb)<br>(gdb) p *current_job<br>$6 = {f = 0x7f24656d6d8d <empty_peer_callback>, msg = 0x7f2469f88d40, orig_peer = 0x7f2469f6ed50, next = 0x0, prev = 0x0}<br>(gdb)<br></font><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Apr 24, 2020 at 1:30 PM Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello,</p>
<p>have you tried the suggestion from Charles in the other response?
It can help figuring out where the problem resides.</p>
<p>Now, from C point of view, I would need the following output from
gdb of the core file:</p>
<p>frame 0<br>
p *reason</p>
<p>frame 1<br>
p *worker<br>
p *current_job</p>
<p>I would also need to know the modparams for dmq and other dmq_*
module, plus the list if modules for which you enabled dmq (eg,
htable, dialog, presence, ...).</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 24.04.20 18:10, SamyGo wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Oops,apologize, missed that:
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace">version: kamailio 5.3.3
(x86_64/linux) 44ccb9-dirty<br>
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS,
DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC,
Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX,
FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER,
USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES<br>
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144,
MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB<br>
poll method support: poll, epoll_lt, epoll_et, sigio_rt,
select.<br>
id: 44ccb9 -dirty<br>
compiled on 17:04:55 Apr 17 2020 with gcc 4.9.2</font><br>
</div>
<div><font face="monospace"><br>
</font></div>
<div><font face="arial, sans-serif">Tried this with version 5.0,
5.2, and now 5.3 same situation.. </font></div>
<div><font face="arial, sans-serif"><br>
</font></div>
<div><font face="arial, sans-serif">Thankyou for looking into
this,</font></div>
<div><font face="arial, sans-serif">Sammy</font></div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Apr 24, 2020 at 2:33
AM Daniel-Constantin Mierla <<a href="mailto:miconda@gmail.com" target="_blank">miconda@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello,</p>
<p>you have to provide the version of kamailio for each
reported kamailio issue, otherwise is hard to match with
the source code. Use 'kamailio -v' to get version details.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 23.04.20 23:36, SamyGo wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi,<br>
<br>
<div>Is there a way to broadcast KDMQ to the cluster but
not expect a reply back !?as far as I've read the
source code dmq_bcast_message is exactly like
dmq_send_message in a way that it expects a callback
to be executed on response i.e expects a reply.</div>
<div><br>
</div>
<div>So, the situation I'm facing is I'm broadcasting
message to cluster and I do not want a reply back. The
following two options result in crash & core dump.</div>
<div><br>
</div>
<div>1 - If my script doesn't respond back, by use of
dmq_handle_message, it marks the destined servers as
"inactive" and stops usrloc sync process which
isn't desirable.</div>
<div>2 - If I respond back with the dmq_handle_message
it crashes the Kamailio which just received this
broadcasted message.</div>
<div><br>
</div>
<div>Here is how its done in script:<br>
</div>
<div><br>
</div>
<div><b>broadcasting message to cluster:</b></div>
<div><font face="monospace">
dmq_bcast_message("userOnline", "$fu",
"text/plain");</font><br>
</div>
<div><font face="arial, sans-serif"><br>
</font></div>
<div><font face="arial, sans-serif"><b>Receiving and
handling a broadcast message:</b></font></div>
<div><font face="monospace">route[DMQ_HANDLE] {</font></div>
<font face="monospace"> if(!(is_method("KDMQ") || $rm
== "KDMQ")) return;<br>
<br>
if(is_method("KDMQ") || $rm == "KDMQ"){<br>
if($rU =~ "userOnline"){<br>
//user came online in cluster,
resume transactions if-any suspended<br>
$avp(remoteUser) = $rb; <br>
}<br>
dmq_handle_message();<br>
exit;<br>
}<br>
}</font>
<div><font face="arial, sans-serif"><br>
</font></div>
<div><font face="arial, sans-serif"><b>Related log
lines:</b></font></div>
<div><font face="monospace">Apr 23 21:15:48
kamailio[916]: ALERT: <script>: [da2c1-2f499]
------ DMQ_HANDLE: UserOnline Event Received ------<br>
Apr 23 21:15:48 kamailio[916]: DEBUG: dmq
[message.c:53]: ki_dmq_handle_message_rc():
dmq_handle_message [KDMQ <a href="http://sip:userOnline@9.8.7.123:5060" target="_blank">sip:userOnline@9.8.7.123:5060</a>]<br>
Apr 23 21:15:48 kamailio[916]: DEBUG: dmq
[message.c:66]: ki_dmq_handle_message_rc():
dmq_handle_message peer found: userOnline<br>
Apr 23 21:15:48 kamailio[916]: DEBUG: <core>
[core/receive.c:437]: receive_msg(): request-route
executed in: 401461 usec<br>
Apr 23 21:15:48 kamailio[935]: DEBUG: dmq
[worker.c:87]: worker_loop(): dmq_worker [2 935]
lock acquired<br>
</font></div>
<div><font face="monospace">and crash/segfault..<br>
<br>
</font></div>
<div><font face="monospace">Core dump: </font><a href="https://pastebin.com/S7ekCPfF" target="_blank">https://pastebin.com/S7ekCPfF</a><br>
<br>
Any help or pointers to solve this would be really
appreciated.</div>
<div><br>
</div>
<div>Best Regards,</div>
<div>Sammy</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Kamailio (SER) - Users Mailing List
<a href="mailto:sr-users@lists.kamailio.org" target="_blank">sr-users@lists.kamailio.org</a>
<a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
</blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank">www.linkedin.com/in/miconda</a></pre>
</div>
</blockquote>
</div>
</blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank">www.linkedin.com/in/miconda</a></pre>
</div>
</blockquote></div>