<div dir="ltr">Hello,<div><br></div><div>It seems you have already solved your issue, but just to add, it's often better to provide an FQDN as notification_address which resolves to multiple IPs (and set "multi_notify" parameter).</div><div><br></div><div><span style="font-size:12.800000190734863px">There should be no issues when shutting down multiple nodes at once, as long as the remaining nodes are still in contact with each other - at least, that is my experience across several long-established (and highly dynamic) clusters. When the disabled nodes are re-enabled, providing they have the address of at least one active node (or better, an FQDN resolving to multiple) they'll rejoin seamlessly.</span><br></div><div><br></div><div>Of course, if others have a different experience, it would be good to learn more about it.</div><div><br></div><div>Cheers,</div><div><span style="font-size:12.800000190734863px"><br></span></div><div><span style="font-size:12.800000190734863px">Charles</span></div><div><span style="font-size:12.800000190734863px"><br></span></div><div class="gmail_extra"><br><div class="gmail_quote">On 1 December 2017 at 11:00, Julien Chavanton <span dir="ltr"><<a href="mailto:jchavanton@gmail.com" target="_blank">jchavanton@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Forgot to mention<br><br></div>192.168.150.225 must have another node as notitifcation peer, not himself else he will not replicate data to other nodes<br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Dec 1, 2017 at 2:33 AM, Julien Chavanton <span dir="ltr"><<a href="mailto:jchavanton@gmail.com" target="_blank">jchavanton@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span class="m_6212761935717496828m_-844923629212803955gmail-message_body">Hi Aydar,<br></span><div><span class="m_6212761935717496828m_-844923629212803955gmail-message_body"><br>You can have only one </span>notification_address in this case the last one will be used "192.168.150.225" this will be you DMQ master node<br><br><div><span class="m_6212761935717496828m_-844923629212803955gmail-message_body">I know the bus can break if you shutdown multiple nodes at the same time and one of them is the master node and then you restart them. (not sure if this is your case)<br><br></span></div><div><span class="m_6212761935717496828m_-844923629212803955gmail-message_body">If this happens the only option I can see is to restart all nodes one by one.<br><br></span></div><div><span class="m_6212761935717496828m_-844923629212803955gmail-message_body">I think this case could be handled better maybe by adding a dmq resync command that can recreate the bus .<br></span></div><div><span class="m_6212761935717496828m_-844923629212803955gmail-message_body"><br></span></div></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_6212761935717496828h5">On Fri, Dec 1, 2017 at 12:32 AM, Aidar Kamalov <span dir="ltr"><<a href="mailto:aidar.kamalov@gmail.com" target="_blank">aidar.kamalov@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_6212761935717496828h5"><div dir="ltr">Hello. I have 4 kamailio nodes with same config:<div><br clear="all"><div><div>modparam("dmq", "server_address", "sip:LOCALIP:5050")</div><div>modparam("dmq", "notification_address", "sip:<a href="http://192.168.8.213:5050" target="_blank">192.168.8.213:5050</a>")<br></div><div>modparam("dmq", "notification_address", "sip:<a href="http://192.168.10.54:5050" target="_blank">192.168.10.54:5050</a>")</div><div>modparam("dmq", "notification_address", "sip:<a href="http://192.168.107.195:5050" target="_blank">192.168.107.195:5050</a>")</div><div>modparam("dmq", "notification_address", "sip:<a href="http://192.168.150.225:5050" target="_blank">192.168.150.225:5050</a>")</div><div>modparam("dmq", "num_workers", 6)<br></div><div>modparam("dmq_usrloc", "enable", 1)</div><div>modparam("dmq_usrloc", "sync", 1)</div><div>modparam("dmq_usrloc", "batch_size", 2000)</div><div>modparam("dmq_usrloc", "batch_usleep", 1000)</div></div><div><br></div><div><br></div><div>regularly (don't know why) I've got issues - replication is broken, for example now at all nodes I have:<br></div><div><div># kamcmd dmq.list_nodes</div><div>{</div><div><span style="white-space:pre-wrap"> </span>host: 192.168.8.213</div><div><span style="white-space:pre-wrap"> </span>port: 5050</div><div><span style="white-space:pre-wrap"> </span>resolved_ip: 192.168.8.213</div><div><span style="white-space:pre-wrap"> </span>status: 2</div><div><span style="white-space:pre-wrap"> </span>last_notification: 0</div><div><span style="white-space:pre-wrap"> </span>local: 0</div><div>}</div><div>{</div><div><span style="white-space:pre-wrap"> </span>host: 192.168.10.54</div><div><span style="white-space:pre-wrap"> </span>port: 5050</div><div><span style="white-space:pre-wrap"> </span>resolved_ip: 192.168.10.54</div><div><span style="white-space:pre-wrap"> </span>status: 2</div><div><span style="white-space:pre-wrap"> </span>last_notification: 0</div><div><span style="white-space:pre-wrap"> </span>local: 0</div><div>}</div><div>{</div><div><span style="white-space:pre-wrap"> </span>host: 192.168.150.225</div><div><span style="white-space:pre-wrap"> </span>port: 5050</div><div><span style="white-space:pre-wrap"> </span>resolved_ip: 192.168.150.225</div><div><span style="white-space:pre-wrap"> </span>status: 8</div><div><span style="white-space:pre-wrap"> </span>last_notification: 0</div><div><span style="white-space:pre-wrap"> </span>local: 0</div><div>}</div><div>{</div><div><span style="white-space:pre-wrap"> </span>host: 192.168.107.195</div><div><span style="white-space:pre-wrap"> </span>port: 5050</div><div><span style="white-space:pre-wrap"> </span>resolved_ip: 192.168.107.195</div><div><span style="white-space:pre-wrap"> </span>status: 2</div><div><span style="white-space:pre-wrap"> </span>last_notification: 0</div><div><span style="white-space:pre-wrap"> </span>local: 1</div><div>}</div></div><div><br></div><div>but at node <a href="http://192.168.150.225" target="_blank">192.168.150.225</a>:<br></div><div><div># kamcmd dmq.list_nodes</div><div>{</div><div><span style="white-space:pre-wrap"> </span>host: 192.168.150.225</div><div><span style="white-space:pre-wrap"> </span>port: 5050</div><div><span style="white-space:pre-wrap"> </span>resolved_ip: 192.168.150.225</div><div><span style="white-space:pre-wrap"> </span>status: 2</div><div><span style="white-space:pre-wrap"> </span>last_notification: 0</div><div><span style="white-space:pre-wrap"> </span>local: 0</div><div>}</div><div>{</div><div><span style="white-space:pre-wrap"> </span>host: 192.168.150.225</div><div><span style="white-space:pre-wrap"> </span>port: 5050</div><div><span style="white-space:pre-wrap"> </span>resolved_ip: 192.168.150.225</div><div><span style="white-space:pre-wrap"> </span>status: 2</div><div><span style="white-space:pre-wrap"> </span>last_notification: 0</div><div><span style="white-space:pre-wrap"> </span>local: 1</div><div>}</div></div><div><br></div><div>nodes restart doesn't help. but if i exec</div><div>systemctl stop kamailio && sleep 15 && systemctl start kamailio</div><div>it is partially solve my issue - dmq.list_nodes show all nodes, but at 192.168.150.225 i see not all user locations: they are added gradually</div><div><br></div><div>There is no errors in logs</div><div><div>Dec 1 08:28:37 sipufa /usr/sbin/kamailio[18252]: NOTICE: <script>: ROOT: KDMQ notification_peer [<a href="mailto:15f3edf9258014d1-7114@192.168.8.213" target="_blank">15f3edf9258014d1-7114@192.168<wbr>.8.213</a>]</div><div>Dec 1 08:28:38 sipufa /usr/sbin/kamailio[18271]: NOTICE: <script>: ROOT: KDMQ notification_peer [<a href="mailto:7acb9eae627482f8-87828@192.168.10.54" target="_blank">7acb9eae627482f8-87828@192.16<wbr>8.10.54</a>]</div><div>Dec 1 08:28:39 sipufa /usr/sbin/kamailio[18250]: NOTICE: <script>: ROOT: KDMQ notification_peer [<a href="mailto:621ad9f474b335e2-18272@192.168.150.225" target="_blank">621ad9f474b335e2-18272@192.16<wbr>8.150.225</a>]</div><div>Dec 1 08:28:40 sipufa /usr/sbin/kamailio[18252]: NOTICE: <script>: ROOT: KDMQ notification_peer [<a href="mailto:7b817f6a6244f923-8872@192.168.107.195" target="_blank">7b817f6a6244f923-8872@192.168<wbr>.107.195</a>]</div><div>Dec 1 08:28:47 sipufa /usr/sbin/kamailio[18271]: NOTICE: <script>: ROOT: KDMQ usrloc [<a href="mailto:7acb9eae627482f2-87794@192.168.10.54" target="_blank">7acb9eae627482f2-87794@192.16<wbr>8.10.54</a>]</div><div>Dec 1 08:29:20 sipufa /usr/sbin/kamailio[18251]: NOTICE: <script>: ROOT: KDMQ usrloc [<a href="mailto:7acb9eae627482ef-87790@192.168.10.54" target="_blank">7acb9eae627482ef-87790@192.16<wbr>8.10.54</a>]</div></div><span class="m_6212761935717496828m_-844923629212803955HOEnZb"><font color="#888888"><div><br></div><div><br></div>-- <br><div class="m_6212761935717496828m_-844923629212803955m_-5583497781466523465gmail_signature">Aydar A. Kamalov</div>
</font></span></div></div>
<br></div></div>______________________________<wbr>_________________<br>
Kamailio (SER) - Users Mailing List<br>
<a href="mailto:sr-users@lists.kamailio.org" target="_blank">sr-users@lists.kamailio.org</a><br>
<a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" rel="noreferrer" target="_blank">https://lists.kamailio.org/cgi<wbr>-bin/mailman/listinfo/sr-users</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
Kamailio (SER) - Users Mailing List<br>
<a href="mailto:sr-users@lists.kamailio.org">sr-users@lists.kamailio.org</a><br>
<a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" rel="noreferrer" target="_blank">https://lists.kamailio.org/<wbr>cgi-bin/mailman/listinfo/sr-<wbr>users</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>
</div></div>
<br>
<div><font color="gray" style="font-size:10pt;font-family:Helvetica,Arial,sans-serif">Sipcentric Ltd.
Company registered in England & Wales no. 7365592.</font><span style="font-size:10pt;font-family:Helvetica,Arial,sans-serif"> </span><font color="gray" style="font-size:10pt;font-family:Helvetica,Arial,sans-serif">Registered
office: Faraday Wharf, Innovation Birmingham Campus, Holt Street, Birmingham Science Park, Birmingham B7 4BB.</font></div>