<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello,</p>
<p>the database connection is not initialized for that process based
on the c code, as it is started with no child init callback, where
the db connection is usually created:<br>
</p>
src/modules/dmq/dmq.c<br>
<p>283: newpid = fork_process(PROC_NOCHLDINIT, "DMQ
WORKER", 0);</p>
<p>Can you try with PROC_RPC instead of PROC_NOCHLDINIT and see if
works ok?</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div class="moz-cite-prefix">On 24.02.20 21:04, Joel Serrano wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAMtXxQ=_apaLZWc5drRSJPKwn60=MmGZjGB1phkd3BAUhuhk7w@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Hi Daniel,
<div><br>
</div>
<div>The process desc says: DMQ WORKER</div>
<div><br>
</div>
<div>Joel.</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Feb 24, 2020 at 11:55
AM Daniel-Constantin Mierla <<a
href="mailto:miconda@gmail.com" moz-do-not-send="true">miconda@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Can you get the type of the process throwing the error?
The pid is printed in the log message and the type you can
find in the description corresponding to the pid in the
output of:</p>
<p>kamctl ps</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 24.02.20 20:42, Joel Serrano wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Ok so my findings so far:
<div><br>
</div>
<div>1- db_mode=2 --> No errors.</div>
<div>2- db_mode=1 --> The reported error.</div>
<div>
<div>3- modparam("usrloc", "server_id_filter", 1)
--> Irrelevant, errors happen with or without it
on db_mode=1.</div>
<div><br>
</div>
<div>Anything else I can try in order to find out what
is happening? My next step was to enable mysql logs
or capture traffic and see what exactly is Kam
sending...</div>
<div><br>
</div>
<div>Let me know your thoughts.</div>
<div><br>
</div>
<div>Thanks, </div>
<div>Joel.</div>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Feb 24, 2020
at 11:26 AM Joel Serrano <<a
href="mailto:joel@textplus.com" target="_blank"
moz-do-not-send="true">joel@textplus.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi Henning,
<div><br>
</div>
<div>
<div>Every time I register, NodeA says OK, sends
location by DMQ to NodeB, nodeB logs the error.</div>
</div>
<div><br>
</div>
<div>I tried moving the VIP from NodeA to NodeB, and
the error stops happening on NodeA, and starts
happening on NodeB.</div>
<div><br>
</div>
<div>I did not see this happening with db_mode=2,
and I *think* it started happening when I enabled
the server_id related params? I'm not 100% sure of
this so I'm going to try some different
combinations to see if I can narrow it down a
little and provide more accurate details for you
guys to find the issue.</div>
<div><br>
</div>
<div>I'll report back shortly...</div>
<div><br>
</div>
<div>Thanks! </div>
<div>Joel.</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Feb 24,
2020 at 11:20 AM Henning Westerholt <<a
href="mailto:hw@skalatan.de" target="_blank"
moz-do-not-send="true">hw@skalatan.de</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div lang="DE">
<div>
<p class="MsoNormal"><span>Hello Joel,</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span lang="EN-GB">Feb 24
13:58:46 csbc02 csbc[9280]: ERROR:
<core> [db.c:481]: db_use_table():
invalid parameter value<br>
Feb 24 13:58:46 csbc02 csbc[9280]: ERROR:
usrloc [ucontact.c:670]:
db_insert_ucontact(): sql use_table failed
for: location</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">So the
table name seems to be ok - this function
also log this error if the DB connection
handle is not initialized. But still the
question why this happens.</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">Cheers,</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">Henning</span><span
lang="EN-GB"></span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">-- </span></p>
<p class="MsoNormal"><span lang="EN-GB">Henning
Westerholt – </span><span><a
href="https://skalatan.de/blog/"
target="_blank" moz-do-not-send="true"><span
style="color:rgb(5,99,193)"
lang="EN-GB">https://skalatan.de/blog/</span></a></span><span
lang="EN-GB"></span></p>
<p class="MsoNormal"><span lang="EN-GB">Kamailio
services – </span><span><a
href="https://gilawa.com/"
target="_blank" moz-do-not-send="true"><span
style="color:rgb(5,99,193)"
lang="EN-GB">https://gilawa.com</span></a></span><span>
<span lang="EN-GB"></span></span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"
style="margin-left:35.4pt"><b>From:</b>
sr-users <<a
href="mailto:sr-users-bounces@lists.kamailio.org"
target="_blank" moz-do-not-send="true">sr-users-bounces@lists.kamailio.org</a>>
<b>On Behalf Of </b>Joel Serrano<br>
<b>Sent:</b> Monday, February 24, 2020 8:10
PM<br>
<b>To:</b> Daniel-Constantin Mierla <<a
href="mailto:miconda@gmail.com"
target="_blank" moz-do-not-send="true">miconda@gmail.com</a>><br>
<b>Cc:</b> Kamailio (SER) - Users Mailing
List <<a
href="mailto:sr-users@lists.kamailio.org"
target="_blank" moz-do-not-send="true">sr-users@lists.kamailio.org</a>><br>
<b>Subject:</b> Re: [SR-Users] Database
errors with usrloc db_mode=1 and dmq_usrloc</p>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Hello Daniel, </p>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">I built from
master and re-tested, debug log:</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Feb 24
13:58:46 csbc02 csbc[9280]: DEBUG:
<core> [core/socket_info.c:635]:
grep_sock_info(): checking if port 5060
(advertise 0) matches port 5060<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:423]:
usrloc_dmq_execute_action(): Received
DMQ_UPDATE. Update contact info...<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:75]:
add_contact(): aor: 1000107<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:76]:
add_contact(): ci->ruid:
uloc-2-5e507d6a-1bd5-3<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:77]:
add_contact(): aorhash: 928379112<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:115]:
add_contact(): '1000107' Not found in
usrloc<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:117]:
add_contact(): Insert record<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: usrloc [ul_callback.h:83]:
run_ul_callbacks():
contact=0x7f3a8cfdf170, callback type
1/15, id 0 entered<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:800]:
dmq_ul_cb_contact(): Callback from
usrloc with type=1<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:826]:
dmq_ul_cb_contact(): Contact received
from DMQ... skip<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
ERROR: <core> [db.c:481]:
db_use_table(): invalid parameter value<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
ERROR: usrloc [ucontact.c:670]:
db_insert_ucontact(): sql use_table
failed for: location<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
ERROR: usrloc [urecord.c:598]:
insert_ucontact(): failed to insert in
database<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:121]:
add_contact(): Insert ucontact<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:124]:
add_contact(): Release record<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dmq_usrloc [usrloc_sync.c:126]:
add_contact(): Unlock udomain<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: sl [sl.c:305]: send_reply():
reply in stateless mode (sl)<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: <core>
[core/msg_translator.c:161]:
check_via_address(): (10.2.1.181,
10.2.1.181, 0)<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: dialog [dlg_cseq.c:116]:
dlg_cseq_prepare_new_msg(): prepare new
msg for cseq update operations<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: <core>
[core/parser/msg_parser.c:620]:
parse_msg(): SIP Reply (status):<br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: <core>
[core/parser/msg_parser.c:622]:
parse_msg(): version: <SIP/2.0><br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: <core>
[core/parser/msg_parser.c:624]:
parse_msg(): status: <200><br>
Feb 24 13:58:46 csbc02 csbc[9280]:
DEBUG: <core>
[core/parser/msg_parser.c:626]:
parse_msg(): reason: <OK></p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Thanks, </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Joel.</p>
</div>
</div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
<div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">On Mon, Feb
24, 2020 at 1:36 AM Daniel-Constantin
Mierla <<a
href="mailto:miconda@gmail.com"
target="_blank" moz-do-not-send="true">miconda@gmail.com</a>>
wrote:</p>
</div>
<blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt
solid rgb(204,204,204);padding:0cm 0cm 0cm
6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p style="margin-left:35.4pt">Hello,</p>
<p style="margin-left:35.4pt">the error
message:</p>
<p style="margin-left:35.4pt">Feb 21
00:11:25 csbc02 csbc[13916]: ERROR:
<core> [db.c:481]:
db_use_table(): invalid parameter
value</p>
<p style="margin-left:35.4pt">Is printed
if the database table name is not
given as parameter (eg, null or
empty). I pushed a patch to usrloc to
print the table in the log message,
can you try with latest master (or
patch from next commit) and send again
the log messages?</p>
<p style="margin-left:35.4pt"> * <a
href="https://github.com/kamailio/kamailio/commit/3843ae18599bfc1806ccb62df44bbc8ad41d60dd"
target="_blank"
moz-do-not-send="true">
https://github.com/kamailio/kamailio/commit/3843ae18599bfc1806ccb62df44bbc8ad41d60dd</a></p>
<p style="margin-left:35.4pt">Cheers,<br>
Daniel</p>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">On
21.02.20 22:40, Joel Serrano wrote:</p>
</div>
<blockquote
style="margin-top:5pt;margin-bottom:5pt">
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Hello,
</p>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">I
have a setup with 2 kamailio
in active/idle, locations are
shared via dmq_usrloc, but
database is also used as a
cache where other systems can
access and query "real-time"
data.</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">The
config we have is:</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">#
----- usrloc params -----<br>
modparam("usrloc", "db_url",
DBURL)<br>
modparam("usrloc", "db_mode",
1)<br>
modparam("usrloc", "db_load",
0)<br>
modparam("usrloc",
"db_insert_update", 1)<br>
modparam("usrloc",
"db_timer_clean", 1)<br>
modparam("usrloc",
"server_id_filter", 1)<br>
modparam("usrloc",
"timer_procs", 2)<br>
modparam("usrloc",
"timer_interval", 60)</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">#
----- dmq_usrloc params -----<br>
modparam("dmq_usrloc",
"enable", 1)<br>
modparam("dmq_usrloc", "sync",
1)<br>
modparam("dmq_usrloc",
"batch_msg_contacts", 50) #
50 contacts / message<br>
modparam("dmq_usrloc",
"batch_size", 10000) #
10000 contacts / batch<br>
modparam("dmq_usrloc",
"batch_usleep", 500000) #
one batch every 500ms</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">With
this setup, we assume:</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">1-
almost real-time updated
location info in the database.</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">2-
as long as one of the two Kam
is up, location information
will not be lost (as they will
sync with each other via DMQ)</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">3-
if both servers go down,
locations are lost (as they
will not reload from database
on startup).</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Our
tests work well with the above
config, with one exception, on
the idle node, we see a lot of
(only happens on the node
receiving location via dmq):</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Feb
21 00:11:25 csbc02
csbc[13916]: ERROR:
<core> [db.c:481]:
db_use_table(): invalid
parameter value<br>
Feb 21 00:11:25 csbc02
csbc[13916]: ERROR: usrloc
[ucontact.c:1136]:
db_update_ucontact_ruid(): sql
use_table failed<br>
Feb 21 00:11:25 csbc02
csbc[13916]: ERROR: usrloc
[ucontact.c:1657]:
update_contact_db(): failed to
update database</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">I
added debug logs to see if I
could understand the reason:</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Feb
21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:478]:
usrloc_dmq_handle_msg(): dmq
message received from <a
href="http://sip:usrloc@10.2.1.181:5050"
target="_blank"
moz-do-not-send="true">sip:usrloc@10.2.1.181:5050</a><br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:423]:
usrloc_dmq_execute_action():
Received DMQ_UPDATE. Update
contact info...<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:75]:
add_contact(): aor: 1063362<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:76]:
add_contact(): ci->ruid:
uloc-1-5e502cd3-5629-1<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:77]:
add_contact(): aorhash:
928261667<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:115]:
add_contact(): '1063362' Not
found in usrloc<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:117]:
add_contact(): Insert record<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: usrloc
[ul_callback.h:83]:
run_ul_callbacks():
contact=0x7fe41ff41f40,
callback type 1/15, id 0
entered<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:800]:
dmq_ul_cb_contact(): Callback
from usrloc with type=1<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:826]:
dmq_ul_cb_contact(): Contact
received from DMQ... skip<br>
Feb 21 14:38:34 csbc02
csbc[30796]: ERROR:
<core> [db.c:481]:
db_use_table(): invalid
parameter value<br>
Feb 21 14:38:34 csbc02
csbc[30796]: ERROR: usrloc
[ucontact.c:669]:
db_insert_ucontact(): sql
use_table failed<br>
Feb 21 14:38:34 csbc02
csbc[30796]: ERROR: usrloc
[urecord.c:598]:
insert_ucontact(): failed to
insert in database<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:121]:
add_contact(): Insert ucontact<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:124]:
add_contact(): Release record<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: dmq_usrloc
[usrloc_sync.c:126]:
add_contact(): Unlock udomain<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG: sl
[sl.c:305]: send_reply():
reply in stateless mode (sl)<br>
Feb 21 14:38:34 csbc02
csbc[30796]: DEBUG:
<core>
[core/msg_translator.c:161]:
check_via_address():
(10.2.1.181, 10.2.1.181, 0)</p>
</div>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Can
anyone help me understand what's
the problem? I have the
server_id setting, and the
modparam server_id_filter, so
shouldn't the receiving node
just ignore any database tasks?</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Thanks, </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt">Joel.</p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
<div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
</div>
</div>
<p class="MsoNormal"
style="margin-left:35.4pt"> </p>
<pre style="margin-left:35.4pt">_______________________________________________</pre>
<pre style="margin-left:35.4pt">Kamailio (SER) - Users Mailing List</pre>
<pre style="margin-left:35.4pt"><a href="mailto:sr-users@lists.kamailio.org" target="_blank" moz-do-not-send="true">sr-users@lists.kamailio.org</a></pre>
<pre style="margin-left:35.4pt"><a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank" moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a></pre>
</blockquote>
<pre style="margin-left:35.4pt">-- </pre>
<pre style="margin-left:35.4pt">Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a></pre>
<pre style="margin-left:35.4pt"><a href="http://www.twitter.com/miconda" target="_blank" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank" moz-do-not-send="true">www.linkedin.com/in/miconda</a></pre>
<pre style="margin-left:35.4pt">Kamailio Advanced Training - March 9-11, 2020, Berlin - <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a></pre>
<pre style="margin-left:35.4pt">Kamailio World Conference - April 27-29, 2020, in Berlin -- <a href="http://www.kamailioworld.com" target="_blank" moz-do-not-send="true">www.kamailioworld.com</a></pre>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank" moz-do-not-send="true">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - March 9-11, 2020, Berlin - <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a>
Kamailio World Conference - April 27-29, 2020, in Berlin -- <a href="http://www.kamailioworld.com" target="_blank" moz-do-not-send="true">www.kamailioworld.com</a></pre>
</div>
</blockquote>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla -- <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/miconda">www.twitter.com/miconda</a> -- <a class="moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - March 9-11, 2020, Berlin - <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a>
Kamailio World Conference - April 27-29, 2020, in Berlin -- <a class="moz-txt-link-abbreviated" href="http://www.kamailioworld.com">www.kamailioworld.com</a></pre>
</body>
</html>