<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Again, not to be blamed later: I am not sure you face the same
kind of issue like reported in the past. I was looking at it from
the perspective that db_mysql and carrierroute modules were not
touched that much in the 5.4.x series recently.</p>
<p>The libssl did a lot of internal/api changes between 1.0 and 1.1,
so for a few past years many libs and apps (including kamailio)
faced some "strange" crashes/behaviours. Now seems stabilized and,
at least on debian 10, there were no related open issues recently.
In the future we may need to make --atexit=no the default (maybe
needs to be set even on latest ubuntu 20.04), it's the last I
remember that surfaced and it appeared after a minor version
upgrade of libssl1.1, iirc.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div class="moz-cite-prefix">On 03.06.21 15:31, Andrew Chen wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAEytwJ2uAgmtrOnDuQ-NqQ-Pw=JtiUWCUhEcFxA=ko+3Bixxkw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Thanks Daniel.
<div>It sounds like upgrading to a newer Ubuntu seems like the
cleanest way of resolving this issue.</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Jun 3, 2021 at 8:58 AM
Daniel-Constantin Mierla <<a
href="mailto:miconda@gmail.com" moz-do-not-send="true">miconda@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello,</p>
<p>it is not related to kamailio tls being compiled with
libssl1.1. The issues were observed on the systems having
both libssl1.0 (which was somehow the default choice) and
libssl1.1. I got the chance to test a bit and it was still
crashing even I compiled manually with libssl1.1. I didn't
dig much in the external libraries given that upgrading OS
seemed to fix it. There is a mod param for db_mysql that
you can try to play with: opt_ssl_mode - I don't recall
solving much for mysql servers requiring tls.<br>
</p>
<p>Otherwise, on a different direction: if you can get
mariadb client library, you can try to compile kamailio
against it (the db_mysql module) -- if the MySQL server is
on a remote system, it should work.</p>
<p>I can't confirm it is same issue, but if you upgraded in
kamailio 5.4.x series, there were no changes to
carrierroute or db_mysql to <br>
</p>
<p>As a side note, for newer operating systems and newer
libssl, it may also require to set --atexit cli parameter
for kamailio.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 03.06.21 14:42, Andrew Chen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Daniel,
<div><br>
</div>
<div>I just confirmed mysql is compiled using
libssl.so.1.1.</div>
<div><br>
</div>
<div>As for the other kamailio packages, I only see two
that are installed where both are depends on libssl1.1</div>
<div><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:11px"><br>
</span></div>
<div><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:11px">kamailio-tls-modules:amd64</span></div>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span
style="font-variant-ligatures:no-common-ligatures"></span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>Depends</b></span><span
style="font-variant-ligatures:no-common-ligatures">:
kamailio (= 5.4.5), libc6 (>= 2.14), libcurl4
(>= 7.16.2), libssl1.1 (>= 1.1.0)</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">kamailio-websocket-modules:amd64</span></p>
<p style="margin:0px;font:11px
Menlo;color:rgb(0,0,0);min-height:13px"><span
style="font-variant-ligatures:no-common-ligatures"></span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>Depends</b></span><span
style="font-variant-ligatures:no-common-ligatures">:
kamailio (= 5.4.5), libc6 (>= 2.14), libssl1.1
(>= 1.1.0), libunistring2 (>= 0.9.7)</span></p>
<div><br>
</div>
<div>So in my case here there shouldn't be any conflict
with mysql client libssl version?</div>
<div><br>
</div>
<div>Thanks.</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Jun 3, 2021 at
3:10 AM Daniel-Constantin Mierla <<a
href="mailto:miconda@gmail.com" target="_blank"
moz-do-not-send="true">miconda@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello,</p>
<p>I think the relevant here are the client library.
I asked about mysql vs mariadb because some distro
had (meta-)packages named mysql but installing
mariadb. Debian at some point switched to
something like default-mysql-client...</p>
<p>Based on backtrace, the crashes would prevent
kamailio to start, as carrierroute_db_open() is
used in mod_init/child_init callbacks.</p>
<p>A variant you could try is the use of mysql-proxy
to bridge from a local unix socket to remote
mysql, so kamailio connects to mysql server via
local unix socket and by that won't use libssl.</p>
<p>First you can check with ldd if mysql client
libraries are compiled against libssl1.1 and the
libssl1.0 is also used for other packages.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 02.06.21 21:56, Andrew Chen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Hi Daniel,
<div>So our DB is in rds and this is what I
can gather:</div>
<div>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">+-------------------------+------------------------------+</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| Variable_name <span>
</span>| Value<span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">+-------------------------+------------------------------+</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| innodb_version<span>
</span>| 5.6.34 <span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| protocol_version<span>
</span>| 10 <span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">|
slave_type_conversions<span> </span>|<span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| version <span>
</span>| 5.6.34-log <span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| version_comment <span>
</span>| MySQL Community Server
(GPL) |</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">|
version_compile_machine | x86_64 <span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| version_compile_os<span>
</span>| Linux<span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">+-------------------------+------------------------------+</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">The mysql
server/client library we received from
my apt-cache madison output earlier.</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">I don't know if
upgrading to 20.04 is an option.</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">Given this said, what
are the symptoms of these crashes? I'm
guessing one of them is a brief
disconnect from the DB for
dialplan/dispatcher/carrierroute lookup?</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">Thanks.</span></p>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Jun
2, 2021 at 3:38 PM Daniel-Constantin Mierla
<<a href="mailto:miconda@gmail.com"
target="_blank" moz-do-not-send="true">miconda@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello,</p>
<p>did you upgrade the Kamailio only or
also mysql server/client library? Is it
Oracle's MySQL or MariaDB?</p>
<p>A while ago, there were a few reports
of problems with mysql-client (Oracle)
library on Ubuntu 16.04 and 18.04, which
typically happened after upgrades of the
library, as I understood, the upgrade
brought in a dependency on libssl 1.1,
while libssl1.0 was used by most of the
other packages, resulting in mixed
deployments. Upgrading to 20.04 seemed
to solve it, as it came with libssl1.1
only.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 02.06.21 20:51, Andrew Chen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi all,
<div><br>
</div>
<div>Today I upgraded Kamailio to
5.4.5 and a random crash was just
reported in our crash monitoring
script. Below is the output of the
crash:</div>
<div><br>
</div>
<div>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#3<span> </span>0x00007fa149ebe58d
in ?? () from
/usr/lib/x86_64-linux-gnu/kamailio/modules/tls.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#4<span> </span>0x00007fa14989e277
in ERR_clear_error () from
/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#5<span> </span>0x00007fa149933d81
in X509_STORE_set_default_paths
() from
/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#6<span> </span>0x00007fa10310d374
in ?? () from
/usr/lib/x86_64-linux-gnu/libmysqlclient.so.20</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#7<span> </span>0x00007fa10310d9f3
in ?? () from
/usr/lib/x86_64-linux-gnu/libmysqlclient.so.20</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#8<span> </span>0x00007fa1030e0427
in mysql_real_connect () from
/usr/lib/x86_64-linux-gnu/libmysqlclient.so.20</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#9<span> </span>0x00007fa10368428c
in db_mysql_new_connection ()
from
/usr/lib/x86_64-linux-gnu/kamailio/modules/db_mysql.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#10
0x00007fa1501f96dc in
db_do_init2 () from
/usr/lib/x86_64-linux-gnu/kamailio/libsrdb1.so.1</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#11
0x00007fa1501f83f9 in db_do_init
() from
/usr/lib/x86_64-linux-gnu/kamailio/libsrdb1.so.1</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#12
0x00007fa103689710 in
db_mysql_init () from
/usr/lib/x86_64-linux-gnu/kamailio/modules/db_mysql.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#13
0x00007fa15062d35f in
carrierroute_db_open () from
/usr/lib/x86_64-linux-gnu/kamailio/modules/carrierroute.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#14
0x00007fa1506719c6 in ?? () from
/usr/lib/x86_64-linux-gnu/kamailio/modules/carrierroute.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
</div>
<div>
<div>Is this crash due to libcrypto
or mysql client?</div>
<div>Currently I have the following
mysql client installed on the
kamailio instance:</div>
<div><br>
</div>
<div>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span>lib</span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">client20:amd64 <span>
</span>5.7.34-0ubuntu0.18.04.1
<span> </span>amd64<span>
</span>MySQL database
client library</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span></span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">-client <span>
</span>5.7.34-0ubuntu0.18.04.1
<span> </span>all<span>
</span>MySQL
database client (metapackage
depending on the latest
version)</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span></span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">-client-5.7 <span>
</span>5.7.34-0ubuntu0.18.04.1
<span> </span>amd64<span>
</span>MySQL database
client binaries</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span></span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">-client-core-5.7<span>
</span>5.7.34-0ubuntu0.18.04.1
<span> </span>amd64<span>
</span>MySQL database
core client binaries</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span></span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">-common <span>
</span>5.8+1.0.4
<span>
</span>all<span>
</span>MySQL database common
files, e.g. /etc/</span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">/my.cnf</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(170,171,37)"><span
style="font-variant-ligatures:no-common-ligatures">root@ashintgtpsg51:/var/lib/cores
# </span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">apt-cache
madison mysql-client</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">mysql-client |
5.7.34-0ubuntu0.18.04.1 | <a
href="http://us-east-1.ec2.archive.ubuntu.com/ubuntu" target="_blank"
moz-do-not-send="true">http://us-east-1.ec2.archive.ubuntu.com/ubuntu</a>
bionic-updates/main amd64
Packages</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">mysql-client |
5.7.34-0ubuntu0.18.04.1 | <a
href="http://security.ubuntu.com/ubuntu" target="_blank"
moz-do-not-send="true">http://security.ubuntu.com/ubuntu</a>
bionic-security/main amd64
Packages</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">mysql-client |
5.7.21-1ubuntu1 | <a
href="http://us-east-1.ec2.archive.ubuntu.com/ubuntu"
target="_blank"
moz-do-not-send="true">http://us-east-1.ec2.archive.ubuntu.com/ubuntu</a>
bionic/main amd64 Packages</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(170,171,37)"><span
style="font-variant-ligatures:no-common-ligatures">root@ashintgtpsg51:/var/lib/cores
#<span> </span></span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(170,171,37)"><br>
</p>
</div>
<div>Any suggestions would be
greatly appreciated.</div>
<div>Thanks in advance.</div>
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr"
style="font-size:12.8px">Andy
Chen</div>
<div dir="ltr"
style="font-size:12.8px">
<div>Sr. Telephony Lead
Engineer</div>
<div><a
href="mailto:achen@thinkingphones.com"
target="_blank"
moz-do-not-send="true">achen@</a><a
href="http://fuze.com"
target="_blank"
moz-do-not-send="true">fuze.com</a></div>
<div><br>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">*Confidentiality
Notice: The information contained in
this e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
may be confidential. If you are not an
intended recipient, you</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">are
hereby notified that any
dissemination, distribution or copying
of this</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">e-mail
is strictly prohibited. If you have
received this e-mail in error,</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">please
notify the sender and permanently
delete the e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
immediately. You should not retain,
copy or use this e-mail or</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">any
attachment for any purpose, nor
disclose all or any part of the</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">contents
to any other person. Thank you.*</span>
<br>
<fieldset></fieldset>
<pre>__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
* <a href="mailto:sr-users@lists.kamailio.org" target="_blank" moz-do-not-send="true">sr-users@lists.kamailio.org</a>
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
* <a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank" moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
</blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank" moz-do-not-send="true">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - Online - June 7-10, 2021 (America Timezone)
* <a href="https://www.asipto.com/sw/kamailio-advanced-training-online/" target="_blank" moz-do-not-send="true">https://www.asipto.com/sw/kamailio-advanced-training-online/</a></pre>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr" style="font-size:12.8px">Andy
Chen</div>
<div dir="ltr" style="font-size:12.8px">
<div>Sr. Telephony Lead Engineer</div>
<div>415 516 5535 (M)</div>
<div><a
href="mailto:achen@thinkingphones.com"
target="_blank"
moz-do-not-send="true">achen@</a><a
href="http://fuze.com"
target="_blank"
moz-do-not-send="true">fuze.com</a></div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">*Confidentiality
Notice: The information contained in this e-mail
and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
may be confidential. If you are not an intended
recipient, you</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">are
hereby notified that any dissemination,
distribution or copying of this</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">e-mail
is strictly prohibited. If you have received
this e-mail in error,</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">please
notify the sender and permanently delete the
e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
immediately. You should not retain, copy or use
this e-mail or</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">any
attachment for any purpose, nor disclose all or
any part of the</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">contents
to any other person. Thank you.*</span> </blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank" moz-do-not-send="true">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - Online - June 7-10, 2021 (America Timezone)
* <a href="https://www.asipto.com/sw/kamailio-advanced-training-online/" target="_blank" moz-do-not-send="true">https://www.asipto.com/sw/kamailio-advanced-training-online/</a></pre>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr" style="font-size:12.8px">Andy Chen</div>
<div dir="ltr" style="font-size:12.8px">
<div>Sr. Telephony Lead Engineer</div>
<div>415 516 5535 (M)</div>
<div><a href="mailto:achen@thinkingphones.com"
target="_blank" moz-do-not-send="true">achen@</a><a
href="http://fuze.com" target="_blank"
moz-do-not-send="true">fuze.com</a></div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">*Confidentiality
Notice: The information contained in this e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
may be confidential. If you are not an intended
recipient, you</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">are
hereby notified that any dissemination, distribution or
copying of this</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">e-mail
is strictly prohibited. If you have received this e-mail
in error,</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">please
notify the sender and permanently delete the e-mail and
any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
immediately. You should not retain, copy or use this
e-mail or</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">any
attachment for any purpose, nor disclose all or any part
of the</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">contents
to any other person. Thank you.*</span> </blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank" moz-do-not-send="true">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - Online - June 7-10, 2021 (America Timezone)
* <a href="https://www.asipto.com/sw/kamailio-advanced-training-online/" target="_blank" moz-do-not-send="true">https://www.asipto.com/sw/kamailio-advanced-training-online/</a></pre>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr" style="font-size:12.8px">Andy Chen</div>
<div dir="ltr" style="font-size:12.8px">
<div>Sr. Telephony Lead Engineer</div>
<div>415 516 5535 (M)</div>
<div><a href="mailto:achen@thinkingphones.com"
target="_blank" moz-do-not-send="true">achen@</a><a
href="http://fuze.com" target="_blank"
moz-do-not-send="true">fuze.com</a></div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">*Confidentiality
Notice: The information contained in this e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
may be confidential. If you are not an intended recipient, you</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">are
hereby notified that any dissemination, distribution or copying
of this</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">e-mail
is strictly prohibited. If you have received this e-mail in
error,</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">please
notify the sender and permanently delete the e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
immediately. You should not retain, copy or use this e-mail or</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">any
attachment for any purpose, nor disclose all or any part of the</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">contents
to any other person. Thank you.*</span>
</blockquote>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla -- <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/miconda">www.twitter.com/miconda</a> -- <a class="moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - Online - June 7-10, 2021 (America Timezone)
* <a class="moz-txt-link-freetext" href="https://www.asipto.com/sw/kamailio-advanced-training-online/">https://www.asipto.com/sw/kamailio-advanced-training-online/</a></pre>
</body>
</html>