<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello,</p>
<p>it is not related to kamailio tls being compiled with libssl1.1.
The issues were observed on the systems having both libssl1.0
(which was somehow the default choice) and libssl1.1. I got the
chance to test a bit and it was still crashing even I compiled
manually with libssl1.1. I didn't dig much in the external
libraries given that upgrading OS seemed to fix it. There is a mod
param for db_mysql that you can try to play with: opt_ssl_mode - I
don't recall solving much for mysql servers requiring tls.<br>
</p>
<p>Otherwise, on a different direction: if you can get mariadb
client library, you can try to compile kamailio against it (the
db_mysql module) -- if the MySQL server is on a remote system, it
should work.</p>
<p>I can't confirm it is same issue, but if you upgraded in kamailio
5.4.x series, there were no changes to carrierroute or db_mysql to
<br>
</p>
<p>As a side note, for newer operating systems and newer libssl, it
may also require to set --atexit cli parameter for kamailio.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div class="moz-cite-prefix">On 03.06.21 14:42, Andrew Chen wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAEytwJ1E86FQjZ+2h8mhLnrGwbHu7w-YvEyBu0g-fjLzCuUNYw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Hi Daniel,
<div><br>
</div>
<div>I just confirmed mysql is compiled using libssl.so.1.1.</div>
<div><br>
</div>
<div>As for the other kamailio packages, I only see two that are
installed where both are depends on libssl1.1</div>
<div><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:11px"><br>
</span></div>
<div><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:11px">kamailio-tls-modules:amd64</span></div>
<p class="gmail-p2"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span
class="gmail-s1"
style="font-variant-ligatures:no-common-ligatures"></span></p>
<p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
class="gmail-s2"
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>Depends</b></span><span
class="gmail-s1"
style="font-variant-ligatures:no-common-ligatures">:
kamailio (= 5.4.5), libc6 (>= 2.14), libcurl4 (>=
7.16.2), libssl1.1 (>= 1.1.0)</span></p>
<p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
class="gmail-s1"
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
class="gmail-s1"
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p class="gmail-p1" style="margin:0px;font:11px
Menlo;color:rgb(0,0,0)"><span class="gmail-s1"
style="font-variant-ligatures:no-common-ligatures">kamailio-websocket-modules:amd64</span></p>
<p class="gmail-p2" style="margin:0px;font:11px
Menlo;color:rgb(0,0,0);min-height:13px"><span class="gmail-s1"
style="font-variant-ligatures:no-common-ligatures"></span></p>
<p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
class="gmail-s1"
style="font-variant-ligatures:no-common-ligatures">
</span></p>
<p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
class="gmail-s2"
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>Depends</b></span><span
class="gmail-s1"
style="font-variant-ligatures:no-common-ligatures">:
kamailio (= 5.4.5), libc6 (>= 2.14), libssl1.1 (>=
1.1.0), libunistring2 (>= 0.9.7)</span></p>
<div><br>
</div>
<div>So in my case here there shouldn't be any conflict with
mysql client libssl version?</div>
<div><br>
</div>
<div>Thanks.</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Jun 3, 2021 at 3:10 AM
Daniel-Constantin Mierla <<a
href="mailto:miconda@gmail.com" moz-do-not-send="true">miconda@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello,</p>
<p>I think the relevant here are the client library. I asked
about mysql vs mariadb because some distro had
(meta-)packages named mysql but installing mariadb. Debian
at some point switched to something like
default-mysql-client...</p>
<p>Based on backtrace, the crashes would prevent kamailio to
start, as carrierroute_db_open() is used in
mod_init/child_init callbacks.</p>
<p>A variant you could try is the use of mysql-proxy to
bridge from a local unix socket to remote mysql, so
kamailio connects to mysql server via local unix socket
and by that won't use libssl.</p>
<p>First you can check with ldd if mysql client libraries
are compiled against libssl1.1 and the libssl1.0 is also
used for other packages.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 02.06.21 21:56, Andrew Chen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Hi Daniel,
<div>So our DB is in rds and this is what I can
gather:</div>
<div>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">+-------------------------+------------------------------+</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| Variable_name <span>
</span>| Value<span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">+-------------------------+------------------------------+</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| innodb_version<span>
</span>| 5.6.34 <span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| protocol_version<span>
</span>| 10 <span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">|
slave_type_conversions<span> </span>|<span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| version <span>
</span>| 5.6.34-log <span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| version_comment <span>
</span>| MySQL Community Server (GPL) |</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">|
version_compile_machine | x86_64 <span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">| version_compile_os<span>
</span>| Linux<span>
</span>|</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">+-------------------------+------------------------------+</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">The mysql
server/client library we received from my
apt-cache madison output earlier.</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">I don't know if
upgrading to 20.04 is an option.</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">Given this said, what
are the symptoms of these crashes? I'm guessing
one of them is a brief disconnect from the DB
for dialplan/dispatcher/carrierroute lookup?</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">Thanks.</span></p>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Jun 2, 2021
at 3:38 PM Daniel-Constantin Mierla <<a
href="mailto:miconda@gmail.com" target="_blank"
moz-do-not-send="true">miconda@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello,</p>
<p>did you upgrade the Kamailio only or also mysql
server/client library? Is it Oracle's MySQL or
MariaDB?</p>
<p>A while ago, there were a few reports of
problems with mysql-client (Oracle) library on
Ubuntu 16.04 and 18.04, which typically happened
after upgrades of the library, as I understood,
the upgrade brought in a dependency on libssl
1.1, while libssl1.0 was used by most of the
other packages, resulting in mixed deployments.
Upgrading to 20.04 seemed to solve it, as it
came with libssl1.1 only.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div>On 02.06.21 20:51, Andrew Chen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi all,
<div><br>
</div>
<div>Today I upgraded Kamailio to 5.4.5 and a
random crash was just reported in our crash
monitoring script. Below is the output of
the crash:</div>
<div><br>
</div>
<div>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#3<span> </span>0x00007fa149ebe58d
in ?? () from
/usr/lib/x86_64-linux-gnu/kamailio/modules/tls.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#4<span> </span>0x00007fa14989e277
in ERR_clear_error () from
/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#5<span> </span>0x00007fa149933d81
in X509_STORE_set_default_paths () from
/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#6<span> </span>0x00007fa10310d374
in ?? () from
/usr/lib/x86_64-linux-gnu/libmysqlclient.so.20</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#7<span> </span>0x00007fa10310d9f3
in ?? () from
/usr/lib/x86_64-linux-gnu/libmysqlclient.so.20</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#8<span> </span>0x00007fa1030e0427
in mysql_real_connect () from
/usr/lib/x86_64-linux-gnu/libmysqlclient.so.20</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#9<span> </span>0x00007fa10368428c
in db_mysql_new_connection () from
/usr/lib/x86_64-linux-gnu/kamailio/modules/db_mysql.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#10
0x00007fa1501f96dc in db_do_init2 ()
from
/usr/lib/x86_64-linux-gnu/kamailio/libsrdb1.so.1</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#11
0x00007fa1501f83f9 in db_do_init () from
/usr/lib/x86_64-linux-gnu/kamailio/libsrdb1.so.1</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#12
0x00007fa103689710 in db_mysql_init ()
from
/usr/lib/x86_64-linux-gnu/kamailio/modules/db_mysql.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#13
0x00007fa15062d35f in
carrierroute_db_open () from
/usr/lib/x86_64-linux-gnu/kamailio/modules/carrierroute.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">#14
0x00007fa1506719c6 in ?? () from
/usr/lib/x86_64-linux-gnu/kamailio/modules/carrierroute.so</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures"><br>
</span></p>
</div>
<div>
<div>Is this crash due to libcrypto or mysql
client?</div>
<div>Currently I have the following mysql
client installed on the kamailio instance:</div>
<div><br>
</div>
<div>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span>lib</span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">client20:amd64 <span>
</span>5.7.34-0ubuntu0.18.04.1
<span> </span>amd64<span>
</span>MySQL database client
library</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span></span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">-client <span>
</span>5.7.34-0ubuntu0.18.04.1
<span> </span>all<span>
</span>MySQL database client
(metapackage depending on the latest
version)</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span></span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">-client-5.7 <span>
</span>5.7.34-0ubuntu0.18.04.1
<span> </span>amd64<span>
</span>MySQL database client
binaries</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span></span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">-client-core-5.7<span>
</span>5.7.34-0ubuntu0.18.04.1
<span> </span>amd64<span>
</span>MySQL database core
client binaries</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">ii<span> </span></span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">-common <span>
</span>5.8+1.0.4
<span> </span>all<span>
</span>MySQL database
common files, e.g. /etc/</span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b>mysql</b></span><span
style="font-variant-ligatures:no-common-ligatures">/my.cnf</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(170,171,37)"><span
style="font-variant-ligatures:no-common-ligatures">root@ashintgtpsg51:/var/lib/cores
# </span><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">apt-cache
madison mysql-client</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">mysql-client |
5.7.34-0ubuntu0.18.04.1 | <a
href="http://us-east-1.ec2.archive.ubuntu.com/ubuntu"
target="_blank"
moz-do-not-send="true">http://us-east-1.ec2.archive.ubuntu.com/ubuntu</a>
bionic-updates/main amd64 Packages</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">mysql-client |
5.7.34-0ubuntu0.18.04.1 | <a
href="http://security.ubuntu.com/ubuntu"
target="_blank"
moz-do-not-send="true">http://security.ubuntu.com/ubuntu</a>
bionic-security/main amd64 Packages</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">mysql-client |
5.7.21-1ubuntu1 | <a
href="http://us-east-1.ec2.archive.ubuntu.com/ubuntu"
target="_blank"
moz-do-not-send="true">http://us-east-1.ec2.archive.ubuntu.com/ubuntu</a>
bionic/main amd64 Packages</span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(170,171,37)"><span
style="font-variant-ligatures:no-common-ligatures">root@ashintgtpsg51:/var/lib/cores
#<span> </span></span></p>
<p
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(170,171,37)"><br>
</p>
</div>
<div>Any suggestions would be greatly
appreciated.</div>
<div>Thanks in advance.</div>
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr"
style="font-size:12.8px">Andy Chen</div>
<div dir="ltr"
style="font-size:12.8px">
<div>Sr. Telephony Lead Engineer</div>
<div><a
href="mailto:achen@thinkingphones.com"
target="_blank"
moz-do-not-send="true">achen@</a><a
href="http://fuze.com"
target="_blank"
moz-do-not-send="true">fuze.com</a></div>
<div><br>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">*Confidentiality
Notice: The information contained in this
e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
may be confidential. If you are not an
intended recipient, you</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">are
hereby notified that any dissemination,
distribution or copying of this</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">e-mail
is strictly prohibited. If you have received
this e-mail in error,</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">please
notify the sender and permanently delete the
e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
immediately. You should not retain, copy or
use this e-mail or</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">any
attachment for any purpose, nor disclose all
or any part of the</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">contents
to any other person. Thank you.*</span> <br>
<fieldset></fieldset>
<pre>__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
* <a href="mailto:sr-users@lists.kamailio.org" target="_blank" moz-do-not-send="true">sr-users@lists.kamailio.org</a>
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
* <a href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank" moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
</blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank" moz-do-not-send="true">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - Online - June 7-10, 2021 (America Timezone)
* <a href="https://www.asipto.com/sw/kamailio-advanced-training-online/" target="_blank" moz-do-not-send="true">https://www.asipto.com/sw/kamailio-advanced-training-online/</a></pre>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr" style="font-size:12.8px">Andy
Chen</div>
<div dir="ltr" style="font-size:12.8px">
<div>Sr. Telephony Lead Engineer</div>
<div>415 516 5535 (M)</div>
<div><a href="mailto:achen@thinkingphones.com"
target="_blank" moz-do-not-send="true">achen@</a><a
href="http://fuze.com" target="_blank"
moz-do-not-send="true">fuze.com</a></div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">*Confidentiality
Notice: The information contained in this e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
may be confidential. If you are not an intended
recipient, you</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">are
hereby notified that any dissemination, distribution or
copying of this</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">e-mail
is strictly prohibited. If you have received this e-mail
in error,</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">please
notify the sender and permanently delete the e-mail and
any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
immediately. You should not retain, copy or use this
e-mail or</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">any
attachment for any purpose, nor disclose all or any part
of the</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">contents
to any other person. Thank you.*</span> </blockquote>
<pre cols="72">--
Daniel-Constantin Mierla -- <a href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a>
<a href="http://www.twitter.com/miconda" target="_blank" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a href="http://www.linkedin.com/in/miconda" target="_blank" moz-do-not-send="true">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - Online - June 7-10, 2021 (America Timezone)
* <a href="https://www.asipto.com/sw/kamailio-advanced-training-online/" target="_blank" moz-do-not-send="true">https://www.asipto.com/sw/kamailio-advanced-training-online/</a></pre>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr" style="font-size:12.8px">Andy Chen</div>
<div dir="ltr" style="font-size:12.8px">
<div>Sr. Telephony Lead Engineer</div>
<div>415 516 5535 (M)</div>
<div><a href="mailto:achen@thinkingphones.com"
target="_blank" moz-do-not-send="true">achen@</a><a
href="http://fuze.com" target="_blank"
moz-do-not-send="true">fuze.com</a></div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">*Confidentiality
Notice: The information contained in this e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
may be confidential. If you are not an intended recipient, you</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">are
hereby notified that any dissemination, distribution or copying
of this</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">e-mail
is strictly prohibited. If you have received this e-mail in
error,</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">please
notify the sender and permanently delete the e-mail and any</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">attachments
immediately. You should not retain, copy or use this e-mail or</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">any
attachment for any purpose, nor disclose all or any part of the</span><br
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">
<span
style="color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">contents
to any other person. Thank you.*</span>
</blockquote>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla -- <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/miconda">www.twitter.com/miconda</a> -- <a class="moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - Online - June 7-10, 2021 (America Timezone)
* <a class="moz-txt-link-freetext" href="https://www.asipto.com/sw/kamailio-advanced-training-online/">https://www.asipto.com/sw/kamailio-advanced-training-online/</a></pre>
</body>
</html>