<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI Emoji";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.E-MailFormatvorlage19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi Noah,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">sure – let me give you some pointers. So basically, the SQL files are generated from the XSL infrastructure in the quoted directory. This file e.g. is for postgres:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><a href="https://github.com/kamailio/kamailio/blob/5.4/doc/stylesheets/dbschema/xsl/postgres.xsl">https://github.com/kamailio/kamailio/blob/5.4/doc/stylesheets/dbschema/xsl/postgres.xsl</a><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">You basically need to copy it to a new file and adapt the types in it to the cockroachdb types. If you execute “make dbschema” in the kamailio source tree, it will generate all the SQL
files. Then you could generate the appropriate SQL files also for your database and it will stay in sync after future changes. There might be also a small extension necessary in the Makefile, but we can have a look to this later on.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">About the questions why the SQL files are then also checked in after creation – because otherwise everybody needs to install the xstl dependencies just for installing Kamailio.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">About the rand()/random() topic – I did not find anything in the LCR module as well. It might be obsolete. I would consider dropping this, maybe after asking on the sr-dev list for this
again.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Cheers,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Henning<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">-- <o:p>
</o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Henning Westerholt –
</span><span style="mso-fareast-language:EN-US"><a href="https://skalatan.de/blog/"><span lang="EN-GB" style="color:#0563C1">https://skalatan.de/blog/</span></a></span><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US">Kamailio services –
</span><span style="mso-fareast-language:EN-US"><a href="https://gilawa.com/"><span lang="EN-GB" style="color:#0563C1">https://gilawa.com</span></a></span><span style="mso-fareast-language:EN-US">
<span lang="EN-GB"><o:p></o:p></span></span></p>
</div>
<p class="MsoNormal"><span lang="EN-GB" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-left:35.4pt"><b>From:</b> Noah Mehl <noahmehl@gmail.com>
<br>
<b>Sent:</b> Sunday, February 21, 2021 8:47 PM<br>
<b>To:</b> Jonathan Hunter <hunterj91@hotmail.com><br>
<b>Cc:</b> Henning Westerholt <hw@skalatan.de>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org><br>
<b>Subject:</b> Re: Cockroachdb and kamailio 5.4<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt">Jon,<o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">I’m not sure what would get my branch accepted. Henning mentioned on 9/16/2020 that the .sql files are generated from XML/XLST scripts, I have found: <a href="https://github.com/kamailio/kamailio/tree/5.4/doc/stylesheets/dbschema">https://github.com/kamailio/kamailio/tree/5.4/doc/stylesheets/dbschema</a> which
was updated just 3 days ago. However, I don’t understand how this is used to generate the .sql files for Postgres. I’m also confused as to why the .sql files are checked into the repository if they’re generated? Henning, can you point me in the right direction?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">That being said, this is what’s <a href="https://github.com/kamailio/kamailio/compare/5.4...reperio:cockroachdb-compat?expand=1">changed in the branch</a>:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">kamdbctl.pgsql<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">- I’ve updated the psql command for my preferences regarding output<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">- I’ve made the function checking more verbose<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">- I’ve added the gen_random_uuid() function (by adding pgcrypto to Postgres, it’s native in CockroachDB), this isn’t required, but we are using UUID for usrloc in production<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">- I’ve updated the GRANT commands so they’re compatible with both Postgres and CockroachDB<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Some things to note:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">- concat() is native to CockroachDB, so the CREATE FUNCTION is only necessary for Postgres<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">- rand() is the native function name in MySQL, but random() is the function name in Postgres and CockroachDB. This is where I’m most concerned because the file says it’s used in the lcr module, but I cannot find
where it is used. Does anyone know how to ascertain this? Anyways, they’re the same function, so it’s a little silly to require a CREATE FUNCTION duplicating the exact functionality of an existing native function.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">The rest of the changes have to do with modifying the create statements to not use SERIAL, but use the more verbose SEQUENCE + nextval(). It’s identical in practice, so there’s 0 risk there.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">I think overall risk is low for the branch, as the branch only changes the utility that creates the DBs. As for production use, it’s worked great for us, there have been no issues.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">~Noah<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:35.4pt">On Feb 18, 2021, at 2:45 PM, Jonathan Hunter <<a href="mailto:hunterj91@hotmail.com">hunterj91@hotmail.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Hi Noah,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Hope you are well?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">I work as a consultant for a company in the UK, and I am building a new hosted telephony platform for them in docker initially, and as we are deploying across multiple servers they want to use cockroachdb to allow
easy management of a cluster environment.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">I could see from your posts you got it working using your own branch, and I wondered what changes you made to make things work correctly and what would be needed to get the kamailio dev’s to accept it into the
main stream of code? (Happy to help here where I can).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Unless you will maintain your branch forever<span class="apple-converted-space"> </span><span style="font-family:"Segoe UI Emoji",sans-serif">😊</span><span class="apple-converted-space"> </span>Or are the changes
small so its not too much of a concern? I just want to assess the risk really, and like you I think mainly we will just be using userloc and dispatcher for DB interaction so its positive to hear they work fine. We would also be using rtpengine, routing data
will be done via API so that should be fine. FYI Id like to run kamailio 5.4 initially.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">I see your comments about table creation (in previous posts), that is the initial problem I am seeing when trying to use postgres based commands direct from a pgdump to create the kamailio database structure,
does your branch contain all the creation scripts for the db/tables that I can use for testing?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Thanks again in advance for the response!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Jon<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Sent from<span class="apple-converted-space"> </span><a href="https://go.microsoft.com/fwlink/?LinkId=550986">Mail</a><span class="apple-converted-space"> </span>for Windows 10<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div>
<p class="MsoNormal" style="margin-left:35.4pt"><b>From:<span class="apple-converted-space"> </span></b><a href="mailto:hw@skalatan.de">Henning Westerholt</a><br>
<b>Sent:<span class="apple-converted-space"> </span></b>17 February 2021 16:18<br>
<b>To:<span class="apple-converted-space"> </span></b><a href="mailto:sr-users@lists.kamailio.org">Kamailio (SER) - Users Mailing List</a><br>
<b>Cc:<span class="apple-converted-space"> </span></b><a href="mailto:hunterj91@hotmail.com">Jonathan Hunter</a>;<span class="apple-converted-space"> </span><a href="mailto:noahmehl@gmail.com">Noah Mehl</a><br>
<b>Subject:<span class="apple-converted-space"> </span></b>RE: Cockroachdb and kamailio 5.4<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Hi Jonathan,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">no – I do not think that there has been more work done so far, apart from the discussion that you referenced below.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">If you are also interested in getting this forward, why not reaching out to the other guy starting this discussion earlier? Just to see if you can maybe join forces to get something of this work into a pull request
for review and later a possible merge into our code base.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Henning<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">--<span class="apple-converted-space"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Henning Westerholt –<span class="apple-converted-space"> </span><a href="https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fskalatan.de%2Fblog%2F&data=04%7C01%7C%7C548d23e7d9544667296a08d8d35fb5d3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637491755307382972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WVL5vzihg1kUYxeDwnDwHPksVaf0EaM6ANXuBBBthZE%3D&reserved=0"><span lang="EN-GB" style="color:#0563C1">https://skalatan.de/blog/</span></a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt">Kamailio services –<span class="apple-converted-space"> </span><a href="https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgilawa.com%2F&data=04%7C01%7C%7C548d23e7d9544667296a08d8d35fb5d3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637491755307392967%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zkxs8506K%2FpvHX%2FPcHEj3fyYBei%2BnpuzPmpHdtkFfn4%3D&reserved=0"><span lang="EN-GB" style="color:#0563C1">https://gilawa.com</span></a><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"><b>From:</b><span class="apple-converted-space"> </span>sr-users <<a href="mailto:sr-users-bounces@lists.kamailio.org">sr-users-bounces@lists.kamailio.org</a>><span class="apple-converted-space"> </span><b>On
Behalf Of<span class="apple-converted-space"> </span></b>Jonathan Hunter<br>
<b>Sent:</b><span class="apple-converted-space"> </span>Wednesday, February 17, 2021 1:59 PM<br>
<b>To:</b><span class="apple-converted-space"> </span>Kamailio (SER) - Users Mailing List <<a href="mailto:sr-users@lists.kamailio.org">sr-users@lists.kamailio.org</a>><br>
<b>Subject:</b><span class="apple-converted-space"> </span>[SR-Users] Cockroachdb and kamailio 5.4<o:p></o:p></p>
</div>
</div>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt">Hi Guys,<o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt">Hope all are well?<o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt">We are looking to implement kamailio with cockroachdb due to the advantages it gives us in terms of postgres management and clustering.<o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt">I can see from a previous string<span class="apple-converted-space"> </span><a href="https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsip-router.1086192.n5.nabble.com%2FCockroachDB-and-Kamailio-td189233.html&data=04%7C01%7C%7C548d23e7d9544667296a08d8d35fb5d3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637491755307392967%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8E15g5pgP%2BpY%2BgEz%2FtP%2BaLcv6r4s60hCwk5MV0yGn4I%3D&reserved=0">http://sip-router.1086192.n5.nabble.com/CockroachDB-and-Kamailio-td189233.html</a><span class="apple-converted-space"> </span>that
someone has attempted this and even created their own branch.<o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt">Has any further work been done on this, and if so has anyone got any advice/tips relating to it? As I am nervous about potentially using a branch and not a general release of kamailio moving forwards.<o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt">I appreciate you cant support all database types but I would just be interested to hear people’s thoughts on the topic.<o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt">Many thanks!<o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt">Jon<o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
</div>
<div style="margin-left:35.4pt">
<p class="MsoNormal" style="margin-left:35.4pt">Sent from<span class="apple-converted-space"> </span><a href="https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgo.microsoft.com%2Ffwlink%2F%3FLinkId%3D550986&data=04%7C01%7C%7C548d23e7d9544667296a08d8d35fb5d3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637491755307402965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tCO%2FU3IF9d01%2Fwc7APzZZvr2kcFiqIsZox3p%2BjwegyM%3D&reserved=0">Mail</a><span class="apple-converted-space"> </span>for
Windows 10<o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
</div>
</div>
</body>
</html>