<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
code
{mso-style-priority:99;
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1214274565;
mso-list-template-ids:210930244;}
@list l0:level1
{mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level4
{mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level7
{mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1
{mso-list-id:1243025805;
mso-list-template-ids:-71408328;}
@list l1:level1
{mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level2
{mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level3
{mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level4
{mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level5
{mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level6
{mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level7
{mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level8
{mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level9
{mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2
{mso-list-id:1442340375;
mso-list-template-ids:-1061394868;}
@list l2:level1
{mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level2
{mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level3
{mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level4
{mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level5
{mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level6
{mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level7
{mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level8
{mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2:level9
{mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FR" link="#0563C1" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">I didn’t know about that parameter… thanks for your help :)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regards,<o:p></o:p></p>
<p class="MsoNormal">Nicolas.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="mso-fareast-language:FR">De :</span></b><span style="mso-fareast-language:FR"> Daniel-Constantin Mierla <miconda@gmail.com>
<br>
<b>Envoyé :</b> mardi 23 novembre 2021 17:37<br>
<br>
</span><o:p></o:p></p>
</div>
</div>
<p>Hello,<o:p></o:p></p>
<p>Kamailio tries to increase it up to 256kB if it is lower. If it is higher, probably stays the same. You can try to set -b cli parameter of maxbuffer core parameter to higher values.<o:p></o:p></p>
<p>Cheers,<br>
Daniel<o:p></o:p></p>
<div>
<p class="MsoNormal">On 23.11.21 16:10, Chaigneau, Nicolas wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">I’m using Kamailio 5.5.2 with UDP sockets, configured with 32 children processes.<o:p></o:p></p>
<p class="MsoNormal">During load tests, some packets seem to be dropped. I’m trying to figure out why, and how to solve (or mitigate) the issue.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">From what I understand :<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><i><span style="font-size:10.0pt">« the SIP messages send on UDP/SCTP are received directly from the buffer in kernel one by one, each being processed once read. »</span></i><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">(this is a message from Daniel-Constantin Mierla, posted on the mailing list back in 2014 – I assume this is still accurate)<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">So Kamailio does not handle its own buffers for incoming messages, but simply pulls one message from the kernel queue whenever a process is available for processing.<o:p></o:p></p>
<p class="MsoNormal">(Correct me if this is not true)<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Hereafter some system parameters that seem interesting to me.<o:p></o:p></p>
<p class="MsoNormal">In particular, « net.core.rmem.default » seem very low (0,33 MB). But « net.core.rmem.max » is much higher (16 MB).<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Looking at the code in function probe_max_receive_buffer (src\core\udp_server.c), it seems Kamailio is trying to increase the buffer size from the default.<o:p></o:p></p>
<p class="MsoNormal">However this does not seem to be working, seeing the following logs upon startup :<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">0(7602) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 349520</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">0(7602) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer(): SO_RCVBUF is finally 349520</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">0(7602) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 349520</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">0(7602) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer(): SO_RCVBUF is finally 349520</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">0(7602) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 349520</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">0(7602) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer(): SO_RCVBUF is finally 349520</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Is there something I don’t understand ? could it be a bug ?<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">One quick fix I can think of would be to increase « net.core.rmem.default ». Considering that I don’t have many sockets (3 used by Kamailio), I think I could set this to 16 MB (my VMs have 8 GB of allocated RAM).<o:p></o:p></p>
<p class="MsoNormal">What do you think ?<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Thanks in advance for your help.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">My system parameters :<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"> <o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">net.core.rmem_default = 349520</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">net.core.rmem_max = 16777216</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas"># => 0,33 MB / 16 MB respectively</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas"> </span><o:p></o:p></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:11.25pt;margin-left:93.9pt;text-indent:-18.0pt;mso-list:l1 level1 lfo2;background:#FDFDFD">
<![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><code><span style="font-size:11.5pt;color:#111111;border:solid #E8E8E8 1.0pt;padding:1.0pt;background:#EEEEFF">net.core.rmem_default, net.core.rmem_max</span></code><span style="font-size:12.0pt;color:black"> – default and max socket
receive buffer size in bytes. Each socket gets </span><code><span style="font-size:11.5pt;color:#111111;border:solid #E8E8E8 1.0pt;padding:1.0pt;background:#EEEEFF">rmem_default</span></code><span style="font-size:12.0pt;color:black"> reveive buffer size by
default, and can request up to </span><code><span style="font-size:11.5pt;color:#111111;border:solid #E8E8E8 1.0pt;padding:1.0pt;background:#EEEEFF">rmem_max</span></code><span style="font-size:12.0pt;color:black"> with <a href="http://man7.org/linux/man-pages/man2/setsockopt.2.html"><span style="color:#2A7AE2">setsockopt</span></a> option </span><code><span style="font-size:11.5pt;color:#111111;border:solid #E8E8E8 1.0pt;padding:1.0pt;background:#EEEEFF">SO_RCVBUF</span></code><span style="font-size:12.0pt;color:black">.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">net.ipv4.udp_mem = 188856 251809 377712</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:93.9pt;text-indent:-18.0pt;mso-list:l0 level1 lfo4;background:#FDFDFD">
<![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><span style="font-size:11.5pt;color:black;background:#EEEEFF;mso-fareast-language:FR">net.ipv4.udp_mem = "min pressure max"</span><span style="font-size:12.0pt;color:black"> – these are numbers of <b>PAGES</b> (4KB) available for all
UDP sockets in the system. min, pressure and max controls how memory is managed, but main point is that max is maximum size in PAGES for all UDP bufers in system. These values are set on boot time (if sysctls are not explicitly set) according to available
RAM size.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas"># 377712 x 4 KB => 1475 MB</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span style="font-size:10.0pt;font-family:Consolas">net.ipv4.udp_rmem_min = 4096</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:93.9pt;text-indent:-18.0pt;mso-list:l2 level1 lfo6;background:#FDFDFD">
<![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><span style="font-size:11.5pt;color:black;background:#EEEEFF;mso-fareast-language:FR">net.ipv4.udp_rmem_min, net.ipv4.udp_wmem</span><span style="font-size:12.0pt;color:black"> – minimal size for receive/send buffers (in bytes), guaranteed
for each socket, even if if buffer size of all UDP sockets exceeds pressure parameter in </span><span style="font-size:11.5pt;color:black;background:#EEEEFF;mso-fareast-language:FR">net.ipv4.udp_mem</span><span style="font-size:12.0pt;color:black">.</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Regards,<o:p></o:p></p>
<p class="MsoNormal">Nicolas.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:0cm;margin-right:36.0pt;margin-bottom:0cm;margin-left:36.0pt;margin-bottom:.0001pt">
<span style="mso-fareast-language:FR"><o:p></o:p></span></p>
</blockquote>
</div>
<span style="font-size: 9px; line-height: 10px;">This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.</span></body>
</html>