<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello,</p>
<p>Kamailio tries to increase it up to 256kB if it is lower. If it
is higher, probably stays the same. You can try to set -b cli
parameter of maxbuffer core parameter to higher values.</p>
<p>Cheers,<br>
Daniel<br>
</p>
<div class="moz-cite-prefix">On 23.11.21 16:10, Chaigneau, Nicolas
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:PR3PR02MB6491847BEE7EF42D862D61A0E6609@PR3PR02MB6491.eurprd02.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style>@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}code
{mso-style-priority:99;
font-family:"Courier New";}.MsoChpDefault
{mso-style-type:export-only;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}ol
{margin-bottom:0cm;}ul
{margin-bottom:0cm;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’m using Kamailio 5.5.2 with UDP sockets,
configured with 32 children processes.<o:p></o:p></p>
<p class="MsoNormal">During load tests, some packets seem to be
dropped. I’m trying to figure out why, and how to solve (or
mitigate) the issue.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">From what I understand :<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><i><span
style="font-size:10.0pt;font-family:"Courier
New";color:black;mso-fareast-language:FR">« the SIP
messages send on UDP/SCTP are received directly from the
buffer in kernel one by one, each being processed once
read. »<o:p></o:p></span></i></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">(this is a message from Daniel-Constantin
Mierla, posted on the mailing list back in 2014 – I assume
this is still accurate)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So Kamailio does not handle its own buffers
for incoming messages, but simply pulls one message from the
kernel queue whenever a process is available for processing.<o:p></o:p></p>
<p class="MsoNormal">(Correct me if this is not true)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hereafter some system parameters that seem
interesting to me.<o:p></o:p></p>
<p class="MsoNormal">In particular, « net.core.rmem.default »
seem very low (0,33 MB). But « net.core.rmem.max » is much
higher (16 MB).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Looking at the code in function
probe_max_receive_buffer (src\core\udp_server.c), it seems
Kamailio is trying to increase the buffer size from the
default.<o:p></o:p></p>
<p class="MsoNormal">However this does not seem to be working,
seeing the following logs upon startup :<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">0(7602) INFO:
<core> [core/udp_server.c:154]:
probe_max_receive_buffer(): SO_RCVBUF is initially 349520<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">0(7602) INFO:
<core> [core/udp_server.c:206]:
probe_max_receive_buffer(): SO_RCVBUF is finally 349520<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">0(7602) INFO:
<core> [core/udp_server.c:154]:
probe_max_receive_buffer(): SO_RCVBUF is initially 349520<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">0(7602) INFO:
<core> [core/udp_server.c:206]:
probe_max_receive_buffer(): SO_RCVBUF is finally 349520<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">0(7602) INFO:
<core> [core/udp_server.c:154]:
probe_max_receive_buffer(): SO_RCVBUF is initially 349520<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">0(7602) INFO:
<core> [core/udp_server.c:206]:
probe_max_receive_buffer(): SO_RCVBUF is finally 349520<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Is there something I don’t understand ?
could it be a bug ?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">One quick fix I can think of would be to
increase « net.core.rmem.default ». Considering that I don’t
have many sockets (3 used by Kamailio), I think I could set
this to 16 MB (my VMs have 8 GB of allocated RAM).<o:p></o:p></p>
<p class="MsoNormal">What do you think ?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks in advance for your help.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My system parameters :<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">net.core.rmem_default
= 349520<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">net.core.rmem_max
= 16777216<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas"># => 0,33
MB / 16 MB respectively<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p
style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:11.25pt;margin-left:93.9pt;text-indent:-18.0pt;mso-list:l0
level1 lfo1;background:#FDFDFD">
<!--[if !supportLists]--><span
style="font-size:10.0pt;font-family:Symbol;color:#111111"><span
style="mso-list:Ignore">·<span style="font:7.0pt
"Times New Roman"">
</span></span></span><!--[endif]--><code><span
style="font-size:11.5pt;color:#111111;border:solid #E8E8E8
1.0pt;padding:1.0pt;background:#EEEEFF">net.core.rmem_default,
net.core.rmem_max</span></code><span
style="font-size:12.0pt;font-family:"Segoe
UI",sans-serif;color:#111111"> – default and max socket
receive buffer size in bytes. Each socket gets </span><code><span
style="font-size:11.5pt;color:#111111;border:solid #E8E8E8
1.0pt;padding:1.0pt;background:#EEEEFF">rmem_default</span></code><span
style="font-size:12.0pt;font-family:"Segoe
UI",sans-serif;color:#111111"> reveive buffer size by
default, and can request up to </span><code><span
style="font-size:11.5pt;color:#111111;border:solid #E8E8E8
1.0pt;padding:1.0pt;background:#EEEEFF">rmem_max</span></code><span
style="font-size:12.0pt;font-family:"Segoe
UI",sans-serif;color:#111111"> with <a
href="http://man7.org/linux/man-pages/man2/setsockopt.2.html"
moz-do-not-send="true"><span style="color:#2A7AE2">setsockopt</span></a> option </span><code><span
style="font-size:11.5pt;color:#111111;border:solid #E8E8E8
1.0pt;padding:1.0pt;background:#EEEEFF">SO_RCVBUF</span></code><span
style="font-size:12.0pt;font-family:"Segoe
UI",sans-serif;color:#111111">.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">net.ipv4.udp_mem
= 188856 251809 377712<o:p></o:p></span></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:93.9pt;text-indent:-18.0pt;mso-list:l1
level1 lfo2;background:#FDFDFD">
<!--[if !supportLists]--><span
style="font-size:10.0pt;font-family:Symbol;color:#111111;mso-fareast-language:FR"><span
style="mso-list:Ignore">·<span style="font:7.0pt
"Times New Roman"">
</span></span></span><!--[endif]--><span
style="font-size:11.5pt;font-family:"Courier
New";color:#111111;border:solid #E8E8E8
1.0pt;padding:1.0pt;background:#EEEEFF;mso-fareast-language:FR">net.ipv4.udp_mem
= "min pressure max"</span><span
style="font-size:12.0pt;font-family:"Segoe
UI",sans-serif;color:#111111;mso-fareast-language:FR"> –
these are numbers of <b>PAGES</b> (4KB) available for all
UDP sockets in the system. min, pressure and max controls
how memory is managed, but main point is that max is maximum
size in PAGES for all UDP bufers in system. These values are
set on boot time (if sysctls are not explicitly set)
according to available RAM size.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas"># 377712 x 4
KB => 1475 MB</span><span
style="font-size:10.0pt;font-family:Consolas"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span
style="font-size:10.0pt;font-family:Consolas">net.ipv4.udp_rmem_min
= 4096<o:p></o:p></span></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:93.9pt;text-indent:-18.0pt;mso-list:l2
level1 lfo3;background:#FDFDFD">
<!--[if !supportLists]--><span
style="font-size:10.0pt;font-family:Symbol;color:#111111;mso-fareast-language:FR"><span
style="mso-list:Ignore">·<span style="font:7.0pt
"Times New Roman"">
</span></span></span><!--[endif]--><span
style="font-size:11.5pt;font-family:"Courier
New";color:#111111;border:solid #E8E8E8
1.0pt;padding:1.0pt;background:#EEEEFF;mso-fareast-language:FR">net.ipv4.udp_rmem_min,
net.ipv4.udp_wmem</span><span
style="font-size:12.0pt;font-family:"Segoe
UI",sans-serif;color:#111111;mso-fareast-language:FR"> –
minimal size for receive/send buffers (in bytes), guaranteed
for each socket, even if if buffer size of all UDP sockets
exceeds pressure parameter in </span><span
style="font-size:11.5pt;font-family:"Courier
New";color:#111111;border:solid #E8E8E8
1.0pt;padding:1.0pt;background:#EEEEFF;mso-fareast-language:FR">net.ipv4.udp_mem</span><span
style="font-size:12.0pt;font-family:"Segoe
UI",sans-serif;color:#111111;mso-fareast-language:FR">.<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regards,<o:p></o:p></p>
<p class="MsoNormal">Nicolas.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p style="font-size: 9px; ">This message contains information that
may be privileged or confidential and is the property of the
Capgemini Group. It is intended only for the person to whom it
is addressed. If you are not the intended recipient, you are not
authorized to read, print, retain, copy, disseminate,
distribute, or use this message or any part thereof. If you
receive this message in error, please notify the sender
immediately and delete all copies of this message.</p>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
* <a class="moz-txt-link-abbreviated" href="mailto:sr-users@lists.kamailio.org">sr-users@lists.kamailio.org</a>
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
* <a class="moz-txt-link-freetext" href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla -- <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/miconda">www.twitter.com/miconda</a> -- <a class="moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda">www.linkedin.com/in/miconda</a>
Kamailio Advanced Training - Online
Feb 21-24, 2022 (America Timezone)
* <a class="moz-txt-link-freetext" href="https://www.asipto.com/sw/kamailio-advanced-training-online/">https://www.asipto.com/sw/kamailio-advanced-training-online/</a></pre>
</body>
</html>