<h3>Description</h3>
<p>Kamailio pipelimit module causes high CPU load (20 - 30 %) on servers with > 48 GB RAM even if Kamailio is idle (no active calls).</p>
<p>strace shows that every second <code>open /pro/stat</code> is called even if timer_interval is set to 10 seconds<br>
<code>modparam("pipelimit", "timer_interval", 10)</code></p>
<p><code>1539343630.677932 open("/proc/stat", O_RDONLY) = 6</code><br>
<code>1539343630.678278 open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 6</code><br>
<code>1539343630.678418 open("/proc/net/udp", O_RDONLY) = 6</code><br>
<code>1539343630.679448 open("/proc/net/tcp", O_RDONLY) = 6</code><br>
<code>1539343630.722522 open("/proc/net/tcp", O_RDONLY) = 6</code><br>
<code>1539343630.753095 open("/proc/net/udp", O_RDONLY) = 6</code><br>
<code>1539343630.753838 open("/proc/net/tcp", O_RDONLY) = 6</code><br>
<code>1539343630.783710 open("/proc/net/tcp", O_RDONLY) = 6</code><br>
<code>1539343631.677966 open("/proc/stat", O_RDONLY) = 6</code><br>
<code>1539343631.678378 open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 6</code><br>
<code>1539343631.678534 open("/proc/net/udp", O_RDONLY) = 6</code><br>
<code>1539343631.679784 open("/proc/net/tcp", O_RDONLY) = 6</code><br>
<code>1539343631.717329 open("/proc/net/tcp", O_RDONLY) = 6</code><br>
<code>1539343631.760763 open("/proc/net/udp", O_RDONLY) = 6</code><br>
<code>1539343631.761786 open("/proc/net/tcp", O_RDONLY) = 6</code><br>
<code>1539343631.805426 open("/proc/net/tcp", O_RDONLY) = 6</code></p>
<p>strace shows that with 8 GB  RAM the read syscall takes 100us (average)  - with 48GB it takes 586us (average)!</p>
<p>see 5 seconds interval strace for 48GB:</p>
<p><code>2018-09-21 07:55:38,201</code><br>
<code>strace: Process 6907 attached</code><br>
<code>strace: Process 6907 detached</code><br>
<code>% time seconds usecs/call calls errors syscall</code><br>
<code>------ ----------- ----------- --------- --------- ----------------</code><br>
<code>99.79 1.259694 586 2148 read</code><br>
<code>0.06 0.000761 19 41 open</code><br>
<code>0.04 0.000469 8 61 61 pause</code><br>
<code>0.04 0.000462 13 36 munmap</code><br>
<code>0.03 0.000336 8 41 close</code><br>
<code>0.02 0.000257 7 36 mmap</code><br>
<code>0.02 0.000234 3 80 61 rt_sigreturn</code><br>
<code>0.01 0.000113 3 36 fstat</code><br>
<code>0.00 0.000017 3 5 kill</code><br>
<code>------ ----------- ----------- --------- --------- ----------------</code><br>
<code>100.00 1.262343 2484 122 total</code><br>
<code>2018-09-21 07:55:43,225</code></p>
<p>In the documentation (<a href="https://kamailio.org/docs/modules/5.1.x/modules/pipelimit.html" rel="nofollow">https://kamailio.org/docs/modules/5.1.x/modules/pipelimit.html</a>) I found the following:</p>
<blockquote>
<p>As reading the CPU load average is relatively expensive (opening /proc/stat, parsing it, etc), this only happens once every timer_interval seconds and consequently the FEEDBACK value is only at these intervals recomputed. This in turn makes it difficult for the drop rate to adjust quickly. Worst case scenarios are request rates going up/down instantly by thousands - it takes up to 20 seconds for the controller to adapt to the new request rate.</p>
</blockquote>
<p>IMHO the doc is misleading because the periodic timer used for this is hardcoded with 1000ms.<br>
see<br>
<code>( 349 pipelimit.c ) timer_add(pl_timer, MS_TO_TICKS(1000)); /* Start it after 1000ms */</code></p>
<p>Conclusion:<br>
Pipelimit polls periodically every second /proc/stats and /proc/net/udp|tcp. The modparam timer_interval does not affect the timer period.<br>
On servers with >48 GB this causes high CPU load.</p>
<h4>Reproduction</h4>
<p>always reproducible.</p>
<h3>Possible Solutions</h3>
<p>Use modparam timer_interval to initialize the <code>pl_timer</code> interval</p>
<h3>Additional Information</h3>
<p>version: kamailio 5.1.2 (x86_64/linux)<br>
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES<br>
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB<br>
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.<br>
id: unknown</p>
<ul>
<li><strong>Operating System</strong>:  Centos 7.4</li>
</ul>

<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/kamailio/kamailio/issues/1675">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AF36Za7AA6FKW5bF9gVjH10FaC5ONjLRks5ukI99gaJpZM4XZaGH">mute the thread</a>.<img src="https://github.com/notifications/beacon/AF36ZUH-jg3zG8XhspDfSiYZoZ8QFacbks5ukI99gaJpZM4XZaGH.gif" height="1" width="1" alt="" /></p>
<script type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/kamailio/kamailio","title":"kamailio/kamailio","subtitle":"GitHub repository","main_image_url":"https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/kamailio/kamailio"}},"updates":{"snippets":[{"icon":"DESCRIPTION","message":"pipelimit: module causes high CPU load due to periodic 1s /pro/net/upd|tcp and /pro/stat system call if server RAM \u003e= 48GB (#1675)"}],"action":{"name":"View Issue","url":"https://github.com/kamailio/kamailio/issues/1675"}}}</script>
<script type="application/ld+json">[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/kamailio/kamailio/issues/1675",
"url": "https://github.com/kamailio/kamailio/issues/1675",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
},
{
"@type": "MessageCard",
"@context": "http://schema.org/extensions",
"hideOriginalBody": "false",
"originator": "AF6C5A86-E920-430C-9C59-A73278B5EFEB",
"title": "pipelimit: module causes high CPU load due to periodic 1s /pro/net/upd|tcp and /pro/stat system call if server RAM \u003e= 48GB (#1675)",
"sections": [
{
"text": "",
"activityTitle": "**tpeham**",
"activityImage": "https://assets-cdn.github.com/images/email/message_cards/avatar.png",
"activitySubtitle": "@tpeham",
"facts": [
{
"name": "Repository: ",
"value": "kamailio/kamailio"
},
{
"name": "Issue #: ",
"value": 1675
}
]
}
],
"potentialAction": [
{
"name": "Add a comment",
"@type": "ActionCard",
"inputs": [
{
"isMultiLine": true,
"@type": "TextInput",
"id": "IssueComment",
"isRequired": false
}
],
"actions": [
{
"name": "Comment",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n\"commandName\": \"IssueComment\",\n\"repositoryFullName\": \"kamailio/kamailio\",\n\"issueId\": 1675,\n\"IssueComment\": \"{{IssueComment.value}}\"\n}"
}
]
},
{
"name": "Close issue",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n\"commandName\": \"IssueClose\",\n\"repositoryFullName\": \"kamailio/kamailio\",\n\"issueId\": 1675\n}"
},
{
"targets": [
{
"os": "default",
"uri": "https://github.com/kamailio/kamailio/issues/1675"
}
],
"@type": "OpenUri",
"name": "View on GitHub"
},
{
"name": "Unsubscribe",
"@type": "HttpPOST",
"target": "https://api.github.com",
"body": "{\n\"commandName\": \"MuteNotification\",\n\"threadId\": 392536455\n}"
}
],
"themeColor": "26292E"
}
]</script>