<p>Ok! I'll wait until it happens again.</p>
<p>Some extra info:</p>
<p>1- Pike is enabled, but we have logging when it blocks something (and no logs appeared).<br>
2- CPU is always < 1% usage.<br>
3- I have enabled kernel.core_uses_pid=1.<br>
4- Our traffic is:</p>
<p>[clients]--> IPv4/IPv6 tls_443 [Kamailio] IPv4 udp_5060 --> IPv4 udp_5060 [dispatcher setid 4].</p>
<p>If I run the bt full right now to a worker process, should we get random stuff?</p>
<p>Becase right now I'm getting mostly the same on each process:</p>
<pre><code>root:~# for i in `kamctl ps | grep PID | awk '{print $2}' | tr -d ","`; do gdb /usr/sbin/kamailio -ex "bt full" --batch $i >> $i.txt 2>&1; done
root:~#
root:~# ls -l
total 140
-rw-r--r-- 1 root root 1769 Jun 29 01:04 23585.txt
-rw-r--r-- 1 root root 3300 Jun 29 01:04 23586.txt
-rw-r--r-- 1 root root 3300 Jun 29 01:04 23587.txt
-rw-r--r-- 1 root root 3301 Jun 29 01:04 23588.txt
-rw-r--r-- 1 root root 3300 Jun 29 01:04 23589.txt
-rw-r--r-- 1 root root 3301 Jun 29 01:04 23590.txt
-rw-r--r-- 1 root root 3300 Jun 29 01:04 23591.txt
-rw-r--r-- 1 root root 3301 Jun 29 01:04 23592.txt
-rw-r--r-- 1 root root 3301 Jun 29 01:04 23593.txt
-rw-r--r-- 1 root root 3432 Jun 29 01:04 23594.txt
-rw-r--r-- 1 root root 3432 Jun 29 01:04 23595.txt
-rw-r--r-- 1 root root 3431 Jun 29 01:04 23596.txt
-rw-r--r-- 1 root root 3432 Jun 29 01:04 23597.txt
-rw-r--r-- 1 root root 3432 Jun 29 01:04 23598.txt
-rw-r--r-- 1 root root 3432 Jun 29 01:04 23599.txt
-rw-r--r-- 1 root root 3432 Jun 29 01:04 23600.txt
-rw-r--r-- 1 root root 3432 Jun 29 01:04 23601.txt
-rw-r--r-- 1 root root 2388 Jun 29 01:04 23602.txt
-rw-r--r-- 1 root root 1836 Jun 29 01:04 23603.txt
-rw-r--r-- 1 root root 2245 Jun 29 01:04 23604.txt
-rw-r--r-- 1 root root 4023 Jun 29 01:04 23605.txt
-rw-r--r-- 1 root root 2950 Jun 29 01:04 23606.txt
-rw-r--r-- 1 root root 2837 Jun 29 01:04 23607.txt
-rw-r--r-- 1 root root 4691 Jun 29 01:04 23608.txt
-rw-r--r-- 1 root root 2740 Jun 29 01:04 23609.txt
-rw-r--r-- 1 root root 2691 Jun 29 01:04 23610.txt
-rw-r--r-- 1 root root 2691 Jun 29 01:04 23611.txt
-rw-r--r-- 1 root root 2691 Jun 29 01:04 23612.txt
-rw-r--r-- 1 root root 2691 Jun 29 01:04 23613.txt
-rw-r--r-- 1 root root 2691 Jun 29 01:04 23614.txt
-rw-r--r-- 1 root root 2691 Jun 29 01:04 23615.txt
-rw-r--r-- 1 root root 2691 Jun 29 01:04 23616.txt
-rw-r--r-- 1 root root 2691 Jun 29 01:04 23617.txt
-rw-r--r-- 1 root root 2151 Jun 29 01:04 23618.txt
root:~#
root:~# grep -n ffff *
23585.txt:31: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23586.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23587.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23588.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23589.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23590.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23591.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23592.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23593.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23594.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23595.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23596.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23597.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23598.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23599.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23600.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23601.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23602.txt:40: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23603.txt:33: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23604.txt:39: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23605.txt:75: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23606.txt:51: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23607.txt:49: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23608.txt:75: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23609.txt:50: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23610.txt:47: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23611.txt:47: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23612.txt:47: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23613.txt:47: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23614.txt:47: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23615.txt:47: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23616.txt:47: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23617.txt:47: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
23618.txt:41: p = 0xffffffff <error: Cannot access memory at address 0xffffffff>
root:~#
</code></pre>
<p>Am I missing something or am I doing anything wrong here?</p>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/kamailio/kamailio/issues/1172#issuecomment-311893980">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AF36Zau1qbv0wKRqTc2D-dsCV4gtf8wKks5sI1uBgaJpZM4OIFBs">mute the thread</a>.<img alt="" height="1" src="https://github.com/notifications/beacon/AF36ZeXb8o1x6OH006xg-7FWITFW1J1Sks5sI1uBgaJpZM4OIFBs.gif" width="1" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
<link itemprop="url" href="https://github.com/kamailio/kamailio/issues/1172#issuecomment-311893980"></link>
<meta itemprop="name" content="View Issue"></meta>
</div>
<meta itemprop="description" content="View this Issue on GitHub"></meta>
</div>
<script type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/kamailio/kamailio","title":"kamailio/kamailio","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/kamailio/kamailio"}},"updates":{"snippets":[{"icon":"PERSON","message":"@joelsdc in #1172: Ok! I'll wait until it happens again. \r\n\r\nSome extra info:\r\n\r\n1- Pike is enabled, but we have logging when it blocks something (and no logs appeared).\r\n2- CPU is always \u003c 1% usage.\r\n3- I have enabled kernel.core_uses_pid=1. \r\n4- Our traffic is:\r\n\r\n[clients]--\u003e IPv4/IPv6 tls_443 [Kamailio] IPv4 udp_5060 --\u003e IPv4 udp_5060 [dispatcher setid 4].\r\n\r\nIf I run the bt full right now to a worker process, should we get random stuff?\r\n\r\nBecase right now I'm getting mostly the same on each process:\r\n\r\n```\r\nroot:~# for i in `kamctl ps | grep PID | awk '{print $2}' | tr -d \",\"`; do gdb /usr/sbin/kamailio -ex \"bt full\" --batch $i \u003e\u003e $i.txt 2\u003e\u00261; done\r\nroot:~# \r\n\r\nroot:~# ls -l\r\ntotal 140\r\n-rw-r--r-- 1 root root 1769 Jun 29 01:04 23585.txt\r\n-rw-r--r-- 1 root root 3300 Jun 29 01:04 23586.txt\r\n-rw-r--r-- 1 root root 3300 Jun 29 01:04 23587.txt\r\n-rw-r--r-- 1 root root 3301 Jun 29 01:04 23588.txt\r\n-rw-r--r-- 1 root root 3300 Jun 29 01:04 23589.txt\r\n-rw-r--r-- 1 root root 3301 Jun 29 01:04 23590.txt\r\n-rw-r--r-- 1 root root 3300 Jun 29 01:04 23591.txt\r\n-rw-r--r-- 1 root root 3301 Jun 29 01:04 23592.txt\r\n-rw-r--r-- 1 root root 3301 Jun 29 01:04 23593.txt\r\n-rw-r--r-- 1 root root 3432 Jun 29 01:04 23594.txt\r\n-rw-r--r-- 1 root root 3432 Jun 29 01:04 23595.txt\r\n-rw-r--r-- 1 root root 3431 Jun 29 01:04 23596.txt\r\n-rw-r--r-- 1 root root 3432 Jun 29 01:04 23597.txt\r\n-rw-r--r-- 1 root root 3432 Jun 29 01:04 23598.txt\r\n-rw-r--r-- 1 root root 3432 Jun 29 01:04 23599.txt\r\n-rw-r--r-- 1 root root 3432 Jun 29 01:04 23600.txt\r\n-rw-r--r-- 1 root root 3432 Jun 29 01:04 23601.txt\r\n-rw-r--r-- 1 root root 2388 Jun 29 01:04 23602.txt\r\n-rw-r--r-- 1 root root 1836 Jun 29 01:04 23603.txt\r\n-rw-r--r-- 1 root root 2245 Jun 29 01:04 23604.txt\r\n-rw-r--r-- 1 root root 4023 Jun 29 01:04 23605.txt\r\n-rw-r--r-- 1 root root 2950 Jun 29 01:04 23606.txt\r\n-rw-r--r-- 1 root root 2837 Jun 29 01:04 23607.txt\r\n-rw-r--r-- 1 root root 4691 Jun 29 01:04 23608.txt\r\n-rw-r--r-- 1 root root 2740 Jun 29 01:04 23609.txt\r\n-rw-r--r-- 1 root root 2691 Jun 29 01:04 23610.txt\r\n-rw-r--r-- 1 root root 2691 Jun 29 01:04 23611.txt\r\n-rw-r--r-- 1 root root 2691 Jun 29 01:04 23612.txt\r\n-rw-r--r-- 1 root root 2691 Jun 29 01:04 23613.txt\r\n-rw-r--r-- 1 root root 2691 Jun 29 01:04 23614.txt\r\n-rw-r--r-- 1 root root 2691 Jun 29 01:04 23615.txt\r\n-rw-r--r-- 1 root root 2691 Jun 29 01:04 23616.txt\r\n-rw-r--r-- 1 root root 2691 Jun 29 01:04 23617.txt\r\n-rw-r--r-- 1 root root 2151 Jun 29 01:04 23618.txt\r\nroot:~#\r\n\r\nroot:~# grep -n ffff *\r\n23585.txt:31: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23586.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23587.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23588.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23589.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23590.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23591.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23592.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23593.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23594.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23595.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23596.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23597.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23598.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23599.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23600.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23601.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23602.txt:40: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23603.txt:33: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23604.txt:39: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23605.txt:75: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23606.txt:51: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23607.txt:49: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23608.txt:75: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23609.txt:50: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23610.txt:47: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23611.txt:47: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23612.txt:47: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23613.txt:47: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23614.txt:47: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23615.txt:47: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23616.txt:47: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23617.txt:47: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\n23618.txt:41: p = 0xffffffff \u003cerror: Cannot access memory at address 0xffffffff\u003e\r\nroot:~#\r\n```\r\n\r\nAm I missing something or am I doing anything wrong here?"}],"action":{"name":"View Issue","url":"https://github.com/kamailio/kamailio/issues/1172#issuecomment-311893980"}}}</script>