[kamailio/kamailio] 100% CPU usage after NTP date update (#2046)
#### Reproduction Install newest Kamailio 5.2. Change date e.g. 2 Years into the future. (Make sure you have no daemon running which prohibits those operations): ``` sudo date -s "2014-12-25 $(date +%H:%M:%S)" ``` ``` sudo date -s "2016-12-25 $(date +%H:%M:%S)" ``` Kamailio now uses the whole CPU to do stuff: ``` top - 08:10:35 up 14 days, 1:29, 2 users, load average: 3.77, 1.83, 1.05 Tasks: 212 total, 11 running, 201 sleeping, 0 stopped, 0 zombie %Cpu(s): 95.9 us, 1.5 sy, 2.2 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.4 si, 0.0 st MiB Mem : 926.1 total, 116.3 free, 382.5 used, 427.3 buff/cache MiB Swap: 100.0 total, 40.4 free, 59.6 used. 423.4 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 13264 root 20 0 89720 3944 2676 R 44.9 0.4 0:05.15 kamailio 13266 root 20 0 89720 4000 2736 R 37.3 0.4 0:04.63 kamailio 13267 root 20 0 89720 4912 3488 R 36.0 0.5 0:04.37 kamailio 13270 root 20 0 89720 4540 3240 R 36.0 0.5 0:05.85 kamailio 13269 root 20 0 89720 4000 2736 R 34.7 0.4 0:04.81 kamailio 13265 root 20 0 89720 3944 2680 R 34.4 0.4 0:04.61 kamailio 13261 root 20 0 89720 3952 2676 R 32.2 0.4 0:04.44 kamailio 13262 root 20 0 89720 3944 2676 R 31.8 0.4 0:04.72 kamailio 13256 root 20 0 89720 4752 3540 R 30.3 0.5 0:07.04 kamailio 13268 root 20 0 89720 4000 2736 R 30.3 0.4 0:04.49 kamailio 18021 root 20 0 40116 11884 5728 D 13.7 1.3 0:01.11 apt-get ``` There are multiple mailing list entries regarding this issue: https://lists.kamailio.org/pipermail/sr-users/2015-October/090465.html https://lists.kamailio.org/pipermail/sr-users/2015-November/090487.html https://lists.kamailio.org/pipermail/sr-users/2019-June/105967.html https://lists.kamailio.org/pipermail/sr-users/2019-August/106428.html I was able to get rid of the issue with uncommenting the mentioned part of the code. But I'm quite sure that's not a fix :)  -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2046
Iiirc from older discussions, here is about the timer trying to catch up to the new current time by executing the timer callbacks for each timer step in between. There were proposals to skip executing timer callbacks for intermediate timer steps, but not sure if something was done. In such case there can be other side effects as some callbacks are no longer executed when it should - e.g., timer to retransmit the requests in active transactions, which should be done in 0.5, 1, 2, 4 seconds steps, if time jumps too much in the future and those timer callbacks are not executed, then some side effects can happen, like transaction left unexpired in memory (leak). But then executing each timer step (iirc, it is like every 0.1sec) quickly in a raw without sleep to fill a long interval like 2 years it's going to use a lot of CPU. I am fine if someone comes with some solution/improvement here, eventually controlled by a core parameter. Otherwise can be be seen as a bug either way: * avoid high CPU - skip callbacks execution for intermediate steps, but some other potential ugly side effects * have high CPU with callbacks execution for intermediate steps, but then maybe other processing gets delayed or not even proper handling of sip traffic Over all, I consider this not to be a bug, but more an enhancement nice to have, because having the time jumping a long interval to the future is not a normal operation mode. If someone has root access can use gdb and change values in memory for kamailio to create infinite loops or other bad things. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2046#issuecomment-526280672
This is more than one year old with nobody taking over to implement any variant. If someone wants to do it, just make a pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2046#issuecomment-729926003
Closed #2046. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2046#event-4012545419
participants (2)
-
Benjamin Marty -
Daniel-Constantin Mierla