[SR-Users] Memory issue?

Hello, I got several errors this morning on a Kamailio instance. The first occurred without a complete crash of Kamailio but 4 coredumps have been created: BFD: Warning: /core.3818 is truncated: expected core file size >= 342618112, found: 61579264. [New Thread 3818] Cannot access memory at address 0x7f66e0557658 BFD: Warning: /core.3821 is truncated: expected core file size >= 343044096, found: 21229568. [New Thread 3821] Cannot access memory at address 0x7f66e0557658 BFD: Warning: /core.3822 is truncated: expected core file size >= 342675456, found: 120066048. [New Thread 3822] Cannot access memory at address 0x7f66e0557658 BFD: Warning: /core.3839 is truncated: expected core file size >= 342401024, found: 2691072. [New Thread 3839] Cannot access memory at address 0x3d6f021168 Cannot access memory at address 0x3d6f021168 Cannot access memory at address 0x3d6f021168 It seems that Kamailio auto recovered without fully crashed during this issue. But the Kamailio FIFO was deleted and kamctl didn't worked anymore. I had to properly restart. I have seen in the logs: Jan 21 10:21:11 /usr/local/sbin/kamailio[3843]: ERROR: <core> [dtrie.c:153]: dtrie_insert(): could not allocate shared memory from available pool Jan 21 10:21:11 /usr/local/sbin/kamailio[3839]: ERROR: <core> [dtrie.c:144]: dtrie_insert(): could not allocate shared memory from available pool Jan 21 10:21:11 /usr/local/sbin/kamailio[3839]: ERROR: userblacklist [db.c:91]: db_build_userbl_tree(): could not insert values into trie. Jan 21 10:21:11 /usr/local/sbin/kamailio[3843]: ERROR: userblacklist [db.c:91]: db_build_userbl_tree(): could not insert values into trie. Jan 21 10:21:11 /usr/local/sbin/kamailio[3831]: ERROR: <core> [dtrie.c:144]: dtrie_insert(): could not allocate shared memory from available pool Jan 21 10:21:11 /usr/local/sbin/kamailio[3831]: ERROR: userblacklist [db.c:91]: db_build_userbl_tree(): could not insert values into trie. Jan 21 10:21:11 /usr/local/sbin/kamailio[3841]: ERROR: <core> [dtrie.c:153]: dtrie_insert(): could not allocate shared memory from available pool Jan 21 10:21:12 /usr/local/sbin/kamailio[3821]: ERROR: <core> [dtrie.c:153]: dtrie_insert(): could not allocate shared memory from available pool Jan 21 10:21:12 /usr/local/sbin/kamailio[3821]: ERROR: userblacklist [db.c:91]: db_build_userbl_tree(): could not insert values into trie. Jan 21 10:21:12 /usr/local/sbin/kamailio[3828]: ERROR: <core> [dtrie.c:153]: dtrie_insert(): could not allocate shared memory from available pool Jan 21 10:21:12 /usr/local/sbin/kamailio[3843]: ERROR: <core> [dtrie.c:153]: dtrie_insert(): could not allocate shared memory from available pool Jan 21 10:21:12 /usr/local/sbin/kamailio[3818]: ERROR: <core> [dtrie.c:144]: dtrie_insert(): could not allocate shared memory from available pool Jan 21 10:21:12 /usr/local/sbin/kamailio[3818]: ERROR: userblacklist [db.c:91]: db_build_userbl_tree(): could not insert values into trie. [.] Jan 21 10:21:13 kernel: kamailio[3818]: segfault at 40 ip 00007f66d8812af2 sp 00007fff09c83380 error 4 in libtrie.so.1.0[7f66d8810000+5000] Jan 21 10:21:13 kernel: kamailio[3822]: segfault at 20 ip 00007f66d8812af2 sp 00007fff09c83380 error 4 in libtrie.so.1.0[7f66d8810000+5000] Jan 21 10:21:14 kernel: kamailio[3821]: segfault at 0 ip 00007f66d8812af2 sp 00007fff09c83380 error 4 in libtrie.so.1.0[7f66d8810000+5000] [.] Jan 21 10:21:14 /usr/local/sbin/kamailio[3807]: ALERT: <core> [main.c:784]: handle_sigs(): child process 3828 exited by a signal 11 Jan 21 10:21:14 /usr/local/sbin/kamailio[3807]: ALERT: <core> [main.c:787]: handle_sigs(): core was not generated Jan 21 10:21:14 /usr/local/sbin/kamailio[3807]: ALERT: <core> [main.c:784]: handle_sigs(): child process 3825 exited by a signal 11 Jan 21 10:21:14 /usr/local/sbin/kamailio[3807]: ALERT: <core> [main.c:787]: handle_sigs(): core was not generated Jan 21 10:21:14 /usr/local/sbin/kamailio[3807]: INFO: <core> [main.c:799]: handle_sigs(): terminating due to SIGCHLD Jan 21 10:21:14 /usr/local/sbin/kamailio[3832]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received Jan 21 10:21:14 /usr/local/sbin/kamailio[3841]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received Jan 21 10:21:14 /usr/local/sbin/kamailio[3876]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received 1 hour later a new crash but here Kamailio has completely restarted with two coredump showing the same error than previous one. And 2 coredumps readable: Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid -m 256 -M 64'. Program terminated with signal 11, Segmentation fault. #0 0x00007f13a1550af2 in dtrie_insert (root=0x7f138d7ffb38, number=0x22ba6c0 "003716716", numberlen=9, data=0x2, branches=10) at dtrie.c:141 141 if (node->child[digit] == NULL) { Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid -m 256 -M 64'. Program terminated with signal 11, Segmentation fault. #0 0x00007f13a1550af2 in dtrie_insert (root=0x7f138d7ff998, number=0x22b57a8 "0035587482", numberlen=10, data=0x2, branches=10) at dtrie.c:141 141 if (node->child[digit] == NULL) { Kamailio is 4.2.6 on this instance. Regards, Igor.