After updating my sip proxy to latest kamailio 6.1 branch running on Debian Trixie , I have noticed several crashes. syslog has this: 2026-04-19T16:41:19.664532+03:00 box kernel: kamailio[944355]: segfault at 0 ip 0000558b1c9ea97e sp 00007fffa1deb610 er\ ror 6 in kamailio[f297e,558b1c926000+3b0000] likely on CPU 2 (core 0, socket 4) 2026-04-19T16:41:19.664633+03:00 box kernel: Code: 00 00 00 48 8b 57 08 48 8b 07 f3 0f 7e 0d 8a 72 3f 00 48 8b 4f 10 64\ 4c 8b 24 25 28 00 00 00 4c 89 a4 24 88 00 00 00 41 89 f4 <48> 89 02 66 0f 6c ca 48 89 50 08 0f 11 07 48 83 c3 10 74 13\ 48 39 2026-04-19T16:41:19.936157+03:00 box /usr/sbin/kamailio[944455]: CRITICAL: <core> [core/pass_fd.c:284]: receive_fd(): E\ OF on 34 2026-04-19T16:41:20.435008+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:824]: handle_sigs(): child proce\ ss 944355 exited by a signal 11 2026-04-19T16:41:20.435143+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:828]: handle_sigs(): core was ge\ nerated 2026-04-19T16:41:20.435191+03:00 box /usr/sbin/kamailio[944292]: INFO: <core> [main.c:851]: handle_sigs(): terminating \ due to SIGCHLD 2026-04-19T16:41:20.435613+03:00 box /usr/sbin/kamailio[944405]: INFO: <core> [main.c:907]: sig_usr(): signal 15 receiv\ ed ... and core dump shows this: #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 warning: 44 ./nptl/pthread_kill.c: No such file or directory (gdb) where #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 #1 0x00007f7f7ef509ff in __pthread_kill_internal (threadid=<optimized out>, signo=6) at ./nptl/pthread_kill.c:89 #2 0x00007f7f7eefbcc2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x00007f7f7eee44ac in __GI_abort () at ./stdlib/abort.c:77 #4 0x0000558b1c926d45 in ?? () #5 <signal handler called> #6 0x00007f7f7efb4907 in __GI_sched_yield () at ../sysdeps/unix/syscall-template.S:120 #7 0x0000558b1cc6c7ed in timer_del_safe () #8 0x0000558b1c9f8899 in destroy_dns_cache () #9 0x0000558b1c932537 in cleanup () #10 0x0000558b1c9332ed in ?? () #11 0x0000558b1c933fd3 in handle_sigs () #12 0x0000558b1c9395fd in main_loop () #13 0x0000558b1c92ae91 in main () Any ideas? -- Juha
You have to install the package with the debugging symbols for getting proper details in the backtrace. However, as it looks now, the corefile is written at Kamailio shutdown, either as an effect to another crash which has the core file overwritten if you didn't enabled core files per pid, or during a stop/restart command. To be sure the core files are not overwritten, enable core files per pid or define core file name pattern to be individual per running process. Kamailio wiki has an article about troubleshooting with corefile where more details are provided on these options. Cheers, Daniel On 19.04.26 16:27, Juha Heinanen via sr-users wrote:
After updating my sip proxy to latest kamailio 6.1 branch running on Debian Trixie , I have noticed several crashes.
syslog has this:
2026-04-19T16:41:19.664532+03:00 box kernel: kamailio[944355]: segfault at 0 ip 0000558b1c9ea97e sp 00007fffa1deb610 er\ ror 6 in kamailio[f297e,558b1c926000+3b0000] likely on CPU 2 (core 0, socket 4) 2026-04-19T16:41:19.664633+03:00 box kernel: Code: 00 00 00 48 8b 57 08 48 8b 07 f3 0f 7e 0d 8a 72 3f 00 48 8b 4f 10 64\ 4c 8b 24 25 28 00 00 00 4c 89 a4 24 88 00 00 00 41 89 f4 <48> 89 02 66 0f 6c ca 48 89 50 08 0f 11 07 48 83 c3 10 74 13\ 48 39 2026-04-19T16:41:19.936157+03:00 box /usr/sbin/kamailio[944455]: CRITICAL: <core> [core/pass_fd.c:284]: receive_fd(): E\ OF on 34 2026-04-19T16:41:20.435008+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:824]: handle_sigs(): child proce\ ss 944355 exited by a signal 11 2026-04-19T16:41:20.435143+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:828]: handle_sigs(): core was ge\ nerated 2026-04-19T16:41:20.435191+03:00 box /usr/sbin/kamailio[944292]: INFO: <core> [main.c:851]: handle_sigs(): terminating \ due to SIGCHLD 2026-04-19T16:41:20.435613+03:00 box /usr/sbin/kamailio[944405]: INFO: <core> [main.c:907]: sig_usr(): signal 15 receiv\ ed ...
and core dump shows this:
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
warning: 44 ./nptl/pthread_kill.c: No such file or directory (gdb) where #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 #1 0x00007f7f7ef509ff in __pthread_kill_internal (threadid=<optimized out>, signo=6) at ./nptl/pthread_kill.c:89 #2 0x00007f7f7eefbcc2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x00007f7f7eee44ac in __GI_abort () at ./stdlib/abort.c:77 #4 0x0000558b1c926d45 in ?? () #5 <signal handler called> #6 0x00007f7f7efb4907 in __GI_sched_yield () at ../sysdeps/unix/syscall-template.S:120 #7 0x0000558b1cc6c7ed in timer_del_safe () #8 0x0000558b1c9f8899 in destroy_dns_cache () #9 0x0000558b1c932537 in cleanup () #10 0x0000558b1c9332ed in ?? () #11 0x0000558b1c933fd3 in handle_sigs () #12 0x0000558b1c9395fd in main_loop () #13 0x0000558b1c92ae91 in main ()
Any ideas?
-- Juha
__________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions -- sr-users@lists.kamailio.org To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender!
-- Daniel-Constantin Mierla (@ asipto.com) twitter.com/miconda -- linkedin.com/in/miconda Kamailio Consultancy, Training and Development Services -- asipto.com Kamailio World Conference, May 7-8, 2026 - Berlin, Germany -- kamailioworld.com
Daniel-Constantin Mierla writes:
You have to install the package with the debugging symbols for getting proper details in the backtrace.
I re-built Kamailio and installed also kamailio-dbgsym package. I'll wait if another crash happens. Below is another old core dump from today. -- Juha Core was generated by `/usr/sbin/kamailio -f /etc/sip-proxy/sip-proxy.cfg -P /run/sip-proxy/sip-proxy.pid -m 128 -M 16 -u root -g root -w /run/sip-proxy -Y /run/sip-proxy -n 8'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000558b1c9ea97e in _dns_hash_remove_entry (e=0x7f7f75085ba0, line=662, fpath=0x558b1ccda66a "core/dns_cache.c") at core/dns_cache.c:501 warning: 501 core/dns_cache.c: No such file or directory (gdb) bt full #0 0x0000558b1c9ea97e in _dns_hash_remove_entry (e=0x7f7f75085ba0, line=662, fpath=0x558b1ccda66a "core/dns_cache.c") at core/dns_cache.c:501 __func__ = "_dns_hash_remove_entry" __llevel = <optimized out> __kld = <optimized out> __llevel = <optimized out> __kld = <optimized out> __llevel = <optimized out> __kld = <optimized out> #1 0x0000558b1c9f0e02 in dns_cache_clean (no=4294967295, expired_only=1) at core/dns_cache.c:662 e = 0x7f7f75085ba0 now = 658262396 n = <optimized out> deleted = <optimized out> l = 0x7f7f75085bb0 tmp = 0x0 __func__ = "dns_cache_clean" #2 dns_timer (ticks=<optimized out>, tl=<optimized out>, data=<optimized out>) at core/dns_cache.c:223 No locals. #3 dns_timer (ticks=<optimized out>, tl=<optimized out>, data=<optimized out>) at core/dns_cache.c:210 No locals. #4 0x0000558b1cc6f61a in slow_timer_main () at core/timer.c:1200 n = <optimized out> ret = <optimized out> tl = 0x7f7f74408120 i = <optimized out> __func__ = "slow_timer_main" #5 0x0000558b1c939455 in main_loop () at ./obj-x86_64-linux-gnu/src/main.c:2007 i = <optimized out> pid = <optimized out> si = <optimized out> sx = <optimized out> si_desc = "udp receiver child=7 sock=[2001:67c:224:666::38]:8002\000\000\000\022\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\2007t\177\177\000\000\002\000\000\000\004\000\000\000\177\000\000\001\213U\000\000}\000\000\000\000\000\000\000\000\277Țs\373,&Sun Apr \t\000\000\000\000\000\000" nrprocs = <optimized out> woneinit = 1 --Type <RET> for more, q to quit, c to continue without paging-- agfound = <optimized out> __func__ = "main_loop" error = <optimized out> #6 0x0000558b1c92ae91 in main (argc=<optimized out>, argv=<optimized out>) at ./obj-x86_64-linux-gnu/src/main.c:3434 cfg_stream = <optimized out> c = <optimized out> r = <optimized out> tmp = 0x7fffa1dede80 "" tmp_len = 0 port = 5060 proto = 0 aproto = 0 ahost = 0x0 socket_name = <optimized out> aport = 0 listen_field_count = <optimized out> listen_fields = {0x2 <error: Cannot access memory at address 0x2>, 0x7fffa1debb58 "", 0x1900000 <error: Cannot access memory at address 0x1900000>} options = 0x558b1ccede20 ":f:cm:M:dVIhEeb:B:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:" ret = -1 debug_save = <optimized out> debug_flag = <optimized out> dont_fork_cnt = 0 n_lst = <optimized out> p = <optimized out> tbuf = 0x7f7f7f1d21fe <handle_intel+142> "H\205\300u\272L\211\341H\211\352D\211\356D\211\377\350\354\374\377\377H\205\300u\244L\211\341H\211\352D\211\366D\211\377\350\326\374\377\377H\205\300u\216A\201", <incomplete sequence \357\277> tbuf_tmp = <optimized out> st = {st_dev = 24, st_ino = 35403, st_nlink = 2, st_mode = 16877, st_uid = 117, st_gid = 124, __pad0 = 0, st_rdev = 0, st_size = 60, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1776605683, tv_nsec = 661187506}, st_mtim = {tv_sec = 1776604769, tv_nsec = 912289336}, st_ctim = { tv_sec = 1776605719, tv_nsec = 209844336}, __glibc_reserved = {0, 0, 0}} l1 = <optimized out> lim = {rlim_cur = 262929407, rlim_max = 100} option_index = 0 long_options = {{name = 0x558b1ccd72d8 "help", has_arg = 0, flag = 0x0, val = 104}, { --Type <RET> for more, q to quit, c to continue without paging-- name = 0x558b1ccd9e5d "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x558b1ccdebf2 "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x558b1ccd72dd "domain", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x558b1ccd72e4 "subst", has_arg = 1, flag = 0x0, val = 1025}, { name = 0x558b1ccd72ea "substdef", has_arg = 1, flag = 0x0, val = 1026}, { name = 0x558b1ccd72f3 "substdefs", has_arg = 1, flag = 0x0, val = 1027}, { name = 0x558b1ccd72fd "server-id", has_arg = 1, flag = 0x0, val = 1028}, { name = 0x558b1ccd7307 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, { name = 0x558b1ccd7312 "modparam", has_arg = 1, flag = 0x0, val = 1030}, { name = 0x558b1ccd731b "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x558b1ccd9f7a "debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x558b1ccd7326 "cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {name = 0x558b1ccd7330 "atexit", has_arg = 1, flag = 0x0, val = 1034}, { name = 0x558b1ccd7337 "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name = 0x558b1ccd7342 "iuid", has_arg = 1, flag = 0x0, val = 1036}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}} __func__ = "main"
However, as it looks now, the corefile is written at Kamailio shutdown, either as an effect to another crash which has the core file overwritten if you didn't enabled core files per pid, or during a stop/restart command. To be sure the core files are not overwritten, enable core files per pid or define core file name pattern to be individual per running process. Kamailio wiki has an article about troubleshooting with corefile where more details are provided on these options.
Cheers, Daniel
On 19.04.26 16:27, Juha Heinanen via sr-users wrote:
After updating my sip proxy to latest kamailio 6.1 branch running on Debian Trixie , I have noticed several crashes.
syslog has this:
2026-04-19T16:41:19.664532+03:00 box kernel: kamailio[944355]: segfault at 0 ip 0000558b1c9ea97e sp 00007fffa1deb610 er\ ror 6 in kamailio[f297e,558b1c926000+3b0000] likely on CPU 2 (core 0, socket 4) 2026-04-19T16:41:19.664633+03:00 box kernel: Code: 00 00 00 48 8b 57 08 48 8b 07 f3 0f 7e 0d 8a 72 3f 00 48 8b 4f 10 64\ 4c 8b 24 25 28 00 00 00 4c 89 a4 24 88 00 00 00 41 89 f4 <48> 89 02 66 0f 6c ca 48 89 50 08 0f 11 07 48 83 c3 10 74 13\ 48 39 2026-04-19T16:41:19.936157+03:00 box /usr/sbin/kamailio[944455]: CRITICAL: <core> [core/pass_fd.c:284]: receive_fd(): E\ OF on 34 2026-04-19T16:41:20.435008+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:824]: handle_sigs(): child proce\ ss 944355 exited by a signal 11 2026-04-19T16:41:20.435143+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:828]: handle_sigs(): core was ge\ nerated 2026-04-19T16:41:20.435191+03:00 box /usr/sbin/kamailio[944292]: INFO: <core> [main.c:851]: handle_sigs(): terminating \ due to SIGCHLD 2026-04-19T16:41:20.435613+03:00 box /usr/sbin/kamailio[944405]: INFO: <core> [main.c:907]: sig_usr(): signal 15 receiv\ ed ...
and core dump shows this:
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
warning: 44 ./nptl/pthread_kill.c: No such file or directory (gdb) where #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 #1 0x00007f7f7ef509ff in __pthread_kill_internal (threadid=<optimized out>, signo=6) at ./nptl/pthread_kill.c:89 #2 0x00007f7f7eefbcc2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x00007f7f7eee44ac in __GI_abort () at ./stdlib/abort.c:77 #4 0x0000558b1c926d45 in ?? () #5 <signal handler called> #6 0x00007f7f7efb4907 in __GI_sched_yield () at ../sysdeps/unix/syscall-template.S:120 #7 0x0000558b1cc6c7ed in timer_del_safe () #8 0x0000558b1c9f8899 in destroy_dns_cache () #9 0x0000558b1c932537 in cleanup () #10 0x0000558b1c9332ed in ?? () #11 0x0000558b1c933fd3 in handle_sigs () #12 0x0000558b1c9395fd in main_loop () #13 0x0000558b1c92ae91 in main ()
Any ideas?
-- Juha
__________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions -- sr-users@lists.kamailio.org To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender!
-- Daniel-Constantin Mierla (@ asipto.com) twitter.com/miconda -- linkedin.com/in/miconda Kamailio Consultancy, Training and Development Services -- asipto.com Kamailio World Conference, May 7-8, 2026 - Berlin, Germany -- kamailioworld.com
Earlier today I backported a commit to dns cache that was supposed to fix some types of queries. Was your build done today? If yes, can you rebuild with latest 6.1 branch, as I just reverted that commit? Daniel On 19.04.26 18:04, Juha Heinanen wrote:
Daniel-Constantin Mierla writes:
You have to install the package with the debugging symbols for getting proper details in the backtrace. I re-built Kamailio and installed also kamailio-dbgsym package. I'll wait if another crash happens. Below is another old core dump from today.
-- Juha
Core was generated by `/usr/sbin/kamailio -f /etc/sip-proxy/sip-proxy.cfg -P /run/sip-proxy/sip-proxy.pid -m 128 -M 16 -u root -g root -w /run/sip-proxy -Y /run/sip-proxy -n 8'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000558b1c9ea97e in _dns_hash_remove_entry (e=0x7f7f75085ba0, line=662, fpath=0x558b1ccda66a "core/dns_cache.c") at core/dns_cache.c:501
warning: 501 core/dns_cache.c: No such file or directory (gdb) bt full #0 0x0000558b1c9ea97e in _dns_hash_remove_entry (e=0x7f7f75085ba0, line=662, fpath=0x558b1ccda66a "core/dns_cache.c") at core/dns_cache.c:501 __func__ = "_dns_hash_remove_entry" __llevel = <optimized out> __kld = <optimized out> __llevel = <optimized out> __kld = <optimized out> __llevel = <optimized out> __kld = <optimized out> #1 0x0000558b1c9f0e02 in dns_cache_clean (no=4294967295, expired_only=1) at core/dns_cache.c:662 e = 0x7f7f75085ba0 now = 658262396 n = <optimized out> deleted = <optimized out> l = 0x7f7f75085bb0 tmp = 0x0 __func__ = "dns_cache_clean" #2 dns_timer (ticks=<optimized out>, tl=<optimized out>, data=<optimized out>) at core/dns_cache.c:223 No locals. #3 dns_timer (ticks=<optimized out>, tl=<optimized out>, data=<optimized out>) at core/dns_cache.c:210 No locals. #4 0x0000558b1cc6f61a in slow_timer_main () at core/timer.c:1200 n = <optimized out> ret = <optimized out> tl = 0x7f7f74408120 i = <optimized out> __func__ = "slow_timer_main" #5 0x0000558b1c939455 in main_loop () at ./obj-x86_64-linux-gnu/src/main.c:2007 i = <optimized out> pid = <optimized out> si = <optimized out> sx = <optimized out> si_desc = "udp receiver child=7 sock=[2001:67c:224:666::38]:8002\000\000\000\022\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\2007t\177\177\000\000\002\000\000\000\004\000\000\000\177\000\000\001\213U\000\000}\000\000\000\000\000\000\000\000\277Țs\373,&Sun Apr \t\000\000\000\000\000\000" nrprocs = <optimized out> woneinit = 1 --Type <RET> for more, q to quit, c to continue without paging-- agfound = <optimized out> __func__ = "main_loop" error = <optimized out> #6 0x0000558b1c92ae91 in main (argc=<optimized out>, argv=<optimized out>) at ./obj-x86_64-linux-gnu/src/main.c:3434 cfg_stream = <optimized out> c = <optimized out> r = <optimized out> tmp = 0x7fffa1dede80 "" tmp_len = 0 port = 5060 proto = 0 aproto = 0 ahost = 0x0 socket_name = <optimized out> aport = 0 listen_field_count = <optimized out> listen_fields = {0x2 <error: Cannot access memory at address 0x2>, 0x7fffa1debb58 "", 0x1900000 <error: Cannot access memory at address 0x1900000>} options = 0x558b1ccede20 ":f:cm:M:dVIhEeb:B:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:" ret = -1 debug_save = <optimized out> debug_flag = <optimized out> dont_fork_cnt = 0 n_lst = <optimized out> p = <optimized out> tbuf = 0x7f7f7f1d21fe <handle_intel+142> "H\205\300u\272L\211\341H\211\352D\211\356D\211\377\350\354\374\377\377H\205\300u\244L\211\341H\211\352D\211\366D\211\377\350\326\374\377\377H\205\300u\216A\201", <incomplete sequence \357\277> tbuf_tmp = <optimized out> st = {st_dev = 24, st_ino = 35403, st_nlink = 2, st_mode = 16877, st_uid = 117, st_gid = 124, __pad0 = 0, st_rdev = 0, st_size = 60, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1776605683, tv_nsec = 661187506}, st_mtim = {tv_sec = 1776604769, tv_nsec = 912289336}, st_ctim = { tv_sec = 1776605719, tv_nsec = 209844336}, __glibc_reserved = {0, 0, 0}} l1 = <optimized out> lim = {rlim_cur = 262929407, rlim_max = 100} option_index = 0 long_options = {{name = 0x558b1ccd72d8 "help", has_arg = 0, flag = 0x0, val = 104}, { --Type <RET> for more, q to quit, c to continue without paging-- name = 0x558b1ccd9e5d "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x558b1ccdebf2 "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x558b1ccd72dd "domain", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x558b1ccd72e4 "subst", has_arg = 1, flag = 0x0, val = 1025}, { name = 0x558b1ccd72ea "substdef", has_arg = 1, flag = 0x0, val = 1026}, { name = 0x558b1ccd72f3 "substdefs", has_arg = 1, flag = 0x0, val = 1027}, { name = 0x558b1ccd72fd "server-id", has_arg = 1, flag = 0x0, val = 1028}, { name = 0x558b1ccd7307 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, { name = 0x558b1ccd7312 "modparam", has_arg = 1, flag = 0x0, val = 1030}, { name = 0x558b1ccd731b "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x558b1ccd9f7a "debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x558b1ccd7326 "cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {name = 0x558b1ccd7330 "atexit", has_arg = 1, flag = 0x0, val = 1034}, { name = 0x558b1ccd7337 "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name = 0x558b1ccd7342 "iuid", has_arg = 1, flag = 0x0, val = 1036}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}} __func__ = "main"
However, as it looks now, the corefile is written at Kamailio shutdown, either as an effect to another crash which has the core file overwritten if you didn't enabled core files per pid, or during a stop/restart command. To be sure the core files are not overwritten, enable core files per pid or define core file name pattern to be individual per running process. Kamailio wiki has an article about troubleshooting with corefile where more details are provided on these options.
Cheers, Daniel
On 19.04.26 16:27, Juha Heinanen via sr-users wrote:
After updating my sip proxy to latest kamailio 6.1 branch running on Debian Trixie , I have noticed several crashes.
syslog has this:
2026-04-19T16:41:19.664532+03:00 box kernel: kamailio[944355]: segfault at 0 ip 0000558b1c9ea97e sp 00007fffa1deb610 er\ ror 6 in kamailio[f297e,558b1c926000+3b0000] likely on CPU 2 (core 0, socket 4) 2026-04-19T16:41:19.664633+03:00 box kernel: Code: 00 00 00 48 8b 57 08 48 8b 07 f3 0f 7e 0d 8a 72 3f 00 48 8b 4f 10 64\ 4c 8b 24 25 28 00 00 00 4c 89 a4 24 88 00 00 00 41 89 f4 <48> 89 02 66 0f 6c ca 48 89 50 08 0f 11 07 48 83 c3 10 74 13\ 48 39 2026-04-19T16:41:19.936157+03:00 box /usr/sbin/kamailio[944455]: CRITICAL: <core> [core/pass_fd.c:284]: receive_fd(): E\ OF on 34 2026-04-19T16:41:20.435008+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:824]: handle_sigs(): child proce\ ss 944355 exited by a signal 11 2026-04-19T16:41:20.435143+03:00 box /usr/sbin/kamailio[944292]: ALERT: <core> [main.c:828]: handle_sigs(): core was ge\ nerated 2026-04-19T16:41:20.435191+03:00 box /usr/sbin/kamailio[944292]: INFO: <core> [main.c:851]: handle_sigs(): terminating \ due to SIGCHLD 2026-04-19T16:41:20.435613+03:00 box /usr/sbin/kamailio[944405]: INFO: <core> [main.c:907]: sig_usr(): signal 15 receiv\ ed ...
and core dump shows this:
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
warning: 44 ./nptl/pthread_kill.c: No such file or directory (gdb) where #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 #1 0x00007f7f7ef509ff in __pthread_kill_internal (threadid=<optimized out>, signo=6) at ./nptl/pthread_kill.c:89 #2 0x00007f7f7eefbcc2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x00007f7f7eee44ac in __GI_abort () at ./stdlib/abort.c:77 #4 0x0000558b1c926d45 in ?? () #5 <signal handler called> #6 0x00007f7f7efb4907 in __GI_sched_yield () at ../sysdeps/unix/syscall-template.S:120 #7 0x0000558b1cc6c7ed in timer_del_safe () #8 0x0000558b1c9f8899 in destroy_dns_cache () #9 0x0000558b1c932537 in cleanup () #10 0x0000558b1c9332ed in ?? () #11 0x0000558b1c933fd3 in handle_sigs () #12 0x0000558b1c9395fd in main_loop () #13 0x0000558b1c92ae91 in main ()
Any ideas?
-- Juha
__________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions -- sr-users@lists.kamailio.org To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender!
-- Daniel-Constantin Mierla (@ asipto.com) twitter.com/miconda -- linkedin.com/in/miconda Kamailio Consultancy, Training and Development Services -- asipto.com Kamailio World Conference, May 7-8, 2026 - Berlin, Germany -- kamailioworld.com
-- Daniel-Constantin Mierla (@ asipto.com) twitter.com/miconda -- linkedin.com/in/miconda Kamailio Consultancy, Training and Development Services -- asipto.com Kamailio World Conference, May 7-8, 2026 - Berlin, Germany -- kamailioworld.com
Daniel-Constantin Mierla writes:
Earlier today I backported a commit to dns cache that was supposed to fix some types of queries. Was your build done today?
If yes, can you rebuild with latest 6.1 branch, as I just reverted that commit?
There was another crash today 18:19 CET using Kamailio that was built from 6.1 today at 17:00 CET. When was your revert done? Below.is bt of that crash. -- Juha (gdb) bt full #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 tid = <optimized out> ret = 0 pd = <optimized out> old_mask = {__val = {140185568063568}} ret = <optimized out> #1 0x00007f7f7ef509ff in __pthread_kill_internal (threadid=<optimized out>, signo=6) at ./nptl/pthread_kill.c:89 No locals. #2 0x00007f7f7eefbcc2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 ret = <optimized out> #3 0x00007f7f7eee44ac in __GI_abort () at ./stdlib/abort.c:77 act = {__sigaction_handler = {sa_handler = 0x558b1cce805f, sa_sigaction = 0x558b1cce805f}, sa_mask = {__val = { 94055972036658, 0, 94057000091456, 94057000188160, 4, 16384, 0, 140735909113696, 140185549331538, 3591772330067, 94057000204672, 94057000204774, 94057000204672, 7468223001285240130, 94057000213584, 18446744073709551520}}, sa_flags = 11, sa_restorer = 0x2} #4 0x0000558b1c926d45 in sig_alarm_abort (signo=<optimized out>) at ./obj-x86_64-linux-gnu/src/main.c:735 __func__ = "sig_alarm_abort" #5 <signal handler called> No locals. #6 0x00007f7f7efb4907 in __GI_sched_yield () at ../sysdeps/unix/syscall-template.S:120 No locals. #7 0x0000558b1cc6c7ed in timer_del_safe (tl=0x7f7f74408120) at core/timer.c:655 ret = -1 again = <optimized out> __func__ = "timer_del_safe" #8 0x0000558b1c9f8899 in destroy_dns_cache () at core/dns_cache.c:233 __func__ = "destroy_dns_cache" #9 0x0000558b1c932537 in cleanup (show_status=show_status@entry=1) at ./obj-x86_64-linux-gnu/src/main.c:608 memlog = <optimized out> __func__ = "cleanup" #10 0x0000558b1c9332ed in shutdown_children (show_status=show_status@entry=1, sig=15) at ./obj-x86_64-linux-gnu/src/main.c:754 __func__ = "shutdown_children" --Type <RET> for more, q to quit, c to continue without paging-- #11 0x0000558b1c933fd3 in handle_sigs () at ./obj-x86_64-linux-gnu/src/main.c:854 chld = <optimized out> chld_status = 139 any_chld_stopped = <optimized out> memlog = <optimized out> __func__ = "handle_sigs" #12 0x0000558b1c9395fd in main_loop () at ./obj-x86_64-linux-gnu/src/main.c:2094 i = <optimized out> pid = <optimized out> si = <optimized out> sx = <optimized out> si_desc = "udp receiver child=7 sock=[2001:67c:224:666::38]:8002\000\000\000\022\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\2007t\177\177\000\000\002\000\000\000\004\000\000\000\177\000\000\001\213U\000\000}\000\000\000\000\000\000\000\000\277Țs\373,&Sun Apr \t\000\000\000\000\000\000" nrprocs = <optimized out> woneinit = 1 agfound = <optimized out> __func__ = "main_loop" error = <optimized out> #13 0x0000558b1c92ae91 in main (argc=<optimized out>, argv=<optimized out>) at ./obj-x86_64-linux-gnu/src/main.c:3434 cfg_stream = <optimized out> c = <optimized out> r = <optimized out> tmp = 0x7fffa1dede80 "" tmp_len = 0 port = 5060 proto = 0 aproto = 0 ahost = 0x0 socket_name = <optimized out> aport = 0 listen_field_count = <optimized out> listen_fields = {0x2 <error: Cannot access memory at address 0x2>, 0x7fffa1debb58 "", 0x1900000 <error: Cannot access memory at address 0x1900000>} --Type <RET> for more, q to quit, c to continue without paging-- options = 0x558b1ccede20 ":f:cm:M:dVIhEeb:B:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:" ret = -1 debug_save = <optimized out> debug_flag = <optimized out> dont_fork_cnt = 0 n_lst = <optimized out> p = <optimized out> tbuf = 0x7f7f7f1d21fe <handle_intel+142> "H\205\300u\272L\211\341H\211\352D\211\356D\211\377\350\354\374\377\377H\205\300u\244L\211\341H\211\352D\211\366D\211\377\350\326\374\377\377H\205\300u\216A\201", <incomplete sequence \357\277> tbuf_tmp = <optimized out> st = {st_dev = 24, st_ino = 35403, st_nlink = 2, st_mode = 16877, st_uid = 117, st_gid = 124, __pad0 = 0, st_rdev = 0, st_size = 60, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1776605683, tv_nsec = 661187506}, st_mtim = {tv_sec = 1776604769, tv_nsec = 912289336}, st_ctim = {tv_sec = 1776605719, tv_nsec = 209844336}, __glibc_reserved = {0, 0, 0}} l1 = <optimized out> lim = {rlim_cur = 262929407, rlim_max = 100} option_index = 0 long_options = {{name = 0x558b1ccd72d8 "help", has_arg = 0, flag = 0x0, val = 104}, { name = 0x558b1ccd9e5d "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x558b1ccdebf2 "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x558b1ccd72dd "domain", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x558b1ccd72e4 "subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x558b1ccd72ea "substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x558b1ccd72f3 "substdefs", has_arg = 1, flag = 0x0, val = 1027}, {name = 0x558b1ccd72fd "server-id", has_arg = 1, flag = 0x0, val = 1028}, { name = 0x558b1ccd7307 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name = 0x558b1ccd7312 "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x558b1ccd731b "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x558b1ccd9f7a "debug", has_arg = 1, flag = 0x0, val = 1032}, { name = 0x558b1ccd7326 "cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {name = 0x558b1ccd7330 "atexit", has_arg = 1, flag = 0x0, val = 1034}, {name = 0x558b1ccd7337 "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name = 0x558b1ccd7342 "iuid", has_arg = 1, flag = 0x0, val = 1036}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}} __func__ = "main"
I'm currently making new build. Your revert was not in the previous one, since the new one showed this: Already on '6.1' Your branch is up to date with 'origin/6.1'. remote: Enumerating objects: 28, done. remote: Counting objects: 100% (26/26), done. remote: Compressing objects: 100% (4/4), done. remote: Total 12 (delta 10), reused 10 (delta 8), pack-reused 0 (from 0) Unpacking objects: 100% (12/12), 1.17 KiB | 20.00 KiB/s, done. From https://github.com/kamailio/kamailio 84ebaa2079..5a281a77ba 6.1 -> origin/6.1 eb53aace0f..8ee596f697 master -> origin/master Updating 84ebaa2079..5a281a77ba Fast-forward src/core/dns_cache.c | 16 +++++----------- src/core/dns_cache.h | 34 +++------------------------------- 2 files changed, 8 insertions(+), 42 deletions(-) After build and install. I'll keep on watching if crashes are gone. -- Juha
Daniel-Constantin Mierla writes:
If yes, can you rebuild with latest 6.1 branch, as I just reverted that commit?
No more crashes during the past 11 hours with the latest 6.1 branch. Please create new tag 6.1.2 so that my other servers can get the fixed version automatically. -- Juha
Thanks for testing/reporting back the results! That patch that was reverted was backported to 6.1 branch yesterday morning, it was not part of 6.1.1 and will probably not be fixed in time to be readded for 6.1.2 (which is planned for this week on Wednesday). Daniel On 20.04.26 06:27, Juha Heinanen wrote:
Daniel-Constantin Mierla writes:
If yes, can you rebuild with latest 6.1 branch, as I just reverted that commit? No more crashes during the past 11 hours with the latest 6.1 branch. Please create new tag 6.1.2 so that my other servers can get the fixed version automatically.
-- Juha
-- Daniel-Constantin Mierla (@ asipto.com) twitter.com/miconda -- linkedin.com/in/miconda Kamailio Consultancy, Training and Development Services -- asipto.com Kamailio World Conference, May 7-8, 2026 - Berlin, Germany -- kamailioworld.com
participants (2)
-
Daniel-Constantin Mierla -
Juha Heinanen