sergey-safarov created an issue (kamailio/kamailio#4356)
### Description
Kamailio core generated when `siptrace` module tries to relay a message generated by `uac` module.
### Troubleshooting
#### Reproduction To reproduce the issue need to start Kamailio with the config file. ``` #!KAMAILIO
#!define L_INFO 2 #!trydef TROUBLESHOOT_NET "" local.trace_troubleshooting_net = TROUBLESHOOT_NET descr "trace all SIP messages from given network"
# IPv4 listeners listen=udp:lan
debug=L_INFO log_stderror=yes
loadmodule "pv.so" loadmodule "tm.so"
####### DATABASE module ########## #!substdef "!KAMAILIO_DB_URL!mysql://kamailio:change_me@127.0.0.1:3306/kamailio!g" loadmodule "db_mysql.so"
loadmodule "uac.so" modparam("uac", "reg_db_url", "KAMAILIO_DB_URL") modparam("uac", "reg_contact_addr", "192.168.1.2:5080") modparam("uac","restore_mode","none")
####### siptrace module ########## loadmodule "siptrace.so" modparam("siptrace", "duplicate_uri", "sip:127.0.0.1:9060") modparam("siptrace", "trace_to_database", 0) modparam("siptrace", "trace_on", 1) modparam("siptrace", "trace_mode", 1)
####### Routing Logic ######## request_route { drop; }
event_route[siptrace:msg] { if ($sel(cfg_get.local.trace_troubleshooting_net) != "") { return; } } ``` In the database, need to inserta record ```sql INSERT INTO `uacreg` VALUES (1,'120','120','example.com','120','10.191.1.21','','120','','8e789f9318c98b9ffd4f0fd9e0b8a747','sip:10.191.1.21;transport=udp',120,0,0,'','');
```
#### Debugging Data
Backtrace for the generated core in the attached file [kamailio-bt-full.txt](https://github.com/user-attachments/files/21671121/kamailio-bt-full.txt)
#### Log Messages ``` / # kamailio -DD --atexit=no -m 64 -M 8 -E 0(43) INFO: <core> [core/sctp_core.c:74]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module Listening on udp: 10.191.1.20:5060 Aliases: udp: ip-10-191-1-20.ec2.internal:5060
0(43) INFO: <core> [core/tcp_main.c:5232]: init_tcp(): using epoll_lt as the io watch method (auto detected) 0(43) INFO: siptrace [siptrace.c:438]: mod_init(): can't load dlg api. Will not install dialog callbacks. 0(43) ERROR: siptrace [../../modules/sl/sl.h:107]: sl_load_api(): cannot find bind_sl 0(43) WARNING: siptrace [siptrace.c:450]: mod_init(): cannot bind to SL API. Will not install sl callbacks. 0(43) INFO: <core> [main.c:3233]: main(): processes (at least): 14 - shm size: 67108864 - pkg size: 8388608 0(43) INFO: <core> [core/udp_server.c:163]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992 for fd 4 0(43) INFO: <core> [core/udp_server.c:233]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984 on fd 4 0(43) INFO: <core> [core/udp_server.c:264]: probe_max_send_buffer(): SO_SNDBUF is initially 212992 for fd 4 0(43) INFO: <core> [core/udp_server.c:334]: probe_max_send_buffer(): SO_SNDBUF is finally 425984 on fd 4 0(43) ALERT: <core> [main.c:805]: handle_sigs(): child process 55 exited by a signal 11 0(43) ALERT: <core> [main.c:809]: handle_sigs(): core was generated 0(43) INFO: <core> [main.c:832]: handle_sigs(): terminating due to SIGCHLD 1(44) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 2(45) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 3(46) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 4(47) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 5(48) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 6(49) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 7(50) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 8(51) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 9(52) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 13(56) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 10(53) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 11(54) INFO: <core> [main.c:888]: sig_usr(): signal 15 received 0(43) INFO: <core> [core/sctp_core.c:53]: sctp_core_destroy(): SCTP API not initialized ``` ### Possible Solutions As temporary workaround we use ```diff diff --git a/src/core/cfg/cfg_select.c b/src/core/cfg/cfg_select.c index f948dd4ab4..1a36ef378a 100644 --- a/src/core/cfg/cfg_select.c +++ b/src/core/cfg/cfg_select.c @@ -207,8 +207,10 @@ int select_cfg_var(str *res, select_t *s, struct sip_msg *msg) group = (cfg_group_t *)s->params[1].v.p; var = (cfg_mapping_t *)s->params[2].v.p;
- if(!group || !var) + if(!group || !var || NULL == *(group->handle)) { + LM_ERR("incorrect input args\n"); return -1; + }
/* use the module's handle to access the variable, so the variables * are read from the local config */ ```
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
``` / # kamailio -v version: kamailio 5.8.4 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled on 14:31:12 Nov 18 2024 with gcc 14.2.0 ```
* **Operating System**:
<!-- Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...; Kernel details (output of `lsb_release -a` and `uname -a`) -->
``` NAME="Alpine Linux" ID=alpine VERSION_ID=3.21.4 PRETTY_NAME="Alpine Linux v3.21" HOME_URL="https://alpinelinux.org/" BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues" ``` uname ``` Linux sbc0-site0-stage.ph.nga911.com 6.15.5-200.fc42.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Jul 6 09:16:17 UTC 2025 x86_64 Linux ```
miconda left a comment (kamailio/kamailio#4356)
Can you attach the gdb backtrace to see what was the execution path when crashing?
sergey-safarov left a comment (kamailio/kamailio#4356)
Full back trace attached at https://github.com/user-attachments/files/21671121/kamailio-bt-full.txt
I have grep and stack look ``` safarov@nout:/tmp$ grep -P "^#" kamailio-bt-full.txt #0 memcpy (__od=0x7ffc101eb128, __os=0x0, __n=16) at /usr/include/fortify/string.h:55 #1 select_cfg_var (res=res@entry=0x7ffc101eb128, s=s@entry=0x7f7c289a5830, msg=msg@entry=0x7ffc101ebe68) at core/cfg/cfg_select.c:236 #2 0x0000558368b02508 in run_select (res=res@entry=0x7ffc101eb128, s=0x7f7c289a5830, msg=msg@entry=0x7ffc101ebe68) at core/select.c:461 #3 0x00007f7c2879ee8e in pv_get_select (msg=0x7ffc101ebe68, param=0x7f7c289a5d48, res=0x7ffc101eb470) at pv_select.c:58 #4 0x0000558368aa4aad in pv_get_spec_value (msg=0x7ffc101ebe68, sp=0x7f7c289a5d30, value=0x7ffc101eb470) at core/pvapi.c:1355 #5 0x0000558368ae90b4 in rval_get_btype (msg=msg@entry=0x7ffc101ebe68, rv=rv@entry=0x7f7c289a5d28, val_cache=val_cache@entry=0x7ffc101eb468, h=0x7ffc101ebd90) at core/rvalue.c:427 #6 0x0000558368aef654 in rval_expr_eval_rvlong (h=h@entry=0x7ffc101ebd90, msg=msg@entry=0x7ffc101ebe68, res_rv=res_rv@entry=0x7ffc101eb420, res_i=res_i@entry=0x7ffc101eb410, #7 0x0000558368af06fe in rval_expr_eval_long (h=h@entry=0x7ffc101ebd90, msg=msg@entry=0x7ffc101ebe68, res=res@entry=0x7ffc101eb580, rve=rve@entry=0x7f7c289a6d10) at core/rvalue.c:1933 #8 0x00005583689deb2a in do_action (h=h@entry=0x7ffc101ebd90, a=a@entry=0x7f7c289a75b0, msg=msg@entry=0x7ffc101ebe68) at core/action.c:1097 #9 0x00005583689d93c8 in run_actions (h=h@entry=0x7ffc101ebd90, a=a@entry=0x7f7c289a75b0, msg=msg@entry=0x7ffc101ebe68) at core/action.c:1618 #10 0x00005583689e5f00 in run_top_route (a=0x7f7c289a75b0, msg=msg@entry=0x7ffc101ebe68, c=c@entry=0x7ffc101ebd90) at core/action.c:1701 #11 0x00007f7c23ed3175 in siptrace_exec_evcb_msg (sto=sto@entry=0x7ffc101ec798) at siptrace.c:2255 #12 0x00007f7c23ee0ec0 in siptrace_net_data_sent (evp=<optimized out>) at siptrace.c:2487 #13 0x00007f7c285821fe in msg_send_buffer (dst=0x7f7c241a3c10, #14 0x00007f7c285822f5 in send_pr_buffer (rb=<optimized out>, buf=<optimized out>, len=<optimized out>) at t_funcs.c:75 #15 0x00007f7c285f8a7d in send_prepared_request_impl (request=<optimized out>, retransmit=retransmit@entry=1, branch=branch@entry=0) at uac.c:707 #16 0x00007f7c285fe40f in t_uac_with_ids (uac_r=uac_r@entry=0x7ffc101ed650, ret_index=ret_index@entry=0x0, ret_label=ret_label@entry=0x0) at uac.c:793 #17 0x00007f7c285fe4aa in t_uac (uac_r=uac_r@entry=0x7ffc101ed650) at uac.c:757 #18 0x00007f7c286031db in request (uac_r=0x7ffc101ed650, ruri=<optimized out>, to=0x7ffc101ed5f0, from=0x7ffc101ed5f0, next_hop=0x7f7c241a3208) at uac.c:1137 #19 0x00007f7c27f33767 in uac_reg_send (reg=0x7f7c241a31a0, tn=1754589861) at uac_reg.c:1114 #20 0x00007f7c27f3428f in uac_reg_timer (ticks=<optimized out>) at uac_reg.c:1198 #21 0x00007f7c27f2726b in child_init (rank=<optimized out>) at uac.c:425 #22 0x0000558368b1fb2a in init_mod_child (m=0x7f7c289a3250, rank=rank@entry=0) at core/sr_module.c:917 #23 0x0000558368b1f878 in init_mod_child (m=0x7f7c289a45b0, rank=rank@entry=0) at core/sr_module.c:909 #24 0x0000558368b239f3 in init_child (rank=<optimized out>, rank@entry=0) at core/sr_module.c:996 #25 0x00005583689d7e38 in main_loop () at main.c:1942 #26 0x00005583689cc603 in main (argc=<optimized out>, argv=<optimized out>) at main.c:3256 ```
github-actions[bot] left a comment (kamailio/kamailio#4356)
This issue is stale because it has been open 6 weeks with no activity. Remove stale label or comment or this will be closed in 2 weeks.
miconda left a comment (kamailio/kamailio#4356)
Likely the cfg-vars framework was not properly initialized when the first UAC-registrations are sent on child-init. I pushed a commit, try to see if solves it.
Closed #4356 as completed.
miconda left a comment (kamailio/kamailio#4356)
Reopen of create a new one if the commits referenced above didn't fix it.