sr-dev July 2024

sr-dev@lists.kamailio.org

7 participants
59 discussions

[kamailio/kamailio] ims_registrar_pcscf,ims_qos: support for trusting bottom via (PR #3901)

by Dragos Vingarzan

#### Pre-Submission Checklist    - [x] Commit message has the format required by CONTRIBUTING guide - [x] Commits are split per component (core, individual modules, libs, utils, ...) - [x] Each component has a single commit (if not, squash them into one commit) - [x] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated) #### Type Of Change - [ ] Small bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality) #### Checklist:  - [ ] PR should be backported to stable branches - [x] Tested changes locally - [x] Related to PR https://github.com/kamailio/kamailio/pull/3891 #### Description  Normally, the IMS P-CSCF should identify the clients (UEs) by the received IP address and ports on Rx. The current code is using a mix of that, plus using Contact and Via headers, with arguable potential security issues. This patch adds a new parameter to `ims_registrar_pcscf` and `ims_qos` modules, allowing for an optional outsource of the IPsec functionality to another element, which is also in charge of checking/enforcing correct UE Via header. The existing code is allowed to work as before, with the default value of the flag being towards that. List of functional changes: - `ims_qos` - added `trust_bottom_via` parameter - used it on `w_rx_aar_register()` - `ims_registrar_pcscf` - added `trust_bottom_via` parameter - used it on `update_contacts()`, `save_pending()`, `check_contact()`, `getContactP()`, `check_service_routes()`, `enforce_service_routes()` - made `ims_ipsec_pcscf` dependency optional, with checks when used - skipped checks of `port-uc` in `checkcontact()` if the `ims_ipsec_pcscf` module was not loaded - added `ignore_contact_rxproto_check` parameter - IMS devices open IPsec Security Associations just between IPs and ports, with both UDP and TCP protocols allowed. The default then was set here to always ignore protocol checks. Before, the `ignore_contact_rxport_check` was used to skip this and still make it work, but that seemed like a typo/mistake with hidden effects. List of indirect changes: - `core/ut.h`: added a `str2ushort()` macro, since code was using some dangerous casting and macros with a larger type - `ims_registrar_pcscf`: added `Route` headers in `SUBSCRIBE` to `reginfo`, with values from `Service-Route`s, as per 3GPP specs. - `ims_usrloc_pcscf`: small log fixes You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/3901 -- Commit Summary -- * core: added str2ushort() to avoid some questionable casting in ims_qos * lib/ims: fixed a typo in a log message * ims_usrloc_pcscf: small log fixes * ims_qos: added trust_bottom_via parameter * ims_registrar_pcscf: added trust_bottom_via parameter, added ignore_contact_rxproto_check, fixes for logging, fixes for SUBSCRIBE to reginfo for including Service-Route, allow use without ims_ipsec_pcscf -- File Changes -- M src/core/ut.h (8) M src/lib/ims/ims_getters.c (2) M src/modules/ims_qos/doc/ims_qos.xml (6) M src/modules/ims_qos/doc/ims_qos_admin.xml (41) M src/modules/ims_qos/ims_qos_mod.c (27) M src/modules/ims_qos/rx_aar.h (4) M src/modules/ims_qos/rx_authdata.h (2) M src/modules/ims_qos/rx_avp.c (2) M src/modules/ims_registrar_pcscf/doc/ims_registrar_pcscf.xml (6) M src/modules/ims_registrar_pcscf/doc/ims_registrar_pcscf_admin.xml (73) M src/modules/ims_registrar_pcscf/ims_registrar_pcscf_mod.c (26) M src/modules/ims_registrar_pcscf/notify.c (2) M src/modules/ims_registrar_pcscf/save.c (80) M src/modules/ims_registrar_pcscf/service_routes.c (136) M src/modules/ims_registrar_pcscf/subscribe.c (74) M src/modules/ims_registrar_pcscf/subscribe.h (4) M src/modules/ims_usrloc_pcscf/udomain.c (4) -- Patch Links -- https://github.com/kamailio/kamailio/pull/3901.patch https://github.com/kamailio/kamailio/pull/3901.diff -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/3901 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/pull/3901(a)github.com>

10 hours, 24 minutes

[kamailio/kamailio] kamailio 5.7.6 startup (Issue #3908)

by alekamailio

Hello, Upgraded the kamailio version from 5.7.5 to 5.7.6 and after restart the service got the follow WARNING, WARNING: db_mysql [km_my_con.c:179]: db_mysql_new_connection(): opt_ssl_ca option not supported by mysql version (value (null)) - ignoring, when downgrade to 5.7.5 dindt happend. Thanks -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3908 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3908(a)github.com>

11 hours, 7 minutes

[kamailio/kamailio] ndb_redis: Error when receiving 'MOVED' response from redis-cluster with TLS (Issue #3894)

by uv-2722

### Description Hello, I am using ndb_redis module with TLS and redis-cluster. kamailio connects to the cluster endpoint when getting data, gets a 'MOVED' response and prints below error, it does not get data from the new host. ### Troubleshooting I have verified that I can connect to the cluster endpoint and the host in 'MOVED' using redis-cli on the system that kamailio is running on. I tried to connect to a self hosted cluster without TLS, it works as expected. It also works as expected with TLS and a single redis host. #### Reproduction Connect to a redis-cluster over TLS #### Log Messages  ``` 2024-06-25T20:48:53.626708963Z 2024-06-25 20:48:53.626584 3(17) NOTICE: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:437]: redisc_get_server(): Hash 31807 (ra) 2024-06-25T20:48:53.626710053Z 2024-06-25 20:48:53.626600 3(17) DEBUG: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:440]: redisc_get_server(): Entry 31807 (ra) 2024-06-25T20:48:53.626711073Z 2024-06-25 20:48:53.626636 3(17) NOTICE: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:444]: redisc_get_server(): Using entry 31807 (ra) 2024-06-25T20:48:53.626712073Z 2024-06-25 20:48:53.626651 3(17) DEBUG: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:1030]: redisc_exec(): rsrv->ctxRedis = 0x556b4456dc20 2024-06-25T20:48:53.626713073Z 2024-06-25 20:48:53.626667 3(17) DEBUG: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:1036]: redisc_exec(): rsrv->ctxRedis = 0x556b4456dc20 2024-06-25T20:48:53.626810463Z 2024-06-25 20:48:53.626762 3(17) DEBUG: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:896]: check_cluster_reply(): Redis replied: "MOVED 3667 testdbredis-masked-hostname.00000000001110000000.130-characters-long.00000000000011111111100000000.availability-zone.amazonaws.com:6379" 2024-06-25T20:48:53.626811523Z 2024-06-25 20:48:53.626777 3(17) DEBUG: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:904]: check_cluster_reply(): Port "6379" [4] => 6379 2024-06-25T20:48:53.626816413Z 2024-06-25 20:48:53.626793 3(17) DEBUG: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:914]: check_cluster_reply(): Host "testdbredis-masked-hostname.00000000001110000000.130-characters-long.000000…" [130] 2024-06-25T20:48:53.626846113Z 2024-06-25 20:48:53.626810 3(17) NOTICE: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:925]: check_cluster_reply(): Name of new connection: testdbredis-masked-hostname.00000000001110000000.130-characters-long.00000000000011111111100000000. 2024-06-25T20:48:53.626851443Z 2024-06-25 20:48:53.626827 3(17) NOTICE: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:437]: redisc_get_server(): Hash 968070864 (testdbredis-masked-hostname.00000000001110000000.130-characters-long.00000000000011111111100000000.) 2024-06-25T20:48:53.626896173Z 2024-06-25 20:48:53.626844 3(17) DEBUG: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:440]: redisc_get_server(): Entry 31807 (ra) 2024-06-25T20:48:53.626897503Z 2024-06-25 20:48:53.626861 3(17) NOTICE: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:450]: redisc_get_server(): No entry found. 2024-06-25T20:48:53.626955363Z 2024-06-25 20:48:53.626877 3(17) ERROR: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:947]: check_cluster_reply(): failed to print server spec string 2024-06-25T20:48:53.626957263Z 2024-06-25 20:48:53.626893 3(17) DEBUG: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:1115]: redisc_exec(): rpl->rplRedis->type:6 2024-06-25T20:48:53.626958443Z 2024-06-25 20:48:53.626908 3(17) ERROR: {Kam 192.168.20.4 503874790_98536808 1 753067 INVITE}ndb_redis [redis_client.c:1117]: redisc_exec(): Redis error:MOVED 3667 testdbredis-masked-hostname.00000000001110000000.130-characters-long.00000000000011111111100000000.availability-zone.amazonaws.com:6379 ``` ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.8.1 (x86_64/linux) 384843 flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 384843 compiled on 19:05:30 Jun 25 2024 with gcc 12.2.0 ``` * Hiredis(v1.2.0) and hiredis-cluster(v0.13.0) are installed from source, both with ssl support. * **Operating System**:  ``` Debian 12 ``` ### Kamailio config snippet ``` loadmodule "ndb_redis.so" . . # ----- ndb_redis params ----- modparam("ndb_redis", "server", "name=ra;addr=cluster.testdbredis-masked-hostname.0000000011111111100000.availability-zone.amazonaws.com;port=6379;tls=1") modparam("ndb_redis", "init_without_redis", 1) modparam("ndb_redis", "cluster", 1) modparam("ndb_redis", "allow_dynamic_nodes", 1) modparam("ndb_redis", "ca_path", "/etc/ssl/certs") modparam("ndb_redis", "debug", 1) . . redis_cmd("ra", "HMGET usr_preferences:entry::%s:%s blocked", "$rU", "$fU", "r"); if($redis(r=>value[0]) == 1) { . } ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3894 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3894(a)github.com>

1 day, 22 hours

[kamailio/kamailio] db_redis: compile issues with TLS + redis-cluster (Issue #3893)

by uv-2722

1 day, 22 hours

[kamailio/kamailio] Kamailio crash with DMQ!? (Issue #3905)

by Benjamin

### Description Kamailio version 5.7.4 crashed with a core file. Maybe it's related to the dmq module. #### Debugging Data ``` (gdb) bt full #0 atomic_inc_int (var=0x7f5700000000) at ../../core/parser/../mem/../atomic/atomic_x86.h:236 No locals. #1 cfg_update_local (no_cbs=<optimized out>) at ../../core/cfg/cfg_struct.h:381 group = <optimized out> last_cb = 0x7f57c99545e0 prev_cb = <optimized out> group = <optimized out> last_cb = <optimized out> prev_cb = <optimized out> __func__ = "cfg_update_local" #2 worker_loop (id=id@entry=2) at worker.c:57 worker = <optimized out> current_job = <optimized out> peer_response = {resp_code = 1, content_type = {s = 0x55ea7b42cfca <init_mod_child+90> "\205\300\017\210\r\002", len = 15}, reason = {s = 0x0, len = 2}, body = { s = 0x7ffcf6347c30 "\001", len = 2069833640}} ret_value = <optimized out> not_parsed = <optimized out> dmq_node = 0x0 __func__ = "worker_loop" #3 0x00007f57d9426a11 in child_init (rank=<optimized out>) at dmq.c:321 i = 2 newpid = <optimized out> __func__ = "child_init" #4 0x000055ea7b42cfca in init_mod_child (m=0x7f57dd005640, rank=rank@entry=0) at core/sr_module.c:911 ret = 0 __func__ = "init_mod_child" #5 0x000055ea7b42cfa4 in init_mod_child (m=0x7f57dd005b90, rank=rank@entry=0) at core/sr_module.c:903 ret = <optimized out> __func__ = "init_mod_child" #6 0x000055ea7b42cfa4 in init_mod_child (m=0x7f57dd006910, rank=rank@entry=0) at core/sr_module.c:903 ret = <optimized out> __func__ = "init_mod_child" #7 0x000055ea7b42cfa4 in init_mod_child (m=0x7f57dd006e70, rank=rank@entry=0) at core/sr_module.c:903 ret = <optimized out> __func__ = "init_mod_child" #8 0x000055ea7b42cfa4 in init_mod_child (m=0x7f57dd008660, rank=rank@entry=0) at core/sr_module.c:903 ret = <optimized out> __func__ = "init_mod_child" #9 0x000055ea7b42cfa4 in init_mod_child (m=0x7f57dd0089d0, rank=rank@entry=0) at core/sr_module.c:903 ret = <optimized out> __func__ = "init_mod_child" #10 0x000055ea7b42cfa4 in init_mod_child (m=0x7f57dd008d70, rank=rank@entry=0) at core/sr_module.c:903 ret = <optimized out> __func__ = "init_mod_child" #11 0x000055ea7b43239e in init_child (rank=rank@entry=0) at core/sr_module.c:990 ret = <optimized out> type = <optimized out> __func__ = "init_child" #12 0x000055ea7b288365 in main_loop () at main.c:1929 i = <optimized out> pid = <optimized out> si = 0x0 si_desc = "udp receiver child=15 sock=172.20.21.4:5060\000\000\000\000\000H\211\064\366\374\177\000\000\301'C{\352U\000\000\276\000\000\000\002\000\000\000\017\247`{\352U\000\000\062\323b{\352U\000\000U\024\000\000\000\000\000\000\004\000_{\352U\000\000\000\272\030D\344\223ԃ\270Rc{\352U\000\000\004\000\000\000\000\000\000" nrprocs = <optimized out> woneinit = 1 __func__ = "main_loop" error = <optimized out> #13 0x000055ea7b27be9e in main (argc=<optimized out>, argv=<optimized out>) at main.c:3213 cfg_stream = <optimized out> c = <optimized out> r = <optimized out> tmp = 0x7ffcf6349e4e "" tmp_len = -554008000 port = 32599 proto = 0 ahost = 0x0 aport = 0 options = 0x55ea7b5f2cc8 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:" ret = -1 seed = 1892320457 rfd = <optimized out> debug_save = <optimized out> debug_flag = <optimized out> dont_fork_cnt = <optimized out> n_lst = <optimized out> p = <optimized out> st = {st_dev = 0, st_ino = 0, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 0, st_blocks = 0, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 0, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {0, 0, 0}} tbuf = '\000' <repeats 20 times>, " ", '\000' <repeats 35 times>, "\006\000\000\000\000\000\000\200", '\000' <repeats 56 times>, "\006\000\000\000\000\000\000\200", '\000' <repeats 168 times>... option_index = 9 long_options = {{name = 0x55ea7b5f12ea "help", has_arg = 0, flag = 0x0, val = 104}, {name = 0x55ea7b5fa98e "version", has_arg = 0, flag = 0x0, val = 118}, { name = 0x55ea7b60a690 "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x55ea7b5f12ef "subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x55ea7b5f12f5 "substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x55ea7b5f12fe "substdefs", has_arg = 1, flag = 0x0, val = 1027}, {name = 0x55ea7b5f1308 "server-id", has_arg = 1, flag = 0x0, val = 1028}, {name = 0x55ea7b5f1312 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name = 0x55ea7b5f131d "modparam", has_arg = 1, flag = 0x0, val = 1030}, { name = 0x55ea7b5f1326 "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x55ea7b5faaab "debug", has_arg = 1, flag = 0x0, val = 1032}, { name = 0x55ea7b5f1331 "cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {name = 0x55ea7b5f133b "atexit", has_arg = 1, flag = 0x0, val = 1034}, { name = 0x55ea7b5f1342 "all-errors", has_arg = 0, flag = 0x0, val = 1035}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}} __func__ = "main" (gdb) info locals No locals. (gdb) list 231 in ../../core/parser/../mem/../atomic/atomic_x86.h ``` #### Log Messages ``` Jul 2 15:24:44 hostname kernel: kamailio[1098]: segfault at 7f5700000000 ip 00007f57d94467a0 sp 00007ffcf6347b00 error 6 in dmq.so[7f57d9426000+23000] ``` ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.7.4 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: compiled with gcc 10.2.1 ``` * **Operating System**: ``` Debian: 11.10 - Bullseye Linux 5.10.0-30-amd64 #1 SMP Debian 5.10.218-1 (2024-06-01) x86_64 GNU/Linux ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3905 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3905(a)github.com>

2 days, 6 hours

[kamailio/kamailio] Base58 not decoding correctly (Issue #3907)

by duarterocha91

### Description When doing base58 encoding and decoding there is some issue with the decoded string. Here is my code to ilustrate the issue : $var(Test) = "test_string"; $var(Test) = $(var(Test){s.encode.base58}); xerr("Test String encoded = STRING_BEGIN$var(Test)STRING_END"); $var(Test) = $(var(Test){s.decode.base58}); xerr("Test String decoded = STRING_BEGIN$var(Test)STRING_END"); It has the following output : ![image](https://github.com/kamailio/kamailio/assets/28625449/f9b10fd3-6e75-4368-9abd-005d44cc1235) I've inserted STRING_BEGIN and STRING_END in the log to be able to know when string ends. After this i've tried to user {s.trim} method with no success. I've noticed that despite not printing correctly, the string's length is calculated correctly, so i did a manual substring and was able to fix it. xerr("Test Base58"); $var(Test) = "test_string"; $var(Test) = $(var(Test){s.encode.base58}); xerr("Test String encoded = STRING_BEGIN$var(Test)STRING_END"); $var(Test) = $(var(Test){s.decode.base58}); xerr("Test String decoded = STRING_BEGIN$var(Test)STRING_END"); $var(Test) = $(var(Test){s.trim}); xerr("Test String decoded and trimmed = STRING_BEGIN$var(Test)STRING_END"); $var(substring_length) = $(var(Test){s.len}) - 1; $var(Test) = $(var(Test){s.substr,0,$var(substring_length)}); xerr("Test String decoded and fixed = STRING_BEGIN$var(Test)STRING_END"); Output : ![image](https://github.com/kamailio/kamailio/assets/28625449/9a100c45-4671-4381-9ddf-3a1c8c9404ee) All of this doesn't happen with method for base64 for example ### Additional Information version: kamailio 5.7.1 (x86_64/linux) 4238e4 flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 4238e4 compiled on 12:39:30 Jun 29 2023 with gcc 4.8.5 * **Operating System**: CentOS 7 -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3907 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3907(a)github.com>

2 days, 6 hours

[kamailio/kamailio] performance regression related to new TCP connection tracking feature with a high number of connections (Issue #3759)

by Henning Westerholt

### Description On systems with a high number of TCP sessions there can be a significant performance regression observed, probably related to the newly added TCP connection tracking feature. ### Troubleshooting #### Reproduction No special configuration is necessary, just install the latest 5.7.x release, e.g. 5.7.3 on a production system with a lot of clients connected over TCP or TLS. You need to have a large number of clients connected to be able to observe the regression. For a high number of connections (e.g. more then 20.000 up to 30.000 connections) the Kamailio servers uses about 30% to 50% more CPU as with the old version. #### Debugging Data Two graphs were attached to this issue. The first shows the CPU load before (less load) and after the upgrade (increased load). The second is a flamegraph that shows that over 80% of the CPU time is spent in the newly added function tcp_connection_limit_srcip(). <img width="372" alt="cpu-load-before-after" src="https://github.com/kamailio/kamailio/assets/6481937/ec92c41b-25e2-4847-a4bc…"> ![flamegraph](https://github.com/kamailio/kamailio/assets/6481937/5c2e3982-2d5b-4acd-a774-bad8fa64bd0a) Most of the CPU time is spend in the TCP main process, as expected. #### Log Messages No special log messages could be observed. #### SIP Traffic ### Possible Solutions The TCP limit feature should probably be optimized to not cause such a large performance regression. It should be also possible to deactivate it completly and therefore getting a comparable performance as before the feature addition. ### Additional Information Kamailio 5.7.3 and probably also git master version. * **Operating System**: Debian 11, Debian 12 -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3759 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3759(a)github.com>

2 days, 7 hours

[kamailio/kamailio] WIP pkg: RPM packaging add kamailio-tls_wolfssl subpackage (PR #3592)

by space88man

2 days, 7 hours

[kamailio/kamailio] tcpops module event_route[tcp:closed] route block $conid is null (Issue #3767)

by liangbaolin

### Description During the process of using version 5.7.4 of Kamailio, I found that the $conid cannot be correctly obtained in the TCP: closed routing block of the TCP module event_route. I am certain that it is normal in version 5.4.8. ### Troubleshooting ``` pv [pv_core.c:2725]: pv_get_tcpconn_id() ``` #### Reproduction The test scenario initiated a register registration process for me, and after completing the disconnection of the TCP connection, triggered this event. #### Log Messages  ``` 2024-02-26T01:35:55.902002512Z 52(98) DEBUG: <core> [core/receive.c:635]: receive_msg(): cleaning up 2024-02-26T01:35:55.902034482Z 52(98) DEBUG: <core> [core/tcp_read.c:280]: tcp_read_data(): EOF on connection 0x7f99a1546a80 (state: 0, flags: c018) - FD 24, bytes 0, rd-flags 10000 ([12.0.1.6]:15821 -> [12.0.1.6]:5060)52(98) DEBUG: <core> [core/tcp_read.c:343]: tcp_read(): read 0 bytes: 2024-02-26T01:35:55.902051542Z 2024-02-26T01:35:55.902058282Z 52(98) DEBUG: <core> [core/tcp_read.c:1532]: tcp_read_req(): read= 0 bytes, parsed=0, state=0, error=1 2024-02-26T01:35:55.902063282Z 52(98) DEBUG: <core> [core/tcp_read.c:1534]: tcp_read_req(): last char=0x00, parsed msg= 2024-02-26T01:35:55.902069402Z 2024-02-26T01:35:55.902074032Z 52(98) DEBUG: <core> [core/tcp_read.c:1544]: tcp_read_req(): EOF 2024-02-26T01:35:55.902101372Z 52(98) DEBUG: <core> [core/io_wait.h:598]: io_watch_del(): DBG: io_watch_del (0x559e8a029740, 24, -1, 0x10) fd_no=2 called 2024-02-26T01:35:55.902153162Z 52(98) DEBUG: <core> [core/tcp_read.c:1927]: handle_io(): removing from list 0x7f99a1546a80 id 2 fd 24, state 2, flags c018, main fd 134, refcnt 2 ([12.0.1.6]:15821 -> [12.0.1.6]:5060) 2024-02-26T01:35:55.902168112Z 52(98) DEBUG: <core> [core/tcp_read.c:1702]: release_tcpconn(): releasing con 0x7f99a1546a80, state -1, fd=24, id=2 ([12.0.1.6]:15821 -> [12.0.1.6]:5060) 2024-02-26T01:35:55.902173372Z 52(98) DEBUG: <core> [core/tcp_read.c:1705]: release_tcpconn(): extra_data (nil) 2024-02-26T01:35:55.902208402Z 83(129) DEBUG: <core> [core/tcp_main.c:3747]: handle_tcp_child(): reader response= 7f99a1546a80, -1 from 1 2024-02-26T01:35:55.902224742Z 83(129) DEBUG: <core> [core/tcp_main.c:3904]: handle_tcp_child(): case CONN_EOF tcp_emit_closed_event before 5 2024-02-26T01:35:55.902229802Z 83(129) DEBUG: <core> [core/tcp_main.c:3671]: tcp_emit_closed_event(): TCP closed event creation triggered (reason: 0) 2024-02-26T01:35:55.902318572Z 83(129) DEBUG: tcpops [tcpops.c:311]: tcpops_handle_tcp_closed(): received TCP closed event 2024-02-26T01:35:55.902331352Z 83(129) DEBUG: tcpops [tcpops.c:261]: tcpops_tcp_closed_run_route(): event reason id: 0 rt: 1 2024-02-26T01:35:55.902335132Z 83(129) ERROR: tcpops [tcpops.c:278]: tcpops_tcp_closed_run_route(): run_top_route() before tev->id: 2 2024-02-26T01:35:55.902480972Z 83(129) DEBUG: <core> [core/tcp_main.c:1713]: _tcpconn_find(): 2 port 0 2024-02-26T01:35:55.902493872Z 83(129) DEBUG: pv [pv_core.c:2725]: pv_get_tcpconn_id(): no conid 2024-02-26T01:35:55.902498762Z 83(129) DEBUG: pv [pv_core.c:2726]: pv_get_tcpconn_id(): msg->rcv.proto_reserved1: 2 2024-02-26T01:35:55.902503592Z 83(129) DEBUG: <core> [core/tcp_main.c:1713]: _tcpconn_find(): 2 port 0 2024-02-26T01:35:55.902508262Z 83(129) DEBUG: <core> [core/pvapi.c:1418]: pv_printf_mode(): final buffer length 111 2024-02-26T01:35:55.902513422Z 83(129) ERROR: <script>: tcp connection closed conid: (<null>), tcp(conid): <null>, Ri: 12.0.1.185, Rp: 5060 , si: 12.0.1.6 , sp: 15821 2024-02-26T01:35:55.902607692Z 83(129) DEBUG: <core> [core/tcp_main.c:1713]: _tcpconn_find(): 2 port 0 2024-02-26T01:35:55.902625622Z 83(129) DEBUG: pv [pv_core.c:2725]: pv_get_tcpconn_id(): no conid 2024-02-26T01:35:55.902630672Z 83(129) DEBUG: pv [pv_core.c:2726]: pv_get_tcpconn_id(): msg->rcv.proto_reserved1: 2 2024-02-26T01:35:55.902635822Z 83(129) ERROR: tcpops [tcpops.c:280]: tcpops_tcp_closed_run_route(): run_top_route() after tev->id: 2 2024-02-26T01:35:58.777642759Z 41(87) DEBUG: ims_dialog [dlg_handlers.c:2026]: print_all_dlgs(): ********************41(87) DEBUG: ims_dialog [dlg_handlers.c:2027]: print_all_dlgs(): printing 65536 dialogs ``` #### SIP Traffic  ``` my test script block event_route[tcp:closed] { xlog("L_ERR", "$proto connection closed conid: ($conid), tcp(conid): $tcp(conid), Ri: $Ri, Rp: $Rp , si: $si , sp: $sp\n"); } file: src/modules/pv/pv_core.c int pv_get_tcpconn_id(struct sip_msg *msg, pv_param_t *param, pv_value_t *res) { ... if((con = tcpconn_get(msg->rcv.proto_reserved1, 0, 0, 0, 0)) == NULL) // return false { LM_DBG("no conid\n"); //additional custom log LM_DBG("msg->rcv.proto_reserved1: %d\n",msg->rcv.proto_reserved1); return pv_get_null(msg, param, res); } .... } file: src/core/tcp_main.c struct tcp_connection *tcpconn_lookup(int id, struct ip_addr *ip, int port, union sockaddr_union *local_addr, int try_local_port, ticks_t timeout) { ... if(unlikely(c == NULL)) { c = _tcpconn_find(id, ip, port, &local_ip, local_port); // retrun c == null } ... } struct tcp_connection *_tcpconn_find( int id, struct ip_addr *ip, int port, struct ip_addr *l_ip, int l_port) { ... if(likely(id)) { hash = tcp_id_hash(id); for(c = tcpconn_id_hash[hash]; c; c = c->id_next) { //tcpconn_id_hash find c is null #ifdef EXTRA_DEBUG LM_DBG("c=%p, c->id=%d, port=%d\n", c, c->id, c->rcv.src_port); print_ip("ip=", &c->rcv.src_ip, "\n"); #endif .... } inline static int tcpconn_try_unhash(struct tcp_connection *tcpconn) { ... _tcpconn_detach(tcpconn); //executed ... lock_release(&tcpconn->write_lock); ... } static inline void _tcpconn_detach(struct tcp_connection *c) { ... tcpconn_listrm(tcpconn_id_hash[c->id_hash], c, id_next, id_prev); //remove c ... } ``` ### Possible Solutions  ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.7.4 (x86_64/linux) 9920d9-dirty flags: , EXTRA_DEBUGUSE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 9920d9 -dirty compiled on 09:59:52 Feb 26 2024 with gcc 7.5.0 ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3767 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3767(a)github.com>

2 days, 7 hours

git:master:6f434afc: ndb_redis: larger size for buffers to create server specs on cluster redirect

by Daniel-Constantin Mierla

Module: kamailio Branch: master Commit: 6f434afc414b7f5b66374a124138990f7a3dd1d1 URL: https://github.com/kamailio/kamailio/commit/6f434afc414b7f5b66374a124138990… Author: Daniel-Constantin Mierla <miconda(a)gmail.com> Committer: Daniel-Constantin Mierla <miconda(a)gmail.com> Date: 2024-07-05T11:47:34+02:00 ndb_redis: larger size for buffers to create server specs on cluster redirect - GH #3894 --- Modified: src/modules/ndb_redis/redis_client.c --- Diff: https://github.com/kamailio/kamailio/commit/6f434afc414b7f5b66374a124138990… Patch: https://github.com/kamailio/kamailio/commit/6f434afc414b7f5b66374a124138990… --- diff --git a/src/modules/ndb_redis/redis_client.c b/src/modules/ndb_redis/redis_client.c index cba94a38c06..74df8efda0b 100644 --- a/src/modules/ndb_redis/redis_client.c +++ b/src/modules/ndb_redis/redis_client.c @@ -886,11 +886,11 @@ int redisc_exec_pipelined(redisc_server_t *rsrv) int check_cluster_reply(redisReply *reply, redisc_server_t **rsrv) { redisc_server_t *rsrv_new; - char buffername[100]; + char buffername[512]; unsigned int port; str addr = {0, 0}, tmpstr = {0, 0}, name = {0, 0}; int server_len = 0; - char spec_new[100]; + char spec_new[512]; if(redis_cluster_param) { LM_DBG("Redis replied: \"%.*s\"\n", (int)reply->len, reply->str);

2 days, 8 hours

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

sr-dev July 2024