sr-dev May 2025

sr-dev@lists.kamailio.org

13 participants
125 discussions

[kamailio/kamailio] ims_diameter_server not freeing shared memory (Issue #4116)
by MGrimby 07 May '25

07 May '25

### Description I have run in to an issue when sending diameter requests using the ims_diameter_server module. Shared memory keeps increasing and are not being freed until restart of kamailio. This leads to all shared memory is being used and no more are able to be allocated. This only happens when the request is sent synchronously. When sent asynchronously shared memory is being freed as far as I can see. Following functions have been used for sending requests. diameter_request([peer], appid, commandcode, message) diameter_request_async([peer], appid, commandcode, message) ### Troubleshooting Sent a bunch of diameter requests and verified that shared memory for the following functions are not freed. do_receive(600): AAATranslateMessage(500): AAACreateAVP(137): #### Reproduction It's possible to reproduce by sending sync diameter request and verify shared memory. Example of diameter request block ``` route[CHECK_LOCATION] { diameter_request("hss.ims.test.local", "16777217", "306", "[ " + "{ \"avpCode\":260, \"vendorId\":0, \"Flags\":64, \"list\":[ { \"avpCode\":266, \"vendorId\":0, \"Flags\":64, \"int32\":10415 }, { \"avpCode\":258, \"vendorId\":0, \"Flags\":64, \"int32\":16777217 } ] }, " + "{ \"avpCode\":277, \"vendorId\":0, \"Flags\":64, \"int32\":1 }, " + "{ \"avpCode\":283, \"vendorId\":0, \"Flags\":64, \"string\":\"ims.test.local\"}, " + "{ \"avpCode\":706, \"vendorId\":10415, \"Flags\":192, \"int32\":0 }, " + "{ \"avpCode\":700, \"vendorId\":10415, \"Flags\":192, \"list\": [ { \"avpCode\":601, \"vendorId\":10415, \"Flags\":192, \"string\":\"tel:" + $fU + "\" }]}, " + "{ \"avpCode\":707, \"vendorId\":10415, \"Flags\":192, \"int32\":0 }, " + "{ \"avpCode\":703, \"vendorId\":10415, \"Flags\":192, \"int32\":14 } ]"); xlog("L_INFO", "Sent Diameter request"); } ``` Stats cdp:replies_received = 343310 shmem:fragments = 142 shmem:free_size = 1221489992 shmem:max_used_size = 948242168 shmem:real_used_size = 925993656 shmem:total_size = 2147483648 shmem:used_size = 558205656 kamcmd mod.stats cdp shm do_receive(600): 256245808 AAATranslateMessage(500): 64420712 AAACreateAVP(137): 218100320 Total: 538774000 #### Debugging Data  ``` (paste your debugging data here) ``` #### Log Messages  ``` (paste your log messages here) ``` #### SIP Traffic  ``` (paste your sip traffic here) ``` ### Possible Solutions  ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.7.4 (x86_64/linux) dc393e flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: dc393e compiled on 13:56:27 Feb 27 2024 with gcc 8.5.0 ``` * **Operating System**:  ``` Linux bmatkamas1 4.18.0-553.16.1.el8_10.x86_64 #1 SMP Thu Aug 1 04:16:12 EDT 2024 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux release 8.10 (Ootpa) ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4116 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4116(a)github.com>

4 5

git:master:f2a60cfa: pkg/kamailio/deb: version set 6.1.0~dev1 [skip ci]
by Victor Seva 06 May '25

06 May '25

Module: kamailio Branch: master Commit: f2a60cfac4cf3a5a6d84e24f7845ab4faeacf45e URL: https://github.com/kamailio/kamailio/commit/f2a60cfac4cf3a5a6d84e24f7845ab4… Author: Victor Seva <linuxmaniac(a)torreviejawireless.org> Committer: Victor Seva <linuxmaniac(a)torreviejawireless.org> Date: 2025-05-06T15:39:08+02:00 pkg/kamailio/deb: version set 6.1.0~dev1 [skip ci] --- Modified: pkg/kamailio/deb/bionic/changelog Modified: pkg/kamailio/deb/bookworm/changelog Modified: pkg/kamailio/deb/bullseye/changelog Modified: pkg/kamailio/deb/buster/changelog Modified: pkg/kamailio/deb/debian/changelog Modified: pkg/kamailio/deb/focal/changelog Modified: pkg/kamailio/deb/jammy/changelog Modified: pkg/kamailio/deb/jessie/changelog Modified: pkg/kamailio/deb/noble/changelog Modified: pkg/kamailio/deb/precise/changelog Modified: pkg/kamailio/deb/sid/changelog Modified: pkg/kamailio/deb/stretch/changelog Modified: pkg/kamailio/deb/trusty/changelog Modified: pkg/kamailio/deb/wheezy/changelog Modified: pkg/kamailio/deb/xenial/changelog --- Diff: https://github.com/kamailio/kamailio/commit/f2a60cfac4cf3a5a6d84e24f7845ab4… Patch: https://github.com/kamailio/kamailio/commit/f2a60cfac4cf3a5a6d84e24f7845ab4…

1 0

git:master:4405af93: modules: readme files regenerated - app_jsdt ... [skip ci]
by Kamailio Dev 06 May '25

06 May '25

Module: kamailio Branch: master Commit: 4405af938ec72c0482e85abb3abea7853c5a22bb URL: https://github.com/kamailio/kamailio/commit/4405af938ec72c0482e85abb3abea78… Author: Kamailio Dev <kamailio.dev(a)kamailio.org> Committer: Kamailio Dev <kamailio.dev(a)kamailio.org> Date: 2025-05-06T14:46:10+02:00 modules: readme files regenerated - app_jsdt ... [skip ci] --- Modified: src/modules/app_jsdt/README --- Diff: https://github.com/kamailio/kamailio/commit/4405af938ec72c0482e85abb3abea78… Patch: https://github.com/kamailio/kamailio/commit/4405af938ec72c0482e85abb3abea78…

1 0

git:master:2e20c74f: app_jsdt: doc - fix section tags for loaddir
by Daniel-Constantin Mierla 06 May '25

06 May '25

Module: kamailio Branch: master Commit: 2e20c74fb03f2442cde5f3f69989a0ed914f542c URL: https://github.com/kamailio/kamailio/commit/2e20c74fb03f2442cde5f3f69989a0e… Author: Daniel-Constantin Mierla <miconda(a)gmail.com> Committer: Daniel-Constantin Mierla <miconda(a)gmail.com> Date: 2025-05-06T14:38:02+02:00 app_jsdt: doc - fix section tags for loaddir --- Modified: src/modules/app_jsdt/doc/app_jsdt_admin.xml --- Diff: https://github.com/kamailio/kamailio/commit/2e20c74fb03f2442cde5f3f69989a0e… Patch: https://github.com/kamailio/kamailio/commit/2e20c74fb03f2442cde5f3f69989a0e… --- diff --git a/src/modules/app_jsdt/doc/app_jsdt_admin.xml b/src/modules/app_jsdt/doc/app_jsdt_admin.xml index 6d139c29428..bbf2148b9b4 100644 --- a/src/modules/app_jsdt/doc/app_jsdt_admin.xml +++ b/src/modules/app_jsdt/doc/app_jsdt_admin.xml @@ -107,35 +107,31 @@ modparam("app_jsdt", "load", "/usr/local/etc/kamailio/js/myscript.js") </programlisting> </example> </section> - <section> - <title>Parameters</title> - <section id="app_jsdt.p.loaddir"> - <title> - <varname>loaddir</varname> (str) - </title> - <para> - Set the path to the folder containing JavaScript files to be loaded at startup. All - .js files in the folder will be loaded and combined into a single javascript script. - Then you can use jsdt_run(function, params) to execute a function from the - script at runtime. If you use it for KEMI configuration, - then it has to include the required functions. - </para> - <para> - <emphasis> - Default value is <quote>null</quote>. - </emphasis> - </para> - <example> - <title> - Set <varname>loaddir</varname> parameter - </title> - <programlisting format="linespecific"> - ... - modparam("app_jsdt", "loaddir", "/usr/local/etc/kamailio/js") - ... - </programlisting> - </example> - </section> + <section id="app_jsdt.p.loaddir"> + <title><varname>loaddir</varname> (str)</title> + <para> + Set the path to the folder containing JavaScript files to be loaded at startup. All + .js files in the folder will be loaded and combined into a single javascript script. + Then you can use jsdt_run(function, params) to execute a function from the + script at runtime. If you use it for KEMI configuration, + then it has to include the required functions. + </para> + <para> + <emphasis> + Default value is <quote>null</quote>. + </emphasis> + </para> + <example> + <title> + Set <varname>loaddir</varname> parameter + </title> + <programlisting format="linespecific"> +... +modparam("app_jsdt", "loaddir", "/usr/local/etc/kamailio/js") +... +</programlisting> + </example> + </section> <section id="app_jsdt.p.mode"> <title><varname>mode</varname> (int)</title> <para> @@ -328,4 +324,3 @@ request_route { </programlisting> </section> </chapter> -

1 0

git:master:7fa8b166: Makefile.defs/CMakeLists.txt: version set to 6.1.0-dev1
by Daniel-Constantin Mierla 06 May '25

06 May '25

Module: kamailio Branch: master Commit: 7fa8b166bfc813270d19a30351326a94a9187852 URL: https://github.com/kamailio/kamailio/commit/7fa8b166bfc813270d19a30351326a9… Author: Daniel-Constantin Mierla <miconda(a)gmail.com> Committer: Daniel-Constantin Mierla <miconda(a)gmail.com> Date: 2025-05-06T14:34:06+02:00 Makefile.defs/CMakeLists.txt: version set to 6.1.0-dev1 --- Modified: CMakeLists.txt Modified: src/Makefile.defs --- Diff: https://github.com/kamailio/kamailio/commit/7fa8b166bfc813270d19a30351326a9… Patch: https://github.com/kamailio/kamailio/commit/7fa8b166bfc813270d19a30351326a9… --- diff --git a/CMakeLists.txt b/CMakeLists.txt index 36451562d6f..6d68b5da980 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -11,7 +11,7 @@ project( ) # Set the version number -set(EXTRAVERSION "-dev0") +set(EXTRAVERSION "-dev1") set(RELEASE "${PROJECT_VERSION}${EXTRAVERSION}") message(STATUS "PROJECT_VERSION: ${PROJECT_VERSION}") diff --git a/src/Makefile.defs b/src/Makefile.defs index 7656029123e..6ff7eeddb51 100644 --- a/src/Makefile.defs +++ b/src/Makefile.defs @@ -107,7 +107,7 @@ INSTALL_FLAVOUR=$(FLAVOUR) VERSION = 6 PATCHLEVEL = 1 SUBLEVEL = 0 -EXTRAVERSION = -dev0 +EXTRAVERSION = -dev1 # memory manager switcher # 0 - f_malloc (fast malloc)

1 0

[kamailio/kamailio] cdp: restructure locking order to prevent rare deadlock (PR #4191)
by Morten Tryfoss 06 May '25

06 May '25

#### Pre-Submission Checklist    - [x] Commit message has the format required by CONTRIBUTING guide - [x] Commits are split per component (core, individual modules, libs, utils, ...) - [x] Each component has a single commit (if not, squash them into one commit) - [x] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated) #### Type Of Change - [x] Small bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality) #### Checklist:  - [ ] PR should be backported to stable branches - [x] Tested changes locally - [ ] Related to issue #XXXX (replace XXXX with an open issue number) #### Description We've been running the ims_charging module in production for quite some time, without any issues. A couple of days ago the whole instance freezed, but I was able to get a dump before the stuck processes before restarting. The traffic volume has gradually increased over time, so it's likely related to that. I then did a summary of the different types of stuck processes. Which locks they're holding, and what they're waiting for: ``` holding lock: AAASessionsLock waiting for lock: lock_get(peer_list_lock); #1 0x00007fe70eba8cde in futex_get (lock=0x7fe6e4a5c680) at ../../core/futexlock.h:108 #2 0x00007fe70ebaaca2 in get_peer_by_fqdn (fqdn=0x7fe6e4a5ba30) at peermanager.c:259 #3 0x00007fe70ebb64b7 in get_first_connected_route (cdp_session=0x7fe6e5cc5350, r=0x7fe6e4a5ba30, app_id=4, vendor_id=10415) at routing.c:115 #4 0x00007fe70ebb9a37 in get_routing_peer (cdp_session=0x7fe6e5cc5350, m=0x7fe6e5447ab0) at routing.c:293 #5 0x00007fe70ebcaf8c in AAASendMessage (message=0x7fe6e5447ab0, callback_f=0x7fe707a9e323 <resume_on_initial_ccr>, callback_param=0x7fe6e5297110) at diameter_comm.c:139 #6 0x00007fe707a9d3e0 in Ro_Send_CCR (msg=0x7fe70f011100, dlg=0x7fe6e5c571f0, dir=0, reservation_units=30, incoming_trunk_id=0x7ffe50efb060, outgoing_trunk_id=0x7ffe50efb050, pani=0x7ffe50efaee0, action=0x7fe70efa4730, tindex=10484, tlabel=677172593) at ims_ro.c:1511 #7 0x00007fe707a8285d in ki_ro_ccr (msg=0x7fe70f011100, s_route_name=0x7ffe50efb080, s_direction=0x7ffe50efb070, reservation_units=30, s_incoming_trunk_id=0x7ffe50efb060, s_outgoing_trunk_id=0x7ffe50efb050) at ims_charging_mod.c:742 #8 0x00007fe707a7bf01 in w_ro_ccr (msg=0x7fe70f011100, c_route_name=0x7fe70ef8b8d0 "\220\311\371\016\347\177", c_direction=0x7fe70ef8b980 "p\240\371\016\347\177", reservation_units=30, c_incoming_trunk_id=0x7fe70ef8ba30 "p\241\371\016\347\177", c_outgoing_trunk_id=0x7fe70ef8bae0 "\360\241\371\016\347\177") at ims_charging_mod.c:507 #9 0x00000000004858d8 in do_action (h=0x7ffe50efb970, a=0x7fe70ef99e20, msg=0x7fe70f011100) at core/action.c:1144 #10 0x00000000004928d6 in run_actions (h=0x7ffe50efb970, a=0x7fe70ef99e20, msg=0x7fe70f011100) at core/action.c:1618 #11 0x0000000000492f52 in run_actions_safe (h=0x7ffe50eff1e0, a=0x7fe70ef99e20, msg=0x7fe70f011100) at core/action.c:1681 #12 0x0000000000450156 in rval_get_long (h=0x7ffe50eff1e0, msg=0x7fe70f011100, i=0x7ffe50efbec8, rv=0x7fe70ef9c1d8, cache=0x0) at core/rvalue.c:973 #13 0x0000000000454d24 in rval_expr_eval_long (h=0x7ffe50eff1e0, msg=0x7fe70f011100, res=0x7ffe50efbec8, rve=0x7fe70ef9c1d0) at core/rvalue.c:1854 #14 0x0000000000454d52 in rval_expr_eval_long (h=0x7ffe50eff1e0, msg=0x7fe70f011100, res=0x7ffe50efc448, rve=0x7fe70ef9b920) at core/rvalue.c:1864 #15 0x00000000004850ce in do_action (h=0x7ffe50eff1e0, a=0x7fe70ef9b070, msg=0x7fe70f011100) at core/action.c:1097 -- holding lock: AAASessionsLock waiting for lock: lock_get(peer_list_lock); #1 0x00007fe70eba8d4f in futex_get (lock=0x7fe6e4a5c680) at ../../core/futexlock.h:121 #2 0x00007fe70ebaaca2 in get_peer_by_fqdn (fqdn=0x7ffe50efab90) at peermanager.c:259 #3 0x00007fe70ebb8e89 in get_routing_peer (cdp_session=0x7fe6e5ab6910, m=0x7fe6e5435be0) at routing.c:252 #4 0x00007fe70ebcaf8c in AAASendMessage (message=0x7fe6e5435be0, callback_f=0x7fe707a95edc <resume_on_termination_ccr>, callback_param=0x0) at diameter_comm.c:139 #5 0x00007fe707a95b02 in send_ccr_stop_with_param (ro_session=0x7fe6e5ab65e0, code=0, reason=0x7ffe50efb060) at ims_ro.c:1181 #6 0x00007fe707a72ff7 in dlg_terminated (dlg=0x7fe6e623d7a0, type=64, termcode=0, reason=0x7fe707ab72b3 "normal call clearing", _params=0x7fe707f67280 <params>) at dialog.c:249 #7 0x00007fe707a6a729 in dlg_callback_received (dlg=0x7fe6e623d7a0, type=64, _params=0x7fe707f67280 <params>) at dialog.c:25 #8 0x00007fe707d341b9 in run_dlg_callbacks (type=64, dlg=0x7fe6e623d7a0, req=0x7fe70f011100, rpl=0x0, dir=1, dlg_data=0x0) at dlg_cb.c:271 #9 0x00007fe707cf4db4 in dlg_terminated (req=0x7fe70f011100, dlg=0x7fe6e623d7a0, dir=1) at dlg_handlers.c:413 #10 0x00007fe707cfddeb in dlg_onroute (req=0x7fe70f011100, route_params=0x7ffe50efb6d0, param=0x0) at dlg_handlers.c:1097 #11 0x00007fe70ad285f6 in run_rr_callbacks (req=0x7fe70f011100, rr_param=0x7ffe50efb7c0) at rr_cb.c:96 #12 0x00007fe70ad3ae92 in after_loose (_m=0x7fe70f011100, preloaded=0) at loose.c:1021 #13 0x00007fe70ad3b5ce in loose_route_mode (_m=0x7fe70f011100, _mode=0) at loose.c:1056 #14 0x00007fe70ad3f74f in w_loose_route (msg=0x7fe70f011100, p1=0x0, p2=0x0) at rr_mod.c:273 #15 0x00000000004855ff in do_action (h=0x7ffe50efc390, a=0x7fe70efe0d40, msg=0x7fe70f011100) at core/action.c:1121 -- holding lock: lock_get(peer_list_lock); waiting for lock: lock_get(p->lock); #1 0x00007fe70eba8d4f in futex_get (lock=0x7fe6e4a5cbd0) at ../../core/futexlock.h:121 #2 0x00007fe70ebab0ae in peer_timer (now=1742807320, ptr=0x0) at peermanager.c:286 #3 0x00007fe70ebd0f39 in timer_loop () at timer.c:116 #4 0x00007fe70ebd21b2 in timer_process (returns=0) at timer.c:216 #5 0x00007fe70eb8ccf8 in diameter_peer_start (blocking=0) at diameter_peer.c:350 #6 0x00007fe70eb7cbb2 in cdp_child_init (rank=0) at cdp_mod.c:272 -- holding lock: lock_get(p->lock) waiting for lock: AAASessionsLock #1 0x00007fe70ebf2597 in futex_get (lock=0x7fe6e4a5d490) at ../../core/futexlock.h:108 #2 0x00007fe70ebf26f1 in AAASessionsLock (hash=0) at session.c:79 #3 0x00007fe70ebf5e6e in cdp_get_session (id=...) at session.c:316 #4 0x00007fe70eba6892 in Snd_Message (p=0x7fe6e4a5c880, msg=0x7fe6e63998d0) at peerstatemachine.c:1237 #5 0x00007fe70eba003e in sm_process (p=0x7fe6e4a5c880, event=Send_Message, msg=0x7fe6e63998d0, peer_locked=0, sock=0) at peerstatemachine.c:429 #6 0x00007fe70ebcbdc6 in AAASendMessage (message=0x7fe6e63998d0, callback_f=0x7fe707a8f208 <resume_on_interim_ccr>, callback_param=0x7fe6e6190a90) at diameter_comm.c:166 #7 0x00007fe707a8edb3 in send_ccr_interim (ro_session=0x7fe6e5399160, used=60, reserve=30) at ims_ro.c:847 #8 0x00007fe707a68bd6 in ro_session_ontimeout (tl=0x7fe6e5399200) at ro_timer.c:513 #9 0x00007fe707a63078 in ro_timer_routine (ticks=114862426, attr=0x0) at ro_timer.c:279 #10 0x00000000004fd33e in compat_old_handler (ti=1837798827, tl=0x7fe6e4cf6260, data=0x7fe6e4cf6260) at core/timer.c:980 #11 0x00000000004fde7a in slow_timer_main () at core/timer.c:1103 #12 0x000000000042e4e7 in main_loop () at main.c:1911 #13 0x000000000043876c in main (argc=10, argv=0x7ffe50f001c8) at main.c:3236 ``` In `get_first_connected_route()` in `routing.c` of the cdp module there are two places `get_peer_by_fqdn()` are called. One of them has an unlock/relock of the session list before and after (and a comment about holding two locks at a time), while the other doesn't. I'm currently testing two version of this. The first by just doing the some relock for the other `get_peer_by_fqdn()`, but this PR got an approach for maybe fixing more latent issues. `sm_process()` also got some strange handling by `Rcv_Process()` after the peer lock is released, which seems to be about the same thing. My deadlock is for the `Snd_Message()` equivalent. I've now removed this queueing behaviour, and instead doing a re-lock of the peer to (hopefully) have the same locking order as the other operations. The problem with this thing is how rare it occurs. Just wanted to share my findings for others with more knowledge to the cdp module for comments, suggestions and hopefully some extra testing. You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/4191 -- Commit Summary -- * cdp: restructure locking order to prevent rare deadlock -- File Changes -- M src/modules/cdp/peerstatemachine.c (25) M src/modules/cdp/routing.c (6) -- Patch Links -- https://github.com/kamailio/kamailio/pull/4191.patch https://github.com/kamailio/kamailio/pull/4191.diff -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/4191 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/pull/4191(a)github.com>

4 13

[kamailio/kamailio] http_client leaks TLS shared memory (Issue #4231)
by Alexander Bakker 06 May '25

06 May '25

alexbakker created an issue (kamailio/kamailio#4231) ### Description Performing an HTTPS request using the ``http_client`` module results in a shared memory leak, seemingly related to TLS. ### Troubleshooting #### Reproduction Our use case for ``http_client`` is to update an htable with the latest banned IP's from APIBAN as described here: https://github.com/apiban/apiban?tab=readme-ov-file#integration-into-kamail…. Normally, we refresh the list of banned IP's once every 15 minutes, so the shared memory usage grows fairly slowly. The issue can be reproduced using the Kamailio configuration file below. To demonstrate the issue and make the shared memory usage grow faster, we set up a timer that calls ``http_client_query`` in a tight loop. To prevent unnecessary load on the APIBAN infrastructure, we request an example file from my personal server instead. ``` #!KAMAILIO debug=2 log_stderror=yes fork=yes enable_tls=1 loadmodule "tls.so" loadmodule "cfg_rpc.so" loadmodule "pv.so" loadmodule "xlog.so" loadmodule "ctl.so" loadmodule "rtimer.so" loadmodule "http_client.so" modparam("tls", "certificate", "") modparam("tls", "private_key", "") modparam("rtimer", "timer", "name=apiban;interval=100u;mode=1;") modparam("rtimer", "exec", "timer=apiban;route=APIBAN;") route[APIBAN] { xinfo("running apiban refresh\n"); http_client_query("https://alexbakker.me/u/7bvjn9jfas.txt", "$var(banned)"); } ``` Start Kamailio: ``` kamailio -f kamailio.cfg -DD ``` And then watch the shared memory usage grow rapidly: ``` watch -n1 -- kamcmd core.shmmem ``` #### Log Messages Eventually, Kamailio will run out of shared memory and print the following messages to the log: ``` 21(184184) INFO: <script>: running apiban refresh 21(184184) ERROR: http_client [functions.c:471]: curL_request_url(): failed to perform curl (56) (url: https://alexbakker.me/u/7bvjn9jfas.txt) 21(184184) INFO: <script>: running apiban refresh 21(184184) ERROR: http_client [functions.c:471]: curL_request_url(): failed to perform curl (56) (url: https://alexbakker.me/u/7bvjn9jfas.txt) 21(184184) INFO: <script>: running apiban refresh 21(184184) WARNING: http_client [functions.c:453]: curL_request_url(): TLS error in curl connection (url: https://alexbakker.me/u/7bvjn9jfas.txt) 21(184184) INFO: <script>: running apiban refresh 21(184184) WARNING: http_client [functions.c:453]: curL_request_url(): TLS error in curl connection (url: https://alexbakker.me/u/7bvjn9jfas.txt) 21(184184) INFO: <script>: running apiban refresh 21(184184) WARNING: http_client [functions.c:453]: curL_request_url(): TLS error in curl connection (url: https://alexbakker.me/u/7bvjn9jfas.txt) 21(184184) INFO: <script>: running apiban refresh 21(184184) WARNING: http_client [functions.c:453]: curL_request_url(): TLS error in curl connection (url: https://alexbakker.me/u/7bvjn9jfas.txt) 21(184184) INFO: <script>: running apiban refresh 21(184184) ERROR: <core> [core/mem/q_malloc.c:758]: qm_realloc(): qm_realloc(0x7585c73c4000, 1024) called from tls: tls_init.c: ser_realloc(372), module: tls; qm_malloc() failed! 21(184184) INFO: <script>: running apiban refresh 21(184184) INFO: <script>: running apiban refresh 21(184184) ERROR: <core> [core/mem/q_malloc.c:758]: qm_realloc(): qm_realloc(0x7585c73c4000, 1024) called from tls: tls_init.c: ser_realloc(372), module: tls; qm_malloc() failed! 21(184184) WARNING: http_client [functions.c:453]: curL_request_url(): TLS error in curl connection (url: https://alexbakker.me/u/7bvjn9jfas.txt) 21(184184) INFO: <script>: running apiban refresh 21(184184) INFO: <script>: running apiban refresh 21(184184) WARNING: http_client [functions.c:463]: curL_request_url(): TLS CA certificate read error (url: https://alexbakker.me/u/7bvjn9jfas.txt) ``` ``mem_dump_shm`` reports a large list of TLS-related allocations. Just sharing the last few lines here, as they're seemingly all from the same location. ``` 20(204914) ALERT: qm_status: qm_status(): 28500. N address=0x705ab3178710 frag=0x705ab31786d0 size=112 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28501. N address=0x705ab31787f0 frag=0x705ab31787b0 size=48 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28502. N address=0x705ab3178890 frag=0x705ab3178850 size=32 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28503. N address=0x705ab3178920 frag=0x705ab31788e0 size=32 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28504. N address=0x705ab31789b0 frag=0x705ab3178970 size=32 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28505. N address=0x705ab3178a40 frag=0x705ab3178a00 size=48 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28506. N address=0x705ab3178ae0 frag=0x705ab3178aa0 size=48 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28507. N address=0x705ab3178b80 frag=0x705ab3178b40 size=144 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28509. N address=0x705ab3178d00 frag=0x705ab3178cc0 size=32 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28510. N address=0x705ab3178d90 frag=0x705ab3178d50 size=32 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_realloc(372) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28512. N address=0x705ab3178ea0 frag=0x705ab3178e60 size=16 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28513. N address=0x705ab3178f20 frag=0x705ab3178ee0 size=32 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28515. N address=0x705ab317a3c0 frag=0x705ab317a380 size=112 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28516. N address=0x705ab317a4a0 frag=0x705ab317a460 size=512 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28517. N address=0x705ab317a710 frag=0x705ab317a6d0 size=1504 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed 20(204914) ALERT: qm_status: qm_status(): 28518. N address=0x705ab317ad60 frag=0x705ab317ad20 size=1040 used=1 20(204914) ALERT: qm_status: qm_status(): alloc'd from tls: tls_init.c: ser_malloc(364) 20(204914) ALERT: qm_status: qm_status(): start check=f0f0f0f0, end check= c0c0c0c0, abcdefed ``` ### Possible Solutions I'm not aware of a workaround for this issue. For our specific use case, I'll move the APIBAN polling out of Kamailio to a separate process for now. ### Additional Information I've verified that this issue is reproducible on ``5.5.7``, ``5.6.6``, ``5.7.6``, ``5.8.6`` and ``6.0.1``. * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 6.0.1 (x86_64/linux) fce50d flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: fce50d compiled on 06:42:29 May 6 2025 with gcc 12.2.0 ``` * **Operating System**: ``` $ lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 12 (bookworm) Release: 12 Codename: bookworm $ uname -a Linux ip-10-146-128-121 6.1.0-32-cloud-arm64 #1 SMP Debian 6.1.129-1 (2025-03-06) aarch64 GNU/Linux ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4231 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4231(a)github.com>

2 2

git:master:12bfe206: modules: readme files regenerated - app_jsdt ... [skip ci]
by Kamailio Dev 06 May '25

06 May '25

Module: kamailio Branch: master Commit: 12bfe206daa6b9c2e8aa73e0d0da416d6593fd4e URL: https://github.com/kamailio/kamailio/commit/12bfe206daa6b9c2e8aa73e0d0da416… Author: Kamailio Dev <kamailio.dev(a)kamailio.org> Committer: Kamailio Dev <kamailio.dev(a)kamailio.org> Date: 2025-05-06T13:46:10+02:00 modules: readme files regenerated - app_jsdt ... [skip ci] --- Modified: src/modules/app_jsdt/README Modified: src/modules/cdp/README Modified: src/modules/stirshaken/README --- Diff: https://github.com/kamailio/kamailio/commit/12bfe206daa6b9c2e8aa73e0d0da416… Patch: https://github.com/kamailio/kamailio/commit/12bfe206daa6b9c2e8aa73e0d0da416…

1 0

[kamailio/kamailio] stirshaken: removed repeated x509 certification path check (PR #4202)
by Felipe Cuadra Plaza 06 May '25

06 May '25

2 2

git:master:f007736b: stirshaken: removed repeated x509 certification path check
by Daniel-Constantin Mierla 06 May '25

06 May '25

Module: kamailio Branch: master Commit: f007736ba18f5cc2114ffdd1e6df2b9b03808fe7 URL: https://github.com/kamailio/kamailio/commit/f007736ba18f5cc2114ffdd1e6df2b9… Author: FelipeCuadra <f.cuadra(a)zaleos.net> Committer: Daniel-Constantin Mierla <miconda(a)gmail.com> Date: 2025-05-06T13:34:41+02:00 stirshaken: removed repeated x509 certification path check - removed a second check of the x509 certificate path from the module, since it is already done earlier in the library and updated documentation --- Modified: src/modules/stirshaken/doc/stirshaken_admin.xml Modified: src/modules/stirshaken/stirshaken_mod.c --- Diff: https://github.com/kamailio/kamailio/commit/f007736ba18f5cc2114ffdd1e6df2b9… Patch: https://github.com/kamailio/kamailio/commit/f007736ba18f5cc2114ffdd1e6df2b9… --- diff --git a/src/modules/stirshaken/doc/stirshaken_admin.xml b/src/modules/stirshaken/doc/stirshaken_admin.xml index ef07e6a7212..41f72e5c1b7 100644 --- a/src/modules/stirshaken/doc/stirshaken_admin.xml +++ b/src/modules/stirshaken/doc/stirshaken_admin.xml @@ -528,6 +528,10 @@ request_route { ... </programlisting> </example> + <para> + To ensure proper functionality, the Kamailio stirshaken module requires a minimum version of libstirshaken that includes the stir_shaken_verify_cert_path function for performing the x509 certificate path check. This functionality was added to libstirshaken around 2020 (<![CDATA[https://github.com/signalwire/libstirshaken/commit/58e740b897ae40e2bb02ada2231a051a7eb55137]]>). + If you're using an older version of libstirshaken that predates this commit, the stirshaken module may not function correctly. + </para> </section> </chapter> diff --git a/src/modules/stirshaken/stirshaken_mod.c b/src/modules/stirshaken/stirshaken_mod.c index 95bbdeb5736..5d0bc744885 100644 --- a/src/modules/stirshaken/stirshaken_mod.c +++ b/src/modules/stirshaken/stirshaken_mod.c @@ -613,23 +613,6 @@ static int ki_stirshaken_check_identity(sip_msg_t *msg) goto fail; } - if(stirshaken_vs_verify_x509_cert_path) { - - LM_DBG("Running X509 certificate path verification\n"); - - if(!vs) { - LM_ERR("Verification Service not started\n"); - goto fail; - } - - if(STIR_SHAKEN_STATUS_OK - != stir_shaken_verify_cert_path(&ss, cert_out, vs->store)) { - LM_ERR("Cert did not pass X509 path validation\n"); - stirshaken_print_error_details(&ss); - goto fail; - } - } - if(stirshaken_vs_pptg_pvname.s != 0) { memset(&val, 0, sizeof(pv_value_t)); val.flags = PV_VAL_STR;

1 0

← Newer
1
...
7
8
9
10
11
12
13
Older →

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

sr-dev May 2025