sr-users April 2023

sr-users@lists.kamailio.org

44 participants
46 discussions

SCSCF hangs randomly
by solomon.thoobe＠smilecoms.com 05 Apr '23

05 Apr '23

Hi All, We currently experience issues where scscf hangs randomly throughout the day. It started hanging once in a month, the issue has gotten worse over time and now it can hang more than 5 times in a day. I have no prior kamailio experience or experience debugging c applications. Kamailio version: 4.4.0 Running ps shows scscf processes are running. Service status also shows scscf is running but it does not respond to sip requests. Testing with sipsack hangs and print timeout errors. root@NGVI-PCSCF2:/var/log # service scscf status * Status of Kamailio SIP Server: * scscf is running sipsak -vv -s sip:NGVI-PCSCF2:6060 received ICMP message received ICMP message** timeout after 500 ms** Below is the extract from log files Apr 5 12:04:46 NGVI-PCSCF2 /usr/sbin/kamailio[5102]: {1 1 ACK isbc505ramjjjr3g23cs02r53cms0uxbg2es(a)10.18.5.64} WARNING: tm [t_lookup.c:1476]: t_unref(): WARNING: script writer didn't release transaction Apr 5 12:04:46 NGVI-PCSCF2 /usr/sbin/kamailio[5104]: {1 3 REGISTER _A9LGREyW2miE7LhPJzv7g..} ERROR: ims_registrar_scscf [save.c:1034]: update_contacts(): Unable to remove contact <sip:+2347020246874@154.66.2.182:5060;rinstance=28bf37d2cce6e4a1;transport=udp> Apr 5 12:04:47 NGVI-PCSCF2 /usr/sbin/kamailio[5131]: ERROR: rr [rr_mod.c:300]: w_record_route(): Double attempt to record-route Apr 5 12:04:47 NGVI-PCSCF2 /usr/sbin/kamailio[5144]: ERROR: rr [rr_mod.c:300]: w_record_route(): Double attempt to record-route Apr 5 12:04:47 NGVI-PCSCF2 /usr/sbin/kamailio[5108]: {1 1 ACK r8ug398d118i8iaaunrouggisntkigkn(a)10.18.5.64} WARNING: tm [t_lookup.c:1476]: t_unref(): WARNING: script writer didn't release transaction Apr 5 12:04:48 NGVI-PCSCF2 /usr/sbin/kamailio[5190]: CRITICAL: <core> [pass_fd.c:275]: receive_fd(): EOF on 23 Apr 5 12:04:48 NGVI-PCSCF2 /usr/sbin/kamailio[5062]: ALERT: <core> [main.c:741]: handle_sigs(): child process 5103 exited by a signal 11 Apr 5 12:04:48 NGVI-PCSCF2 /usr/sbin/kamailio[5062]: ALERT: <core> [main.c:744]: handle_sigs(): core was generated Apr 5 12:06:12 NGVI-PCSCF2 kamailio: WARNING: <core> [daemonize.c:348]: daemonize(): pid file contains old pid, replacing pid Apr 5 12:06:13 NGVI-PCSCF2 /usr/sbin/kamailio[15060]: WARNING: interconnectroute [ported_cache.c:113]: initPortedNumberCount(): Requested records[726049]: Total allocated[871258]: Fat 20% :Total MemUsed [26 MB divided between primary[0x7f22b8667d60] and secondary[0x7f22b93b3390]] pr 5 04:38:24 NGVI-PCSCF2 /usr/sbin/kamailio[14018]: message repeated 107 times: [ ERROR: ims_usrloc_scscf [subscribe.c:430]: free_subscriber(): record not found in hash table] Apr 5 12:50:38 NGVI-PCSCF2 /usr/sbin/kamailio[2604]: WARNING: interconnectroute [icncache_eventprocessor.c:76]: icnRefreshProcess(): total size 2147483648 qm_real_used 700707576 qm_used 387064680 1st_frag 0x7fd5f6d6a690 last_frag 0x7fd676d30fd0 Apr 5 13:00:25 NGVI-PCSCF2 /usr/sbin/kamailio[2437]: ERROR: ims_usrloc_scscf [subscribe.c:430]: free_subscriber(): record not found in hash table Apr 5 13:00:26 NGVI-PCSCF2 /usr/sbin/kamailio[2437]: message repeated 125 times: [ ERROR: ims_usrloc_scscf [subscribe.c:430]: free_subscriber(): record not found in hash table] Apr 5 13:01:25 NGVI-PCSCF2 /usr/sbin/kamailio[2437]: CRITICAL: <core> [main.c:650]: sig_alarm_abort(): shutdown timeout triggered, dying... Apr 5 13:02:30 NGVI-PCSCF2 kamailio: WARNING: <core> [daemonize.c:348]: daemonize(): pid file contains old pid, replacing pid Apr 5 13:02:30 NGVI-PCSCF2 /usr/sbin/kamailio[27629]: WARNING: interconnectroute [ported_cache.c:113]: initPortedNumberCount(): Requested records[726049]: Total allocated[871258]: Fat 20% :Total MemUsed [26 MB divided between primary[0x7f3093a3fd60] and secondary[0x7f309478b390]] Thank you in advance. Regards, Solly Thoobe

3 2

urn:emergency:service:sos as RURI
by a.izquierdo＠zaleos.net 05 Apr '23

05 Apr '23

Hello, I'm running a Kamailio instance with some routing logic in it. When a UA sends an INVITE with urn:emergency:service:sos as RURI, Kamailio responds with a 400 Bad Request URI. I have set debug log level and it looks like the problem is in the parse_uri.c when sanity_check("1511", "7") is called: parse_uri(): bad char ':' in state 3 parsed: <urn:emergency:service> (21) / <urn:emergency:service:sos> (25) According to RFC8141 (https://www.rfc-editor.org/rfc/rfc8141), this is a correct URN, but maybe I'm missing something. Is there a way to make Kamailio parse this RURI? Kamailio version used: 5.6.4 Thank you!

3 2

segfault during Identity verification (stirshaken)
by Yuriy Nasida 04 Apr '23

04 Apr '23

Hello, It's kamailio 5.6.4. I can say that there is no segfault in case Identity is OK but bad attempts sometimes are the reason for segfault. Not sure I can duplicate but it happens several times per day. Please look at logs. in messages: Apr 3 09:40:02 kam1 kernel: [6381565.762987] kamailio[9141]: segfault at 746c75616665 ip 00007f20c4cab6df sp 00007ffe7ec51470 error 4 in libcrypto.so.1.1[7f20c4c3b000+19e000] kamailio.log: Apr 3 09:40:02 kam1 kamailio[9141]: NOTICE: <script>: it is INVITE with Identity. Lets check Identity Apr 3 09:40:02 kam1 kamailio[9141]: CRITICAL: <core> [core/mem/q_malloc.c:501]: qm_free(): BUG: bad pointer 0x645f6d6574737973 (out of memory block!) called from tls: tls_init.c: ser_free(323) - ignoring Apr 3 09:40:02 kam1 kamailio[9182]: CRITICAL: <core> [core/pass_fd.c:277]: receive_fd(): EOF on 53 Apr 3 09:40:02 kam1 kamailio[9100]: ALERT: <core> [main.c:774]: handle_sigs(): child process 9141 exited by a signal 11 Apr 3 09:40:02 kam1 kamailio[9100]: ALERT: <core> [main.c:777]: handle_sigs(): core was not generated Apr 3 09:40:02 kam1 kamailio[9100]: CRITICAL: <core> [core/mem/q_malloc.c:501]: qm_free(): BUG: bad pointer 0x645f6d6574737973 (out of memory block!) called from tls: tls_init.c: ser_free(323) - ignoring loadmodule "stirshaken.so" ####### STIRSHAKEN ###### modparam("stirshaken", "vs_verify_x509_cert_path", 1) modparam("stirshaken", "vs_ca_dir", "/etc/kamailio/stir_CA") #modparam("stirshaken", "vs_crl_dir", "/etc/kamailio/CRL") modparam("stirshaken", "as_default_key", "/etc/kamailio/key/private_key.pem") modparam("stirshaken", "vs_cache_certificates", 1) modparam("stirshaken", "vs_cache_dir", "/tmp/cert_cache") modparam("stirshaken", "vs_cache_expire_s", 100) if (is_method("INVITE") && is_present_hf("Identity")) { xlog("L_NOTICE", "it is INVITE with Identity. Lets check Identity\n"); if (1 == stirshaken_check_identity()) { xlog("L_NOTICE", "Shaken Identity is OK\n"); } else { xlog("L_NOTICE", "Shaken Identity is invalid\n"); } } Please give me some advice.

2 2

Handling stateless replies with kemi
by Ovidiu Sas 03 Apr '23

03 Apr '23

Hello all, How can we handle stateless replies with kemi? tm has the following param: modparam("tm", "on_sl_reply", "stateless_replies") What's the proper way to map the "stateless_reply" onreply_route to a kemi function. I'm playing with the app_jsdt module. Thanks, Ovidiu

5 7

How to check, if call is already handled by rtpengine module?
by Benoît Panizzon 02 Apr '23

02 Apr '23

Hi Running Kamailio Registrar Node + rtpengine without dialog module, only TRX module. Situation: Some customer are in ranges to which RTP can directly be routed, some are not. So rtpengine shall be engaged only when backhauling of the rtp stream is required. On a message from a CPE, this is done by checking $si and on a message to the CPE by checking $(ulc(aor=>addr)[$T_branch_idx]{uri.host}); On a BYE, rtpengine_delete() is called no matter if RTP was backhauled or not. If rtpengine does not find the call to delete, no harm happens. But now I struggle with Session Refresh Re-Invites TO the CPE as they are routed very early on and do not pass the whole location lookup stuff, therefore I have no access to the aor=>addr to determine if the destination needs backhauling or not. So the obvious solution which came to my mind is to call rtpengine_query() and see if it is finding the call and if so, call rtpengine_manage() to keep backhauling traffic on the Re-Invites refrreshing the session. Unfortunately, I found no way to check if rtpengine_query finds the call. Did I miss something? -- Mit freundlichen Grüssen -Benoît Panizzon- @ HomeOffice und normal erreichbar -- I m p r o W a r e A G - Leiter Commerce Kunden ______________________________________________________ Zurlindenstrasse 29 Tel +41 61 826 93 00 CH-4133 Pratteln Fax +41 61 826 93 01 Schweiz Web http://www.imp.ch ______________________________________________________

2 1

Germany unsure about whether Kamailio will survive next winter
by Alex Balashov 01 Apr '23

01 Apr '23

For immediate release: ATLANTA, GA (1 April 2023)--The Federal Ministry for Economic Affairs and Energy of Germany was forced to disclose today that it has been tasked with assessing whether Germany will be able to operate its Kamailio through the next winter. This initiative comes amidst considerable uncertainty, shared in some other EU member states, about whether the enormous fossil fuel energy footprint of Kamailio is sustainable after the Russian invasion of Ukraine in early 2022. Pioneering Atlanta energy market analysts Evariste Systems were tapped to assist with forecasting whether German strategic natural gas reserves and liquefied natural gas (LNG) imports could keep Kamailio's enormous turbines spinning under a variety of stress-test scenarios outlined by regulators, including SIPP and SIPVicious. Alex Balashov, Evariste's principal, said in a news conference earlier this week: "Simply put, the political leadership wanted a dispassionate, technocratic kind of analysis, data-driven and the rest, free of the inflection of ideological bias and tendentious policy angles in charged German domestic politics. Whether it's the centre-right or the Greens, there are strong ideas about what to do with energy-greedy megaprojects like Kamailio." Florida-based energy turnaround vet Fred Posner, parachuted in to assist with the fast-tracked study, agreed: "It's well-understood that the German GDP is almost a perfectly linear function of gross annual SIP packets routed. The Nord Stream 1 shut-off situation in September only exposed Germany to further Kamailio-related energy vulnerabilities." At issue are the basic physics of SIP message routing in Kamailio. Due to a fixed-size worker process pool, Kamailio routing consumes about 1500 BTUs per packet-kilometre travelled. While 1500 BTUs/packet-kilometre is efficient in distance terms as compared to a typical automobile, which consumes roughly 3800 BTUs of fossil fuel energy per passenger-kilometre travelled (at typical load factors), the dizzying number of SIP packets routed through German territory in a typical business day greatly outstrips passenger-kilometres travelled. Balashov noted that sometimes, the activation energy requirement can be higher than 1500 BTUs when INVITEs with large SDPs are launched toward the next hop, though this varies with the altitude of the destination above mean sea level, wind direction, weather and other factors involved in ballistics. "It's a bit of a fool's errand to play these guessing games with averages. The energy budget can vary enormously depending on whether there is DTLS, video codecs, PASSporTs and other stuff. I find it helpful to think in kilowatt-hour terms: sometimes it's less than half a kWh, so like US $0.20 if you're counting the beans, and sometimes it's more like $0.38, we just don't know. The Bundestag always wants these big, round numbers, but if you've ever looked at the TM module, you know that's not how this works." A milder-than-expected winter, a 9% year-over-year increase in US LNG exports (to about 300 million cubic metres per day) in 2022, dependable imports from Norway and the Netherlands, and other favourable factors gave German SIP regulators a reprieve. Despite energy market volatility, premium LNG spot prices, and occasional 408 Request Timed Out scenarios, the country dodged a widely-feared Kamailio energy crisis. However, it is difficult to say whether energy market conditions will be as propitious next year. "Then there's the whole climate change goals thing. Nobody even wants to talk about that, especially for the WebRTC gateway side," says Posner. "There's a persistent fantasy that we can just power Kamailio with solar or wind if we just had enough generating capacity. I think that's really missing the forest for the trees. The worker processes stay running whether you need them or not, it all depends on how many listeners you have set and what the children config value is. Either way, I've got two words for you: base load. Even if we completely ignore that the energy density is just not there per hectare of solar or wind installation, how do you provide the constant power to the child processes?" Past feasibility studies published jointly by Balashov and Posner support this assessment. One such study, initiated in 2018 and concluded during the COVID-19 global pandemic, found that an area equivalent to the size of the entire state of North Rhine-Westphalia would need to be devoted to batteries, even using the latest lithium ferrous phosphate (LFP) technology. "But wait, there's more!" says Posner: "Show me the amperage." "The packet forwarding mechanism is similar to a railgun, at least if you're doing stateful. Where are you going to get the millions of amps? It's either setting up huge banks of capacitors god-knows-where, or the pulsed power system that's undergirded by the original OpenSER turbines. I think we already know the answer to that." Balashov noted that, even beyond the political and economic challenges that Kamailio downtime would introduce, there are other tightropes to walk. As the most economically dynamic, export-oriented EU member state, Germany is prone to flirt with SIP proxy privatisation schemes in its national discourse, all while messaging a stronger Euro to the ECB. "There is a vocal minority," says Balashov, "who push for research into more modular, regionally sited SIP gateways. They're always talking about sclerotic, unresponsive federal regulations and listless, bumbling Brussels bureaucrats and making it all smaller and leaner. Listen, I'm all for the Invisible Hand, but setting the Request URI and adding custom headers requires a truly integrated, national-scale infrastructure. It's the stuff of public-private partnerships and megaprojects. Even if you're a market zealot who is not sold on basic INVITEs as a public good, think about Presence or IMS. You really need the full capacity of a nation-state on deck for that." If devolution to smaller, more numerous SIP routing sites is not feasible--at least, at the unit cost and reliability level demanded by advanced economies--then excessive complexity offers a cautionary tale from the opposite extreme. In a particularly notorious example of technical and policy failure, the Electric Reliability Council of Texas (ERCOT), following the guidance of northern Californian advisors Unicorn & Moonshot, attempted to scale down traditional Kamailio turbine blades into newer, so-called Kubernetes "Pod" chasses to increase exhaust recapture. This made for a much more intricate installation with an exponential growth of moving parts, telemetry and site support skill requirements. This proved unmanageable, and the now-famous explosion that followed the deterioration of the site led to unprecedented casualties in US history. Furthermore, it also resulted in the long-term humidification of large expanses of East Texas, Louisiana, Mississippi and southern Alabama, rendering most of these areas uninhabitable. These events discredited the Unicorn & Moonshot approach in the eyes of most industry analysts. While it will take some months for Evariste to reach a conclusion regarding the resilience of the German Kamailio through the winter and beyond, and it will take still longer to issue policy recommendations, one factor is universally agreed upon already: this prolonged test of Germany's resilience is part of a Russian strategic calculation. A conservative MP from the CDU/CSU faction of the Bundestag, speaking on condition of anonomity, offered this summary: "The Russians think they can wait this out. They are waiting for us to fold and switch to an OpenSIPS reactor, knowing full well this will keep us busy with troubleshooting and diminish our ability to support Ukraine militarily." -- Alex Balashov Principal Consultant Evariste Systems LLC Web: https://evaristesys.com Tel: +1-706-510-6800

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

sr-users April 2023