### Description
I have created a broken config with construction ``` if (is_request()) { } ```
When I try to check config I get the core
``` 0(66437) CRITICAL: <core> [core/cfg.y:3665]: yyerror_at(): parse error in config file proxy-media.cfg, line 18, column 5: syntax error 0(66437) CRITICAL: <core> [core/cfg.y:3665]: yyerror_at(): parse error in config file proxy-media.cfg, line 18, column 5: bad command 0(66437) CRITICAL: <core> [core/cfg.y:3662]: yyerror_at(): parse error in config file proxy-media.cfg, line 20, column 5-6: ERROR: bad config file (3 errors) (parsing code: 1) 0(66437) ERROR: <core> [core/ppcfg.c:243]: pp_ifdef_level_error(): different number of preprocessor directives: 1 more #!if[n]def as #!endif 0(66437) INFO: <core> [core/sctp_core.c:53]: sctp_core_destroy(): SCTP API not initialized
Program received signal SIGSEGV, Segmentation fault. 0x00007fffee8589ca in BN_clear_free () from /lib64/libcrypto.so.1.1 Missing separate debuginfos, use: yum debuginfo-install brotli-1.0.6-2.el8.x86_64 cyrus-sasl-lib-2.1.27-5.el8.x86_64 hiredis-0.13.3-13.el8.x86_64 jansson-2.11-3.el8.x86_64 keyutils-libs-1.5.10-6.el8.x86_64 krb5-libs-1.18.2-5.el8.x86_64 libcom_err-1.45.6-1.el8.x86_64 libcurl-7.61.1-14.el8.x86_64 libidn2-2.2.0-1.el8.x86_64 libnghttp2-1.33.0-3.el8_2.1.x86_64 libpsl-0.20.2-6.el8.x86_64 libselinux-2.9-4.el8_3.x86_64 libssh-0.9.4-2.el8.x86_64 libunistring-0.9.9-3.el8.x86_64 libxcrypt-4.1.1-4.el8.x86_64 libxml2-2.9.7-8.el8.x86_64 openldap-2.4.46-15.el8.x86_64 openssl-libs-1.1.1g-11.el8.x86_64 pcre2-10.32-2.el8.x86_64 xz-libs-5.2.4-3.el8.x86_64 zlib-1.2.11-16.el8_2.x86_64 (gdb) bt full #0 0x00007fffee8589ca in BN_clear_free () from /lib64/libcrypto.so.1.1 No symbol table info available. #1 0x00007fffecec5825 in ssh_dh_finalize.part () from /lib64/libssh.so.4 No symbol table info available. #2 0x00007fffeceb6e4d in libssh_destructor () from /lib64/libssh.so.4 No symbol table info available. #3 0x00007ffff7de4386 in _dl_fini () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #4 0x00007ffff708df8c in __run_exit_handlers () from /lib64/libc.so.6 No symbol table info available. #5 0x00007ffff708e0c0 in exit () from /lib64/libc.so.6 No symbol table info available. #6 0x00007ffff70777ba in __libc_start_main () from /lib64/libc.so.6 No symbol table info available. #7 0x000000000041c39e in _start () No symbol table info available. (gdb) ``` On host installed ``` [root@bcf-a0 config]# rpm -qf /lib64/libcrypto.so.1.1 openssl-libs-1.1.1g-11.el8.x86_64 ``` ### Additional Information
* **Kamailio Version** - output of `kamailio -v`
``` [root@bcf-a0 config]# kamailio -v version: kamailio 5.5.0-dev3 (x86_64/linux) aff06f flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: aff06f compiled on 04:01:30 Jan 7 2021 with gcc 8.3.1 ```
* **Operating System**:
``` [root@host ~]# cat /etc/os-release NAME="CentOS Linux" VERSION="8" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="8" PLATFORM_ID="platform:el8" PRETTY_NAME="CentOS Linux 8" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:8" HOME_URL="https://centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-8" CENTOS_MANTISBT_PROJECT_VERSION="8" ```
You have to install debugging symbols to have any useful lead. On the other hand, problems are startup can appear any time, trying to catch everything wrong there could bring a lot of complexity.
For an interpreter that focuses to be fast at runtime and offer a quick restart, trying to cover all the errors at startup that can crash (instead of a smoother shut down) may bring costs in performance that can impact the restart with good configs. So these two benefits have to be balanced:
* smooth shutdown on starting with broken config * quick start with valid config
We still have to see what is the cause of the crash here, just saying it needs proper consideration of benefits vs drawbacks.
@miconda debug files installed, but a stack does not contain any Kamailio libs. I can try to install debug symbols for `openssl-libs` and `gcc`.
If this will help.
With the trace pasted here, there is nothing we can relate to kamailio and have a point of start for investigation, is completely useless.
@sergey-safarov - try to minimize the cfg that crashes by removing blocks and trying it after each change. If you have a sufficient minimal cfg you can post it here for easier reproducing.
@henningw - see the description of the report (first comment), there is a config snippet if you want to reproduce yourself.
Thanks @miconda - i missed that as I thought its too small.
I found for reproduction important load of `tls` and `http_client` modules
Full kamailio config ``` listen=tls:eth0:5061 enable_tls=1
####### tls module ########## loadmodule "tls.so" loadmodule "http_client.so"
request_route { if (is_request()) { }
exit; } ```
My `tls.cfg` looks like ``` [server:default] method = SSLv23 verify_certificate = yes require_certificate = no private_key = /etc/kamailio/tls/server.key certificate = /etc/kamailio/tls/server.pem
[server:any] method = SSLv23 verify_certificate = yes require_certificate = no private_key = /etc/kamailio/tls/server.key certificate = /etc/kamailio/tls/server.pem server_name = safarov-dell.home server_name_mode = 1 ```
I found the issue. The build CenOS version is older than the target CentOS version. I go to refresh the build CentOS and RHEL versions.
@sergey-safarov - if you solved the issue with the older build than target centos, then close it.
If not, try with latest master branch from today and see if it is fixed.
yes, not solved. When I make a compilation from sources than not crash. When I install from RPMs files compiled, then I have a core.
Now I want to compare each RPM file version on build and target host. Think this give me hooks to find the important difference.
I forgot to mention that you have to start kamailio with `--atexit=no`. If you start it from systemd, be sure you change the kamailio.service file.
I have tested commit d8643c on CentOS 8 with the latest updates. Used config ``` [root@bcf-a0 tmp]# cat /root/kam.cfg listen=tls:eth0:25061 enable_tls=1
####### tls module ########## loadmodule "tls.so" loadmodule "http_client.so"
request_route { if (is_request()) { }
exit; } ```
yes `--atexit=no` now resolve the issue.
Should I add `--atexit=no` to systemd kamailio unit file?
It has to be added, indeed, but only to the unit files of newer Linux distros.
Added commit 9a35a5b95e8144dbdb18003f40e49e5b497274f3. Added `--atexit=no` option for `Fedora`, `OpenSUSE`, `CentOS 8` and other `RHEL 8` based dists.
Should I backport this commit into `5.4`, `5.3` and `5.2` branches?
Only 5.4 has these fixes now. 5.2 is out of maintenance.
Not sure if backporting to 5.3 is straightforward, given that this issue is specific to newer linux distributions, so we can even skip it to backport our older branches. However, if done, backporting the change in systemd service file must be done after the backports in the c code.
Closing this one here, decisions to backport to be done when planning next 5.3.x release.
Closed #2599.
-
Daniel-Constantin Mierla -
Henning Westerholt
-
sergey-safarov