<!-- Kamailio Project uses GitHub Issues only for bugs in the code or feature requests. Please use this template only for bug reports.
If you have questions about using Kamailio or related to its configuration file, ask on sr-users mailing list:
* http://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
If you have questions about developing extensions to Kamailio or its existing C code, ask on sr-dev mailing list:
* http://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Please try to fill this template as much as possible for any issue. It helps the developers to troubleshoot the issue.
If there is no content to be filled in a section, the entire section can be removed.
You can delete the comments from the template sections when filling.
You can delete next line and everything above before submitting (it is a comment). -->
### Description
<!-- Explain what you did, what you expected to happen, and what actually happened. --> Ubuntu Bionic 18.04.02 LTS ships with "OpenSSL 1.1.1 11 Sep 2018" (0x1010100f), whereas the `kamailio-tls-modules` package is compiled with "OpenSSL 1.1.0g 2 Nov 2017" (0x1010007f).
I installed Kamailio from the Kamailio repositories (not from the Ubuntu repositories).
This leads to Kamailio being unable to start as it complains about the OpenSSL versions being too different from each other.
Overriding the OpenSSl version check by enabling `tls_force_run` does not solve the issue, instead, it leads to Kamailio emitting multiple errors.
<!-- ### Troubleshooting -->
#### Reproduction
I installed Kamailio from the official Kamailio apt sources (nightly build, the same occurs for the latest stable version 5.2).
``` deb http://deb.kamailio.org/kamailiodev-nightly bionic main deb-src http://deb.kamailio.org/kamailiodev-nightly bionic main ```
I enabled TLS and edited the configuration files accordingly.
<!-- If the issue can be reproduced, describe how it can be done. -->
<!-- #### Debugging Data -->
<!-- If you got a core dump, use gdb to extract troubleshooting data - full backtrace, local variables and the list of the code at the issue location.
gdb /path/to/kamailio /path/to/corefile bt full info locals list
If you are familiar with gdb, feel free to attach more of what you consider to be relevant. -->
``` (paste your debugging data here) ```
#### Log Messages
``` CRITICAL: tls [tls_init.c:677]: init_tls_h(): installed openssl library version is too different from the library the kamailio tls module was compiled with: installed "OpenSSL 1.1.1 11 Sep 2018" (0x1010100f), compiled "OpenSSL 1.1.0g 2 Nov 2017" (0x1010007f).#012 Please make sure a compatible version is used (tls_force_run in kamailio.cfg will override this check) ```
<!-- #### SIP Traffic -->
### Possible Solutions
Recompile and publish the `kamailio-tls-modules` package compiled with OpenSSL 1.1.1b.
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
``` version: kamailio 5.3.0-dev6 (x86_64/linux) flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 7.3.0 ```
* **Operating System**:
<!-- Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...; Kernel details (output of `uname -a`) -->
``` Linux hostname 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Description: Ubuntu 18.04.2 LTS Release: 18.04 ```
Maybe @linuxmaniac can get a bit of time to check if OS used to build the packages is properly up to date in this case.
confirmed: ``` 03:28:37 Get: 1 http://archive.ubuntu.com/ubuntu bionic/main amd64 libssl1.1 amd64 1.1.0g-2ubuntu4 [1128 kB] ```
We don't use <dist>-updates nor <dist>-security repositories
Shouldn't that be advisable though?
So the only option we have is downgrading openssl to 1.1.0g?
So the only option we have is downgrading openssl to 1.1.0g?
Until We build the debs with those repositories added, yes. I hope it would be not too long to solve this.
I think we'll rather go with plain TCP then as other tools are relying on >=1.1.1 and we cannot downgrade them, too.
I hope it would be not too long to solve this.
That's important for us. That's likely gonna be a hard time with non-encrypted SIP. Looking forward to seeing that build in the repos soon.
As a follow-up, now, when using the nightly build, I'm getting a different error:
```log ERROR: tls [tls_util.h:42]: tls_err_ret(): TLS accept:error:1408F10B:SSL routines:ssl3_get_record:wrong version number Aug 20 22:46:01 ubuntu-server /usr/sbin/kamailio[31861]: ERROR: <core> [core/tcp_read.c:1505]: tcp_read_req(): ERROR: tcp_read_req: error reading - c: 0x7f16f4abed70 r: 0x7f16f4abedf0 (-1) Aug 20 22:46:02 ubuntu-server /usr/sbin/kamailio[31862]: ERROR: tls [tls_util.h:42]: tls_err_ret(): TLS accept:error:1408F10B:SSL routines:ssl3_get_record:wrong version number Aug 20 22:46:02 ubuntu-server /usr/sbin/kamailio[31862]: ERROR: <core> [core/tcp_read.c:1505]: tcp_read_req(): ERROR: tcp_read_req: error reading - c: 0x7f16f4abed70 r: 0x7f16f4abedf0 (-1) ```
After the changes, we are using the latest version ``` Get: 170 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.4 [1566 kB] ``` @welljsjs What is the version you have installed?
``` version: kamailio 5.3.0-dev7 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 7.4.0 ``` This is the extract from the logs: ``` Sep 04 14:39:41 ubuntu-server systemd[1]: Started Kamailio (OpenSER) - the Open Source SIP Server. Sep 04 20:17:41 ubuntu-server /usr/sbin/kamailio[2057]: ERROR: tls [tls_util.h:42]: tls_err_ret(): TLS accept:error:1408F10B:SSL routines:ssl3_get_record:wrong version number Sep 04 20:17:41 ubuntu-server /usr/sbin/kamailio[2057]: ERROR: <core> [core/tcp_read.c:1505]: tcp_read_req(): ERROR: tcp_read_req: error reading - c: 0x7fb548fd6d70 r: 0x7fb548fd6df0 (-1) ```
@welljsjs What is the version you have installed?
Sorry I meant what openssl version do you have in that system
Sorry, my mistake.
``` OpenSSL 1.1.1 11 Sep 2018 built on: Thu Jun 20 17:36:28 2019 UTC platform: debian-amd64 options: bn(64,64) rc4(16x,int) des(int) blowfish(ptr) compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-cn9tZy/openssl-1.1.1=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2 OPENSSLDIR: "/usr/lib/ssl" ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1" Seeding source: os-specific ```
Closed #2018.
The error looks now related to runtime operations, no longer related to the initial compilation with an outdated version.
@welljsjs - open a new issue and it would be good if you can attach the logs with debug=3 in kamailio.cfg. Also, try to use kamailio 5.3.0-pre1 that has new code for dealing with libssl 1.1+.
I'm seeing this exact behavior with kamailio/5.3.3 on Ubuntu/18.04.4. ``` $ cat /etc/apt/sources.list.d/kamailio.list deb http://deb.kamailio.org/kamailio53 bionic main deb-src http://deb.kamailio.org/kamailio53 bionic main $ $ dpkg -l | awk '/kam/ { print $2 " " $3 }' kamailio 5.3.3+bionic kamailio-extra-modules:amd64 5.3.3+bionic kamailio-mysql-modules:amd64 5.3.3+bionic kamailio-snmpstats-modules:amd64 5.3.3+bionic kamailio-tls-modules:amd64 5.3.3+bionic kamailio-websocket-modules:amd64 5.3.3+bionic $ $ /usr/sbin/kamailio -v version: kamailio 5.3.3 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 7.3.0 $ $ /usr/bin/openssl version OpenSSL 1.1.1 11 Sep 2018 ```
``` Apr 14 17:18:39 kam-01 /usr/sbin/kamailio[22073]: CRITICAL: tls [tls_init.c:677]: init_tls_h(): installed openssl library version is too different from the library the kamailio tls module was compiled with: installed "OpenSSL 1.1.1 11 Sep 2018" (0x1010100f), compiled "OpenSSL 1.1.0g 2 Nov 2017" (0x1010007f).#012 Please make sure a compatible version is used (tls_force_run in kamailio.cfg will override this check) Apr 14 17:18:39 kam-01 /usr/sbin/kamailio[22073]: CRITICAL: <core> [main.c:2768]: main(): could not initialize tls, exiting... ```
Should this be a new bug? Is the build host for these packages broken again?