Description

Kamailio crashes on shutdown in geoip2_destroy_pv in main process process if geoip2 database was reloaded.

Troubleshooting

libmaxminddb.so.0 uses calloc() internally to allocate memory for metadata languages and descriptions. On reload via e.g. ctl metadata memory is allocated in ctl process but main process tries to cleanup it.

Reproduction

Load geoip2 module

loadmodule "geoip2.so"
modparam("geoip2", "path", "/path/to/your/geolite2.mmdb")

Reload database and stop/restart Kamailio

kamcmd geoip2.reload
service kamailio restart

Debugging Data

(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1  0x00007f117bc59e8f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  0x00007f117bc0afb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007f117bbf5472 in __GI_abort () at ./stdlib/abort.c:79
#4  0x00007f117bc4e430 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7f117bd68459 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#5  0x00007f117bc637aa in malloc_printerr (str=str@entry=0x7f117bd660b1 "free(): invalid pointer") at ./malloc/malloc.c:5660
#6  0x00007f117bc65534 in _int_free (av=<optimized out>, p=<optimized out>, have_lock=have_lock@entry=0) at ./malloc/malloc.c:4435
#7  0x00007f117bc67e8f in __GI___libc_free (mem=<optimized out>) at ./malloc/malloc.c:3385
#8  0x00007f1158e6e454 in free_languages_metadata (mmdb=0x7f11592db390) at ./src/maxminddb.c:1860
#9  free_mmdb_struct (mmdb=0x7f11592db390) at ./src/maxminddb.c:1843
#10 0x00007f1158e708a5 in MMDB_close (mmdb=<optimized out>) at ./src/maxminddb.c:1803
#11 0x00007f1158e7ebc9 in geoip2_destroy_pv () at ./src/modules/geoip2/geoip2_pv.c:567
#12 0x000056094fd5f743 in destroy_modules () at core/sr_module.c:881
#13 0x000056094fbb7910 in cleanup (show_status=show_status@entry=1) at ./src/main.c:587
#14 0x000056094fbb86ad in shutdown_children (show_status=show_status@entry=1, sig=15) at ./src/main.c:735
#15 0x000056094fbb932b in handle_sigs () at ./src/main.c:835
#16 0x000056094fbbf1e9 in main_loop () at ./src/main.c:2002
#17 0x000056094fbb353c in main (argc=<optimized out>, argv=<optimized out>) at ./src/main.c:3256
(gdb) f 8
#8  0x00007f1158e6e454 in free_languages_metadata (mmdb=0x7f11592db390) at ./src/maxminddb.c:1860
1860        FREE_AND_SET_NULL(mmdb->metadata.languages.names);
(gdb) p *mmdb
$1 = {flags = 1, filename = 0x0, file_size = 652795, file_content = 0x7f1158dcd000 <error: Cannot access memory at address 0x7f1158dcd000>, data_section = 0x7f1158e6c430 <error: Cannot access memory at address 0x7f1158e6c430>, data_section_size = 459,
  metadata_section = 0x7f1158e6c546 <error: Cannot access memory at address 0x7f1158e6c546>, metadata_section_size = 181, full_record_byte_size = 6, depth = 32, ipv4_start_node = {netmask = 0, node_value = 0}, metadata = {node_count = 108720, record_size = 24, ip_version = 4,
    database_type = 0x0, languages = {count = 1, names = 0x560951ca6250}, binary_format_major_version = 2, binary_format_minor_version = 0, build_epoch = 1648557301, description = {count = 1, descriptions = 0x560951ca5470}}}
(gdb) f 9
#9  free_mmdb_struct (mmdb=0x7f11592db390) at ./src/maxminddb.c:1843
1843        free_languages_metadata(mmdb);
(gdb) f 10
#10 0x00007f1158e708a5 in MMDB_close (mmdb=<optimized out>) at ./src/maxminddb.c:1803
1803    void MMDB_close(MMDB_s *const mmdb) { free_mmdb_struct(mmdb); }
(gdb) f 11
#11 0x00007f1158e7ebc9 in geoip2_destroy_pv () at ./src/modules/geoip2/geoip2_pv.c:567
567             MMDB_close(_handle_GeoIP);

Additional Information

/usr/sbin/kamailio -V
version: kamailio 5.8.1 (x86_64/linux)
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled with gcc 12.2.0
cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"


Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: <kamailio/kamailio/issues/3861@github.com>