<!-- Kamailio Project uses GitHub Issues only for bugs in the code or feature requests. Please use this template only for bug reports.
If you have questions about using Kamailio or related to its configuration file, ask on sr-users mailing list:
* https://lists.kamailio.org/mailman3/postorius/lists/sr-users.lists.kamailio....
If you have questions about developing extensions to Kamailio or its existing C code, ask on sr-dev mailing list:
* https://lists.kamailio.org/mailman3/postorius/lists/sr-dev.lists.kamailio.or...
Please try to fill this template as much as possible for any issue. It helps the developers to troubleshoot the issue.
Note that an issue report may be closed automatically after about 2 months if there is no interest from developers or community users on pursuing it, being considered expired. In such case, it can be reopened by writing a comment that includes the token `/notexpired`. About two weeks before considered expired, the issue is marked with the label `stale`, trying to notify the submitter and everyone else that might be interested in it. To remove the label `stale`, write a comment that includes the token `/notstale`. Also, any comment postpone the `expire` timeline, being considered that there is interest in pursuing the issue.
If there is no content to be filled in a section, the entire section can be removed.
You can delete the comments from the template sections when filling.
You can delete next line and everything above before submitting (it is a comment). -->
### Description Kamailio crashes on shutdown in `geoip2_destroy_pv` in `main process` process if geoip2 database was reloaded.
<!-- Explain what you did, what you expected to happen, and what actually happened. -->
### Troubleshooting `libmaxminddb.so.0` uses `calloc()` internally to allocate memory for metadata languages and descriptions. On reload via e.g. `ctl` metadata memory is allocated in `ctl` process but `main process` tries to cleanup it.
#### Reproduction Load `geoip2` module ``` loadmodule "geoip2.so" modparam("geoip2", "path", "/path/to/your/geolite2.mmdb") ``` Reload database and stop/restart Kamailio ``` kamcmd geoip2.reload service kamailio restart ``` <!-- If the issue can be reproduced, describe how it can be done. -->
#### Debugging Data ``` (gdb) bt #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 #1 0x00007f117bc59e8f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78 #2 0x00007f117bc0afb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x00007f117bbf5472 in __GI_abort () at ./stdlib/abort.c:79 #4 0x00007f117bc4e430 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7f117bd68459 "%s\n") at ../sysdeps/posix/libc_fatal.c:155 #5 0x00007f117bc637aa in malloc_printerr (str=str@entry=0x7f117bd660b1 "free(): invalid pointer") at ./malloc/malloc.c:5660 #6 0x00007f117bc65534 in _int_free (av=<optimized out>, p=<optimized out>, have_lock=have_lock@entry=0) at ./malloc/malloc.c:4435 #7 0x00007f117bc67e8f in __GI___libc_free (mem=<optimized out>) at ./malloc/malloc.c:3385 #8 0x00007f1158e6e454 in free_languages_metadata (mmdb=0x7f11592db390) at ./src/maxminddb.c:1860 #9 free_mmdb_struct (mmdb=0x7f11592db390) at ./src/maxminddb.c:1843 #10 0x00007f1158e708a5 in MMDB_close (mmdb=<optimized out>) at ./src/maxminddb.c:1803 #11 0x00007f1158e7ebc9 in geoip2_destroy_pv () at ./src/modules/geoip2/geoip2_pv.c:567 #12 0x000056094fd5f743 in destroy_modules () at core/sr_module.c:881 #13 0x000056094fbb7910 in cleanup (show_status=show_status@entry=1) at ./src/main.c:587 #14 0x000056094fbb86ad in shutdown_children (show_status=show_status@entry=1, sig=15) at ./src/main.c:735 #15 0x000056094fbb932b in handle_sigs () at ./src/main.c:835 #16 0x000056094fbbf1e9 in main_loop () at ./src/main.c:2002 #17 0x000056094fbb353c in main (argc=<optimized out>, argv=<optimized out>) at ./src/main.c:3256 ``` ``` (gdb) f 8 #8 0x00007f1158e6e454 in free_languages_metadata (mmdb=0x7f11592db390) at ./src/maxminddb.c:1860 1860 FREE_AND_SET_NULL(mmdb->metadata.languages.names); (gdb) p *mmdb $1 = {flags = 1, filename = 0x0, file_size = 652795, file_content = 0x7f1158dcd000 <error: Cannot access memory at address 0x7f1158dcd000>, data_section = 0x7f1158e6c430 <error: Cannot access memory at address 0x7f1158e6c430>, data_section_size = 459, metadata_section = 0x7f1158e6c546 <error: Cannot access memory at address 0x7f1158e6c546>, metadata_section_size = 181, full_record_byte_size = 6, depth = 32, ipv4_start_node = {netmask = 0, node_value = 0}, metadata = {node_count = 108720, record_size = 24, ip_version = 4, database_type = 0x0, languages = {count = 1, names = 0x560951ca6250}, binary_format_major_version = 2, binary_format_minor_version = 0, build_epoch = 1648557301, description = {count = 1, descriptions = 0x560951ca5470}}} (gdb) f 9 #9 free_mmdb_struct (mmdb=0x7f11592db390) at ./src/maxminddb.c:1843 1843 free_languages_metadata(mmdb); (gdb) f 10 #10 0x00007f1158e708a5 in MMDB_close (mmdb=<optimized out>) at ./src/maxminddb.c:1803 1803 void MMDB_close(MMDB_s *const mmdb) { free_mmdb_struct(mmdb); } (gdb) f 11 #11 0x00007f1158e7ebc9 in geoip2_destroy_pv () at ./src/modules/geoip2/geoip2_pv.c:567 567 MMDB_close(_handle_GeoIP); ``` <!-- If you got a core dump, use gdb to extract troubleshooting data - full backtrace, local variables and the list of the code at the issue location.
gdb /path/to/kamailio /path/to/corefile bt full info locals list
If you are familiar with gdb, feel free to attach more of what you consider to be relevant. -->
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
``` /usr/sbin/kamailio -V version: kamailio 5.8.1 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 12.2.0 ```
* **Operating System**:
<!-- Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...; Kernel details (output of `lsb_release -a` and `uname -a`) -->
``` cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/" ```
Should be fixed by a series of commits in master. Can you test and provide feedback on results? They will be backported if all ok.
Should be fixed by a series of commits in master. Can you test and provide feedback on results? They will be backported if all ok.
Thanks. It does not crash anymore. However db is not reloaded anymore. Works for me with changes from #3867
Closed #3861 as completed.
Thanks, to be backported, closing!