### Description Kamailio crashes while querying for unexpired contacts in a location table while it is saving contacts to db. Seems to be a regression, since this didn't happened with Kamailio 4.4.
#### Reproduction Constantly flush many locations and simultaneously get location statistics from `usrloc` module via `kamcmd ul.db_users`: ```sh shell1 % while kamcmd ul.flush ; do : ; done ```
```sh shell2 % curl -H content-type:\ application/json -s -d '{"jsonrpc":"2.0", "method":"ul.db_contacts", "params": ["location"]}' localhost:5060/JSONRPC ```
#### Debugging Data
``` #0 0x00007f600e3345ad in ul_rpc_db_contacts (rpc=0x7f6004f8b960 <func_param>, ctx=0x7f6004f8b8e0 <_jsonrpc_ctx>) at ul_rpc.c:774 #1 0x00007f6004d6c15c in jsonrpc_dispatch (msg=0x7ffc7fc4e250, s1=0x0, s2=0x0) at jsonrpcs_mod.c:1251 #2 0x000055c674d60eb9 in do_action (h=0x7ffc7fc4e170, a=0x7f6012607828, msg=0x7ffc7fc4e250) at core/action.c:1054 #3 0x000055c674d6ea4d in run_actions (h=0x7ffc7fc4e170, a=0x7f6012607828, msg=0x7ffc7fc4e250) at core/action.c:1552 #4 0x000055c674d60e25 in do_action (h=0x7ffc7fc4e170, a=0x7f6012607ac8, msg=0x7ffc7fc4e250) at core/action.c:1045 #5 0x000055c674d6ea4d in run_actions (h=0x7ffc7fc4e170, a=0x7f6012600b28, msg=0x7ffc7fc4e250) at core/action.c:1552 #6 0x000055c674d60e25 in do_action (h=0x7ffc7fc4e170, a=0x7f6012607d00, msg=0x7ffc7fc4e250) at core/action.c:1045 #7 0x000055c674d6ea4d in run_actions (h=0x7ffc7fc4e170, a=0x7f60125fecf8, msg=0x7ffc7fc4e250) at core/action.c:1552 #8 0x00007f6004f913b4 in xhttp_process_request (orig_msg=0x7f6012622410, new_buf=0x7f6012638ad0 "POST /JSONRPC HTTP/1.1\r\nVia: SIP/2.0/TCP 127.0.0.1:58590\r\nHost: localhost:5060\r\nUser-Agent: curl/7.52.1\r\nAccept: */*\r\nContent-Length: 67\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n{"jsonrpc": "2.0", "method":"ul.db_contacts","params":["location"]}", new_len=256) at xhttp_mod.c:282 #9 0x00007f6004f9253b in xhttp_handler (msg=0x7f6012622410) at xhttp_mod.c:357 #10 0x000055c674e4be83 in nonsip_msg_run_hooks (msg=0x7f6012622410) at core/nonsip_hooks.c:111 #11 0x000055c674e944d7 in receive_msg (buf=0x7f5feac686a8 "POST /JSONRPC HTTP/1.1\r\nHost: localhost:5060\r\nUser-Agent: curl/7.52.1\r\nAccept: */*\r\nContent-Length: 67\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n{"jsonrpc": "2.0", "method":"ul.db_contacts","params":["location"]}", len=222, rcv_info=0x7f5feac683c8) at core/receive.c:202 #12 0x000055c674f2e43c in receive_tcp_msg (tcpbuf=0x7f5feac686a8 "POST /JSONRPC HTTP/1.1\r\nHost: localhost:5060\r\nUser-Agent: curl/7.52.1\r\nAccept: */*\r\nContent-Length: 67\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n{"jsonrpc": "2.0", "method":"ul.db_contacts","params":["location"]}", len=222, rcv_info=0x7f5feac683c8, con=0x7f5feac683b0) at core/tcp_read.c:1263 #13 0x000055c674f30a70 in tcp_read_req (con=0x7f5feac683b0, bytes_read=0x7ffc7fc4ee68, read_flags=0x7ffc7fc4ee6c) at core/tcp_read.c:1478 #14 0x000055c674f339bb in handle_io (fm=0x7f60126506b0, events=1, idx=-1) at core/tcp_read.c:1652 #15 0x000055c674f23d9b in io_wait_loop_epoll (h=0x55c6753e0fa0 <io_w>, t=2, repeat=0) at core/io_wait.h:1065 #16 0x000055c674f359b0 in tcp_receive_loop (unix_sock=54) at core/tcp_read.c:1822 #17 0x000055c674e14095 in tcp_init_children () at core/tcp_main.c:4802 #18 0x000055c674d216e1 in main_loop () at main.c:1713 #19 0x000055c674d286a9 in main (argc=13, argv=0x7ffc7fc4f4f8) at main.c:2642 ```
#### Log Messages
``` ERROR: usrloc [urecord.c:431]: wb_timer(): updating contact in db failed (aor: 1234@example.com) ERROR: db_mysql [km_dbase.c:127]: db_mysql_submit_query(): driver error on query: Commands out of sync; you can't run this command now (2014) ERROR: <core> [db_query.c:339]: db_do_update(): error while submitting query ERROR: usrloc [ucontact.c:1378]: db_update_ucontact_instance(): updating database failed ```
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
``` version: kamailio 5.0.7 (x86_64/linux) flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 6.3.0 ```
* **Operating System**:
``` Debian 9.5 ```
Can you try with the patch referenced above? For now it is only in master branch, if solves it, then I will backport.
The patch is with safety checks for db result to avoid a crash. Code in 4.4, 5.0 and even master is pretty much the same for this rpc function, so I guess is rather a matter of getting mysql out of sync, than the version of kamailio. You are sending rpc commands at high rate in a loop in shell, many of the rpc commands were designed more for human/admin interaction, not considering high rate processing. I haven't developed the db rpc function in usrloc, never used them before, so I am not sure about their purpose. They are also restricted to sql backends, using raw query internally, something that is not recommended to be done in C code.
Closing here. Reopen if still an issue after the patch pushed for it.
Closed #1900.