Hi,
(Kamailio 4.2.3)
Working to get dialogs closed and cleaned up nicely I'm looking into the ka_interval and ka_timer parameter of the dialog module. The following is now happening.
Scenario: 1. Call gets setup 2. Callee's internet connection drops 3. Call gets terminated (nicely) due to ka_timer/ka_interval 4. Log fills with "freeing a free fragment" after about 10 seconds
Digging into this I saw that the dialog is being closed and cleaned properly in step 3. But it fails to remove/clean the timers used for the keep-alive. That's what triggers step 4, which in turn tries to clean the (or some of the) dialog space again. Is this expected behaviour when using the keep-alive options, or am I running into somekind of (known) bug?
Thanks in advance for any help.
Hello,
thanks for reporting and troubleshooting.
Can you refer to the code (file+lines) for step 3 and 4? It will save me some time to look into the code and fix the issue.
Cheers, Daniel
On 07/05/15 08:46, Dirk Teurlings - SIGNET B.V. wrote:
Hi,
(Kamailio 4.2.3)
Working to get dialogs closed and cleaned up nicely I'm looking into the ka_interval and ka_timer parameter of the dialog module. The following is now happening.
Scenario:
- Call gets setup
- Callee's internet connection drops
- Call gets terminated (nicely) due to ka_timer/ka_interval
- Log fills with "freeing a free fragment" after about 10 seconds
Digging into this I saw that the dialog is being closed and cleaned properly in step 3. But it fails to remove/clean the timers used for the keep-alive. That's what triggers step 4, which in turn tries to clean the (or some of the) dialog space again. Is this expected behaviour when using the keep-alive options, or am I running into somekind of (known) bug?
Thanks in advance for any help.
Hi again,
File dlg_req_within.c function bye_reply_cb line: 166-236 functino dlg_ka_cb line: 240-275
To help you out a bit more, this is important part of the log.
May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:1066]: next_state_dlg(): dialog 0x7f812f2585e8 changed from state 4 to state 5, due event 7 (ref 5) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:195]: bye_reply_cb(): removing dialog with h_entry 1231 and h_id 1142 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7f812f2585e8 [1231:1142] with clid '6fcb9a946131aad41e1deda730c6b76a@192.168.10.245:5060' and tags 'as57b6a153' 'c42610a13600fe22i0' May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_cb.c:256]: run_dlg_callbacks(): dialog=0x7f812f2585e8, type=64 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:219]: bye_reply_cb(): first final reply May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:872]: dlg_unref(): unref dlg 0x7f812f2585e8 with 2 -> 3 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f2529f0) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: tm [t_reply.c:1653]: cleanup_uac_timers(): DEBUG: cleanup_uac_timers: RETR/FR timers reset May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_var.c:81]: cb_dlg_locals_reset(): resetting the local dialog shortcuts on script callback: 2147483652
...
May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [xavp.c:448]: xavp_destroy_list(): destroying xavp list (nil) May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [receive.c:298]: receive_msg(): receive_msg: cleaning up May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:1294]: t_should_relay_response(): ->>>>>>>>> T_code=0, new_code=408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2080]: local_reply(): DEBUG: local_reply: branch=0, save=0, winner=0 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2117]: local_reply(): DEBUG: local transaction completed May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_hooks.c:288]: run_trans_callbacks_internal(): DBG: trans=0x7f812f256fd0, callback type 1024, id 0 entered May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_req_within.c:255]: dlg_ka_cb(): receiving a final reply 408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f252ba0) May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7f812f252ad0/0x7f812f252b08) - ignore May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 etc...
Cheers, Dirk Teurlings
On 07-05-15 09:03, Daniel-Constantin Mierla wrote:
Hello,
thanks for reporting and troubleshooting.
Can you refer to the code (file+lines) for step 3 and 4? It will save me some time to look into the code and fix the issue.
Cheers, Daniel
On 07/05/15 08:46, Dirk Teurlings - SIGNET B.V. wrote:
Hi,
(Kamailio 4.2.3)
Working to get dialogs closed and cleaned up nicely I'm looking into the ka_interval and ka_timer parameter of the dialog module. The following is now happening.
Scenario:
- Call gets setup
- Callee's internet connection drops
- Call gets terminated (nicely) due to ka_timer/ka_interval
- Log fills with "freeing a free fragment" after about 10 seconds
Digging into this I saw that the dialog is being closed and cleaned properly in step 3. But it fails to remove/clean the timers used for the keep-alive. That's what triggers step 4, which in turn tries to clean the (or some of the) dialog space again. Is this expected behaviour when using the keep-alive options, or am I running into somekind of (known) bug?
Thanks in advance for any help.
Hello,
I couldn't correlate the log message for double free with the code for bye_reply_cb and dlg_ka_cb. The logs from pid 24493 shows only dialog lookups and that double free message.
Do you have the instance still running? Can you send the output of:
kamctl ps
Also, would be helpful to see some other logs messages printed just before by pid 24493.
By analyzing the code I found an improvement to be done, already committed in master via:
- https://github.com/kamailio/kamailio/commit/04410aef38f5ed40c8f4a6796d956584...
It is about being sure the dialog is still in confirmed state when the keepalive callback is handled.
But I can't be sure yet if that was the case resulting in double free.
Cheers, Daniel
On 07/05/15 09:34, Dirk Teurlings - SIGNET B.V. wrote:
Hi again,
File dlg_req_within.c function bye_reply_cb line: 166-236 functino dlg_ka_cb line: 240-275
To help you out a bit more, this is important part of the log.
May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:1066]: next_state_dlg(): dialog 0x7f812f2585e8 changed from state 4 to state 5, due event 7 (ref 5) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:195]: bye_reply_cb(): removing dialog with h_entry 1231 and h_id 1142 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7f812f2585e8 [1231:1142] with clid '6fcb9a946131aad41e1deda730c6b76a@192.168.10.245:5060' and tags 'as57b6a153' 'c42610a13600fe22i0' May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_cb.c:256]: run_dlg_callbacks(): dialog=0x7f812f2585e8, type=64 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:219]: bye_reply_cb(): first final reply May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:872]: dlg_unref(): unref dlg 0x7f812f2585e8 with 2 -> 3 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f2529f0) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: tm [t_reply.c:1653]: cleanup_uac_timers(): DEBUG: cleanup_uac_timers: RETR/FR timers reset May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_var.c:81]: cb_dlg_locals_reset(): resetting the local dialog shortcuts on script callback: 2147483652
...
May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [xavp.c:448]: xavp_destroy_list(): destroying xavp list (nil) May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [receive.c:298]: receive_msg(): receive_msg: cleaning up May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:1294]: t_should_relay_response(): ->>>>>>>>> T_code=0, new_code=408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2080]: local_reply(): DEBUG: local_reply: branch=0, save=0, winner=0 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2117]: local_reply(): DEBUG: local transaction completed May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_hooks.c:288]: run_trans_callbacks_internal(): DBG: trans=0x7f812f256fd0, callback type 1024, id 0 entered May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_req_within.c:255]: dlg_ka_cb(): receiving a final reply 408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f252ba0) May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7f812f252ad0/0x7f812f252b08) - ignore May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 etc...
Cheers, Dirk Teurlings
On 07-05-15 09:03, Daniel-Constantin Mierla wrote:
Hello,
thanks for reporting and troubleshooting.
Can you refer to the code (file+lines) for step 3 and 4? It will save me some time to look into the code and fix the issue.
Cheers, Daniel
On 07/05/15 08:46, Dirk Teurlings - SIGNET B.V. wrote:
Hi,
(Kamailio 4.2.3)
Working to get dialogs closed and cleaned up nicely I'm looking into the ka_interval and ka_timer parameter of the dialog module. The following is now happening.
Scenario:
- Call gets setup
- Callee's internet connection drops
- Call gets terminated (nicely) due to ka_timer/ka_interval
- Log fills with "freeing a free fragment" after about 10 seconds
Digging into this I saw that the dialog is being closed and cleaned properly in step 3. But it fails to remove/clean the timers used for the keep-alive. That's what triggers step 4, which in turn tries to clean the (or some of the) dialog space again. Is this expected behaviour when using the keep-alive options, or am I running into somekind of (known) bug?
Thanks in advance for any help.
Hi Daniel,
Good to see you found an improvement, not sure yet if it'll help us.
Here is the non-debug log first. Just now I noticed that the initial dialog doesn't get ended as well now, 5fa4dc3670596fb626269f083b0c9480 in the log.
The setup is like this
Leg 1 Kamailio -> Asterisk Leg 2 (initialized by Asterisk) -> Kamailio -> Gateway
First the output of ps
Process:: ID=0 PID=21528 Type=attendant Process:: ID=1 PID=21530 Type=udp receiver child=0 sock=127.0.0.1:5060 Process:: ID=2 PID=21531 Type=udp receiver child=0 sock=192.168.10.246:5060 Process:: ID=3 PID=21532 Type=udp receiver child=0 sock=31.223.168.246:5060 Process:: ID=4 PID=21533 Type=udp receiver child=0 sock=[::1]:5060 Process:: ID=5 PID=21534 Type=udp receiver child=0 sock=[2001:4cb8:34a:10::246]:5060 Process:: ID=6 PID=21535 Type=slow timer Process:: ID=7 PID=21536 Type=timer Process:: ID=8 PID=21537 Type=MI FIFO Process:: ID=9 PID=21538 Type=ctl handler Process:: ID=10 PID=21539 Type=TIMER NH Process:: ID=11 PID=21540 Type=Dialog KA Timer Process:: ID=12 PID=21541 Type=Dialog Clean Timer Process:: ID=13 PID=21542 Type=WLCC TB TIMER Process:: ID=14 PID=21543 Type=LLCC TB TIMER Process:: ID=15 PID=21544 Type=tcp receiver (generic) child=0 Process:: ID=16 PID=21545 Type=tcp main process
And the log
May 7 18:54:43 vps-host /usr/sbin/kamailio[21532]: INFO: <script>: 5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060 - CC, dialog:start May 7 18:54:44 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:start May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - CC, dialog:end May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - WLCC: ENDCALL May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7fa76e546600 [869:4705] with clid '5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060' and tags 'as155a87b8' '95b0a07565f99244i0' May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:end May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore
Is this enough, or do you need the full debug log? It's a bit heavy on the log because of continuous database lookups. But I could filter them down a bit if needed.
Cheers, Dirk Teurlings
On 07-05-15 13:31, Daniel-Constantin Mierla wrote:
Hello,
I couldn't correlate the log message for double free with the code for bye_reply_cb and dlg_ka_cb. The logs from pid 24493 shows only dialog lookups and that double free message.
Do you have the instance still running? Can you send the output of:
kamctl ps
Also, would be helpful to see some other logs messages printed just before by pid 24493.
By analyzing the code I found an improvement to be done, already committed in master via:
https://github.com/kamailio/kamailio/commit/04410aef38f5ed40c8f4a6796d956584...
It is about being sure the dialog is still in confirmed state when the keepalive callback is handled.
But I can't be sure yet if that was the case resulting in double free.
Cheers, Daniel
On 07/05/15 09:34, Dirk Teurlings - SIGNET B.V. wrote:
Hi again,
File dlg_req_within.c function bye_reply_cb line: 166-236 functino dlg_ka_cb line: 240-275
To help you out a bit more, this is important part of the log.
May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:1066]: next_state_dlg(): dialog 0x7f812f2585e8 changed from state 4 to state 5, due event 7 (ref 5) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:195]: bye_reply_cb(): removing dialog with h_entry 1231 and h_id 1142 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7f812f2585e8 [1231:1142] with clid '6fcb9a946131aad41e1deda730c6b76a@192.168.10.245:5060' and tags 'as57b6a153' 'c42610a13600fe22i0' May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_cb.c:256]: run_dlg_callbacks(): dialog=0x7f812f2585e8, type=64 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:219]: bye_reply_cb(): first final reply May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:872]: dlg_unref(): unref dlg 0x7f812f2585e8 with 2 -> 3 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f2529f0) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: tm [t_reply.c:1653]: cleanup_uac_timers(): DEBUG: cleanup_uac_timers: RETR/FR timers reset May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_var.c:81]: cb_dlg_locals_reset(): resetting the local dialog shortcuts on script callback: 2147483652
...
May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [xavp.c:448]: xavp_destroy_list(): destroying xavp list (nil) May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [receive.c:298]: receive_msg(): receive_msg: cleaning up May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:1294]: t_should_relay_response(): ->>>>>>>>> T_code=0, new_code=408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2080]: local_reply(): DEBUG: local_reply: branch=0, save=0, winner=0 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2117]: local_reply(): DEBUG: local transaction completed May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_hooks.c:288]: run_trans_callbacks_internal(): DBG: trans=0x7f812f256fd0, callback type 1024, id 0 entered May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_req_within.c:255]: dlg_ka_cb(): receiving a final reply 408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f252ba0) May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7f812f252ad0/0x7f812f252b08) - ignore May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 etc...
Cheers, Dirk Teurlings
On 07-05-15 09:03, Daniel-Constantin Mierla wrote:
Hello,
thanks for reporting and troubleshooting.
Can you refer to the code (file+lines) for step 3 and 4? It will save me some time to look into the code and fix the issue.
Cheers, Daniel
On 07/05/15 08:46, Dirk Teurlings - SIGNET B.V. wrote:
Hi,
(Kamailio 4.2.3)
Working to get dialogs closed and cleaned up nicely I'm looking into the ka_interval and ka_timer parameter of the dialog module. The following is now happening.
Scenario:
- Call gets setup
- Callee's internet connection drops
- Call gets terminated (nicely) due to ka_timer/ka_interval
- Log fills with "freeing a free fragment" after about 10 seconds
Digging into this I saw that the dialog is being closed and cleaned properly in step 3. But it fails to remove/clean the timers used for the keep-alive. That's what triggers step 4, which in turn tries to clean the (or some of the) dialog space again. Is this expected behaviour when using the keep-alive options, or am I running into somekind of (known) bug?
Thanks in advance for any help.
Hello,
the message "freeing a free fragment ..." is from Dialog KA Timer and that process doesn't handle any SIP reply or the transmission timeout callback. I will look at the code and see if I can spot where the double free happens.
But now I understand that the dialog is actually not ended, right?
Cheers, Daniel
On 07/05/15 17:05, Dirk Teurlings - SIGNET B.V. wrote:
Hi Daniel,
Good to see you found an improvement, not sure yet if it'll help us.
Here is the non-debug log first. Just now I noticed that the initial dialog doesn't get ended as well now, 5fa4dc3670596fb626269f083b0c9480 in the log.
The setup is like this
Leg 1 Kamailio -> Asterisk Leg 2 (initialized by Asterisk) -> Kamailio -> Gateway
First the output of ps
Process:: ID=0 PID=21528 Type=attendant Process:: ID=1 PID=21530 Type=udp receiver child=0 sock=127.0.0.1:5060 Process:: ID=2 PID=21531 Type=udp receiver child=0 sock=192.168.10.246:5060 Process:: ID=3 PID=21532 Type=udp receiver child=0 sock=31.223.168.246:5060 Process:: ID=4 PID=21533 Type=udp receiver child=0 sock=[::1]:5060 Process:: ID=5 PID=21534 Type=udp receiver child=0 sock=[2001:4cb8:34a:10::246]:5060 Process:: ID=6 PID=21535 Type=slow timer Process:: ID=7 PID=21536 Type=timer Process:: ID=8 PID=21537 Type=MI FIFO Process:: ID=9 PID=21538 Type=ctl handler Process:: ID=10 PID=21539 Type=TIMER NH Process:: ID=11 PID=21540 Type=Dialog KA Timer Process:: ID=12 PID=21541 Type=Dialog Clean Timer Process:: ID=13 PID=21542 Type=WLCC TB TIMER Process:: ID=14 PID=21543 Type=LLCC TB TIMER Process:: ID=15 PID=21544 Type=tcp receiver (generic) child=0 Process:: ID=16 PID=21545 Type=tcp main process
And the log
May 7 18:54:43 vps-host /usr/sbin/kamailio[21532]: INFO: <script>: 5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060 - CC, dialog:start May 7 18:54:44 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:start May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - CC, dialog:end May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - WLCC: ENDCALL May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7fa76e546600 [869:4705] with clid '5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060' and tags 'as155a87b8' '95b0a07565f99244i0' May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:end May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore
Is this enough, or do you need the full debug log? It's a bit heavy on the log because of continuous database lookups. But I could filter them down a bit if needed.
Cheers, Dirk Teurlings
On 07-05-15 13:31, Daniel-Constantin Mierla wrote:
Hello,
I couldn't correlate the log message for double free with the code for bye_reply_cb and dlg_ka_cb. The logs from pid 24493 shows only dialog lookups and that double free message.
Do you have the instance still running? Can you send the output of:
kamctl ps
Also, would be helpful to see some other logs messages printed just before by pid 24493.
By analyzing the code I found an improvement to be done, already committed in master via:
https://github.com/kamailio/kamailio/commit/04410aef38f5ed40c8f4a6796d956584...
It is about being sure the dialog is still in confirmed state when the keepalive callback is handled.
But I can't be sure yet if that was the case resulting in double free.
Cheers, Daniel
On 07/05/15 09:34, Dirk Teurlings - SIGNET B.V. wrote:
Hi again,
File dlg_req_within.c function bye_reply_cb line: 166-236 functino dlg_ka_cb line: 240-275
To help you out a bit more, this is important part of the log.
May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:1066]: next_state_dlg(): dialog 0x7f812f2585e8 changed from state 4 to state 5, due event 7 (ref 5) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:195]: bye_reply_cb(): removing dialog with h_entry 1231 and h_id 1142 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7f812f2585e8 [1231:1142] with clid '6fcb9a946131aad41e1deda730c6b76a@192.168.10.245:5060' and tags 'as57b6a153' 'c42610a13600fe22i0' May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_cb.c:256]: run_dlg_callbacks(): dialog=0x7f812f2585e8, type=64 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:219]: bye_reply_cb(): first final reply May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:872]: dlg_unref(): unref dlg 0x7f812f2585e8 with 2 -> 3 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f2529f0) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: tm [t_reply.c:1653]: cleanup_uac_timers(): DEBUG: cleanup_uac_timers: RETR/FR timers reset May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_var.c:81]: cb_dlg_locals_reset(): resetting the local dialog shortcuts on script callback: 2147483652
...
May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [xavp.c:448]: xavp_destroy_list(): destroying xavp list (nil) May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [receive.c:298]: receive_msg(): receive_msg: cleaning up May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:1294]: t_should_relay_response(): ->>>>>>>>> T_code=0, new_code=408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2080]: local_reply(): DEBUG: local_reply: branch=0, save=0, winner=0 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2117]: local_reply(): DEBUG: local transaction completed May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_hooks.c:288]: run_trans_callbacks_internal(): DBG: trans=0x7f812f256fd0, callback type 1024, id 0 entered May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_req_within.c:255]: dlg_ka_cb(): receiving a final reply 408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f252ba0) May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7f812f252ad0/0x7f812f252b08) - ignore May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 etc...
Cheers, Dirk Teurlings
On 07-05-15 09:03, Daniel-Constantin Mierla wrote:
Hello,
thanks for reporting and troubleshooting.
Can you refer to the code (file+lines) for step 3 and 4? It will save me some time to look into the code and fix the issue.
Cheers, Daniel
On 07/05/15 08:46, Dirk Teurlings - SIGNET B.V. wrote:
Hi,
(Kamailio 4.2.3)
Working to get dialogs closed and cleaned up nicely I'm looking into the ka_interval and ka_timer parameter of the dialog module. The following is now happening.
Scenario:
- Call gets setup
- Callee's internet connection drops
- Call gets terminated (nicely) due to ka_timer/ka_interval
- Log fills with "freeing a free fragment" after about 10 seconds
Digging into this I saw that the dialog is being closed and cleaned properly in step 3. But it fails to remove/clean the timers used for the keep-alive. That's what triggers step 4, which in turn tries to clean the (or some of the) dialog space again. Is this expected behaviour when using the keep-alive options, or am I running into somekind of (known) bug?
Thanks in advance for any help.
Following quickly up, I pushed another patch that will remove the dialog from keepalive list as soon as 'deleted' state is discovered and also sends keepalive options only if dialog is in state 'confirmed'.
Can you test the patch and see how it works?
- https://github.com/kamailio/kamailio/commit/0e22abe2b89be8936df4b8230955fbaf...
Cheers, Daniel
On 11/05/15 16:55, Daniel-Constantin Mierla wrote:
Hello,
the message "freeing a free fragment ..." is from Dialog KA Timer and that process doesn't handle any SIP reply or the transmission timeout callback. I will look at the code and see if I can spot where the double free happens.
But now I understand that the dialog is actually not ended, right?
Cheers, Daniel
On 07/05/15 17:05, Dirk Teurlings - SIGNET B.V. wrote:
Hi Daniel,
Good to see you found an improvement, not sure yet if it'll help us.
Here is the non-debug log first. Just now I noticed that the initial dialog doesn't get ended as well now, 5fa4dc3670596fb626269f083b0c9480 in the log.
The setup is like this
Leg 1 Kamailio -> Asterisk Leg 2 (initialized by Asterisk) -> Kamailio -> Gateway
First the output of ps
Process:: ID=0 PID=21528 Type=attendant Process:: ID=1 PID=21530 Type=udp receiver child=0 sock=127.0.0.1:5060 Process:: ID=2 PID=21531 Type=udp receiver child=0 sock=192.168.10.246:5060 Process:: ID=3 PID=21532 Type=udp receiver child=0 sock=31.223.168.246:5060 Process:: ID=4 PID=21533 Type=udp receiver child=0 sock=[::1]:5060 Process:: ID=5 PID=21534 Type=udp receiver child=0 sock=[2001:4cb8:34a:10::246]:5060 Process:: ID=6 PID=21535 Type=slow timer Process:: ID=7 PID=21536 Type=timer Process:: ID=8 PID=21537 Type=MI FIFO Process:: ID=9 PID=21538 Type=ctl handler Process:: ID=10 PID=21539 Type=TIMER NH Process:: ID=11 PID=21540 Type=Dialog KA Timer Process:: ID=12 PID=21541 Type=Dialog Clean Timer Process:: ID=13 PID=21542 Type=WLCC TB TIMER Process:: ID=14 PID=21543 Type=LLCC TB TIMER Process:: ID=15 PID=21544 Type=tcp receiver (generic) child=0 Process:: ID=16 PID=21545 Type=tcp main process
And the log
May 7 18:54:43 vps-host /usr/sbin/kamailio[21532]: INFO: <script>: 5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060 - CC, dialog:start May 7 18:54:44 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:start May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - CC, dialog:end May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - WLCC: ENDCALL May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7fa76e546600 [869:4705] with clid '5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060' and tags 'as155a87b8' '95b0a07565f99244i0' May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:end May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore
Is this enough, or do you need the full debug log? It's a bit heavy on the log because of continuous database lookups. But I could filter them down a bit if needed.
Cheers, Dirk Teurlings
On 07-05-15 13:31, Daniel-Constantin Mierla wrote:
Hello,
I couldn't correlate the log message for double free with the code for bye_reply_cb and dlg_ka_cb. The logs from pid 24493 shows only dialog lookups and that double free message.
Do you have the instance still running? Can you send the output of:
kamctl ps
Also, would be helpful to see some other logs messages printed just before by pid 24493.
By analyzing the code I found an improvement to be done, already committed in master via:
https://github.com/kamailio/kamailio/commit/04410aef38f5ed40c8f4a6796d956584...
It is about being sure the dialog is still in confirmed state when the keepalive callback is handled.
But I can't be sure yet if that was the case resulting in double free.
Cheers, Daniel
On 07/05/15 09:34, Dirk Teurlings - SIGNET B.V. wrote:
Hi again,
File dlg_req_within.c function bye_reply_cb line: 166-236 functino dlg_ka_cb line: 240-275
To help you out a bit more, this is important part of the log.
May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:1066]: next_state_dlg(): dialog 0x7f812f2585e8 changed from state 4 to state 5, due event 7 (ref 5) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:195]: bye_reply_cb(): removing dialog with h_entry 1231 and h_id 1142 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7f812f2585e8 [1231:1142] with clid '6fcb9a946131aad41e1deda730c6b76a@192.168.10.245:5060' and tags 'as57b6a153' 'c42610a13600fe22i0' May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_cb.c:256]: run_dlg_callbacks(): dialog=0x7f812f2585e8, type=64 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:219]: bye_reply_cb(): first final reply May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:872]: dlg_unref(): unref dlg 0x7f812f2585e8 with 2 -> 3 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f2529f0) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: tm [t_reply.c:1653]: cleanup_uac_timers(): DEBUG: cleanup_uac_timers: RETR/FR timers reset May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_var.c:81]: cb_dlg_locals_reset(): resetting the local dialog shortcuts on script callback: 2147483652
...
May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [xavp.c:448]: xavp_destroy_list(): destroying xavp list (nil) May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [receive.c:298]: receive_msg(): receive_msg: cleaning up May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:1294]: t_should_relay_response(): ->>>>>>>>> T_code=0, new_code=408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2080]: local_reply(): DEBUG: local_reply: branch=0, save=0, winner=0 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2117]: local_reply(): DEBUG: local transaction completed May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_hooks.c:288]: run_trans_callbacks_internal(): DBG: trans=0x7f812f256fd0, callback type 1024, id 0 entered May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_req_within.c:255]: dlg_ka_cb(): receiving a final reply 408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f252ba0) May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7f812f252ad0/0x7f812f252b08) - ignore May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 etc...
Cheers, Dirk Teurlings
On 07-05-15 09:03, Daniel-Constantin Mierla wrote:
Hello,
thanks for reporting and troubleshooting.
Can you refer to the code (file+lines) for step 3 and 4? It will save me some time to look into the code and fix the issue.
Cheers, Daniel
On 07/05/15 08:46, Dirk Teurlings - SIGNET B.V. wrote:
Hi,
(Kamailio 4.2.3)
Working to get dialogs closed and cleaned up nicely I'm looking into the ka_interval and ka_timer parameter of the dialog module. The following is now happening.
Scenario:
- Call gets setup
- Callee's internet connection drops
- Call gets terminated (nicely) due to ka_timer/ka_interval
- Log fills with "freeing a free fragment" after about 10 seconds
Digging into this I saw that the dialog is being closed and cleaned properly in step 3. But it fails to remove/clean the timers used for the keep-alive. That's what triggers step 4, which in turn tries to clean the (or some of the) dialog space again. Is this expected behaviour when using the keep-alive options, or am I running into somekind of (known) bug?
Thanks in advance for any help.
Hi Daniel,
This fixes the problem, thanks very much,
Will this fix be included in the next (4.2 and/or 4.3) stable release?
Cheers, Dirk Teurlings
On 11-05-15 17:25, Daniel-Constantin Mierla wrote:
Following quickly up, I pushed another patch that will remove the dialog from keepalive list as soon as 'deleted' state is discovered and also sends keepalive options only if dialog is in state 'confirmed'.
Can you test the patch and see how it works?
https://github.com/kamailio/kamailio/commit/0e22abe2b89be8936df4b8230955fbaf...
Cheers, Daniel
On 11/05/15 16:55, Daniel-Constantin Mierla wrote:
Hello,
the message "freeing a free fragment ..." is from Dialog KA Timer and that process doesn't handle any SIP reply or the transmission timeout callback. I will look at the code and see if I can spot where the double free happens.
But now I understand that the dialog is actually not ended, right?
Cheers, Daniel
On 07/05/15 17:05, Dirk Teurlings - SIGNET B.V. wrote:
Hi Daniel,
Good to see you found an improvement, not sure yet if it'll help us.
Here is the non-debug log first. Just now I noticed that the initial dialog doesn't get ended as well now, 5fa4dc3670596fb626269f083b0c9480 in the log.
The setup is like this
Leg 1 Kamailio -> Asterisk Leg 2 (initialized by Asterisk) -> Kamailio -> Gateway
First the output of ps
Process:: ID=0 PID=21528 Type=attendant Process:: ID=1 PID=21530 Type=udp receiver child=0 sock=127.0.0.1:5060 Process:: ID=2 PID=21531 Type=udp receiver child=0 sock=192.168.10.246:5060 Process:: ID=3 PID=21532 Type=udp receiver child=0 sock=31.223.168.246:5060 Process:: ID=4 PID=21533 Type=udp receiver child=0 sock=[::1]:5060 Process:: ID=5 PID=21534 Type=udp receiver child=0 sock=[2001:4cb8:34a:10::246]:5060 Process:: ID=6 PID=21535 Type=slow timer Process:: ID=7 PID=21536 Type=timer Process:: ID=8 PID=21537 Type=MI FIFO Process:: ID=9 PID=21538 Type=ctl handler Process:: ID=10 PID=21539 Type=TIMER NH Process:: ID=11 PID=21540 Type=Dialog KA Timer Process:: ID=12 PID=21541 Type=Dialog Clean Timer Process:: ID=13 PID=21542 Type=WLCC TB TIMER Process:: ID=14 PID=21543 Type=LLCC TB TIMER Process:: ID=15 PID=21544 Type=tcp receiver (generic) child=0 Process:: ID=16 PID=21545 Type=tcp main process
And the log
May 7 18:54:43 vps-host /usr/sbin/kamailio[21532]: INFO: <script>: 5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060 - CC, dialog:start May 7 18:54:44 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:start May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - CC, dialog:end May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - WLCC: ENDCALL May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7fa76e546600 [869:4705] with clid '5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060' and tags 'as155a87b8' '95b0a07565f99244i0' May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:end May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore
Is this enough, or do you need the full debug log? It's a bit heavy on the log because of continuous database lookups. But I could filter them down a bit if needed.
Cheers, Dirk Teurlings
On 07-05-15 13:31, Daniel-Constantin Mierla wrote:
Hello,
I couldn't correlate the log message for double free with the code for bye_reply_cb and dlg_ka_cb. The logs from pid 24493 shows only dialog lookups and that double free message.
Do you have the instance still running? Can you send the output of:
kamctl ps
Also, would be helpful to see some other logs messages printed just before by pid 24493.
By analyzing the code I found an improvement to be done, already committed in master via:
https://github.com/kamailio/kamailio/commit/04410aef38f5ed40c8f4a6796d956584...
It is about being sure the dialog is still in confirmed state when the keepalive callback is handled.
But I can't be sure yet if that was the case resulting in double free.
Cheers, Daniel
On 07/05/15 09:34, Dirk Teurlings - SIGNET B.V. wrote:
Hi again,
File dlg_req_within.c function bye_reply_cb line: 166-236 functino dlg_ka_cb line: 240-275
To help you out a bit more, this is important part of the log.
May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:1066]: next_state_dlg(): dialog 0x7f812f2585e8 changed from state 4 to state 5, due event 7 (ref 5) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:195]: bye_reply_cb(): removing dialog with h_entry 1231 and h_id 1142 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7f812f2585e8 [1231:1142] with clid '6fcb9a946131aad41e1deda730c6b76a@192.168.10.245:5060' and tags 'as57b6a153' 'c42610a13600fe22i0' May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_cb.c:256]: run_dlg_callbacks(): dialog=0x7f812f2585e8, type=64 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:219]: bye_reply_cb(): first final reply May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:872]: dlg_unref(): unref dlg 0x7f812f2585e8 with 2 -> 3 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f2529f0) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: tm [t_reply.c:1653]: cleanup_uac_timers(): DEBUG: cleanup_uac_timers: RETR/FR timers reset May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_var.c:81]: cb_dlg_locals_reset(): resetting the local dialog shortcuts on script callback: 2147483652
...
May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [xavp.c:448]: xavp_destroy_list(): destroying xavp list (nil) May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [receive.c:298]: receive_msg(): receive_msg: cleaning up May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:1294]: t_should_relay_response(): ->>>>>>>>> T_code=0, new_code=408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2080]: local_reply(): DEBUG: local_reply: branch=0, save=0, winner=0 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2117]: local_reply(): DEBUG: local transaction completed May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_hooks.c:288]: run_trans_callbacks_internal(): DBG: trans=0x7f812f256fd0, callback type 1024, id 0 entered May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_req_within.c:255]: dlg_ka_cb(): receiving a final reply 408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f252ba0) May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7f812f252ad0/0x7f812f252b08) - ignore May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 etc...
Cheers, Dirk Teurlings
On 07-05-15 09:03, Daniel-Constantin Mierla wrote:
Hello,
thanks for reporting and troubleshooting.
Can you refer to the code (file+lines) for step 3 and 4? It will save me some time to look into the code and fix the issue.
Cheers, Daniel
On 07/05/15 08:46, Dirk Teurlings - SIGNET B.V. wrote: > Hi, > > (Kamailio 4.2.3) > > Working to get dialogs closed and cleaned up nicely I'm looking into > the ka_interval and ka_timer parameter of the dialog module. The > following is now happening. > > Scenario: > 1. Call gets setup > 2. Callee's internet connection drops > 3. Call gets terminated (nicely) due to ka_timer/ka_interval > 4. Log fills with "freeing a free fragment" after about 10 seconds > > Digging into this I saw that the dialog is being closed and cleaned > properly in step 3. But it fails to remove/clean the timers used for > the keep-alive. That's what triggers step 4, which in turn tries to > clean the (or some of the) dialog space again. Is this expected > behaviour when using the keep-alive options, or am I running into > somekind of (known) bug? > > Thanks in advance for any help. >
Hello,
master is still the code to be 4.3, we will make a dedicated branch sometime soon.
The patch will be backported to 4.2 as well. Thanks for helping to troubleshoot and testing.
Cheers, Daniel
On 12/05/15 10:03, Dirk Teurlings - SIGNET B.V. wrote:
Hi Daniel,
This fixes the problem, thanks very much,
Will this fix be included in the next (4.2 and/or 4.3) stable release?
Cheers, Dirk Teurlings
On 11-05-15 17:25, Daniel-Constantin Mierla wrote:
Following quickly up, I pushed another patch that will remove the dialog from keepalive list as soon as 'deleted' state is discovered and also sends keepalive options only if dialog is in state 'confirmed'.
Can you test the patch and see how it works?
https://github.com/kamailio/kamailio/commit/0e22abe2b89be8936df4b8230955fbaf...
Cheers, Daniel
On 11/05/15 16:55, Daniel-Constantin Mierla wrote:
Hello,
the message "freeing a free fragment ..." is from Dialog KA Timer and that process doesn't handle any SIP reply or the transmission timeout callback. I will look at the code and see if I can spot where the double free happens.
But now I understand that the dialog is actually not ended, right?
Cheers, Daniel
On 07/05/15 17:05, Dirk Teurlings - SIGNET B.V. wrote:
Hi Daniel,
Good to see you found an improvement, not sure yet if it'll help us.
Here is the non-debug log first. Just now I noticed that the initial dialog doesn't get ended as well now, 5fa4dc3670596fb626269f083b0c9480 in the log.
The setup is like this
Leg 1 Kamailio -> Asterisk Leg 2 (initialized by Asterisk) -> Kamailio -> Gateway
First the output of ps
Process:: ID=0 PID=21528 Type=attendant Process:: ID=1 PID=21530 Type=udp receiver child=0 sock=127.0.0.1:5060 Process:: ID=2 PID=21531 Type=udp receiver child=0 sock=192.168.10.246:5060 Process:: ID=3 PID=21532 Type=udp receiver child=0 sock=31.223.168.246:5060 Process:: ID=4 PID=21533 Type=udp receiver child=0 sock=[::1]:5060 Process:: ID=5 PID=21534 Type=udp receiver child=0 sock=[2001:4cb8:34a:10::246]:5060 Process:: ID=6 PID=21535 Type=slow timer Process:: ID=7 PID=21536 Type=timer Process:: ID=8 PID=21537 Type=MI FIFO Process:: ID=9 PID=21538 Type=ctl handler Process:: ID=10 PID=21539 Type=TIMER NH Process:: ID=11 PID=21540 Type=Dialog KA Timer Process:: ID=12 PID=21541 Type=Dialog Clean Timer Process:: ID=13 PID=21542 Type=WLCC TB TIMER Process:: ID=14 PID=21543 Type=LLCC TB TIMER Process:: ID=15 PID=21544 Type=tcp receiver (generic) child=0 Process:: ID=16 PID=21545 Type=tcp main process
And the log
May 7 18:54:43 vps-host /usr/sbin/kamailio[21532]: INFO: <script>: 5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060 - CC, dialog:start May 7 18:54:44 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:start May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - CC, dialog:end May 7 18:55:57 vps-host /usr/sbin/kamailio[21535]: INFO: <script>: 123 - WLCC: ENDCALL May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7fa76e546600 [869:4705] with clid '5fa4dc3670596fb626269f083b0c9480@192.168.10.245:5060' and tags 'as155a87b8' '95b0a07565f99244i0' May 7 18:55:57 vps-host /usr/sbin/kamailio[21531]: INFO: <script>: 4d263e35-89ea921c@172.16.0.205 - CC, dialog:end May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54a4a8/0x7fa76e54a4e0) - ignore May 7 18:56:17 vps-host /usr/sbin/kamailio[21540]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7fa76e54aeb8/0x7fa76e54aef0) - ignore
Is this enough, or do you need the full debug log? It's a bit heavy on the log because of continuous database lookups. But I could filter them down a bit if needed.
Cheers, Dirk Teurlings
On 07-05-15 13:31, Daniel-Constantin Mierla wrote:
Hello,
I couldn't correlate the log message for double free with the code for bye_reply_cb and dlg_ka_cb. The logs from pid 24493 shows only dialog lookups and that double free message.
Do you have the instance still running? Can you send the output of:
kamctl ps
Also, would be helpful to see some other logs messages printed just before by pid 24493.
By analyzing the code I found an improvement to be done, already committed in master via:
https://github.com/kamailio/kamailio/commit/04410aef38f5ed40c8f4a6796d956584...
It is about being sure the dialog is still in confirmed state when the keepalive callback is handled.
But I can't be sure yet if that was the case resulting in double free.
Cheers, Daniel
On 07/05/15 09:34, Dirk Teurlings - SIGNET B.V. wrote:
Hi again,
File dlg_req_within.c function bye_reply_cb line: 166-236 functino dlg_ka_cb line: 240-275
To help you out a bit more, this is important part of the log.
May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:1066]: next_state_dlg(): dialog 0x7f812f2585e8 changed from state 4 to state 5, due event 7 (ref 5) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:195]: bye_reply_cb(): removing dialog with h_entry 1231 and h_id 1142 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: WARNING: dialog [dlg_req_within.c:212]: bye_reply_cb(): inconsitent dlg timer data on dlg 0x7f812f2585e8 [1231:1142] with clid '6fcb9a946131aad41e1deda730c6b76a@192.168.10.245:5060' and tags 'as57b6a153' 'c42610a13600fe22i0' May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_cb.c:256]: run_dlg_callbacks(): dialog=0x7f812f2585e8, type=64 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_req_within.c:219]: bye_reply_cb(): first final reply May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_hash.c:872]: dlg_unref(): unref dlg 0x7f812f2585e8 with 2 -> 3 May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f2529f0) May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: tm [t_reply.c:1653]: cleanup_uac_timers(): DEBUG: cleanup_uac_timers: RETR/FR timers reset May 4 18:09:42 vps-host /usr/sbin/kamailio[24484]: DEBUG: dialog [dlg_var.c:81]: cb_dlg_locals_reset(): resetting the local dialog shortcuts on script callback: 2147483652
...
May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [xavp.c:448]: xavp_destroy_list(): destroying xavp list (nil) May 4 18:10:02 vps-host /usr/sbin/kamailio[24484]: DEBUG: <core> [receive.c:298]: receive_msg(): receive_msg: cleaning up May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:1294]: t_should_relay_response(): ->>>>>>>>> T_code=0, new_code=408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2080]: local_reply(): DEBUG: local_reply: branch=0, save=0, winner=0 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_reply.c:2117]: local_reply(): DEBUG: local transaction completed May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: tm [t_hooks.c:288]: run_trans_callbacks_internal(): DBG: trans=0x7f812f256fd0, callback type 1024, id 0 entered May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_req_within.c:255]: dlg_ka_cb(): receiving a final reply 408 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24488]: DEBUG: dialog [dlg_handlers.c:324]: dlg_iuid_sfree(): freeing dlg iuid [1231:1142] (0x7f812f252ba0) May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=1142 found on entry 1231 May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: INFO: <core> [mem/f_malloc.c:599]: fm_free(): freeing a free fragment (0x7f812f252ad0/0x7f812f252b08) - ignore May 4 18:10:02 vps-host /usr/sbin/kamailio[24493]: DEBUG: dialog [dlg_hash.c:663]: dlg_lookup(): no dialog id=11917 found on entry 68 etc...
Cheers, Dirk Teurlings
On 07-05-15 09:03, Daniel-Constantin Mierla wrote: > Hello, > > thanks for reporting and troubleshooting. > > Can you refer to the code (file+lines) for step 3 and 4? It will > save me > some time to look into the code and fix the issue. > > Cheers, > Daniel > > On 07/05/15 08:46, Dirk Teurlings - SIGNET B.V. wrote: >> Hi, >> >> (Kamailio 4.2.3) >> >> Working to get dialogs closed and cleaned up nicely I'm looking >> into >> the ka_interval and ka_timer parameter of the dialog module. The >> following is now happening. >> >> Scenario: >> 1. Call gets setup >> 2. Callee's internet connection drops >> 3. Call gets terminated (nicely) due to ka_timer/ka_interval >> 4. Log fills with "freeing a free fragment" after about 10 seconds >> >> Digging into this I saw that the dialog is being closed and >> cleaned >> properly in step 3. But it fails to remove/clean the timers >> used for >> the keep-alive. That's what triggers step 4, which in turn >> tries to >> clean the (or some of the) dialog space again. Is this expected >> behaviour when using the keep-alive options, or am I running into >> somekind of (known) bug? >> >> Thanks in advance for any help. >>
Hello,
Following up on this issue, though KA seems to work fine now. We're running into another problem. We use a fork of CNXCC that terminates all calls when the user doesn't have enough credit. But with these patches that doesn't work anymore and the current thread even locks up completely.
We've tracked it down to these changes:
diff --git a/modules/dialog/dlg_req_within.c b/modules/dialog/dlg_req_within.c index 206d13e..5668c3c 100644 --- a/modules/dialog/dlg_req_within.c +++ b/modules/dialog/dlg_req_within.c @@ -214,7 +214,7 @@ void bye_reply_cb(struct cell* t, int type, struct tmcb_params* ps){ unref++; } /* dialog terminated (BYE) */ - run_dlg_callbacks( DLGCB_TERMINATED, dlg, ps->req, ps->rpl, DLG_DIR_NONE, 0); + run_dlg_callbacks( DLGCB_TERMINATED_CONFIRMED, dlg, ps->req, ps->rpl, DLG_DIR_NONE, 0);
LM_DBG("first final reply\n"); /* derefering the dialog */ @@ -521,6 +525,9 @@ int dlg_bye_all(struct dlg_cell *dlg, str *hdrs) str all_hdrs = { 0, 0 }; int ret;
+ /* run dialog terminated callbacks */ + run_dlg_callbacks( DLGCB_TERMINATED, dlg, NULL, NULL, DLG_DIR_NONE, 0); + if ((build_extra_hdr(dlg, hdrs, &all_hdrs)) != 0) { LM_ERR("failed to build dlg headers\n");
This is a diff between 4.2.3 and 4.2.5, once reverted, everthing works fine.
Can you tell me the reason for these additions? What is the expected difference in behaviour?
Cheers, Dirk
On 12-05-15 10:49, Daniel-Constantin Mierla wrote:
Hello,
master is still the code to be 4.3, we will make a dedicated branch sometime soon.
The patch will be backported to 4.2 as well. Thanks for helping to troubleshoot and testing.
Cheers, Daniel
Hello,
the respective patch was done to fix generating CDRs on local BYE. Maybe cnxcc is locking slots as well, I am not familiar with the module at all.
Perhaps the best is to change the mutexes to be re-entrant, to avoid such conflicts between dialog and upper layer modules relying on it. I will push a patch for it.
Cheers, Daniel
On 16/06/15 13:59, Dirk Teurlings - SIGNET B.V. wrote:
Hello,
Following up on this issue, though KA seems to work fine now. We're running into another problem. We use a fork of CNXCC that terminates all calls when the user doesn't have enough credit. But with these patches that doesn't work anymore and the current thread even locks up completely.
We've tracked it down to these changes:
diff --git a/modules/dialog/dlg_req_within.c b/modules/dialog/dlg_req_within.c index 206d13e..5668c3c 100644 --- a/modules/dialog/dlg_req_within.c +++ b/modules/dialog/dlg_req_within.c @@ -214,7 +214,7 @@ void bye_reply_cb(struct cell* t, int type, struct tmcb_params* ps){ unref++; } /* dialog terminated (BYE) */
run_dlg_callbacks( DLGCB_TERMINATED, dlg, ps->req,ps->rpl, DLG_DIR_NONE, 0);
run_dlg_callbacks( DLGCB_TERMINATED_CONFIRMED, dlg,ps->req, ps->rpl, DLG_DIR_NONE, 0);
LM_DBG("first final reply\n"); /* derefering the dialog */@@ -521,6 +525,9 @@ int dlg_bye_all(struct dlg_cell *dlg, str *hdrs) str all_hdrs = { 0, 0 }; int ret;
/* run dialog terminated callbacks */run_dlg_callbacks( DLGCB_TERMINATED, dlg, NULL, NULL,DLG_DIR_NONE, 0);
if ((build_extra_hdr(dlg, hdrs, &all_hdrs)) != 0) { LM_ERR("failed to build dlg headers\n");This is a diff between 4.2.3 and 4.2.5, once reverted, everthing works fine.
Can you tell me the reason for these additions? What is the expected difference in behaviour?
Cheers, Dirk
On 12-05-15 10:49, Daniel-Constantin Mierla wrote:
Hello,
master is still the code to be 4.3, we will make a dedicated branch sometime soon.
The patch will be backported to 4.2 as well. Thanks for helping to troubleshoot and testing.
Cheers, Daniel
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
I just pushed the patch -- commit link is:
- https://github.com/kamailio/kamailio/commit/9c3ea838b31039ac067e17d519df67b6...
Testing and reporting the results will be really appreciated.
Cheers, Daniel
On 16/06/15 14:32, Daniel-Constantin Mierla wrote:
Hello,
the respective patch was done to fix generating CDRs on local BYE. Maybe cnxcc is locking slots as well, I am not familiar with the module at all.
Perhaps the best is to change the mutexes to be re-entrant, to avoid such conflicts between dialog and upper layer modules relying on it. I will push a patch for it.
Cheers, Daniel
On 16/06/15 13:59, Dirk Teurlings - SIGNET B.V. wrote:
Hello,
Following up on this issue, though KA seems to work fine now. We're running into another problem. We use a fork of CNXCC that terminates all calls when the user doesn't have enough credit. But with these patches that doesn't work anymore and the current thread even locks up completely.
We've tracked it down to these changes:
diff --git a/modules/dialog/dlg_req_within.c b/modules/dialog/dlg_req_within.c index 206d13e..5668c3c 100644 --- a/modules/dialog/dlg_req_within.c +++ b/modules/dialog/dlg_req_within.c @@ -214,7 +214,7 @@ void bye_reply_cb(struct cell* t, int type, struct tmcb_params* ps){ unref++; } /* dialog terminated (BYE) */
run_dlg_callbacks( DLGCB_TERMINATED, dlg, ps->req,ps->rpl, DLG_DIR_NONE, 0);
run_dlg_callbacks( DLGCB_TERMINATED_CONFIRMED, dlg,ps->req, ps->rpl, DLG_DIR_NONE, 0);
LM_DBG("first final reply\n"); /* derefering the dialog */@@ -521,6 +525,9 @@ int dlg_bye_all(struct dlg_cell *dlg, str *hdrs) str all_hdrs = { 0, 0 }; int ret;
/* run dialog terminated callbacks */run_dlg_callbacks( DLGCB_TERMINATED, dlg, NULL, NULL,DLG_DIR_NONE, 0);
if ((build_extra_hdr(dlg, hdrs, &all_hdrs)) != 0) { LM_ERR("failed to build dlg headers\n");This is a diff between 4.2.3 and 4.2.5, once reverted, everthing works fine.
Can you tell me the reason for these additions? What is the expected difference in behaviour?
Cheers, Dirk
On 12-05-15 10:49, Daniel-Constantin Mierla wrote:
Hello,
master is still the code to be 4.3, we will make a dedicated branch sometime soon.
The patch will be backported to 4.2 as well. Thanks for helping to troubleshoot and testing.
Cheers, Daniel
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi Daniel,
Just applied this patch on 4.2.5, debian wheezy stable from kamailio.org. Unfortunately it doesn't fix the problem.
Looking into CNXCC master code I noticed that there are quite a few changes now. I will try and backport CNXCC from master to 4.2.5 first before coming back to this issue.
Thanks, Dirk
On 16-06-15 15:34, Daniel-Constantin Mierla wrote:
I just pushed the patch -- commit link is:
https://github.com/kamailio/kamailio/commit/9c3ea838b31039ac067e17d519df67b6...
Testing and reporting the results will be really appreciated.
Cheers, Daniel
On 16/06/15 14:32, Daniel-Constantin Mierla wrote:
Hello,
the respective patch was done to fix generating CDRs on local BYE. Maybe cnxcc is locking slots as well, I am not familiar with the module at all.
Perhaps the best is to change the mutexes to be re-entrant, to avoid such conflicts between dialog and upper layer modules relying on it. I will push a patch for it.
Cheers, Daniel
On 16/06/15 13:59, Dirk Teurlings - SIGNET B.V. wrote:
Hello,
Following up on this issue, though KA seems to work fine now. We're running into another problem. We use a fork of CNXCC that terminates all calls when the user doesn't have enough credit. But with these patches that doesn't work anymore and the current thread even locks up completely.
We've tracked it down to these changes:
diff --git a/modules/dialog/dlg_req_within.c b/modules/dialog/dlg_req_within.c index 206d13e..5668c3c 100644 --- a/modules/dialog/dlg_req_within.c +++ b/modules/dialog/dlg_req_within.c @@ -214,7 +214,7 @@ void bye_reply_cb(struct cell* t, int type, struct tmcb_params* ps){ unref++; } /* dialog terminated (BYE) */
run_dlg_callbacks( DLGCB_TERMINATED, dlg, ps->req,ps->rpl, DLG_DIR_NONE, 0);
run_dlg_callbacks( DLGCB_TERMINATED_CONFIRMED, dlg,ps->req, ps->rpl, DLG_DIR_NONE, 0);
LM_DBG("first final reply\n"); /* derefering the dialog */@@ -521,6 +525,9 @@ int dlg_bye_all(struct dlg_cell *dlg, str *hdrs) str all_hdrs = { 0, 0 }; int ret;
/* run dialog terminated callbacks */run_dlg_callbacks( DLGCB_TERMINATED, dlg, NULL, NULL,DLG_DIR_NONE, 0);
if ((build_extra_hdr(dlg, hdrs, &all_hdrs)) != 0) { LM_ERR("failed to build dlg headers\n");This is a diff between 4.2.3 and 4.2.5, once reverted, everthing works fine.
Can you tell me the reason for these additions? What is the expected difference in behaviour?
Cheers, Dirk
On 12-05-15 10:49, Daniel-Constantin Mierla wrote:
Hello,
master is still the code to be 4.3, we will make a dedicated branch sometime soon.
The patch will be backported to 4.2 as well. Thanks for helping to troubleshoot and testing.
Cheers, Daniel
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-
Daniel-Constantin Mierla -
Dirk Teurlings - SIGNET B.V.