Hello,

On 02.06.21 19:00, Trevor Hemsley wrote:
Hi

We've seen a few occurrences of this crash since we implemented kamailio 5.4.5 for our inbound call traffic.

to clarify: have you made an update to kamailio 5.4.5 from an older version and since then you started to have such crashes? If yes, what was the previous version? I quickly checked and there were no major changes to dialog module in 5.4 branch.

If you started with this version, is it under heavy load? Have you captured the SIP traffic? It would be useful to see the sip messages for such call.


Preceding the crash we always get a message in the logs about "dialog [dlg_hash.c:1182]: next_state_dlg(): bogus event 2 in state 5 for dlg" and whenever we see that message, we get the crash.

JunĀ  1 10:12:14 thissystem /usr/sbin/kamailio[20001]: CRITICAL: {2 1 INVITE 6787142-3831531134-1330894187@some.telco.domain} dialog [dlg_hash.c:1182]: next_state_dlg(): bogus event 2 in state 5 for dlg 0x7fae153d3cf0 [3973:6059] with clid '82608924-3831089984-1833452161@some.telco.domain' and tags '3831089984-684203260' ''

The crash happens on line 879 of https://github.com/kamailio/kamailio/blob/master/src/modules/dialog/dlg_db_handler.c

You reference to code in master branch, not in 5.4 branch, so unless you did the remapping, the lines might not be the same.

If you have the core dump files, the best is to get the output of the next gdb commands:

"""
bt full

list

info locals
""

You can eventually open an issue in the bug tracker to collect the details there and assist with troubleshooting.

Cheers,
Daniel




SET_STR_VALUE(values+7, cell->bind_addr[DLG_CALLER_LEG]->sock_str);

Looking at gdb, I see that cell->bind_addr[DLG_CALLER_LEG] == 0 so trying to use it results in the segfault. I have thought about patching that to check for bind_addr being set and bypassing that line if not but I do not think it's the right fix though it probably would avoid the current crash (and do odd things later instead!). I did check that everything else in that function that uses DLG_CALLER_LEG is set correctly and it appears that it is - or at least if not correctly then they are at least set!

The "bogus event 2 in state 5" message is coming out from line 1143 in src/modules/ims_dialog/dlg_hash.c

        case DLG_EVENT_RPL1xx:
            switch (dlg->state) {
                case DLG_STATE_UNCONFIRMED:
                case DLG_STATE_EARLY:
                    dlg->state = DLG_STATE_EARLY;
                    break;
                default:
                    log_next_state_dlg(event, dlg);
            }
            break;

(I didn't really understand why it would set dlg->state = DLG_STATE_EARLY if it's already DLG_STATE_EARLY but...)

I have a couple of core dumps of this from the last day or two - usually seems to crash at least once a week on our pair of kamailio inbound servers. If there's any more info I can provide to help fix this properly rather than my proposed bodge then feel free to ask.

Trevor Hemsley


Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here.


__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
  * sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
  * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- 
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - Online - June 7-10, 2021 (America Timezone)
  * https://www.asipto.com/sw/kamailio-advanced-training-online/