<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Hello,</p>
    <p>that's strange, but a while ago someone else reported an issue
      with same backtrace.</p>
    <p>So the crash happens at the last line in the next snippet from
      reply_received() function in the tm module:</p>
    <p>    uac=&t->uac[branch];<br>
          LM_DBG("org. status uas=%d, uac[%d]=%d local=%d
      is_invite=%d)\n",<br>
              t->uas.status, branch, uac->last_received,<br>
              is_local(t), is_invite(t));<br>
          last_uac_status=uac->last_received;<br>
      <br>
      The backtrace and info locals say that uac is null (0x0).
      According to my knowledge, the address of a field in a structure
      cannot be null and uac is set to &t->uac[branch]. Moreover,
      uac->last_received is printed in the LM_DBG() above the line of
      crash, if uac was 0x0, the crash should have happened there.</p>
    <p>Then uac is a local variable, so it is on the stack of the
      process, in its private memory. There is no other assign or copy
      operation between the line of code where the uac is set and the
      crash. So overall, should be no race condition there. Either the
      kernel was doing something wrong, or maybe the coredump was
      somehow corrupted.<br>
    </p>
    <p>What was the value of debug level you had during the crash (debug
      parameter in kamailio.cfg)?</p>
    <p>Could there have been any freeze of the operating system for long
      time and then a resume?</p>
    <p>Can you give the output of command:</p>
    <p>uname -a</p>
    <p>What kind of linux distro and version you are running?</p>
    <p>Cheers,<br>
      Daniel<br>
    </p>
    <div class="moz-cite-prefix">On 25.02.19 15:27, Abdoul Osséni wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CABc7nxs=-UzVwU3qOZ8gEsKsYwPkkGM_waBkQeuwWtBGpffJxg@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div dir="ltr">
          <div>Hello,</div>
          <div><br>
          </div>
          <div>Please see attached the output of the
            gdb commands.</div>
          <div><br>
          </div>
          <div>
            <p>Can you check with all core files and see if the
              backtrace is the same?</p>
            <p>--> Yes the backtrace is the same.</p>
            <p>Sorry, I use kamailio v5.2<br>
            </p>
            <p>root@sbc:/var/cores# kamailio -V<br>
              version: kamailio 5.2.1 (x86_64/linux) cd2583<br>
              flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS,
              USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK,
              SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC,
              TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX,
              FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER,
              USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES<br>
              ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144
              MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB<br>
              poll method support: poll, epoll_lt, epoll_et, sigio_rt,
              select.<br>
              id: cd2583<br>
              compiled on 07:33:25 Jan 31 2019 with gcc 4.9.2<br>
            </p>
            <p><br>
            </p>
            <p>Kamailio is running on a bare metal server. <br>
            </p>
            <p><br>
            </p>
          </div>
          <div>Thanks</div>
          <div><br>
          </div>
          <div dir="ltr">
            <div>
              <div dir="ltr" class="gmail_signature">
                <div dir="ltr">
                  <div>
                    <div dir="ltr">
                      <div>
                        <div dir="ltr">
                          <div><span style="font-size:12.8px">Abdoul <br>
                            </span></div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
            <br>
          </div>
          <br>
          <div class="gmail_quote">
            <div dir="ltr" class="gmail_attr">Le lun. 25 févr. 2019
              à 14:40, Daniel-Constantin Mierla <<a
                href="mailto:miconda@gmail.com" moz-do-not-send="true">miconda@gmail.com</a>>
              a écrit :<br>
            </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px
              0.8ex;border-left:1px solid
              rgb(204,204,204);padding-left:1ex">
              <div bgcolor="#FFFFFF">
                <p>Hello,</p>
                <p>can you give the output for next gdb commands:</p>
                <p>bt full</p>
                <p>info locals</p>
                <p>list</p>
                <p>Can you check with all core files and see if the
                  backtrace is the same?</p>
                <p>What is the version of Kamailio? Is it running on a
                  bare metal server or a virtual machine/container?</p>
                <p>Cheers,<br>
                  Daniel<br>
                </p>
                <div class="gmail-m_4771917602702595822moz-cite-prefix">On
                  25.02.19 14:21, Abdoul Osséni wrote:<br>
                </div>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div>Hello,</div>
                      <div><br>
                      </div>
                      <div>Hello dear list,<br>
                        <br>
                        Today, I have had mutiples crashes. It seems it
                        linked to tm.so module.<br>
                        <br>
                        -rw------- 1 root kamailio 4299702272 Feb 25
                        13:08 core.kamailio.sig11.29204<br>
                        -rw------- 1 root kamailio 1453023232 Feb 25
                        13:12 core.kamailio.sig11.29203<br>
                        -rw------- 1 root kamailio 1416065024 Feb 25
                        13:12 core.kamailio.sig11.29207<br>
                        -rw------- 1 root kamailio 4299681792 Feb 25
                        13:16 core.kamailio.sig11.19047<br>
                        -rw------- 1 root kamailio 2108506112 Feb 25
                        13:20 core.kamailio.sig11.19043<br>
                        -rw------- 1 root kamailio 4299689984 Feb 25
                        13:34 core.kamailio.sig11.19247<br>
                        -rw------- 1 root kamailio 4299681792 Feb 25
                        13:34 core.kamailio.sig11.19246<br>
                        -rw------- 1 root kamailio 4299698176 Feb 25
                        13:35 core.kamailio.sig11.19248<br>
                        -rw------- 1 root kamailio 4299689984 Feb 25
                        13:35 core.kamailio.sig11.19243<br>
                        -rw------- 1 root kamailio 4299685888 Feb 25
                        13:35 core.kamailio.sig11.19244<br>
                        -rw------- 1 root kamailio 4299689984 Feb 25
                        13:36 core.kamailio.sig11.19242<br>
                        <br>
                        root@sbc:/var/cores# gdb
                        /usr/local/sbin/kamailio
                        core.kamailio.sig11.29204<br>
                        GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1<br>
                        Copyright (C) 2014 Free Software Foundation,
                        Inc.<br>
                        License GPLv3+: GNU GPL version 3 or later <<a
                          href="http://gnu.org/licenses/gpl.html"
                          target="_blank" moz-do-not-send="true">http://gnu.org/licenses/gpl.html</a>><br>
                        This is free software: you are free to change
                        and redistribute it.<br>
                        There is NO WARRANTY, to the extent permitted by
                        law.  Type "show copying"<br>
                        and "show warranty" for details.<br>
                        This GDB was configured as "x86_64-linux-gnu".<br>
                        Type "show configuration" for configuration
                        details.<br>
                        For bug reporting instructions, please see:<br>
                        <<a
                          href="http://www.gnu.org/software/gdb/bugs/"
                          target="_blank" moz-do-not-send="true">http://www.gnu.org/software/gdb/bugs/</a>>.<br>
                        Find the GDB manual and other documentation
                        resources online at:<br>
                        <<a
                          href="http://www.gnu.org/software/gdb/documentation/"
                          target="_blank" moz-do-not-send="true">http://www.gnu.org/software/gdb/documentation/</a>>.<br>
                        For help, type "help".<br>
                        Type "apropos word" to search for commands
                        related to "word"...<br>
                        Reading symbols from
                        /usr/local/sbin/kamailio...done.<br>
                        [New LWP 29204]<br>
                        <br>
                        warning: .dynamic section for
                        "/lib/x86_64-linux-gnu/libpthread.so.0" is not
                        at the expected address (wrong library or
                        version mismatch?)<br>
                        Warning: couldn't activate thread debugging
                        using libthread_db: Cannot find new threads:
                        generic error<br>
                        <br>
                        warning: File "/lib/x86_64-linux-gnu/<a
                          href="http://libthread_db-1.0.so"
                          target="_blank" moz-do-not-send="true">libthread_db-1.0.so</a>"
                        auto-loading has been declined by your
                        `auto-load safe-path' set to
                        "$debugdir:$datadir/auto-load".<br>
                        To enable execution of this file add<br>
                                add-auto-load-safe-path
                        /lib/x86_64-linux-gnu/<a
                          href="http://libthread_db-1.0.so"
                          target="_blank" moz-do-not-send="true">libthread_db-1.0.so</a><br>
                        line to your configuration file
                        "/root/.gdbinit".<br>
                        To completely disable this security protection
                        add<br>
                                set auto-load safe-path /<br>
                        line to your configuration file
                        "/root/.gdbinit".<br>
                        For more information about this security
                        protection see the<br>
                        "Auto-loading safe path" section in the GDB
                        manual.  E.g., run from the shell:<br>
                                info "(gdb)Auto-loading safe path"<br>
                        <br>
                        warning: Unable to find libthread_db matching
                        inferior's thread library, thread debugging will
                        not be available.<br>
                        Warning: couldn't activate thread debugging
                        using libthread_db: Cannot find new threads:
                        generic error<br>
                        <br>
                        warning: File "/lib/x86_64-linux-gnu/<a
                          href="http://libthread_db-1.0.so"
                          target="_blank" moz-do-not-send="true">libthread_db-1.0.so</a>"
                        auto-loading has been declined by your
                        `auto-load safe-path' set to
                        "$debugdir:$datadir/auto-load".<br>
                        <br>
                        warning: Unable to find libthread_db matching
                        inferior's thread library, thread debugging will
                        not be available.<br>
                        Core was generated by `/usr/local/sbin/kamailio
                        -P /var/run/kamailio/kamailio.pid -f
                        /usr/local/etc/ka'.<br>
                        Program terminated with signal SIGSEGV,
                        Segmentation fault.<br>
                        #0  0x00007f68b9bb4515 in reply_received
                        (p_msg=0x7f693ccd5cc8) at t_reply.c:2240<br>
                        2240           
                        last_uac_status=uac->last_received;<br>
                        (gdb)<br>
                        <br>
                        <br>
                        Any ideas?<br>
                        <br>
                        Regards<br>
                        <br>
                      </div>
                      <div>
                        <div>
                          <div dir="ltr"
                            class="gmail-m_4771917602702595822gmail_signature">
                            <div dir="ltr">
                              <div>
                                <div dir="ltr">
                                  <div>
                                    <div dir="ltr">
                                      <div><span
                                          style="font-size:12.8px">Abdoul<br>
                                        </span></div>
                                    </div>
                                  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                  <br>
                  <fieldset
                    class="gmail-m_4771917602702595822mimeAttachmentHeader"></fieldset>
                  <pre class="gmail-m_4771917602702595822moz-quote-pre">_______________________________________________
Kamailio (SER) - Users Mailing List
<a class="gmail-m_4771917602702595822moz-txt-link-abbreviated" href="mailto:sr-users@lists.kamailio.org" target="_blank" moz-do-not-send="true">sr-users@lists.kamailio.org</a>
<a class="gmail-m_4771917602702595822moz-txt-link-freetext" href="https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users" target="_blank" moz-do-not-send="true">https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users</a>
</pre>
                </blockquote>
                <pre class="gmail-m_4771917602702595822moz-signature" cols="72">-- 
Daniel-Constantin Mierla -- <a class="gmail-m_4771917602702595822moz-txt-link-abbreviated" href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a>
<a class="gmail-m_4771917602702595822moz-txt-link-abbreviated" href="http://www.twitter.com/miconda" target="_blank" moz-do-not-send="true">www.twitter.com/miconda</a> -- <a class="gmail-m_4771917602702595822moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda" target="_blank" moz-do-not-send="true">www.linkedin.com/in/miconda</a>
Kamailio World Conference - May 6-8, 2019 -- <a class="gmail-m_4771917602702595822moz-txt-link-abbreviated" href="http://www.kamailioworld.com" target="_blank" moz-do-not-send="true">www.kamailioworld.com</a>
Kamailio Advanced Training - Mar 4-6, 2019 in Berlin; Mar 25-27, 2019, in Washington, DC, USA -- <a class="gmail-m_4771917602702595822moz-txt-link-abbreviated" href="http://www.asipto.com" target="_blank" moz-do-not-send="true">www.asipto.com</a></pre>
              </div>
            </blockquote>
          </div>
        </div>
      </div>
    </blockquote>
    <pre class="moz-signature" cols="72">-- 
Daniel-Constantin Mierla -- <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/miconda">www.twitter.com/miconda</a> -- <a class="moz-txt-link-abbreviated" href="http://www.linkedin.com/in/miconda">www.linkedin.com/in/miconda</a>
Kamailio World Conference - May 6-8, 2019 -- <a class="moz-txt-link-abbreviated" href="http://www.kamailioworld.com">www.kamailioworld.com</a>
Kamailio Advanced Training - Mar 4-6, 2019 in Berlin; Mar 25-27, 2019, in Washington, DC, USA -- <a class="moz-txt-link-abbreviated" href="http://www.asipto.com">www.asipto.com</a></pre>
  </body>
</html>