<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE></TITLE>
<META content="MSHTML 6.00.6000.16981" name=GENERATOR></HEAD>
<BODY text=#000000 bgColor=#ffffff>
<DIV>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB
style="mso-ansi-language: EN-GB">Hello,<?xml:namespace prefix = o ns =
"urn:schemas-microsoft-com:office:office" /><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB
style="mso-ansi-language: EN-GB">I have tried a lot of times with different
processes, but backtrace shows always only this.<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB
style="mso-ansi-language: EN-GB"> <o:p></o:p></SPAN></P><SPAN lang=EN-GB
style="FONT-SIZE: 12pt; FONT-FAMILY: 'Times New Roman'; mso-ansi-language: EN-GB; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: IT; mso-bidi-language: AR-SA">Daniele</SPAN></DIV>
<BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader dir=ltr align=left><FONT face=Tahoma
size=2>-----Messaggio originale-----<BR><B>Da:</B> Daniel-Constantin Mierla
[mailto:miconda@gmail.com]<BR><B>Inviato:</B> venerdì 26 marzo 2010
19.40<BR><B>A:</B> Zappasodi Daniele<BR><B>Cc:</B> users@lists.kamailio.org;
sr-dev<BR><B>Oggetto:</B> Re: R: [Kamailio-Users] Fast lock loop with
arm<BR><BR></FONT></DIV>Hello,<BR><BR>On 3/26/10 4:13 PM, Zappasodi Daniele
wrote:
<BLOCKQUOTE
cite=mid:14A9059A37EF9A45BC2520DE64C30F60075256@slttex002.seltatel.local
type="cite">
<META content="MSHTML 6.00.6000.16981" name=GENERATOR>
<DIV>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN
lang=EN-GB>Hello,<O:P></O:P></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB>this is what
I get with gdb:<O:P></O:P></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN
lang=EN-GB><O:P></O:P></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB>(gdb) bt
full<O:P></O:P></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN
lang=EN-GB>#0<SPAN> </SPAN>0x40137e54 in sched_yield () from
/usr/local/lib2/libc.so.6<O:P></O:P></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB>No symbol
table info available.<O:P></O:P></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN
lang=EN-GB><O:P></O:P></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB>I don’t
think that it can help, but I am not able to load the symbol table for
openser<SPAN class=292020015-26032010> on the
server</SPAN>.</SPAN></P></DIV></BLOCKQUOTE>hmm, strange. Did you try with
many processes? Sam result in the backtrace?<BR><BR>Cheers,<BR>Daniel<BR><BR>
<BLOCKQUOTE
cite=mid:14A9059A37EF9A45BC2520DE64C30F60075256@slttex002.seltatel.local
type="cite">
<DIV>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN
lang=EN-GB><O:P></O:P></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><SPAN
class=292020015-26032010></SPAN></SPAN><SPAN lang=EN-GB></SPAN> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><SPAN
class=292020015-26032010><FONT face=Arial color=#0000ff
size=2>thanks,</FONT></SPAN></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><O:P><SPAN
class=292020015-26032010><FONT face=Arial color=#0000ff
size=2>Daniele</FONT></SPAN></O:P></SPAN></P></DIV>
<BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader dir=ltr align=left><FONT face=Tahoma
size=2>-----Messaggio originale-----<BR><B>Da:</B> Daniel-Constantin
Mierla [<A class=moz-txt-link-freetext
href="mailto:miconda@gmail.com">mailto:miconda@gmail.com</A>]<BR><B>Inviato:</B>
mercoledì 24 marzo 2010 16.47<BR><B>A:</B> Zappasodi Daniele<BR><B>Cc:</B>
<A class=moz-txt-link-abbreviated
href="mailto:users@lists.kamailio.org">users@lists.kamailio.org</A>;
sr-dev<BR><B>Oggetto:</B> Re: [Kamailio-Users] Fast lock loop with
arm<BR><BR></FONT></DIV>Hello,<BR><BR>what version of kamailio do you
use?<BR><BR>Can you grab a backtrace with gdb once you get this high cpu
usage? Spot one of the processes, take the pid and do:<BR><BR>gdb
/path/to/kamailio _pid_<BR><BR>Then:<BR><BR>bt<BR><BR>You should see
the bactrace of executed function getting to deadloc. Make sure you get at
least one SIP worker process backtrace (do kamctl ps to see the type of
kamailio process).<BR><BR>Cheers,<BR>Daniel<BR><BR>On 3/24/10 4:29 PM,
Zappasodi Daniele wrote:
<BLOCKQUOTE
cite=mid:14A9059A37EF9A45BC2520DE64C30F60075252@slttex002.seltatel.local
type="cite">
<META content="MS Exchange Server version 6.5.7654.12" name=Generator><!-- Converted from text/rtf format -->
<P><SPAN lang=en-gb><FONT face="Times New Roman">Hi,</FONT></SPAN>
<BR><SPAN lang=en-gb><FONT face="Times New Roman">I have a big problem
with fast lock mutex on arm CPU: </FONT></SPAN><BR><SPAN
lang=en-gb><FONT face="Times New Roman">sometimes I find one or more
children that go in an infinite loop, in the while loop of get_lock
function. </FONT></SPAN><BR><SPAN lang=en-gb><FONT
face="Times New Roman">They remain in Run for long time (sometimes
hours) and openser keeps 100% CPU.</FONT></SPAN> </P>
<P><SPAN lang=en-gb><FONT face="Times New Roman">Now I have changed the
functions get_lock and tsl in order to obtain more info and
</FONT></SPAN><BR><SPAN lang=en-gb><FONT face="Times New Roman">I
observe that (if and) when the process finish the cycle, it have
done a big amount of cycles. </FONT></SPAN></P>
<P><SPAN lang=en-gb><FONT face="Times New Roman">This the log with my
added info:</FONT></SPAN> <BR><SPAN lang=en-gb><FONT
face="Times New Roman">Mar 26 20:29:08 SAM-IP openser[1645]:
NOTICE:tm:_lock: ret 1 (cycle)</FONT></SPAN> <BR><SPAN lang=en-gb><FONT
face="Times New Roman">Mar 26 20:29:51 SAM-IP openser[1645]:
NOTICE:tm:_lock: ret 1 (cycle)</FONT></SPAN> <BR><SPAN lang=en-gb><FONT
face="Times New Roman">Mar 26 20:29:55 SAM-IP openser[1645]:
NOTICE:tm:_lock: ret 1 (cycle)</FONT></SPAN> </P>
<P><SPAN lang=en-gb><FONT face="Times New Roman">Mar 26 20:29:55 SAM-IP
openser[1645]: NOTICE:tm:t_retransmit_reply: MYTM lock
</FONT></SPAN><BR><SPAN lang=en-gb><FONT face="Times New Roman">[process
in loop]</FONT></SPAN> <BR><SPAN lang=en-gb><FONT
face="Times New Roman">Mar 26 20:33:46 SAM-IP openser[1645]:
NOTICE:tm:_lock: ret 59374917 (cycle)</FONT></SPAN> <BR><SPAN
lang=en-gb><FONT face="Times New Roman">[after 4 minutes and 59374917
cycles, this is an example with a "short" loop]</FONT></SPAN> <BR><SPAN
lang=en-gb><FONT face="Times New Roman">Mar 26 20:33:46 SAM-IP
openser[1645]: NOTICE:tm:t_retransmit_reply: MYTM lock done
</FONT></SPAN></P>
<P><SPAN lang=en-gb><FONT face="Times New Roman">I’m not able to
recognize the call flow that generates the loop (I know only that it
always happens when t_retransmit_reply calls LOCK_REPLIES),
</FONT></SPAN></P>
<P><SPAN lang=en-gb><FONT face="Times New Roman">but I urgently need a
work around to escape from the loop.</FONT></SPAN> </P>
<P><SPAN lang=en-gb><FONT face="Times New Roman">This is the get_lock
function with my counter j:</FONT></SPAN> </P>
<P><SPAN lang=en-gb><FONT face="Times New Roman">inline static int
get_lock(fl_lock_t* lock)</FONT></SPAN> <BR><SPAN lang=en-gb><FONT
face="Times New Roman">{</FONT></SPAN> <BR><SPAN lang=en-gb><FONT
face="Times New Roman">#ifdef ADAPTIVE_WAIT</FONT></SPAN> <BR><SPAN
lang=en-gb> <FONT
face="Times New Roman">int i=ADAPTIVE_WAIT_LOOPS;</FONT></SPAN>
<BR><SPAN lang=en-gb> <FONT
face="Times New Roman">int
j=1;
/***** my change
****/</FONT></SPAN> <BR><SPAN lang=en-gb><FONT
face="Times New Roman">#endif</FONT></SPAN> <BR><SPAN
lang=en-gb> </SPAN><BR><SPAN
lang=en-gb> <FONT
face="Times New Roman">while(tsl(lock)){</FONT></SPAN> <BR><SPAN
lang=en-gb><FONT face="Times New Roman">#ifdef BUSY_WAIT</FONT></SPAN>
<BR><SPAN lang=en-gb><FONT face="Times New Roman">#elif defined
ADAPTIVE_WAIT</FONT></SPAN> <BR><SPAN
lang=en-gb>
<FONT
face="Times New Roman">j++;
/***** my change
****/</FONT></SPAN> <BR><SPAN
lang=en-gb>
<FONT
face="Times New Roman">if (i>0) i--;</FONT></SPAN> <BR><SPAN
lang=en-gb>
<FONT
face="Times New Roman">else sched_yield();</FONT></SPAN> <BR><SPAN
lang=en-gb><FONT face="Times New Roman">#else</FONT></SPAN> <BR><SPAN
lang=en-gb>
<FONT
face="Times New Roman">sched_yield();</FONT></SPAN> <BR><SPAN
lang=en-gb><FONT face="Times New Roman">#endif</FONT></SPAN> <BR><SPAN
lang=en-gb> <FONT
face="Times New Roman">}</FONT></SPAN> <BR><SPAN
lang=en-gb> <FONT
face="Times New Roman">return(j);</FONT></SPAN> <BR><SPAN
lang=en-gb><FONT face="Times New Roman">}</FONT></SPAN> </P>
<P><SPAN lang=en-gb><FONT face="Times New Roman">Can I break the lock
when my counter reaches a big value? </FONT></SPAN><BR><SPAN
lang=en-gb><FONT face="Times New Roman">Should I call the unlock
function after the break?</FONT></SPAN> <BR><SPAN lang=en-gb><FONT
face="Times New Roman">which value can be considered too
big?</FONT></SPAN> </P>
<P><SPAN lang=en-gb><FONT face="Times New Roman">Thanks,</FONT></SPAN>
<BR><SPAN lang=en-gb><FONT face="Times New Roman">Daniele
</FONT></SPAN></P><BR><BR>
<P>**********************************************************************
The information in this message is confidential and may be legally
privileged. It is intended solely for the addressee. Access to this
message by anyone else is unauthorized. If you are not the intended
recipient, any disclosure, copying, or distribution of the message, or
any action or omission taken by you in reliance on it, is prohibited and
may be unlawful. Please immediately contact the sender if you have
received this message in error.
**********************************************************************</P><PRE wrap=""><FIELDSET class=mimeAttachmentHeader></FIELDSET>
_______________________________________________
Kamailio (OpenSER) - Users mailing list
<A class=moz-txt-link-abbreviated href="mailto:Users@lists.kamailio.org" moz-do-not-send="true">Users@lists.kamailio.org</A>
<A class=moz-txt-link-freetext href="http://lists.kamailio.org/cgi-bin/mailman/listinfo/users" moz-do-not-send="true">http://lists.kamailio.org/cgi-bin/mailman/listinfo/users</A>
<A class=moz-txt-link-freetext href="http://lists.openser-project.org/cgi-bin/mailman/listinfo/users" moz-do-not-send="true">http://lists.openser-project.org/cgi-bin/mailman/listinfo/users</A></PRE></BLOCKQUOTE><BR>
<DIV class=moz-signature>-- <BR>Daniel-Constantin Mierla * <A
class=moz-txt-link-freetext href="http://www.asipto.com/"
moz-do-not-send="true">http://www.asipto.com/</A> </DIV></BLOCKQUOTE>
<P>**********************************************************************
The information in this message is confidential and may be legally
privileged. It is intended solely for the addressee. Access to this message
by anyone else is unauthorized. If you are not the intended recipient, any
disclosure, copying, or distribution of the message, or any action or
omission taken by you in reliance on it, is prohibited and may be unlawful.
Please immediately contact the sender if you have received this message in
error.
**********************************************************************</P></BLOCKQUOTE><BR>
<DIV class=moz-signature>-- <BR>Daniel-Constantin Mierla * <A
class=moz-txt-link-freetext
href="http://www.asipto.com/">http://www.asipto.com/</A>
</DIV></BLOCKQUOTE><p>**********************************************************************
The information in this message is confidential and may be legally
privileged. It is intended solely for the addressee. Access to this
message
by anyone else is unauthorized. If you are not the intended
recipient, any
disclosure, copying, or distribution of the message, or any action
or
omission taken by you in reliance on it, is prohibited and may be
unlawful.
Please immediately contact the sender if you have received this
message in error.
**********************************************************************</p>
</BODY></HTML>