[Kamailio-Users] R: R: Fast lock loop with arm

Daniel-Constantin Mierla miconda at gmail.com
Mon Mar 29 11:00:48 CEST 2010


Hello,

On 3/29/10 8:53 AM, Zappasodi Daniele wrote:
>
> Hello,
>
> I have tried a lot of times with different processes, but backtrace 
> shows always only this.
>

this is really strange. do you use mi_fifo? if yes, when you start 
openser run 'openserctl ps'

Spot a udp worker process, the mi fifo process, the main process and the 
timer process. when the locking happens, attach to each of them and get 
the backtrace.

Thanks,
Daniel

> Daniele
>
>     -----Messaggio originale-----
>     *Da:* Daniel-Constantin Mierla [mailto:miconda at gmail.com]
>     *Inviato:* venerdì 26 marzo 2010 19.40
>     *A:* Zappasodi Daniele
>     *Cc:* users at lists.kamailio.org; sr-dev
>     *Oggetto:* Re: R: [Kamailio-Users] Fast lock loop with arm
>
>     Hello,
>
>     On 3/26/10 4:13 PM, Zappasodi Daniele wrote:
>>
>>     Hello,
>>
>>     this is what I get with gdb:
>>
>>     (gdb) bt full
>>
>>     #0 0x40137e54 in sched_yield () from /usr/local/lib2/libc.so.6
>>
>>     No symbol table info available.
>>
>>     I don't think that it can help, but I am not able to load the
>>     symbol table for openser on the server.
>>
>     hmm, strange. Did you try with many processes? Sam result in the
>     backtrace?
>
>     Cheers,
>     Daniel
>
>>     thanks,
>>
>>     Daniele
>>
>>         -----Messaggio originale-----
>>         *Da:* Daniel-Constantin Mierla [mailto:miconda at gmail.com]
>>         *Inviato:* mercoledì 24 marzo 2010 16.47
>>         *A:* Zappasodi Daniele
>>         *Cc:* users at lists.kamailio.org; sr-dev
>>         *Oggetto:* Re: [Kamailio-Users] Fast lock loop with arm
>>
>>         Hello,
>>
>>         what version of kamailio do you use?
>>
>>         Can you grab a backtrace with gdb once you get this high cpu
>>         usage? Spot one of the processes, take the pid and do:
>>
>>         gdb  /path/to/kamailio  _pid_
>>
>>         Then:
>>
>>         bt
>>
>>         You should see the bactrace of executed function getting to
>>         deadloc. Make sure you get at least one SIP worker process
>>         backtrace (do kamctl ps to see the type of kamailio process).
>>
>>         Cheers,
>>         Daniel
>>
>>         On 3/24/10 4:29 PM, Zappasodi Daniele wrote:
>>>
>>>         Hi,
>>>         I have a big problem with fast lock mutex on arm CPU:
>>>         sometimes I find one or more children that go in an infinite
>>>         loop, in the while loop of get_lock function.
>>>         They remain in Run for long time (sometimes hours) and
>>>         openser keeps 100% CPU.
>>>
>>>         Now I have changed the functions get_lock and tsl in order
>>>         to obtain more info and
>>>         I observe that  (if and) when the process finish the cycle,
>>>         it have done a big amount of cycles.
>>>
>>>         This the log with my added info:
>>>         Mar 26 20:29:08 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 1
>>>         (cycle)
>>>         Mar 26 20:29:51 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 1
>>>         (cycle)
>>>         Mar 26 20:29:55 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 1
>>>         (cycle)
>>>
>>>         Mar 26 20:29:55 SAM-IP openser[1645]:
>>>         NOTICE:tm:t_retransmit_reply: MYTM lock
>>>         [process in loop]
>>>         Mar 26 20:33:46 SAM-IP openser[1645]: NOTICE:tm:_lock: ret
>>>         59374917 (cycle)
>>>         [after 4 minutes and 59374917 cycles, this is an example
>>>         with a "short" loop]
>>>         Mar 26 20:33:46 SAM-IP openser[1645]:
>>>         NOTICE:tm:t_retransmit_reply: MYTM lock done
>>>
>>>         I'm not able to recognize the call flow that generates the
>>>         loop (I know only that it always happens when
>>>         t_retransmit_reply calls LOCK_REPLIES),
>>>
>>>         but I urgently need a work around to escape from the loop.
>>>
>>>         This is the get_lock function with my counter j:
>>>
>>>         inline static int get_lock(fl_lock_t* lock)
>>>         {
>>>         #ifdef ADAPTIVE_WAIT
>>>         int i=ADAPTIVE_WAIT_LOOPS;
>>>         int j=1;                                        /***** my
>>>         change ****/
>>>         #endif
>>>
>>>         while(tsl(lock)){
>>>         #ifdef BUSY_WAIT
>>>         #elif defined ADAPTIVE_WAIT
>>>         j++;                            /***** my change ****/
>>>         if (i>0) i--;
>>>         else sched_yield();
>>>         #else
>>>         sched_yield();
>>>         #endif
>>>         }
>>>         return(j);
>>>         }
>>>
>>>         Can I break the lock when my counter reaches a big value?
>>>         Should I call the unlock function after the break?
>>>         which value can be considered too big?
>>>
>>>         Thanks,
>>>         Daniele
>>>
>>>
>>>
>>>         **********************************************************************
>>>         The information in this message is confidential and may be
>>>         legally privileged. It is intended solely for the addressee.
>>>         Access to this message by anyone else is unauthorized. If
>>>         you are not the intended recipient, any disclosure, copying,
>>>         or distribution of the message, or any action or omission
>>>         taken by you in reliance on it, is prohibited and may be
>>>         unlawful. Please immediately contact the sender if you have
>>>         received this message in error.
>>>         **********************************************************************
>>>
>>>
>>>         _______________________________________________
>>>         Kamailio (OpenSER) - Users mailing list
>>>         Users at lists.kamailio.org
>>>         http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
>>>         http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
>>
>>         -- 
>>         Daniel-Constantin Mierla * http://www.asipto.com/
>>
>>     **********************************************************************
>>     The information in this message is confidential and may be
>>     legally privileged. It is intended solely for the addressee.
>>     Access to this message by anyone else is unauthorized. If you are
>>     not the intended recipient, any disclosure, copying, or
>>     distribution of the message, or any action or omission taken by
>>     you in reliance on it, is prohibited and may be unlawful. Please
>>     immediately contact the sender if you have received this message
>>     in error.
>>     **********************************************************************
>>
>
>     -- 
>     Daniel-Constantin Mierla * http://www.asipto.com/
>
> ********************************************************************** 
> The information in this message is confidential and may be legally 
> privileged. It is intended solely for the addressee. Access to this 
> message by anyone else is unauthorized. If you are not the intended 
> recipient, any disclosure, copying, or distribution of the message, or 
> any action or omission taken by you in reliance on it, is prohibited 
> and may be unlawful. Please immediately contact the sender if you have 
> received this message in error. 
> **********************************************************************
>
>
> _______________________________________________
> Kamailio (OpenSER) - Users mailing list
> Users at lists.kamailio.org
> http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
> http://lists.openser-project.org/cgi-bin/mailman/listinfo/users

-- 
Daniel-Constantin Mierla * http://www.asipto.com/ * 
http://twitter.com/miconda * 
http://www.linkedin.com/in/danielconstantinmierla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/users/attachments/20100329/69304efe/attachment.htm>


More information about the Users mailing list