[Devel] openser hangs

[Klaus Darilion]

Hi Bogdan!

Meanwhile I restarted openser.

Bogdan-Andrei Iancu wrote:
> which process hangs? is the TCP listener or one of the UDP listeners?

How will I know? Which of the treads are udp/TCP?

I start with 4 childrens, which lead to 12 threads.

If I order the threads by process-id, I would guess:

1. thread:     opens the sockets, does nothing
2. thread:     fifo
3.-6. thread:  UDP processing
7. thread:     something for 3-6 ?
8.-11. thread: TCP processing
12. thread:    something for TCP?

Is this correct? Is it safe to identify the threads by ordering by 
process-id?

If above is correct, the UDP processing was hanging.

Here is the strace of an UDP thread while openser was hanging:
philipp:/export/log# strace -p5243
Process 5243 attached - interrupt to quit
sched_yield()                           = 0
sched_yield()                           = 0
sched_yield()                           = 0
...and so on...

Here is the strace of an UDP thread while openser works fine:
philipp:/export/log# strace -p21210
Process 21210 attached - interrupt to quit
recvfrom(7,
...waiting for an UDP paccket ...

Thus, I guess there were problems with UDP processing.


Maybe it will be easier if you will test yourself. Are you familiar with 
the protos testsuite? Just download it and fire it against one of your 
proxies and wait till they are crashing ;-)

regards
klaus


> 
> can you describe the test if possible? also a full debug output will be 
> helpful..
> 
> so, please send me (privately to avoid bloating the list):
>     ps output
>     netstat output
>     openser full debug output
> 
> try also to kill -SIGSEGV the hanged process in order to get a core and 
> backtrace (if you are not able to attach via gdb to it)
> 
> 
> regards,
> bogdan
> 
> PS: it will be difficult for me to set the scenario?
> 
> 
> Klaus Darilion wrote:
> 
>> Bogdan-Andrei Iancu wrote:
>>
>>> Hi Klaus,
>>>
>>> are you running in fork mode? - asking since you are referring to 
>>> only one process listening on the network.
>>
>>
>>
>> Yes, fork mode. Started via the init.d script.
>>
>> I just found out, that it is another thread that receives from the 
>> sockets:
>>
>> philipp:/export/log# netstat -anp|grep 5060|grep openser
>> tcp        0      0 83.136.32.83:5060       0.0.0.0:* LISTEN     
>> 5239/openser
>> udp   129360      0 83.136.32.83:5060       0.0.0.0:*        5239/openser
>>
>>> are you able to attach to it with gdb and get a backtrace?
>>
>>
>>
>> which thread?
>> philipp:/export/log# gdb 5237
>> (gdb) bt
>> No stack.
>> (gdb) file /usr/sbin/openser
>> Reading symbols from /usr/sbin/openser...done.
>> Using host libthread_db library "/lib/tls/libthread_db.so.1".
>> (gdb) bt
>> No stack.
>> (gdb)
>>
>>> what kind of tests were you running? stress or conformity?
>>
>>
>>
>> conformity
>>
>> regards
>> klaus
>>
>>>
>>> regards,
>>> bogdan
>>>
>>> Klaus Darilion wrote:
>>>
>>>> Hi!
>>>>
>>>> I did again some protos tests and openser hangs. It is still 
>>>> running, but does not respond to SIP packets anymore. The process 
>>>> which listens on the network (5237) is in "pause".
>>>>
>>>> The FIFO is still working:
>>>>   philipp:/# openserctl fifo lcr_dump
>>>>   sip:1.2.32.167:5060:tcp
>>>>   sip:1.2.164.123:5060:tcp
>>>>   sip:1.2.32.168:5060:tcp
>>>>   sip:1.2.33.4:5060:tcp
>>>>
>>>> Any glues how to debug this issue?
>>>>
>>>> klaus
>>>>
>>>> philipp:/export/log# ps -Al|grep openser
>>>> 1 S   106  5239  5237  0  76   0 - 10052 pipe_w ?        00:00:00 
>>>> openser
>>>> 1 R   106  5240  5237  0  85   0 - 10138 -      ?        00:07:47 
>>>> openser
>>>> 1 R   106  5243  5237  0  85   0 - 10138 -      ?        00:08:05 
>>>> openser
>>>> 1 R   106  5245  5237  0  85   0 - 10138 -      ?        00:08:02 
>>>> openser
>>>> 1 R   106  5246  5237  0  85   0 - 10172 -      ?        00:07:49 
>>>> openser
>>>> 1 S   106  5248  5237  0  76   0 - 10052 schedu ?        00:00:36 
>>>> openser
>>>> 1 S   106  5250  5237  0  76   0 - 10052 schedu ?        00:00:00 
>>>> openser
>>>> 1 S   106  5252  5237  0  76   0 - 10052 schedu ?        00:00:00 
>>>> openser
>>>> 1 S   106  5253  5237  0  76   0 - 10052 schedu ?        00:00:00 
>>>> openser
>>>> 1 S   106  5254  5237  0  76   0 - 10052 schedu ?        00:00:00 
>>>> openser
>>>> 1 S   106  5255  5237  0  76   0 - 10018 schedu ?        00:00:00 
>>>> openser
>>>> 5 S   106  5237     1  0  76   0 - 10018 pause  ?        00:00:00 
>>>> openser
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel at openser.org
>>>> http://openser.org/cgi-bin/mailman/listinfo/devel
>>>>
>>>
>>>
>>
>>
> 
>