[Users] Daemons and killing processes...

Bogdan-Andrei Iancu bogdan at voice-system.ro
Mon Mar 13 15:59:51 CET 2006


Hi Helge,

I manage to find the bug - please update and see if the problem is solved.

regards,
bogdan

Helge Waastad wrote:

>Hi,
>I do not know if this is sufficient enough, but anyway:
>
>(I belive there is a memory problem, cause it's crashing the moment a
>REGISTER comes, a replication from another OpenSER....)
>
>br hw
>
>
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: SIP Request:
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]:  method:  <REGISTER>
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]:  uri:
><sip:smartnet.no>
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]:  version: <SIP/2.0>
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
>flags=2
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: Found param type 232,
><branch> = <z9hG4bKc68c.bdbe9a8.0>; state=16
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: end of header
>reached, state=5
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: Via
>found, flags=2
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: this
>is the first via
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: After parse_msg...
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: preparing to run
>routing scripts...
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
>flags=80
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: Found param type 232,
><branch> = <z9hG4bKc68c.61bc4535.0>; state=16
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: end of header
>reached, state=5
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: Via
>found, flags=80
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: this
>is the second via
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: Found param type 232,
><branch> = <z9hG4bK78F48B37>; state=16
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: end of header
>reached, state=5
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: Via
>found, flags=80
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: get_hdr_field: cseq
><CSeq>: <135> <REGISTER>
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: DEBUG:parse_to:end of
>header reached, state=9
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: DEBUG: get_hdr_field:
><To> [44]; uri=[sip:67512391 at smartnet.no]
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: DEBUG: to body
>["Helge Waastad" <sip:67512391 at smartnet.no>^M ]
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: DEBUG: get_hdr_body :
>content_length=0
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: subst_run: running.
>r=0
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: subst_run: matched
>(16, 133):
>[<sip:67512391 at 80.203.98.197:59086;transport=udp>;methods="INVITE,
>MESSAGE, INFO, SUBSCRIBE, OPTIONS, BYE, CANCEL, NOTIFY, ACK, REFER"]
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: avpops:ops_subst:
>subst to 1 avps
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: REGISTER
>195.18.134.147:5080 -> 195.18.134.150:5080 Replication (no-nat) ("Helge
>Waastad" <sip:67512391 at 80.203.98.197:59086;transport=udp>)
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
>flags=ffffffffffffffff
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: found end of header
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
>flags=8000000
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
>flags=ffffffffffffffff
>Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: build_contact():
>Created Contact HF: Contact:
><sip:67512391 at 80.203.98.197:59086;transport=udp>;q=0;expires=300^M
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22201]: ERROR: receive_fd:
>EOF on 11
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22201]: DBG:
>handle_ser_child: dead child 2, pid 22188 (shutting down?)
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22201]: DBG: io_watch_del
>(0x80f0d60, 11, -1, 0x0) fd_no=10 called
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22171]: child process 22188
>exited by a signal 11
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22171]: core was generated
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22171]: INFO: terminating due
>to SIGCHLD
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22191]: INFO: signal 15
>received
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]: INFO: signal 15
>received
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22191]: Memory status (pkg):
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]: Memory status (pkg):
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]: qm_status
>(0x8122b00):
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]:  heap size= 1048576
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]:  used= 129088, used
>+overhead=177312, free=871264
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]:  max used
>(+overhead)= 181344
>Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]: dumping all alloc'ed.
>fragments:
>
>
>
>fre, 10,.03.2006 kl. 21.21 +0200, skrev Bogdan-Andrei Iancu:
>  
>
>>Hi Helge,
>>
>>one you compiled the mem debug support (remove F_MALLOC and add 
>>DBG_QM_MALLOC) if you set debug level to a high level (9 for ex),you 
>>will get as output a lot of info msgs about the memory ops.
>>
>>regards,
>>bogdan
>>
>>Helge Waastad wrote:
>>
>>    
>>
>>>Hi,
>>>bt is as follows:
>>>
>>>(gdb) bt
>>>#0  0x08098ebf in qm_free ()
>>>#1  0x007aaf97 in update_contacts (_m=0x814f308, _r=0xb61920d8,
>>>_c=0x8139548) at save.c:627
>>>#2  0x007ab67d in save_real (_m=0x814f308, _t=0xb618fe88, doreply=1) at
>>>save.c:662
>>>#3  0x0805087b in do_action ()
>>>#4  0x08051a0c in do_action ()
>>>#5  0x08051bd2 in run_actions ()
>>>#6  0x0807534f in receive_msg ()
>>>#7  0x08092ca3 in udp_rcv_loop ()
>>>#8  0x08065056 in main_loop ()
>>>#9  0x08065835 in main ()
>>>(gdb)
>>>
>>>I'm not sure how to get even more memory debugs.
>>>
>>>br hw
>>>
>>>fre, 10,.03.2006 kl. 11.44 +0200, skrev Bogdan-Andrei Iancu:
>>> 
>>>
>>>      
>>>
>>>>Hi,
>>>>
>>>>you mean that after first register the proxy dies? if so, it seams like 
>>>>a mem corruption to me...any logs? the memory debugger messages are very 
>>>>useful.
>>>>
>>>>regards,
>>>>bogdan
>>>>
>>>>Helge Waastad wrote:
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>Hi,
>>>>>this is the output after compiling with the extra options.
>>>>>
>>>>>Anyway, that really killed the daemon..
>>>>>One register, and it dumps.
>>>>>
>>>>>br hw
>>>>>
>>>>>
>>>>>(gdb) bt
>>>>>#0  0x002727a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
>>>>>#1  0x00338d8c in sched_yield () from /lib/tls/libc.so.6
>>>>>#2  0x00e4b100 in lock_udomain (_d=0xb618fe58) at ../../fastlock.h:166
>>>>>#3  0x00e4db9a in mem_timer_udomain (_d=0xb618fe58) at udomain.c:677
>>>>>#4  0x00e47f05 in synchronize_all_udomains () at dlist.c:488
>>>>>#5  0x00e50a15 in destroy () at ul_mod.c:285
>>>>>#6  0x08085dce in destroy_modules ()
>>>>>#7  0x080638ba in cleanup ()
>>>>>#8  0x080644f6 in handle_sigs ()
>>>>>#9  0x08065215 in main_loop ()
>>>>>#10 0x08065835 in main ()
>>>>>
>>>>>
>>>>>Mvh,
>>>>>Helge Waastad
>>>>>Senior Engineer
>>>>>Smartnet
>>>>>tlf: 67830017
>>>>>
>>>>>
>>>>>
>>>>>-----Original Message-----
>>>>>From: Bogdan-Andrei Iancu [mailto:bogdan at voice-system.ro]
>>>>>Sent: Tue 3/7/2006 9:17 PM
>>>>>To: Helge Waastad
>>>>>Cc: Andreas Granig; users at openser.org
>>>>>Subject: Re: [Users] Daemons and killing processes...
>>>>>
>>>>>Hi Helge,
>>>>>
>>>>>I suspect a problem related to memory - the remaining process may cycle
>>>>>with no control trying to print the mem status (which may be broken).
>>>>>
>>>>>to see if it;s the case, compile mem debug support (remove F_MALLOC and
>>>>>add DBG_QM_MALLOC and recompile everything)
>>>>>
>>>>>regards,
>>>>>bogdan
>>>>>
>>>>>Helge Waastad wrote:
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>Hi,
>>>>>>I've been waiting fo 12 min now, and are preparing to put then kettle on
>>>>>>to fix me a cup of coffee...I guess I have time enough :-)
>>>>>>
>>>>>>...is there a "debugging" manual avaliable? I guess it would be nice
>>>>>>having feedback on the dev list already including the backtrace?
>>>>>>
>>>>>>
>>>>>>BTW, the output for the hanging process is:
>>>>>>
>>>>>>(gdb) bt
>>>>>>#0  0x080a0ab9 in fm_status ()
>>>>>>#1  0x080663e2 in sig_usr ()
>>>>>>#2  <signal handler called>
>>>>>>#3  0x007787a0 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
>>>>>>#4  0x008594d1 in recvfrom () from /lib/tls/libc.so.6
>>>>>>#5  0x08098a87 in udp_rcv_loop ()
>>>>>>#6  0x08066df2 in main_loop ()
>>>>>>#7  0x08067585 in main ()
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>tir, 07,.03.2006 kl. 18.56 +0100, skrev Andreas Granig:
>>>>>>
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>>>Helge Waastad wrote:
>>>>>>> 
>>>>>>>
>>>>>>>         
>>>>>>>
>>>>>>>              
>>>>>>>
>>>>>>>>Hi,
>>>>>>>>I have somewhat trouble to get a core dump from the init script.
>>>>>>>>I get core dumps when I run it in shell, but not as init script.
>>>>>>>>
>>>>>>>>I'll try to be creative later this evening.
>>>>>>>>   
>>>>>>>>
>>>>>>>>           
>>>>>>>>
>>>>>>>>                
>>>>>>>>
>>>>>>>The backtrace for the processes looks like this:
>>>>>>>
>>>>>>>#0  0x0808eb0d in fm_status (qm=0x811d940) at mem/f_malloc.c:515
>>>>>>>#1  0x08065fee in sig_usr (signo=1077141592) at main.c:565
>>>>>>>#2  <signal handler called>
>>>>>>>#3  0x4010d534 in recvfrom () from /lib/libc.so.6
>>>>>>>#4  0x08089433 in udp_rcv_loop () at udp_server.c:415
>>>>>>>#5  0x08063de4 in main_loop () at main.c:919
>>>>>>>#6  0x0806522e in main (argc=1, argv=0xbfbc54e4) at main.c:1472
>>>>>>>
>>>>>>>But it shuts down after quite some (long) time...
>>>>>>>
>>>>>>>Andy
>>>>>>> 
>>>>>>>
>>>>>>>         
>>>>>>>
>>>>>>>              
>>>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>





More information about the sr-users mailing list