[Users] Daemons and killing processes...

Helge Waastad helge at smartnet.no
Tue Mar 21 10:01:52 CET 2006


Hi,
a little late but just now I got to update by cvs.

everythings OK now.

Thanks,

br hw

man, 13,.03.2006 kl. 16.59 +0200, skrev Bogdan-Andrei Iancu:
> Hi Helge,
> 
> I manage to find the bug - please update and see if the problem is solved.
> 
> regards,
> bogdan
> 
> Helge Waastad wrote:
> 
> >Hi,
> >I do not know if this is sufficient enough, but anyway:
> >
> >(I belive there is a memory problem, cause it's crashing the moment a
> >REGISTER comes, a replication from another OpenSER....)
> >
> >br hw
> >
> >
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: SIP Request:
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]:  method:  <REGISTER>
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]:  uri:
> ><sip:smartnet.no>
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]:  version: <SIP/2.0>
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
> >flags=2
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: Found param type 232,
> ><branch> = <z9hG4bKc68c.bdbe9a8.0>; state=16
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: end of header
> >reached, state=5
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: Via
> >found, flags=2
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: this
> >is the first via
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: After parse_msg...
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: preparing to run
> >routing scripts...
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
> >flags=80
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: Found param type 232,
> ><branch> = <z9hG4bKc68c.61bc4535.0>; state=16
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: end of header
> >reached, state=5
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: Via
> >found, flags=80
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: this
> >is the second via
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: Found param type 232,
> ><branch> = <z9hG4bK78F48B37>; state=16
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: end of header
> >reached, state=5
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers: Via
> >found, flags=80
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: get_hdr_field: cseq
> ><CSeq>: <135> <REGISTER>
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: DEBUG:parse_to:end of
> >header reached, state=9
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: DEBUG: get_hdr_field:
> ><To> [44]; uri=[sip:67512391 at smartnet.no]
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: DEBUG: to body
> >["Helge Waastad" <sip:67512391 at smartnet.no>^M ]
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: DEBUG: get_hdr_body :
> >content_length=0
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: subst_run: running.
> >r=0
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: subst_run: matched
> >(16, 133):
> >[<sip:67512391 at 80.203.98.197:59086;transport=udp>;methods="INVITE,
> >MESSAGE, INFO, SUBSCRIBE, OPTIONS, BYE, CANCEL, NOTIFY, ACK, REFER"]
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: avpops:ops_subst:
> >subst to 1 avps
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: REGISTER
> >195.18.134.147:5080 -> 195.18.134.150:5080 Replication (no-nat) ("Helge
> >Waastad" <sip:67512391 at 80.203.98.197:59086;transport=udp>)
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
> >flags=ffffffffffffffff
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: found end of header
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
> >flags=8000000
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: parse_headers:
> >flags=ffffffffffffffff
> >Mar 10 22:49:06 proxy-02 /usr/sbin/openser[22188]: build_contact():
> >Created Contact HF: Contact:
> ><sip:67512391 at 80.203.98.197:59086;transport=udp>;q=0;expires=300^M
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22201]: ERROR: receive_fd:
> >EOF on 11
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22201]: DBG:
> >handle_ser_child: dead child 2, pid 22188 (shutting down?)
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22201]: DBG: io_watch_del
> >(0x80f0d60, 11, -1, 0x0) fd_no=10 called
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22171]: child process 22188
> >exited by a signal 11
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22171]: core was generated
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22171]: INFO: terminating due
> >to SIGCHLD
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22191]: INFO: signal 15
> >received
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]: INFO: signal 15
> >received
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22191]: Memory status (pkg):
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]: Memory status (pkg):
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]: qm_status
> >(0x8122b00):
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]:  heap size= 1048576
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]:  used= 129088, used
> >+overhead=177312, free=871264
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]:  max used
> >(+overhead)= 181344
> >Mar 10 22:49:07 proxy-02 /usr/sbin/openser[22190]: dumping all alloc'ed.
> >fragments:
> >
> >
> >
> >fre, 10,.03.2006 kl. 21.21 +0200, skrev Bogdan-Andrei Iancu:
> >  
> >
> >>Hi Helge,
> >>
> >>one you compiled the mem debug support (remove F_MALLOC and add 
> >>DBG_QM_MALLOC) if you set debug level to a high level (9 for ex),you 
> >>will get as output a lot of info msgs about the memory ops.
> >>
> >>regards,
> >>bogdan
> >>
> >>Helge Waastad wrote:
> >>
> >>    
> >>
> >>>Hi,
> >>>bt is as follows:
> >>>
> >>>(gdb) bt
> >>>#0  0x08098ebf in qm_free ()
> >>>#1  0x007aaf97 in update_contacts (_m=0x814f308, _r=0xb61920d8,
> >>>_c=0x8139548) at save.c:627
> >>>#2  0x007ab67d in save_real (_m=0x814f308, _t=0xb618fe88, doreply=1) at
> >>>save.c:662
> >>>#3  0x0805087b in do_action ()
> >>>#4  0x08051a0c in do_action ()
> >>>#5  0x08051bd2 in run_actions ()
> >>>#6  0x0807534f in receive_msg ()
> >>>#7  0x08092ca3 in udp_rcv_loop ()
> >>>#8  0x08065056 in main_loop ()
> >>>#9  0x08065835 in main ()
> >>>(gdb)
> >>>
> >>>I'm not sure how to get even more memory debugs.
> >>>
> >>>br hw
> >>>
> >>>fre, 10,.03.2006 kl. 11.44 +0200, skrev Bogdan-Andrei Iancu:
> >>> 
> >>>
> >>>      
> >>>
> >>>>Hi,
> >>>>
> >>>>you mean that after first register the proxy dies? if so, it seams like 
> >>>>a mem corruption to me...any logs? the memory debugger messages are very 
> >>>>useful.
> >>>>
> >>>>regards,
> >>>>bogdan
> >>>>
> >>>>Helge Waastad wrote:
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>>>Hi,
> >>>>>this is the output after compiling with the extra options.
> >>>>>
> >>>>>Anyway, that really killed the daemon..
> >>>>>One register, and it dumps.
> >>>>>
> >>>>>br hw
> >>>>>
> >>>>>
> >>>>>(gdb) bt
> >>>>>#0  0x002727a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
> >>>>>#1  0x00338d8c in sched_yield () from /lib/tls/libc.so.6
> >>>>>#2  0x00e4b100 in lock_udomain (_d=0xb618fe58) at ../../fastlock.h:166
> >>>>>#3  0x00e4db9a in mem_timer_udomain (_d=0xb618fe58) at udomain.c:677
> >>>>>#4  0x00e47f05 in synchronize_all_udomains () at dlist.c:488
> >>>>>#5  0x00e50a15 in destroy () at ul_mod.c:285
> >>>>>#6  0x08085dce in destroy_modules ()
> >>>>>#7  0x080638ba in cleanup ()
> >>>>>#8  0x080644f6 in handle_sigs ()
> >>>>>#9  0x08065215 in main_loop ()
> >>>>>#10 0x08065835 in main ()
> >>>>>
> >>>>>
> >>>>>Mvh,
> >>>>>Helge Waastad
> >>>>>Senior Engineer
> >>>>>Smartnet
> >>>>>tlf: 67830017
> >>>>>
> >>>>>
> >>>>>
> >>>>>-----Original Message-----
> >>>>>From: Bogdan-Andrei Iancu [mailto:bogdan at voice-system.ro]
> >>>>>Sent: Tue 3/7/2006 9:17 PM
> >>>>>To: Helge Waastad
> >>>>>Cc: Andreas Granig; users at openser.org
> >>>>>Subject: Re: [Users] Daemons and killing processes...
> >>>>>
> >>>>>Hi Helge,
> >>>>>
> >>>>>I suspect a problem related to memory - the remaining process may cycle
> >>>>>with no control trying to print the mem status (which may be broken).
> >>>>>
> >>>>>to see if it;s the case, compile mem debug support (remove F_MALLOC and
> >>>>>add DBG_QM_MALLOC and recompile everything)
> >>>>>
> >>>>>regards,
> >>>>>bogdan
> >>>>>
> >>>>>Helge Waastad wrote:
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>>>Hi,
> >>>>>>I've been waiting fo 12 min now, and are preparing to put then kettle on
> >>>>>>to fix me a cup of coffee...I guess I have time enough :-)
> >>>>>>
> >>>>>>...is there a "debugging" manual avaliable? I guess it would be nice
> >>>>>>having feedback on the dev list already including the backtrace?
> >>>>>>
> >>>>>>
> >>>>>>BTW, the output for the hanging process is:
> >>>>>>
> >>>>>>(gdb) bt
> >>>>>>#0  0x080a0ab9 in fm_status ()
> >>>>>>#1  0x080663e2 in sig_usr ()
> >>>>>>#2  <signal handler called>
> >>>>>>#3  0x007787a0 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
> >>>>>>#4  0x008594d1 in recvfrom () from /lib/tls/libc.so.6
> >>>>>>#5  0x08098a87 in udp_rcv_loop ()
> >>>>>>#6  0x08066df2 in main_loop ()
> >>>>>>#7  0x08067585 in main ()
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>tir, 07,.03.2006 kl. 18.56 +0100, skrev Andreas Granig:
> >>>>>>
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>>>Helge Waastad wrote:
> >>>>>>> 
> >>>>>>>
> >>>>>>>         
> >>>>>>>
> >>>>>>>              
> >>>>>>>
> >>>>>>>>Hi,
> >>>>>>>>I have somewhat trouble to get a core dump from the init script.
> >>>>>>>>I get core dumps when I run it in shell, but not as init script.
> >>>>>>>>
> >>>>>>>>I'll try to be creative later this evening.
> >>>>>>>>   
> >>>>>>>>
> >>>>>>>>           
> >>>>>>>>
> >>>>>>>>                
> >>>>>>>>
> >>>>>>>The backtrace for the processes looks like this:
> >>>>>>>
> >>>>>>>#0  0x0808eb0d in fm_status (qm=0x811d940) at mem/f_malloc.c:515
> >>>>>>>#1  0x08065fee in sig_usr (signo=1077141592) at main.c:565
> >>>>>>>#2  <signal handler called>
> >>>>>>>#3  0x4010d534 in recvfrom () from /lib/libc.so.6
> >>>>>>>#4  0x08089433 in udp_rcv_loop () at udp_server.c:415
> >>>>>>>#5  0x08063de4 in main_loop () at main.c:919
> >>>>>>>#6  0x0806522e in main (argc=1, argv=0xbfbc54e4) at main.c:1472
> >>>>>>>
> >>>>>>>But it shuts down after quite some (long) time...
> >>>>>>>
> >>>>>>>Andy
> >>>>>>> 
> >>>>>>>
> >>>>>>>         
> >>>>>>>
> >>>>>>>              
> >>>>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> 
-- 
Helge Waastad
Senior Konsulent
Smartnet




More information about the sr-users mailing list