Re: [Users] memory leak in presence module?

Hi! I also tried now with QM_JOIN_FREE but the same result: The shared mem (64M) is full after ~8000 PUBLISH (=8000 NOTIFY) and does not get freed: darilion@ds3000:~/software/openser-svn-1.2/openser$ sudo openserctl fifo get_statistics all core:rcv_requests = 8743 core:rcv_replies = 8198 core:fwd_requests = 0 core:fwd_replies = 0 core:drop_requests = 0 core:drop_replies = 0 core:err_requests = 0 core:err_replies = 0 core:bad_URIs_rcvd = 0 core:unsupported_methods = 0 core:bad_msg_hdr = 0 shmem:total_size = 67108864 shmem:used_size = 65865712 shmem:real_used_size = 67004532 shmem:max_used_size = 67108376 shmem:free_size = 104332 shmem:fragments = 239 sl:1xx_replies = 0 sl:2xx_replies = 8221 sl:3xx_replies = 0 sl:4xx_replies = 196 sl:5xx_replies = 186 sl:6xx_replies = 0 sl:sent_replies = 8603 sl:sent_err_replies = 186 sl:received_ACKs = 0 tm:received_replies = 8198 tm:relayed_replies = 8198 tm:local_replies = 0 tm:UAS_transactions = 8218 tm:UAC_transactions = 8200 tm:2xx_transactions = 8198 tm:3xx_transactions = 0 tm:4xx_transactions = 0 tm:5xx_transactions = 0 tm:6xx_transactions = 0 tm:inuse_transactions = 8217 usrloc:registered_users = 0 usrloc:location-users = 0 usrloc:location-contacts = 0 usrloc:location-expires = 0 registrar:max_expires = 0 registrar:max_contacts = 0 registrar:default_expire = 3600 registrar:accepted_regs = 1 registrar:rejected_regs = 0 darilion@ds3000:~/software/openser-svn-1.2/openser$ regards klaus Klaus Darilion wrote: > memory logs: > > kill -SIGUSR1: > http://pernau.at/kd/openser/debug-kill-usr1.txt > > openser stop: > http://pernau.at/kd/openser/debug-stop3.txt > > regards > klaus > > Klaus Darilion wrote: >> >> Openser runs out of mem: >> $ sudo openserctl fifo get_statistics all >> core:rcv_requests = 13893 >> core:rcv_replies = 18007 >> core:fwd_requests = 0 >> core:fwd_replies = 0 >> core:drop_requests = 0 >> core:drop_replies = 0 >> core:err_requests = 0 >> core:err_replies = 0 >> core:bad_URIs_rcvd = 0 >> core:unsupported_methods = 0 >> core:bad_msg_hdr = 0 >> shmem:total_size = 67108864 >> shmem:used_size = 65351684 >> shmem:real_used_size = 66591880 >> shmem:max_used_size = 66953980 >> shmem:free_size = 516984 >> shmem:fragments = 1052 >> sl:1xx_replies = 0 >> sl:2xx_replies = 8878 >> sl:3xx_replies = 0 >> sl:4xx_replies = 2322 >> sl:5xx_replies = 2308 >> sl:6xx_replies = 0 >> sl:sent_replies = 13508 >> sl:sent_err_replies = 2308 >> sl:received_ACKs = 0 >> tm:received_replies = 18007 >> tm:relayed_replies = 17652 >> tm:local_replies = 1 >> tm:UAS_transactions = 8861 >> tm:UAC_transactions = 17653 >> tm:2xx_transactions = 17652 >> tm:3xx_transactions = 0 >> tm:4xx_transactions = 1 >> tm:5xx_transactions = 0 >> tm:6xx_transactions = 0 >> tm:inuse_transactions = 8861 >> usrloc:registered_users = 2 >> usrloc:location-users = 2 >> usrloc:location-contacts = 2 >> usrloc:location-expires = 0 >> registrar:max_expires = 0 >> registrar:max_contacts = 0 >> registrar:default_expire = 3600 >> registrar:accepted_regs = 12 >> registrar:rejected_regs = 0 >> >> >> after 1h of inactivity: >> $ sudo openserctl fifo get_statistics all >> Password: >> core:rcv_requests = 14285 >> core:rcv_replies = 18007 >> core:fwd_requests = 0 >> core:fwd_replies = 0 >> core:drop_requests = 0 >> core:drop_replies = 0 >> core:err_requests = 0 >> core:err_replies = 0 >> core:bad_URIs_rcvd = 0 >> core:unsupported_methods = 0 >> core:bad_msg_hdr = 0 >> shmem:total_size = 67108864 >> shmem:used_size = 65351684 >> shmem:real_used_size = 66591880 >> shmem:max_used_size = 66953980 >> shmem:free_size = 516984 >> shmem:fragments = 1052 >> sl:1xx_replies = 0 >> sl:2xx_replies = 8922 >> sl:3xx_replies = 0 >> sl:4xx_replies = 2481 >> sl:5xx_replies = 2497 >> sl:6xx_replies = 0 >> sl:sent_replies = 13900 >> sl:sent_err_replies = 2497 >> sl:received_ACKs = 0 >> tm:received_replies = 18007 >> tm:relayed_replies = 17652 >> tm:local_replies = 1 >> tm:UAS_transactions = 8861 >> tm:UAC_transactions = 17653 >> tm:2xx_transactions = 17652 >> tm:3xx_transactions = 0 >> tm:4xx_transactions = 1 >> tm:5xx_transactions = 0 >> tm:6xx_transactions = 0 >> tm:inuse_transactions = 8861 >> usrloc:registered_users = 2 >> usrloc:location-users = 2 >> usrloc:location-contacts = 2 >> usrloc:location-expires = 0 >> registrar:max_expires = 0 >> registrar:max_contacts = 0 >> registrar:default_expire = 3600 >> registrar:accepted_regs = 51 >> registrar:rejected_regs = 0 >> >> >> >> regards >> klaus >> >> Daniel-Constantin Mierla wrote: >>> my mistake: >>> >>> # openserctl fifo get_statistics all >>> >>> Cheers, >>> Daniel >>> >>> >>> On 04/26/07 12:36, Klaus Darilion wrote: >>>> # openserctl fifo get_statistics >>>> 400 Too few or too many arguments >>>> >>>> :-( >>>> >>>> I'm using 1.2 >>>> >>>> >>>> regards >>>> klaus >>>> >>>> >>>> >>>> Klaus Darilion wrote: >>>>> >>>>> >>>>> Daniel-Constantin Mierla wrote: >>>>>> Hello Klaus, >>>>>> >>>>>> if that is the full log, seems no mem leak in shm. What is the >>>>>> shm memory size you use? >>>>> >>>>> I don't know - I use the default values. But the size should be >>>>> irrelevant as the "no memory left" is still there after 20 >>>>> minutes of inactivity. >>>>> >>>>>> Once you get out of memory can you issue" openserctl fifo >>>>>> get_statistics" and send output? >>>>> ok >>>>>> >>>>>> I don't think is fragmentation, but you can try to compile with >>>>>> -DQM_JOIN_FREE >>>>>> >>>>> ok >>>>>> Do you do the tests over TLS? >>>>> ok >>>>>> >>>>>> Cheers, >>>>>> Daniel >>>>>> >>>>>> >>>>>> On 04/26/07 10:17, Klaus Darilion wrote: >>>>>>> btw: there is no >>>>>>> "PRESENCE:notify: ERROR in function tmb.t_request_within" >>>>>>> in my log file, thus I guess the problem is somewhere else. >>>>>>> >>>>>>> regards >>>>>>> klaus >>>>>>> >>>>>>> Klaus Darilion wrote: >>>>>>>> Hi Anca! >>>>>>>> >>>>>>>> There is still a leak: >>>>>>>> http://pernau.at/kd/openser/debug-stop2.txt >>>>>>>> >>>>>>>> regards >>>>>>>> klaus >>>>>>>> >>>>>>>> Anca-Maria Vamanu wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I have made a fix. Please test again with sipp and say if it >>>>>>>>> works now. >>>>>>>>> >>>>>>>>> regards, >>>>>>>>> >>>>>>>>> Anca >>>>>>>>> >>>>>>>>> Klaus Darilion wrote: >>>>>>>>> >>>>>>>>>> Hi! >>>>>>>>>> >>>>>>>>>> Openser 1.2 runs out of memory during presence load tests >>>>>>>>>> with sipp. >>>>>>>>>> >>>>>>>>>> Config snippet: >>>>>>>>>> >>>>>>>>>> if (is_method("PUBLISH")) { >>>>>>>>>> xlog("L_INFO","$ci PUBLISH detected ... outside >>>>>>>>>> t_newtran\n"); >>>>>>>>>> if (t_newtran()) { >>>>>>>>>> xlog("L_INFO","$ci PUBLISH detected ... inside >>>>>>>>>> t_newtran\n"); >>>>>>>>>> handle_publish(); >>>>>>>>>> xlog("L_INFO","$ci PUBLISH detected, handle_publish ... >>>>>>>>>> done\n"); >>>>>>>>>> exit; >>>>>>>>>> } else { >>>>>>>>>> sl_reply_error(); >>>>>>>>>> exit; >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> Memory log after openser stop (after 20 minutes of >>>>>>>>>> inactivity): >>>>>>>>>> http://pernau.at/kd/openser/debug-stop.txt >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> regards >>>>>>>>>> klaus Klaus Darilion wrote: >>>>>>>>>> >>>>>>>>>>> Hi Cesc! >>>>>>>>>>> >>>>>>>>>>> Where do you have t_newtran() - in ser or openser? >>>>>>>>>>> >>>>>>>>>>> I have it here in openser just before handle_publish(). It >>>>>>>>>>> works but now openser has run out of mem during my sipp >>>>>>>>>>> performance tests :-( >>>>>>>>>>> >>>>>>>>>>> ...more to come.... >>>>>>>>>>> >>>>>>>>>>> Cesc wrote: >>>>>>>>>>> >>>>>>>>>>>> I think I can answer that one ... stateless? >>>>>>>>>>>> >>>>>>>>>>>> I have a presence openser proxied by a SER, both on same >>>>>>>>>>>> machine ... >>>>>>>>>>>> For easier configuration purposes, openser and ser >>>>>>>>>>>> communicate via >>>>>>>>>>>> 127.0.0.1 ... >>>>>>>>>>>> >>>>>>>>>>>> The result is that OK from the PUAs do not get to openser >>>>>>>>>>>> (i guess SER >>>>>>>>>>>> gets confussed by the vias of openser being 0.0.0.0:5060 >>>>>>>>>>>> and the >>>>>>>>>>>> 127.0.0.1 stuff) ... So, openser gets no OK, but it does >>>>>>>>>>>> not matter >>>>>>>>>>>> ... >>>>>>>>>>>> BTW, i have a t_newtran() in there ... >>>>>>>>>>>> >>>>>>>>>>>> Oh, for sure my configuration is messy and improvable ... >>>>>>>>>>>> but for now >>>>>>>>>>>> it works steady :) But the OKs not going through is >>>>>>>>>>>> worrying me a bit >>>>>>>>>>>> ... and also openser not "timing out" or giving an error. >>>>>>>>>>>> >>>>>>>>>>>> Cesc >>>>>>>>>>>> >>>>>>>>>>>> On 4/25/07, Klaus Darilion <klaus.mailinglists(a)pernau.at> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi! >>>>>>>>>>>>> >>>>>>>>>>>>> Does presence module handles PUBLISH/SUBSCRIBE stateful >>>>>>>>>>>>> or stateless? >>>>>>>>>>>>> >>>>>>>>>>>>> Thus, how should retransmission be handled. Will they be >>>>>>>>>>>>> detected by >>>>>>>>>>>>> presence module or is it possible to wrap the >>>>>>>>>>>>> handle_publish inside >>>>>>>>>>>>> t_newtran()? >>>>>>>>>>>>> >>>>>>>>>>>>> regards >>>>>>>>>>>>> klaus >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> Users mailing list >>>>>>>>>>>>> Users(a)openser.org >>>>>>>>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Users mailing list >>>>>>>>>>> Users(a)openser.org >>>>>>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Users mailing list >>>>>>>>>> Users(a)openser.org >>>>>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Users mailing list >>>>>>>> Users(a)openser.org >>>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Users mailing list >>>>>>> Users(a)openser.org >>>>>>> http://openser.org/cgi-bin/mailman/listinfo/users >>>>>>> >>>>> >>>>> _______________________________________________ >>>>> Users mailing list >>>>> Users(a)openser.org >>>>> http://openser.org/cgi-bin/mailman/listinfo/users >>>> >> >> _______________________________________________ >> Users mailing list >> Users(a)openser.org >> http://openser.org/cgi-bin/mailman/listinfo/users