[Kamailio-Users] Can openser.cfg lead to pkg memory problem?

[mayamatakeshi]

On Tue, Sep 23, 2008 at 6:00 PM, Daniel-Constantin Mierla <miconda at gmail.com
> wrote:

> Hello,
>
> On 09/23/08 10:31, mayamatakeshi wrote:
>
>> Hello,
>> we have openser 1.3.3 running in production (current rev.: 4943).
>> For 3 times in 50 days we had to restart openser to correct pkg memory
>> problem.
>>
> openser 1.3.3 was released 3 weeks ago, so I guess you were running
>  previous version before, but it happened again since you upgraded to 1.3.3,
> right?
>
>  After some time logging messages like this:
>> /openser.log:Aug 19 10:39:18 ipx022 /usr/local/sbin/openser[16991]:
>> ERROR:core:new_credentials: no pkg memory left,
>> openser will eventually run out of pkg memory and refuse all subsequent
>> requests.
>>
>> We are trying to recreate this in our lab so that we can follow memory
>> troubleshooting instructions at
>> http://kamailio.net/dokuwiki/doku.php/troubleshooting:memory, but so far
>> we were unable to do it even when generating millions of calls and
>> registration transactions (we are using SIPp to generate normal call flows
>> and even abnormal call flows detected when reading openser.log, like
>> 'invalid cseq for aor', malformed SIP messages etc).
>>
> We can spot memory leaks even the "out of memory" message is not printed.
> Just archive the logs (the most important is the shut down time) and made
> them available for download so they can be investigated.
>
> There could be two reasons:
> - there is memory leak but happens in some cases that you don't reproduce
> in lab, but they are in the production environment
> - you get memory fragmentation
>
> Let's see first the debug messages...


Hello,
here are the link for openser.log and cfg files:
http://www.yousendit.com/download/bVlEV0o4R3NoeWJIRGc9PQ

After compilation with debug flags for memory manager, I left openser
running in production for 24 hours. Then, I moved all traffic to another
host and waited for more than 30 minutes before stopping openser.
In the openser.cfg, I set debug=2. If you need, I can run it again with a
higher value (but I hope it doesn't have to be too high, due to overhead
concerns).

regards,
takeshi

>
>
>
>  And this is much more than in our production environment, with just 600
>> subscribers and about 2000 calls a day.
>>
>> The frequency the problem happens is increasing with the number of
>> subscribers, so we are performing periodic restart of openser (actually,
>> what we do is to switch over to the standby server). We already recompiled
>> openser with pkg memory  pool size set to 4MB so that this will not have to
>> be done frequently.
>>
>> Since we cannot recreate this in our lab, we suspect there is a situation
>> happening in production that might not be having been properly handled by
>> openser.cfg. So my question is: would it be possible to an overlooked detail
>> in openser.cfg to cause pkg memory problem?
>>
>>
>> In case someone could take a look at it, here's our cfg file:
>>
>>  [...]
>
> --
> Daniel-Constantin Mierla
> http://www.asipto.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kamailio.org/pipermail/users/attachments/20080926/3332a8ab/attachment.htm