[Kamailio-Users] Can openser.cfg lead to pkg memory problem?

[mayamatakeshi]

On Fri, Sep 26, 2008 at 6:24 PM, mayamatakeshi <mayamatakeshi at gmail.com>wrote:

>
> On Tue, Sep 23, 2008 at 6:00 PM, Daniel-Constantin Mierla <
> miconda at gmail.com> wrote:
>
>> Hello,
>>
>> On 09/23/08 10:31, mayamatakeshi wrote:
>>
>>> Hello,
>>> we have openser 1.3.3 running in production (current rev.: 4943).
>>> For 3 times in 50 days we had to restart openser to correct pkg memory
>>> problem.
>>>
>> openser 1.3.3 was released 3 weeks ago, so I guess you were running
>>  previous version before, but it happened again since you upgraded to 1.3.3,
>> right?
>>
>>  After some time logging messages like this:
>>> /openser.log:Aug 19 10:39:18 ipx022 /usr/local/sbin/openser[16991]:
>>> ERROR:core:new_credentials: no pkg memory left,
>>> openser will eventually run out of pkg memory and refuse all subsequent
>>> requests.
>>>
>>> We are trying to recreate this in our lab so that we can follow memory
>>> troubleshooting instructions at
>>> http://kamailio.net/dokuwiki/doku.php/troubleshooting:memory, but so far
>>> we were unable to do it even when generating millions of calls and
>>> registration transactions (we are using SIPp to generate normal call flows
>>> and even abnormal call flows detected when reading openser.log, like
>>> 'invalid cseq for aor', malformed SIP messages etc).
>>>
>> We can spot memory leaks even the "out of memory" message is not printed.
>> Just archive the logs (the most important is the shut down time) and made
>> them available for download so they can be investigated.
>>
>> There could be two reasons:
>> - there is memory leak but happens in some cases that you don't reproduce
>> in lab, but they are in the production environment
>> - you get memory fragmentation
>>
>> Let's see first the debug messages...
>
>
> Hello,
> here are the link for openser.log and cfg files:
> http://www.yousendit.com/download/bVlEV0o4R3NoeWJIRGc9PQ
>
> After compilation with debug flags for memory manager, I left openser
> running in production for 24 hours. Then, I moved all traffic to another
> host and waited for more than 30 minutes before stopping openser.
> In the openser.cfg, I set debug=2. If you need, I can run it again with a
> higher value (but I hope it doesn't have to be too high, due to overhead
> concerns).
>

Sorry, I forgot to tell one thing: the last revision that showed this
problem was 4809, so we reverted back to that revision before performing the
above.


>
> regards,
> takeshi
>
>>
>>
>>
>>  And this is much more than in our production environment, with just 600
>>> subscribers and about 2000 calls a day.
>>>
>>> The frequency the problem happens is increasing with the number of
>>> subscribers, so we are performing periodic restart of openser (actually,
>>> what we do is to switch over to the standby server). We already recompiled
>>> openser with pkg memory  pool size set to 4MB so that this will not have to
>>> be done frequently.
>>>
>>> Since we cannot recreate this in our lab, we suspect there is a situation
>>> happening in production that might not be having been properly handled by
>>> openser.cfg. So my question is: would it be possible to an overlooked detail
>>> in openser.cfg to cause pkg memory problem?
>>>
>>>
>>> In case someone could take a look at it, here's our cfg file:
>>>
>>>  [...]
>>
>> --
>> Daniel-Constantin Mierla
>> http://www.asipto.com
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kamailio.org/pipermail/users/attachments/20080926/943b554d/attachment.htm