26 Sep
2008
26 Sep
'08
9:24 a.m.
On Tue, Sep 23, 2008 at 6:00 PM, Daniel-Constantin Mierla <miconda(a)gmail.com
wrote:
Hello,
On 09/23/08 10:31, mayamatakeshi wrote:
Hello,
here are the link for openser.log and cfg files:
http://www.yousendit.com/download/bVlEV0o4R3NoeWJIRGc9PQ
After compilation with debug flags for memory manager, I left openser
running in production for 24 hours. Then, I moved all traffic to another
host and waited for more than 30 minutes before stopping openser.
In the openser.cfg, I set debug=2. If you need, I can run it again with a
higher value (but I hope it doesn't have to be too high, due to overhead
concerns).
regards,
takeshi
Hello,
we have openser 1.3.3 running in production (current rev.: 4943).
For 3 times in 50 days we had to restart openser to correct pkg memory
problem.
openser 1.3.3 was released 3 weeks ago, so I guess you were running
previous version before, but it happened again since you upgraded to 1.3.3,
right?
After some time logging messages like this:
/openser.log:Aug 19 10:39:18 ipx022
/usr/local/sbin/openser[16991]:
ERROR:core:new_credentials: no pkg memory left,
openser will eventually run out of pkg memory and refuse all subsequent
requests.
We are trying to recreate this in our lab so that we can follow memory
troubleshooting instructions at
http://kamailio.net/dokuwiki/doku.php/troubleshooting:memory, but so far
we were unable to do it even when generating millions of calls and
registration transactions (we are using SIPp to generate normal call flows
and even abnormal call flows detected when reading openser.log, like
'invalid cseq for aor', malformed SIP messages etc).
We can spot memory leaks even the "out of memory" message is not
printed.
Just archive the logs (the most important is the shut down time) and made
them available for download so they can be investigated.
There could be two reasons:
- there is memory leak but happens in some cases that you don't reproduce
in lab, but they are in the production environment
- you get memory fragmentation
Let's see first the debug messages...
And this is much more than in our production environment, with just 600
subscribers and about 2000 calls a day.
The frequency the problem happens is increasing with the number of
subscribers, so we are performing periodic restart of openser (actually,
what we do is to switch over to the standby server). We already recompiled
openser with pkg memory pool size set to 4MB so that this will not have to
be done frequently.
Since we cannot recreate this in our lab, we suspect there is a situation
happening in production that might not be having been properly handled by
openser.cfg. So my question is: would it be possible to an overlooked detail
in openser.cfg to cause pkg memory problem?
In case someone could take a look at it, here's our cfg file:
[...]
--
Daniel-Constantin Mierla
http://www.asipto.com