12 Jun
2007
12 Jun
'07
4:10 a.m.
Hello,
We have exactly the same problem. Openser after a few hours starts loging "out of
memory" errors and soon after that it cannot process any messages. I am using 1.2
branch from SVN and the last export that I got was at the begining of June, the revision
number was 2333. Today I was able to catch the problem while it is happening and I did a
memory dump on the process that run out of memory (process 4197). ps shows that this
process is a receiver with id=1 and child=0. I have not restarted the server yet, but
pretty soon I will have to restart because more processes will run out of memory. If there
is some additional information (statistics, ngrep etc) that I could gather while the
server is still running, please let me know. I don't have quick access to a public ftp
site, so if someone can take a look at the log file and the memory dump, please let me
know and I will email it (1MB).
thank you for any help.
George
Date: Mon, 11 Jun 2007 17:52:08 +0200
From: Klaus Darilion <klaus.mailinglists(a)pernau.at>
Subject: Re: [Users] Memory issues
To: Christian <mailing-lists(a)switzernet.com>
Cc: users(a)openser.org
Message-ID: <466D6FA8.9010201(a)pernau.at>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Christian wrote:
Thank you Bogdan,
For the moment I am not very sure to switch to SVN branch (this is a
production server, so every minor bug can be dramatic)
SVN != unstable
SVN is the code repository. In the repository there are different branches. E.g. there is
1.1: openser 1.1 branch
1.2: openser 1.2 branch
trunk: openser development branch
Thus, you should update to the 1.2 branch from SVN. This is the currents table release
with all bugfixes:
But I will at least upgrade to 1.2.1 asap, and see if
the issue gets
solved.
1.2.1 is a snapshot taken from 1.2 branch
regards
klaus
The main issue here is that the memory leak
doesn't crash openser, but
puts it in a state where messages are accepted but not relayed.
Shouldn't it rather send a SIP error reply (500 I guess), so that
handover on other servers gets properly handled?
Regards,
Christian
Bogdan-Andrei Iancu a ιcrit :
Disclaimer
The information in this e-mail and any attachments is confidential. It is intended solely
for the attention and use of the named addressee(s). If you are not the intended
recipient, or person responsible for delivering this information to the intended
recipient, please notify the sender immediately. Unless you are the intended recipient or
his/her representative you are not authorized to, and must not, read, copy, distribute,
use or retain this message or any part of it. E-mail transmission cannot be guaranteed to
be secure or error-free as information could be intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain viruses.
it looks like PKG memory problem.
Christian, try to update to the latest 1.2 version from SVN and see
if the problem disappears - there were some pkg mem leak fixed
since the 1.2 release.
regards,
bogdan
Klaus Darilion wrote:
_______________________________________________
Users mailing list
Users(a)openser.org
http://openser.org/cgi-bin/mailman/listinfo/users
<http://openser.org/cgi-bin/mailman/listinfo/users>
http://www.openser.org/dokuwiki/doku.php/troubleshooting:memory
<http://www.openser.org/dokuwiki/doku.php/troubleshooting:memory>
Make sure to use newest 1.2 (make a SVN checkout of /branches/1.2/ )
regards
klaus
Christian wrote:
Hi All,
I have a recurrent memory problem with openser 1.2.0. In this case
it only acts as a SIP router. No registration, accounting etc. and
only approx. 20 simultaneous calls.
Here is the problem: everything works fine (calls from/to PSTN
through different SIP servers), but after a few days, "out of
memory" errors start to occur. After this, calls don't work
anymore, even if openser continues to run.
Here is a part of the log, showing the first errors:
Jun 5 17:25:43 localhost SER[27772]: ERROR:
build_res_buf_from_sip_res: out of mem Jun 5 17:25:43 localhost
SER[27772]: ERROR:tm:relay_reply: no mem for outbound reply buffer
Jun 5 17:25:43 localhost SER[27772]: ERROR: via_builder: out of
memory Jun 5 17:25:43 localhost SER[27772]: ERROR: build_local: no
via header got from builder Jun 5 17:25:43 localhost SER[27772]:
ERROR: attempt to build a CANCEL failed Jun 5 17:25:50 localhost
SER[27772]: ERROR:
build_res_buf_from_sip_res: out of mem Jun 5 17:25:50 localhost
SER[27772]: ERROR:tm:relay_reply: no mem for outbound reply buffer
Jun 5 17:25:50 localhost SER[27772]: ERROR:
build_res_buf_from_sip_req: out of memory ; needs 410 Jun 5
17:26:11 localhost SER[27772]: ERROR:
build_res_buf_from_sip_res: out of mem Jun 5 17:26:11 localhost
SER[27772]: ERROR:tm:relay_reply: no mem for outbound reply buffer
Jun 5 17:26:11 localhost SER[27772]: ERROR:
build_res_buf_from_sip_req: out of memory ; needs 404 Jun 5
17:26:25 localhost SER[27772]: eval_elem: no more memory Jun 5
17:26:25 localhost SER[27772]: do_assign: no value in right
expression Jun 5 17:26:25 localhost SER[27772]: ERROR:
build_res_buf_from_sip_res: out of mem Jun 5 17:26:25 localhost
SER[27772]: ERROR:tm:relay_reply: no mem for outbound reply buffer
Jun 5 17:26:25 localhost SER[27772]: ERROR:
build_res_buf_from_sip_req: out of memory ; needs 420 Jun 5
17:26:25 localhost SER[27771]: ERROR:forward_reply: no 2nd via
found in reply Jun 5 17:26:31 localhost SER[27772]: ERROR:
build_res_buf_from_sip_res: out of mem Jun 5 17:26:31 localhost
SER[27772]: ERROR:forward_reply: building rpl from req failed Jun
5 17:26:33 localhost SER[27772]: ERROR:
build_res_buf_from_sip_res: out of mem Jun 5 17:26:33 localhost
SER[27772]: ERROR:forward_reply: building rpl from req failed Jun
5 17:26:38 localhost SER[27772]: ERROR: insert_new_lump_after:
out of memory
Jun 5 17:26:38 localhost SER[27772]: ERROR: insert_new_lump_before:
out of memory
Jun 5 17:26:38 localhost SER[27772]: insert_RR(): Error while
inserting conditional lump Jun 5 17:26:38 localhost SER[27772]:
ERROR: subst_str: mem.
allocation error (res->s)
Jun 5 17:26:38 localhost SER[27772]: ERROR: do_action: memory
allocation failure Jun 5 17:26:38 localhost SER[27772]: ERROR:
build_res_buf_from_sip_req: out of memory ; needs 375 Jun 5
17:26:38 localhost SER[27772]: ERROR: mk_proxy: memory allocation
failure Jun 5 17:26:38 localhost SER[27772]:
ERROR:tm:t_forward_nonack:
failure to add branches
At this point, similar errors continue to appear and the only way
to recover is to restart openser.
Nothing unusual seems to cause the initial error. The output of
"top" appears to be fine after the "crash", and there is still
plenty free memory on the server.
Could anybody give me some hints on how to solve the problem? I
suspect a portion of the config file where the record-route headers
are split on multiple lines (for compatibility issues with another
SIP proxy). It's the only place where variables are used. You can
find it here:
http://switzernet.com/people/christian-lathion/files/070605-openser
<http://switzernet.com/people/christian-lathion/files/070605-openser>
-config-part.txt
Otherwise the config file is quite usual, with basic processing and
routing of SIP packets.
Regards,
Christian
_______________________________________________
Users mailing list
Users(a)openser.org
http://openser.org/cgi-bin/mailman/listinfo/users
<http://openser.org/cgi-bin/mailman/listinfo/users>
_______________________________________________
Users mailing list
Users(a)openser.org
http://openser.org/cgi-bin/mailman/listinfo/users
<http://openser.org/cgi-bin/mailman/listinfo/users>