[OpenSER-Devel] [ openser-Bugs-1960561 ] Package Memory Leak in Parser

[SourceForge.net]

Bugs item #1960561, was opened at 2008-05-08 18:04
Message generated for change (Comment added) made by smullen27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=743020&aid=1960561&group_id=139143

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: core
Group: ver 1.3.x
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Scott Mullen (smullen27)
Assigned to: Bogdan-Andrei Iancu (bogdan_iancu)
Summary: Package Memory Leak in Parser

Initial Comment:
I'm running openser 1.3.0 currently and I think I've discovered a memory leak in the parser code.

I've recompiled the code already to use the debug option DBG_QM_MALLOC.  I've taken an initial snapshot of the memory right after startup.  I've also taken 4 additional snapshots after some load has been running on the box.  For each of the additional snapshots, I've let all of the load run off of the box prior to taking the snapshot.

A gzip'd tar file can be downloaded from:
http://mullen.dyndns.org/openser/OpenserMemory.tar.gz

threadDump.0508-HHMMSS-1.txt (actual dump from kill signal)
threadAnalysis.0508-HHMMSS-1.txt (counts of each instance in pkg memory from the dump file)


threadAnalysis.0508-153951-1.txt
threadAnalysis.0508-164349-1.txt
threadAnalysis.0508-164600-1.txt
threadAnalysis.0508-165217-1.txt
threadAnalysis.0508-170038-1.txt
threadAnalysis.0508-181908-1.txt
threadDump.0508-153951-1.txt
threadDump.0508-164349-1.txt
threadDump.0508-164600-1.txt
threadDump.0508-170038-1.txt
threadDump.0508-181908-1.txt
threadDump.0508-165217-1.txt


I can also attach gdb to the process to print out anything that could be helpful as well.

----------------------------------------------------------------------

>Comment By: Scott Mullen (smullen27)
Date: 2008-05-10 20:11

Message:
Logged In: YES 
user_id=2082554
Originator: YES

Hi Bogdan


I am not using any perl modules at all in this setup.  Currently the
config is 1368 lines long, although this call is only traversing through a
small subset of that.

The text files that I submitted with the bug report contain the memory
information for all of the threads of these processes.  The threadAnalysis
files contain the counts of each instance of allocation of package memory
for all of the processes by file and line.  (basically I've parsed the
threadDump file that was generated with a kill -SIGUSR1 for each thread of
the openser process into something a little more manageable to see what was
leaking)   The first file by time (files are in format
threadDump.MMYY-HHMMSS-t.txt)is right at start-up with no traffic on the
system.  Each file there after is another snapshot of the memory after some
 calls have run through the proxy but have been allowed to drain off.  Most
all of the memory is stable except for the code in the parser.  This
continues to grow after each time.

Please let me know if I can grab any other data to help resolve this
issue.

Regards

Scott

----------------------------------------------------------------------

Comment By: Bogdan-Andrei Iancu (bogdan_iancu)
Date: 2008-05-10 17:59

Message:
Logged In: YES 
user_id=1275325
Originator: NO

Hi Scott,

First of all, two questions:
  1) are you using perl modules?
  2) how many lines your cfg script has?

You mention in your report a leak in the parser code - could point out an
example of parse leak?

Regards,
Bogdan

----------------------------------------------------------------------

Comment By: Bogdan-Andrei Iancu (bogdan_iancu)
Date: 2008-05-09 15:44

Message:
Logged In: YES 
user_id=1275325
Originator: NO

Hi Scott,

I will investigate this. Thanks for the report.

Regards,
Bogdan

----------------------------------------------------------------------

Comment By: Scott Mullen (smullen27)
Date: 2008-05-09 14:51

Message:
Logged In: YES 
user_id=2082554
Originator: YES

I've attached two call flows that are causing the memory leaks.  Looks
like there are "bad" UA's on either side of openser for this, but
none-the-less this should not cause a leak.

CallFlow1.txt:   All Invites below the Initial Invite->200 OK->ACK are
Re-Invites with the same callid, same To tag, same From tag and same CSEQ.

CallFlow2.txt:   All of the Invites in this call flow are Initial Invites
with the same callid, same From tag, Same CSEQ and NO To tag.

Sorry about the formatting of these text files.  They are dumped out of
wireshark to a text file and the format is not perfect, but you can make
sense of them.

Scott

----------------------------------------------------------------------

Comment By: Scott Mullen (smullen27)
Date: 2008-05-09 14:46

Message:
Logged In: YES 
user_id=2082554
Originator: YES

File Added: CallFlow2.txt

----------------------------------------------------------------------

Comment By: Scott Mullen (smullen27)
Date: 2008-05-09 14:45

Message:
Logged In: YES 
user_id=2082554
Originator: YES

File Added: CallFlow1.txt

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=743020&aid=1960561&group_id=139143