[Serusers] SER Reports "out of memory"

[Java Rockx]

Zeus,

Thanks for the info. I did change that config.h define and now it works 
well. 

My newest problem is the ser start time. In my very non-scientific test it 
took ser about 25 minutes before it began serving requests because it was 
loading usrloc information.

That was using 500000 records in the location table. The MySQL server was 
running on the same box as SER, which is also my workstation, so stuff like 
Firefox, X, etc, were in use.

But this does bring up an interesting problem namely - how can ser service 
SIP clients while loading large number of usrloc records? I'm kind of 
thinking that this could be a big problem. When dealing with massive user 
bases there is no such thing as a "quick restart".

We do have LVS fully "sip-aware" so we are doing true UDP load balancing 
based on the Call-ID header, but this is still a problem [potentially] with 
replication ucontact records while the server is starting up.

I wonder if it is possible to modify the behaviour of usrloc so that it 
loads in the background while ser is processing SIP messages. And when 
lookup("location") is executed, usrloc searching the the ser cache and then 
MySQL if no hit is found in cache -- or something like that.

Can anyone on serusers give some tips as to how to get ser to load usrloc 
entries optimized? I know the usual stuff like faster MySQL disks, faster 
network connection, dedicated app servers, etc, etc. But I'm looking for ser 
and/or MySQL tweaking hacks.

Regards,
Paul

On 5/24/05, Zeus Ng <zeus.ng at isquare.com.au> wrote:
> 
> Paul,
> 
> Like you, I'm not totally familiar with SER's memory management. I just
> happened to bookmark that link which I think is useful.
> 
> Before we jump into conclusion that SER memory management is at flaw, can
> you just recompile SER and confirm that change fix your problem. I would
> like to know the result too.
> 
> The loading of location table is done in the mod_init() function, not
> child_init() function. Base on that information, I would assume that only
> the master process has to allocate that much memory. The child processes
> need only know the pointer to that list. Thus, there is no need of 4G as 
> in
> your example. Again, this is only my assumption. It's better to raise
> another thread to ask the maintainer.
> 
> Regards,
> 
> Zeus
> 
> 
> >
> > Zeus...thanks for the post.
> >
> > However, it does a raise new (and possibly disturbing)
> > question about SER's memory usage architecture.
> >
> > Here is the question; If I have 500000 records in the
> > location table and for the sake of this example, let's assume
> > that each usrloc contact occupies 1024 bytes of memory. To
> > load all 500000 usrloc records we would need (500000 x 1024)
> > bytes of memory plus the structure overhead. So let's just
> > estimate it to be a total of 512MB of memory for discussions
> > sake. (I realize that the actual memory requirements will be
> > much lower).
> >
> > Now when SER starts up it needs to load all usrloc records in
> > to memory. Per the email in the archives, SER will do this in
> > private memory space. I assume that after all usrloc records
> > are loaded in to private memory that they are then copied to
> > shared memory. The amount of private memory is specified in
> > config.h as PKG_MEM_POOL_SIZE and affects __ALL__ ser processes.
> >
> > This then implies that in order to load a large number of
> > usrloc records I need to increase PKG_MEM_POOL_SIZE to
> > approximately 512MB which then affects __ALL__ ser processes.
> > If I specify ser to use 8 child processes in my ser.cfg file
> > then will ser attempt to allocate 512MB x 8 or (4GB) of
> > memory just to start up???
> >
> > If this is the case when the fundamental architecture is
> > flawed as it would make much more sense to have a single
> > process load usrloc records in to shared memory and then fork
> > the children with a much smaller amount of private memory.
> >
> > Am I totally missing something here?
> >
> > Regards,
> > Paul
> >
> >
> >
> > On 5/23/05, Zeus Ng <zeus.ng at isquare.com.au> wrote:
> >
> > This may help.
> >
> >
> > http://lists.iptel.org/pipermail/serusers/2004-December/013593.html
> >
> > > -----Original Message-----
> > > From: serusers-bounces at lists.iptel.org
> > > [mailto:serusers-bounces at lists.iptel.org] On Behalf Of Java Rockx
> > > Sent: Tuesday, 24 May 2005 11:09 AM
> > > To: serusers
> > > Subject: [Serusers] SER Reports "out of memory"
> > >
> > >
> > > Hi All.
> > >
> > > I'm using ser-0.9.2 and testing how it works when the
> > > location table has many thousands of rows.
> > >
> > > ser will not start when the location table has many rows. SER
> > > loads the first 1344 records and then pukes on the 1345th
> > > record with an out of memory error. I've started ser with "-m
> > > 512" but that doesn't seem to make any difference.
> > >
> > > Am I doing something wrong?
> > >
> > > Also, for my "test" data I simply created a bunch of bogus
> > > records in the location table. I did not do anything with the
> > > subscriber table. Does this matter? If so, what needs to be
> > > done in order to create a large number of "registered" users?
> > >
> > > Regards,
> > > Paul
> > >
> > >
> >
> >
> >
> >
> >
> 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20050524/272eca7b/attachment.htm>