[OpenSER-Devel] [ openser-Patches-1998043 ] Fetch Support for Dialog-Module

[Henning Westerholt]

On Tuesday 24 June 2008, Dan Pascu wrote:
> > > IMO, this is a bug fix, more than a new feature and it should be
> > > applied to 1.4 before released. We had similar problems in the past
> > > with loading usrloc contacts and was fixed in a similar manner.
> > >
> > > Not being able to load more than 300 dialogs at startup is a serious
> > > bug not a missing feature.
> > >
> > > At this point I'm so tempted to use the system malloc instead of
> > > pkg_malloc as I had so many issues with not enough memory or memory
> > > fragmentation in the past and even now.
> >
> > this is a limitiation,
>
> It's a very serious limitation. I even call it a bug. If you have to flush
> you dialog state table to be able to restart openser, the whole ability
> to preserve dialog state over restarts is gone out on the window. Under
> these circumstances, why would I even bother to save dialog state if I
> can't reload it.

You can of course work around this problem with increasing the pkg_memory size 
depending on the size of the dialog set.

> > but the patch is IMHO to extensive to be applied
> > in the current state of development.
>
> I don't think that is the unit to measure it. If it's a bug fix it doesn't
> matter how large the fix is. If you remember the amount of code that was
> necessary to fix the problems with mi_xmlrpc in 1.3 (and which was pushed
> during the freeze), you'd probably consider this a walk on the beach.

I remember this one for sure. I just think we need stability more at the 
moment. But i did not work that much in this area, so its not my turn to 
decide this in the end.

> > I'm really curious to see a performance comparison between pkg_malloc
> > and system malloc for recent glibc implementations. I'm not sure how
> > much work they've invest after the merge of ptmalloc2 that went into
> > 2.3.0, but the only actual issues i'm aware of are some regressions [1]
> > and some multithreading problems [2], which probably not apply for us.
>
> It's not only about performance, for me it's about limitations like this
> that make pkg_malloc unfriendly. Also the openser memory management has a
> very nasty problem with fragmentation. Not that visible with shm_malloc
> since that has a much larger pool to play with, but very visible with
> pkg_malloc. Try to run openserctl fifo ul_dump when you have more than
> 2000 contacts. The command will try to build the list to return and
> somewhere along the process will fail because it has not enough pkg
> memory. It'll then release all allocated memory and return an error code.
> You'd think it ends there, but after this, any openserctl command you try
> to run will fail with pkm memory allocation failures, even though almost
> all of the pkm memory is available. But it's so fragmented that it cannot
> allocate even a small chunk for something like openserctl ps
> To fix this you have to restart openser.

This sounds indeed not nice. I've not seen this fragmentation issues, perhaps 
because we're using more pkg_memory, and don't cache that much data in the 
server. If this problem is that easy to reproduce then its a serious issue.

I'm no fan of this pkg_mem stuff either. I don't think we need to maintain our 
own memory allocator. For debugging there exists plugins that does double 
free checks, bound checks and such. Even the plain glibc malloc does now such 
things, i think. Removing this would also ease the debugging, it would be 
possible to use valgrind. 

I already have some core cleanup patches in my repository for 1.5, and plan to 
do some work in this area, apart from some IO stuff.

> I don't know how the system malloc handles this, but I guess the OS has
> some sort of memory defragmentater, otherwise the system wouldn't be able
> to run for long with all the applications doing constant memory
> allocations and deallocations.

AFAIK there is no memory defragmentation in the linux kernel. There were some 
work/ patches in this area, but none of them get in (yet). What is in 2.6.24 
is fragmentation avoidance [1]. Its a know problem for some workloads i think 
[2], but i don't think we'll be able to trigger problems with our server.

Cheers,

Henning


[1] http://lwn.net/Articles/224829/
[2] http://linux-mm.org/ProblemWorkloads