[OpenSER-Devel] [ openser-Patches-1998043 ] Fetch Support for Dialog-Module

[Dan Pascu]

On Tuesday 24 June 2008, Henning Westerholt wrote:
> On Friday 20 June 2008, Dan Pascu wrote:
> > IMO, this is a bug fix, more than a new feature and it should be
> > applied to 1.4 before released. We had similar problems in the past
> > with loading usrloc contacts and was fixed in a similar manner.
> >
> > Not being able to load more than 300 dialogs at startup is a serious
> > bug not a missing feature.
> >
> > At this point I'm so tempted to use the system malloc instead of
> > pkg_malloc as I had so many issues with not enough memory or memory
> > fragmentation in the past and even now.
>
> Hello Dan,
>
> this is a limitiation,

It's a very serious limitation. I even call it a bug. If you have to flush 
you dialog state table to be able to restart openser, the whole ability 
to preserve dialog state over restarts is gone out on the window. Under 
these circumstances, why would I even bother to save dialog state if I 
can't reload it.

I had similar problems with usrloc before the code was changed and it was 
not pretty to restart openser. Having to flush the usrloc table goes 
against having the ability to persistently store contacts over restarts.

> but the patch is IMHO to extensive to be applied 
> in the current state of development.

I don't think that is the unit to measure it. If it's a bug fix it doesn't 
matter how large the fix is. If you remember the amount of code that was 
necessary to fix the problems with mi_xmlrpc in 1.3 (and which was pushed 
during the freeze), you'd probably consider this a walk on the beach.

>
> I'm really curious to see a performance comparison between pkg_malloc
> and system malloc for recent glibc implementations. I'm not sure how
> much work they've invest after the merge of ptmalloc2 that went into
> 2.3.0, but the only actual issues i'm aware of are some regressions [1]
> and some multithreading problems [2], which probably not apply for us.

It's not only about performance, for me it's about limitations like this 
that make pkg_malloc unfriendly. Also the openser memory management has a 
very nasty problem with fragmentation. Not that visible with shm_malloc 
since that has a much larger pool to play with, but very visible with 
pkg_malloc. Try to run openserctl fifo ul_dump when you have more than 
2000 contacts. The command will try to build the list to return and 
somewhere along the process will fail because it has not enough pkg 
memory. It'll then release all allocated memory and return an error code. 
You'd think it ends there, but after this, any openserctl command you try 
to run will fail with pkm memory allocation failures, even though almost 
all of the pkm memory is available. But it's so fragmented that it cannot 
allocate even a small chunk for something like openserctl ps
To fix this you have to restart openser.

I don't know how the system malloc handles this, but I guess the OS has 
some sort of memory defragmentater, otherwise the system wouldn't be able 
to run for long with all the applications doing constant memory 
allocations and deallocations.

-- 
Dan