[sr-dev] Adding new memory manager with the new API

[Daniel-Constantin Mierla]

Hello,

during Kamailio World I and Camille discussed briefly about enhancing
kamailio to support selecting the memory manager without recompilation.
This morning I pushed a bunch of patches allowing this via command line
parameters.

I am writing now to describe shortly the process of adding a new memory
manager in the list of active ones.

1) a define should be available in the Makefile.defs to enable disable
that memory manger completely during compilation (right now there are
Q_MALLOC and F_MALLOC). The files with the code of the memory manager
must be wrapped in such define

2) each memory manager has to export a set of function to do pkg and shm
memory operations -- they are different as shm management requires some
'unsafe' functions (when locking must not be done). There are two API
structures defined in mem/memapi.h

3) the new memory manger has to export two functions to initialize the
memory management for pkg and shm. Theyr prototypes have to be added in
mem/memcore.h. The implementation of those two functions must be done in
the files with the code of the memory manager

4) extend memory manger conditions with the new name inside mem/pkg.c:
pkg_init_manager(...)  and mem/shm.c:shm_init_manager(...).

5) code of the memory manger has to be extended to export the two apis
for pkg and shm -- looking at the last part of mem/f_malloc.c should
give rather straightforward code to follow

6) debugging mode for memory manager must be enabled if DBG_SR_MEMORY is
defined, because the API prototypes depend on it. For example,
mem/f_malloc.h at the top enables debugging for f_malloc via:

#ifdef DBG_SR_MEMORY
#define DBG_F_MALLOC
#endif

General remarks:

- it is theoretically possible to use one memory manger for shm and
another one for pkg (e.g., pkg can use system malloc)
- shm has some common api to allocate the pool of shared memory and
destroy it, the manager should request the pool from the core api.
Destroying of the shared pool is done also in the common code.
- pkg is allocating directly with malloc the pool, so it is done inside
each manager. Destroying at shut down has to be done there as well
- the plan is to extend the code to allow loading the memory manager
from a module -- it may need some time, for the moment existing memory
managers can be converted and be left in the core

Short term plans/goals:

- hopefully Camille will find some spare time to convert TLSF to the new
memory management api -- I can assist when needed
- I plan to add a system malloc custom manager to be able to use system
memory without recompilation. This will be different than compiling
kamailio without PKG_MALLOC (which does inline replacement with
malloc/free/...)
- maybe resurrect other memory managers that are implemented in mem/
folder -- at least Doug Lea malloc seems to worth the troubles
- said already, enhance so the code for memory malloc can be inside a
module, to move out of core some of those managers

The new memory management api is at beginning, suggestions to adjust it
are welcome.

Cheers,
Daniel

-- 
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Book: SIP Routing With Kamailio - http://www.asipto.com
Kamailio Advanced Training, Sep 28-30, 2015, in Berlin - http://asipto.com/u/kat