[sr-dev] new memory allocator proposal: TLSF

[Camille Oudot]

Hi,

== Short story:

Currently, there is a (corner case) performance issue in kamailio when
lots of fragments of about the same size are freed in a short time.

== Long story:

I've recently hit a wall using kamailio, dealing with ~150k TLS
connections in parallel: when I stop my injector (sipp), all the
connections are closed at the same time. The TCP main process took ~20
minutes to dispose of all the objects, and was unable to accept any new
connection in the meantime.

The bottleneck lies in the shm_free() code (whether f_malloc or q_malloc
is used), when a free fragment is be inserted in an ordered linked list
(which is O(n) if n is the size of the list). Enabling mem_join did
help a little (less fragments, shorter lists), and the 150k
disconnections scenario went down to ~5 minutes unavailability for the
TCP main process. But this is still big.

== TLSF:

I looked for alternatives to f/q_malloc, and I found TLSF:

  * http://www.gii.upv.es/tlsf/

This allocator has O(1) malloc and free even in every case. This means
no surprise like the one I had previously.

I took the implementation found here:

  * http://tlsf.baisoku.org/

and integrated it in kamailio. My 150k disconnections scenario went
down to 12 seconds, and the fragmentation is as good as q_malloc() with
mem_join.

I've published a branch with TLSF here:

  * https://github.com/kamailio/kamailio/commits/tmp/TLSF_malloc

What do you think of integrating it in kamailio?

By the way, I've tried to play with dl_malloc, ll_malloc and sf_malloc,
but none of these currently compile. Should we maintain them or remove
them from the codebase?

-- 
Camille