[SR-Users] PDT ran out of pkg memory using pdt_list

[JR Richardson]

>>> [..]
>>> When I added 180K records in the database, I got the "no more pkg mem"
>>> error again.
>>> I increased again:
>>> #define PKG_MEM_POOL_SIZE 32*1024*1024
>>> This allowed me to execute pdt_list with 180K records loaded.
>>>
>>> I increased database record count to 240K and got the "no more pkg
>>> mem" error again.
>>>
>>> So I don't think it is prudent to just keep increasing
>>> PKG_MEM_POOL_SIZE.  Is this an architectural limitation with fifo
>>> pkg_mem, shouldn't this be a dynamic allocation if it's not within
>>> shmem and has no affect on core sip-router function?  While the
>>> pdt_reload and pdt_list is going on (takes a few seconds to load and
>>> list), I don't see any problems with the sip-router executions.  I
>>> guess I can just use the old fashion database query to look up routes
>>> instead of fifo pdt_list.
>>>
>>
>> Hi Jr,
>>
>> if you don't use the PKG_MALLOC MM in your system at least the kamailio
>> 1.5
>> will fallback to the system malloc, which don't have this size
>> restrictions.
>> For 3.x/ sr there is an option to build the MI interface with system
>> malloc as
>> well, it this commit:
>>
>> commit 85f2302e8448ee20edb12bc3b4e2911ca3b111ec
>> Author: Daniel-Constantin Mierla<miconda at gmail.com>
>> Date:   Mon Jun 29 12:34:08 2009 +0200
>>
>>     lib/kmi: option to build it with system malloc
>>
>>     - MI lib can be built independently of core with system malloc for pkg
>>       memory (see Makefile)
>>
>
> this is the option added to overcome the MI design limitations - while most
> of dumped structures are in shared memory, MI builds the output in PKG
> memory, resulting in insufficient space since shm is much bigger than pkg -
> also this sometime ended as well in lot of mem fragmentation for mi process
> in the past.
>
> It needs to edit lib/kmi/Makefile and enable system malloc, recompile and
> reinstall.
>
> Regards,
> Ramona
>
Hi Ramona,

Thanks for your suggestions.  I started over with default mem
parameters in config.h and compiled with:
## uncomment next line for using system malloc with MI
DEFS+= -DMI_SYSTEM_MALLOC

I loaded the pdt database table with 360K records and also started
kamailio with '-m 512' but the table still has a max record
limitation, will not load propperly and I get these errors:
 0(22456) ERROR: pdt [pdtree.c:283]: bad parameters
 0(22456) INFO: pdt [pdt.c:490]: no prefix found in [2000171212]

Of course wihout a propper table load, pdt_list does not produce any
results (but i don;t get the pkg_mem error any more).

sip-router2:~# kamctl fifo get_statistics all
shmem:total_size = 536870912
shmem:used_size = 1229472
shmem:real_used_size = 1247560
shmem:max_used_size = 1255376
shmem:free_size = 535623352

There seems to still be a bottleneck with shmem, even through I have
512M+ available, I only seem to be using ~1.2M.  Is there an internal
shmem hard limit that can be adjusted?

Thanks

JR
-- 
JR Richardson
Engineering for the Masses