[SR-Users] reload to memory is much slower and has problms after upgrade to 3.3.2 and using mem_join=1

Uri Shacked ushacked at gmail.com
Wed Nov 21 13:33:49 CET 2012


Hi,

I recompiled with MEMDBG=1 and installed.
here are the results for reloading 5 million rows with MTREE:

mem_join=1 -->takes 56 seconds and the real_used_size of shmem is around
1.2Gb.
mem_join=0 --> takes 10 seconds and the real_used_size of shmem is around
2Gb
does it seems normal?
56 seconds is a lot of time......

by the way, when the f_malloc was used, the size of the real_used shmem was
twice smaller.

Thanks.
On Tue, Nov 20, 2012 at 9:45 PM, Daniel-Constantin Mierla <miconda at gmail.com
> wrote:

>  Hello,
>
>
> On 11/20/12 7:34 PM, Uri Shacked wrote:
>
>  Hi,
>
> can you be a litle more specific of the steps of the install and where do
> i make the changes?
>
>
> in the source tree, edit the file Makefile.defs and set:
>
> MEMDBG=1
>
> then run:
>
> make all
> make install
>
>
>  some words of what is the diff between f_malloc and q_malloc will be
> great :-).
>
>
> q_malloc is more debugging purposes, keeping more information for each
> chunk, therefore the overhead is a bit higher than with f_malloc, but
> because keeps more details, it is faster to find the fragments that can be
> joined.
>
> Cheers,
> Daniel
>
>
>
> thanks,
> Uri
>
>  On Tue, Nov 20, 2012 at 6:26 PM, Daniel-Constantin Mierla <
> miconda at gmail.com> wrote:
>
>>  Hello,
>>
>> ok, I will look over it. At this moment the f_malloc (which is enabled
>> for 3.3) has a pretty inefficient mem join implementation, can you try with
>> q_malloc? Edit Makefile.defs and set:
>>
>> MEMDBG=1
>>
>> Then compile and install.
>>
>> The join operation should be faster, let's see if you get blocking issues
>> with this one.
>>
>> Cheers,
>> Daniel
>>
>>
>> On 11/20/12 2:57 PM, Uri Shacked wrote:
>>
>>  Daniel hi,
>>
>> I attached 2 txt files.
>> One with mem_join=1, the other with mem_join=0, and the info you asked
>> for.
>> Let me know if it is OK.
>>
>> Thanks,
>> Uri
>>
>>  On Mon, Nov 19, 2012 at 10:50 AM, Daniel-Constantin Mierla <
>> miconda at gmail.com> wrote:
>>
>>>  Hello,
>>>
>>> if you set memjoin to 0, do you see any difference?
>>>
>>> Can you try again (with memjoin 1 as well as 0) and send the output of:
>>>
>>> kamctl mi get_statistics shmem:
>>>
>>> before executing the reload commands?
>>>
>>> When it gets to 100%, can you see which process is using the cpu and
>>> attach to it with:
>>>
>>> gdb /path/to/kamailio PID
>>>
>>> then do:
>>>
>>> bt full
>>>
>>> and send output here?
>>>
>>> Cheers,
>>> Daniel
>>>
>>>
>>> On 11/18/12 4:09 PM, Uri Shacked wrote:
>>>
>>>   After some testing I notice the following:
>>> First reload of 5 million records after kamailio started took about 9
>>> sec.
>>> Second reload (4 minutes after the first one) took 60 sec.
>>> The third one (again about 4 minutes after the secind) got kamailio to
>>> use 100% cpu and after 13 minutes! i killed it.....
>>>
>>> I can understand that the memory manger works harder, still, any ideas
>>> on how to use mem_join and keep on reloading data.
>>> (in real life our data loads 5 million records once a day when almost no
>>> traffic. still after a few days it stops...)
>>>
>>> Thanks,
>>> Uri
>>>
>>>
>>>
>>> On Sun, Nov 18, 2012 at 11:52 AM, Uri Shacked <ushacked at gmail.com>wrote:
>>>
>>>>   Hi,
>>>>
>>>> I am using MTREE and DIALPLAN modules to load lots of info to kamailio.
>>>> (6 million rows).
>>>>
>>>> When kamailio was running with 3.2.1 (no mem_join=1 option), the used
>>>> size was increasing but the process of loading the data was fast eanough.
>>>>
>>>> I upgraded to 3.3.2 and set mem_join=1. Now the loading process take
>>>> about 10 time longer and sometimes stops kamailio from responding to
>>>> traffic.
>>>>
>>>> Any ideas?
>>>>
>>>> Thanks,
>>>>
>>>> Uri
>>>>
>>>
>>>
>>>
>>>  _______________________________________________
>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users at lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>>
>>> --
>>> Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
>>>
>>>
>>
>> --
>> Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
>>
>>
>
> --
> Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20121121/47ff34e8/attachment.htm>


More information about the sr-users mailing list