Hi,
 
I recompiled with MEMDBG=1 and installed.
here are the results for reloading 5 million rows with MTREE:
 
mem_join=1 -->takes 56 seconds and the real_used_size of shmem is around 1.2Gb.
mem_join=0 --> takes 10 seconds and the real_used_size of shmem is around 2Gb
does it seems normal?
56 seconds is a lot of time......
 
by the way, when the f_malloc was used, the size of the real_used shmem was twice smaller.
 
Thanks.
On Tue, Nov 20, 2012 at 9:45 PM, Daniel-Constantin Mierla <miconda@gmail.com> wrote:
Hello,


On 11/20/12 7:34 PM, Uri Shacked wrote:
Hi,
 
can you be a litle more specific of the steps of the install and where do i make the changes?

in the source tree, edit the file Makefile.defs and set:

MEMDBG=1

then run:

make all
make install


some words of what is the diff between f_malloc and q_malloc will be great :-).

q_malloc is more debugging purposes, keeping more information for each chunk, therefore the overhead is a bit higher than with f_malloc, but because keeps more details, it is faster to find the fragments that can be joined.

Cheers,
Daniel


 
thanks,
Uri

On Tue, Nov 20, 2012 at 6:26 PM, Daniel-Constantin Mierla <miconda@gmail.com> wrote:
Hello,

ok, I will look over it. At this moment the f_malloc (which is enabled for 3.3) has a pretty inefficient mem join implementation, can you try with q_malloc? Edit Makefile.defs and set:

MEMDBG=1

Then compile and install.

The join operation should be faster, let's see if you get blocking issues with this one.

Cheers,
Daniel


On 11/20/12 2:57 PM, Uri Shacked wrote:
Daniel hi,
 
I attached 2 txt files.
One with mem_join=1, the other with mem_join=0, and the info you asked for.
Let me know if it is OK.
 
Thanks,
Uri

On Mon, Nov 19, 2012 at 10:50 AM, Daniel-Constantin Mierla <miconda@gmail.com> wrote:
Hello,

if you set memjoin to 0, do you see any difference?

Can you try again (with memjoin 1 as well as 0) and send the output of:

kamctl mi get_statistics shmem:

before executing the reload commands?

When it gets to 100%, can you see which process is using the cpu and attach to it with:

gdb /path/to/kamailio PID

then do:

bt full

and send output here?

Cheers,
Daniel


On 11/18/12 4:09 PM, Uri Shacked wrote:
After some testing I notice the following:
First reload of 5 million records after kamailio started took about 9 sec.
Second reload (4 minutes after the first one) took 60 sec.
The third one (again about 4 minutes after the secind) got kamailio to use 100% cpu and after 13 minutes! i killed it.....
 
I can understand that the memory manger works harder, still, any ideas on how to use mem_join and keep on reloading data.
(in real life our data loads 5 million records once a day when almost no traffic. still after a few days it stops...)
 
Thanks,
Uri


 
On Sun, Nov 18, 2012 at 11:52 AM, Uri Shacked <ushacked@gmail.com> wrote:

Hi,

I am using MTREE and DIALPLAN modules to load lots of info to kamailio. (6 million rows).

When kamailio was running with 3.2.1 (no mem_join=1 option), the used size was increasing but the process of loading the data was fast eanough.

I upgraded to 3.3.2 and set mem_join=1. Now the loading process take about 10 time longer and sometimes stops kamailio from responding to traffic.

Any ideas?

Thanks,

Uri




_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- 
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda


-- 
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda


-- 
Daniel-Constantin Mierla - http://www.asipto.com
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda