[SR-Users] mtree on db Berkeley DB?

[Javier Gallart]

Hi

On Wed, Nov 16, 2011 at 8:13 AM, Daniel-Constantin Mierla <miconda at gmail.com
> wrote:

>  Hello,
>
>
> On 11/15/11 10:09 PM, Javier Gallart wrote:
>
>  Hello
>>
>> coming back to the topic related to mtree, to be able to set values via
>> mi/rpc -- it won't be that difficult to add such functionality. Usually
>> with a tree is mainly reading, due to fast matching on tree indexing.
>> The question is how many times/often do you need to change values and
>> how many of them at the same time (more or less).
>>
>>  I was also thinking about that: our application works in such a way
> that the full tree is recalculated every 15 minutes. Currently we have
> aound 60 branches with 150000 entries each.
>
>> I assume many times the changes will be somewhere down the tree, since
>> the first part of the number is usually the same (e.g., country code and
>> operator prefix). To update the tree at runtime, while there are reads
>> on it, there must be used a lock to be safe an consistent. If you do lot
>> of writes and very often, then you keep the tree structure locked a lot,
>> slowing the search.
>>
>
>  That's the case, tname is rarely updated, it's tvalue the column that we
> normally update.
>
>
> I guess you meant actually tprefix is not updated -- tname is table name,
> tprefix is the prefix used to build the tree. What would be the percentage
> of tvalue updates, do you have add/remove of tprefix-es?
>
You're right, sorry for the wrong names. The tprefixes are only updated
once a day, and only a tiny percentage of them (less than 1%) actually
changes.

>
>
>
>> Can you estimate the number of writes and how often they would be on a
>> daily basis? There might be other solutions to look at, if writes are
>> very often.
>>
>
>  From the numbers above, let's say that we need to update around 8M
> records every 15 minutes (we expect this number will keep steadily
> increasing...)
>
>
> But not all of them change the value, right?
>
I've been checking this, and  on average about 40% are updated in each
reload.

>
>
>
>  As a side note, I've looked at how to implement the mt_match equivalent
> in redis and it does't look that hard using kamailio s.prefixed
> transformation (as you suggested) and redis sorted sets. I'll need to make
> more tests to check the performance, I'll share the results.
>
>
> I have used this solution with mysql (using a table structure in memory of
> mysql server), since the changes could have been done in mysql in normal
> way, in one update query, which is a sync operation. Does redis do any
> caching or is using always the disk file db?
>
All the data in Redis stays in memory, you can dump periodic snapshots to
disk. Some time ago they worked in a feature called disktore, with some
cache mechanism between disk and memory, but as far as I know that line was
abandoned.

>
> Cheers,
> Daniel
>
>
>  Thanks!
>
>  Javi
>
>>
>> Cheers,
>> Daniel
>> >
>> > Regards
>> >
>> > Javi
>>
>
>
> _______________________________________________
> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users at lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>
>
> --
> Daniel-Constantin Mierla -- http://www.asipto.com
> Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kathttp://linkedin.com/in/miconda -- http://twitter.com/miconda
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20111116/23badcac/attachment.htm>