Hi
On Wed, Nov 16, 2011 at 8:13 AM, Daniel-Constantin Mierla <miconda@gmail.com
wrote:
Hello,
On 11/15/11 10:09 PM, Javier Gallart wrote:
Hello
coming back to the topic related to mtree, to be able to set values via mi/rpc -- it won't be that difficult to add such functionality. Usually with a tree is mainly reading, due to fast matching on tree indexing. The question is how many times/often do you need to change values and how many of them at the same time (more or less).
I was also thinking about that: our application works in such a way
that the full tree is recalculated every 15 minutes. Currently we have aound 60 branches with 150000 entries each.
I assume many times the changes will be somewhere down the tree, since the first part of the number is usually the same (e.g., country code and operator prefix). To update the tree at runtime, while there are reads on it, there must be used a lock to be safe an consistent. If you do lot of writes and very often, then you keep the tree structure locked a lot, slowing the search.
That's the case, tname is rarely updated, it's tvalue the column that we normally update.
I guess you meant actually tprefix is not updated -- tname is table name, tprefix is the prefix used to build the tree. What would be the percentage of tvalue updates, do you have add/remove of tprefix-es?
You're right, sorry for the wrong names. The tprefixes are only updated once a day, and only a tiny percentage of them (less than 1%) actually changes.
Can you estimate the number of writes and how often they would be on a daily basis? There might be other solutions to look at, if writes are very often.
From the numbers above, let's say that we need to update around 8M records every 15 minutes (we expect this number will keep steadily increasing...)
But not all of them change the value, right?
I've been checking this, and on average about 40% are updated in each reload.
As a side note, I've looked at how to implement the mt_match equivalent in redis and it does't look that hard using kamailio s.prefixed transformation (as you suggested) and redis sorted sets. I'll need to make more tests to check the performance, I'll share the results.
I have used this solution with mysql (using a table structure in memory of mysql server), since the changes could have been done in mysql in normal way, in one update query, which is a sync operation. Does redis do any caching or is using always the disk file db?
All the data in Redis stays in memory, you can dump periodic snapshots to disk. Some time ago they worked in a feature called disktore, with some cache mechanism between disk and memory, but as far as I know that line was abandoned.
Cheers, Daniel
Thanks!
Javi
Cheers, Daniel
Regards
Javi
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- http://www.asipto.com Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kathttp://linkedin.com/in/miconda -- http://twitter.com/miconda