Hello,
so far nathelper was using id column (auto-incremented for mysql) from database table location for deciding what records will be used for sending keepalive requests on a timer execution.
Practically, it was like:
id % max_partitions = current_iteration
max_partitions is computed based on number of nat ping processes and nat ping interval.
We changed the way we mark the records for sending keepalives, in order to have proper indexing -- by adding the keepalive column.
Initially I thought that can be used for keeping the partition index as well, but there is one thing that prevents it: pinging all contacts. So no matter is behind nat or not, there is the option to ping the contact. In this case, the natted state is no longer related to keepalive flag and the keepalive column cannot be used.
The solution is to add a dedicated column for partition index. I already implemented it in a separate branch, named tmp/usrloc-partion. The new code is rather small, because is only about inserting into database, the value is not used when having records in memory, therefore not needed to be loaded back in memory or updated. But it changes the database structure, by adding a new column to location table.
I think this is a fix and still ok to apply in this testing phase. There is no new feature comparing with previous stable, but making it be the same in functionality. However, I wanted to check with all devs and see other opinions. And again, it is only about db_mode=DB_ONLY for usrloc.
The alternative will be to add back the selection based on modulo over id column (that was based on a raw query), which makes it work only with mysql and postgress (like it was so in the old versions), plus, it is not suitable for indexing.
If nobody has anything against, I will merge it.
Cheers, Daniel
Hi Daniel,
just to state that i agree with the implementation of the new column.
Best ________________________________________ From: sr-dev [sr-dev-bounces@lists.sip-router.org] on behalf of Daniel-Constantin Mierla [miconda@gmail.com] Sent: Wednesday, May 06, 2015 9:26 AM To: sr-dev Subject: [sr-dev] nat keepalive partitioning in usrloc db only mode
Hello,
so far nathelper was using id column (auto-incremented for mysql) from database table location for deciding what records will be used for sending keepalive requests on a timer execution.
Practically, it was like:
id % max_partitions = current_iteration
max_partitions is computed based on number of nat ping processes and nat ping interval.
We changed the way we mark the records for sending keepalives, in order to have proper indexing -- by adding the keepalive column.
Initially I thought that can be used for keeping the partition index as well, but there is one thing that prevents it: pinging all contacts. So no matter is behind nat or not, there is the option to ping the contact. In this case, the natted state is no longer related to keepalive flag and the keepalive column cannot be used.
The solution is to add a dedicated column for partition index. I already implemented it in a separate branch, named tmp/usrloc-partion. The new code is rather small, because is only about inserting into database, the value is not used when having records in memory, therefore not needed to be loaded back in memory or updated. But it changes the database structure, by adding a new column to location table.
I think this is a fix and still ok to apply in this testing phase. There is no new feature comparing with previous stable, but making it be the same in functionality. However, I wanted to check with all devs and see other opinions. And again, it is only about db_mode=DB_ONLY for usrloc.
The alternative will be to add back the selection based on modulo over id column (that was based on a raw query), which makes it work only with mysql and postgress (like it was so in the old versions), plus, it is not suitable for indexing.
If nobody has anything against, I will merge it.
Cheers, Daniel
-- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, May 27-29, 2015 Berlin, Germany - http://www.kamailioworld.com
_______________________________________________ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Daniel-Constantin Mierla writes:
I think this is a fix and still ok to apply in this testing phase. There is no new feature comparing with previous stable, but making it be the same in functionality. However, I wanted to check with all devs and see other opinions. And again, it is only about db_mode=DB_ONLY for usrloc.
what is the performance impact of this compared to the present raw query implementation that fetches id modulo x from location table?
also, is there any performance implications on normal lookups and saves in db only mode?
-- juha
On 06/05/15 21:20, Juha Heinanen wrote:
Daniel-Constantin Mierla writes:
I think this is a fix and still ok to apply in this testing phase. There is no new feature comparing with previous stable, but making it be the same in functionality. However, I wanted to check with all devs and see other opinions. And again, it is only about db_mode=DB_ONLY for usrloc.
what is the performance impact of this compared to the present raw query implementation that fetches id modulo x from location table?
It allows to make clear indexes (no bitwise and no module operations in match expression), that speeds up a lot. Anyhow, the main issue was that the old implementation was working for mysql (and perhaps postgress) only.
also, is there any performance implications on normal lookups and saves in db only mode?
lookup is not affected at all. update of existing record is not affected as well. first registration (insert) will set an extra column with integer value.
Daniel
Daniel-Constantin Mierla writes:
The solution is to add a dedicated column for partition index. I already implemented it in a separate branch, named tmp/usrloc-partion. The new code is rather small, because is only about inserting into database, the value is not used when having records in memory, therefore not needed to be loaded back in memory or updated. But it changes the database structure, by adding a new column to location table.
how is this new column going to gets its value when location table db structure is upgraded?
-- juha
On 06/05/15 21:22, Juha Heinanen wrote:
Daniel-Constantin Mierla writes:
The solution is to add a dedicated column for partition index. I already implemented it in a separate branch, named tmp/usrloc-partion. The new code is rather small, because is only about inserting into database, the value is not used when having records in memory, therefore not needed to be loaded back in memory or updated. But it changes the database structure, by adding a new column to location table.
how is this new column going to gets its value when location table db structure is upgraded?
Default is 0, which means the records will be keepalived as part of partition 0 (like in the old code their id % partition_size == 0). Anyhow, there are other new columns in location table already, so if one is using db only mode should consider lowering the max expire, wait a bit and then flush the db for a clean upgrade. After than the max expired can be restored to older value.
Cheers, Daniel
On 05/06/2015 06:26 PM, Daniel-Constantin Mierla wrote:
I think this is a fix and still ok to apply in this testing phase. There is no new feature comparing with previous stable, but making it be the same in functionality. However, I wanted to check with all devs and see other opinions. And again, it is only about db_mode=DB_ONLY for usrloc.
+1 for merging
To conclude the discussion: code was merged a bit ago, upgrade wiki page was updated.
Cheers, Daniel
On 07/05/15 07:49, Victor Seva wrote:
On 05/06/2015 06:26 PM, Daniel-Constantin Mierla wrote:
I think this is a fix and still ok to apply in this testing phase. There is no new feature comparing with previous stable, but making it be the same in functionality. However, I wanted to check with all devs and see other opinions. And again, it is only about db_mode=DB_ONLY for usrloc.
+1 for merging
sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
-
Daniel-Constantin Mierla -
Juha Heinanen -
Luis Azedo -
Victor Seva