[SR-Users] DMQ re-ordering concern

[Julien Chavanton]

If we were to implement queue and forward, we could use the mqueue
module/API
This a very simple and elegant module, we could later improve this module,
this would be sinergetic.

On Fri, May 10, 2019 at 12:03 PM Julien Chavanton <jchavanton at gmail.com>
wrote:

> Hi Charles,
>
> By the way, great work implementing all of this !
> As far as I am concerned, this is the best solution to create a cluster of
> registrar, working great with well shared state overall.
>
> Considering the usage of TCP to solve this problem for us, I can see that
> this may be quite complicated.
>
> If I am correct, modules like dmq_userloc are using the dmq api:
> bcast_message and/or send_message and the request is actually sent from the
> current worker process using tmb.t_request.
> (If we did use TCP, than another process would take care of async connect)
>
> It seems there will be no strait forward way to force the same process in
> this case because DMQ is not controlling anything.
>
> Maybe DMQ could expose generic a queue and forward functionality, when
> queuing it would be possible to specify an argument/logic to make sure
> order is preserved.
> We can imagine that hashing over call-id would  for dialog and
> registration  and htable name, etc.
> (Then it could become an option to use TCP several connections to several
> servers, etc.)
>
> I may need to do some test and spend more time to validate my assumptions
> ... (or you may correct me to save some time)
>
> Anyway I am not thinking about finding a quick solution for this, I
> beleive there is no urgency.
> Regards,
> Julien
>
>
>
> On Thu, May 9, 2019 at 5:46 AM Charles Chance <
> charles.chance at sipcentric.com> wrote:
>
>> Hi Julien,
>>
>> It's a good point and I agree it could be a concern in some cases.
>>
>> I'm not sure, however, the DMQ module itself is the place to do it. DMQ
>> is purely a transport mechanism and has no knowledge or context of what's
>> being relayed. This limits its scope to the ordering of messages only,
>> rather than the replicated data.
>>
>> Limiting DMQ to a single process could have performance implications, and
>> at the same time does nothing to affect the ordering of messages sent to or
>> received from multiple nodes (in a cluster of more than two) with varying
>> distance/latency between them.
>>
>> The sensible thing to do, in my opinion, is to implement ordering/version
>> checking within each of the modules that need it. Most of them do anyway,
>> in some way or another, for messages received locally. It would not be a
>> big job to extend that to replicated messages and provide some kind of
>> feedback/reverse-sync to the sending node if the local version is higher or
>> the timestamp is later.
>>
>> What do you/others think?
>>
>> Cheers,
>>
>> Charles
>>
>>
>> On Thu, 9 May 2019 at 11:57, Julien Chavanton <jchavanton at gmail.com>
>> wrote:
>>
>>> I guess, we could simply use tcp with t_relay using only one process in
>>> the module.
>>>
>>> On Thu, May 9, 2019, 04:52 Julien Chavanton <jchavanton at gmail.com>
>>> wrote:
>>>
>>>>
>>>> Yesterday at Kamailio World there was an interesting question, raising
>>>> a concern about replication of user location and DMQ in general, we should
>>>> track it in the mailing list and use it as a feature request and I can
>>>> share my initial thoughts on the topic.
>>>>
>>>> The concern was about DMQ reordering transactions.
>>>>
>>>> With SIP registration this seems like a minor concern, high
>>>> responsiveness at the cost of potential small inconsistency seems like a
>>>> good option.
>>>> Considering that, the state will automatically correct itself and that
>>>> this can be controlled using the expiration timer.  This will become even
>>>> more insignificant when the replicas can only be used as primary server
>>>> failover. (it becomes a very small concern only when the primary server
>>>> fails)
>>>>
>>>> With other type of data re-ordering may result in more problematic use
>>>> cases.
>>>> My impression is that DMQ should try to be good only with volatile
>>>> data, data that will expire anyway, like caching with expiration,
>>>> registration, Dialogs, etc.
>>>>
>>>> I think it may be worth looking at adding support for ordering/queuing
>>>> in DMQ.
>>>>
>>>> Performance and simplicity could be maintained by doing mainly
>>>> transactions in parallel with a configurable re-ordering best effort to
>>>> minimize the impact of the problem.
>>>>
>>>> For example :
>>>> 1- trying to re-order for a configurable amount of time. (32 seconds to
>>>> match the RFC for SIP re-transmissions)
>>>> 2- adding tractability for failed transactions.
>>>> 3- the possibility to trigger a re-sync.
>>>>
>>>> Another option is to look at preserving strict ordering but I can
>>>> imagine this could add more problems in some cases ...
>>>>
>>>>
>>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users at lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>
>>
>> --
>> *Charles Chance*
>> Managing Director
>>
>> t. 0330 120 1200    m. 07932 063 891
>>
>> Sipcentric Ltd. Company registered in England & Wales no. 7365592. Registered
>> office: Faraday Wharf, Innovation Birmingham Campus, Holt Street,
>> Birmingham Science Park, Birmingham B7 4BB.
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20190511/aa8d0c52/attachment.html>