[SR-Users] Kamailio propagates 180 and 200 OK OUT OF ORDER

[Alex Balashov]

Hi Luis,

Rather confusingly, there is an 'async_workers' parameter in the core as
well, which needs to be set:

https://www.kamailio.org/wiki/cookbooks/5.3.x/core#async_workers

There is some relationship between 'async_workers' in the core and the
'workers' modparam in the 'async' module which is explained by Daniel
somewhere in a past mailing list thread, but I do not remember it
offhand. I really should dig it out and update the documentation with
this nuance.

Having said that, I did not use the 'async' framework for my fix, but
rather 'mqueue' and 'rtimer'. I have no real justification for that;
just custom, habit and comfort with those mechanisms. I grew accustomed
to them at a time when I had some issues with 100% CPU utilisation in a
virtualised (Xen) environment when using early versions of the 'async'
concepts.

1) The first thing is to create a reinvite queue; a single one will do,
since it's specifically designed to be multiprocess-safe:

loadmodule "mqueue"
modparam("mqueue", "mqueue", "name=reinvite_q");

2) Then I create 12 'rtimer' processes to consume this queue, each
having a 10,000 usec re-invocation delay:

loadmodule "rtimer"
modparam("rtimer", "timer", "name=reinvite_q1;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q1;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q2;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q2;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q3;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q3;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q4;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q4;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q5;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q5;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q6;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q6;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q7;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q7;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q8;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q8;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q9;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q9;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q10;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q10;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q11;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q11;route=REINVITE_DEQUEUE")
modparam("rtimer", "timer", "name=reinvite_q12;interval=10000u;mode=1;")
modparam("rtimer", "exec", "timer=reinvite_q12;route=REINVITE_DEQUEUE")

3) I then add this handling for reinvites to the loose_route() block in
the main request route -- logging and other extraneous matter omitted:

---
   if(has_totag()) {
      if(loose_route()) {
         if(is_method("INVITE")) {
              if(!t_suspend()) {
                 sl_send_reply("500", "Internal Server Error");
                 exit;
              }

              mq_add("reinvite_q", "$T(id_index):$T(id_label)", "");
          } else {
	     # Normal in-dialog request handling, t_relay() and that.

             route(IN_DLG_REQ);
	  }
       }
   }
---

And the handler on the other side, when the transaction is reanimated,
as it were:

---
route[REINVITE_DEQUEUE] {
   while(mq_fetch("reinvite_q")) {
      xlog("L_INFO", "[R-REINVITE-DEQUEUE:$ci] -> Resuming re-invite handling ($TV(Sn)) in PID $pp\n");

      $var(id) = $(mqk(reinvite_q){s.select,0,:}{s.int});
      $var(label) = $(mqk(reinvite_q){s.select,1,:}{s.int});

      xlog("L_INFO", "[R-REINVITE-DEQUEUE:$ci] -> Resuming re-invite handling ($TV(Sn)) in PID $pp\n");

      # Call route[IN_DLG_REQ] to do what we otherwise would have done
      # immediately, were it any other kind of in-dialog request.

      t_continue("$var(id)", "$var(label)", "IN_DLG_REQ");
      return;
   }
}
---

The idea behind 12 whole 'rtimer' processes is to massively
overprovision the amount of reinvite handlers available relative to the
actual number of reinvites passing through the system, even at high
volumes. 

The danger with too few processes is that the intended 10 ms delay may
not apply, because while(mq_fetch(...)) will just spin, always
dequeueing new reinvites and never allowing the execution route to
return control to 'rtimer'. 

As a practical matter, the system is not extremely high-volume and this
is a very academic concern. 

Overall, I would describe this method as crude but effective. It was
developed some years ago. 'async' provides some simplification and
removes some of the manual labour around this kind of management, from
what I understand, and overall I think you'd find it easier to use that.

¡Suerte! I'd be happy to answer any additional questions.

-- Alex

-- 
Alex Balashov | Principal | Evariste Systems LLC

Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/