Pre-Submission Checklist

Type Of Change

Checklist:

Description

This commit is introducing JSON format for ACC events.
2 way to output the ACC events are proposed/implemented (syslog and mqueue)

The option to use db_flastore was considered and found less attractive, mainly because the DB abstraction layer is not making much sense, example reformatting the data many times ...
and other outputs could benefit from JSON format.

1- syslog (directly from the module, because it may be valuable to run Kamailio using runit or in docker and not "daemionizing" it anymore while still using syslog for acc events)

2- mqueue this way you may be able to control the acc event from Kamailio routing script, and example is provided using the http_client module to publish to NSQD

This seemed to be the best approach to remain as compatible as possible with the ACC module and benefit from it, while introducing the JSON format.

Many parameters are reused like time_mode, log_extra, log_facility, it seemed like reusing was making sense, this may be questionable.

If jansson or mqueue is not available or loaded the module will still build and run disabling the extra features

Usage example

where we use mqueue and process acc events in other worker processes.
example also inluded in the doc

# example using json_mqueue/http_client to publish to NSQD
max_while_loops=100000
modparam("mqueue", "mqueue", "name=acc_events;size=100000")
modparam("acc", "json_mqueue", "acc_events")
modparam("acc", "json_flag", 2)
modparam("acc", "log_extra", "caller_ip_port=$avp(caller_ip_port);")
modparam("rtimer", "timer", "name=nsqt;interval=1;mode=1;")
modparam("rtimer", "exec", "timer=nsqt;route=RUN_CDR_PUBLISH")
modparam("http_client", "keep_connections", 1)
modparam("http_client", "httpcon", "nsqd=>http://localhost:4151/pub?topic=acc")

route[RUN_CDR_PUBLISH] {
   $var(count) = 0;
   while (mq_fetch("acc_events")) {
      $var(q_size) = mq_size("acc_events");
      $var(count) = $var(count) + 1;
      xinfo("[RUN_CDR_PUBLISH][$var(q_size)][$var(count)][$mqk(acc_events)][$mqv(acc_events)]\n");
      $var(res) = http_connect("nsqd", "", "application/json", $mqv(acc_events), "$var(nsq_res)");
      if ($var(res) != "200") {
         mq_add("acc_events", "acc_key", "$mqv(acc_events)");
         return;
      }
   }
   if ($var(count) > 0 ) {
      xinfo("[RUN_CDR_PUBLISH]done count[$var(count)]\n");
   }
}

Load testing

Load tests where conducted with both json_syslog and json_mqueue/http to NSQD, latency was very stable it was high because the test server used was on West coast
California >> New-Jersey >> California

image
average message/response latency graph in ms/sec

Using voip_perf to send SIP compliant traffic (INVITE < 100 < 180 < 200 > ACK || BYE < 200)
7M SIP messages calls sent > 800/sec
https://github.com/jchavanton/voip_perf

Total 1000000 INVITE calls sent in 1037082 ms at rate of 819/sec 
Total 1000000 responses received in 1037881 ms at rate of 818/sec:
                                                                  
Detailed responses received:                                    
 - 200 responses:   1000000     (OK)                             
                    ------                                      
 TOTAL responses:   1000000 (rate=818/sec)

All the ACC events where queued in NSQD and written to SYSLOG without introducing delay.

curl -v http://127.0.0.1:4151/stats
 
nsqd v1.0.0-compat (built w/go1.8)
start_time 2018-02-12T17:10:58Z
uptime 33m4.181861175s
 
Health: OK
 
   [acc            ] depth: 3000004 be-depth: 2990004 msgs: 3000004  e2e%:

wc -l /var/log/json_acc.log
 
3000004 /var/log/json_acc.log

You can view, comment on, or merge this pull request online at:

  https://github.com/kamailio/kamailio/pull/1437

Commit Summary

File Changes

Patch Links:

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.