<!-- Kamailio Pull Request Template -->
<!-- IMPORTANT: - for detailed contributing guidelines, read: https://github.com/kamailio/kamailio/blob/master/.github/CONTRIBUTING.md - pull requests must be done to master branch, unless they are backports of fixes from master branch to a stable branch - backports to stable branches must be done with 'git cherry-pick -x ...' - code is contributed under BSD for core and main components (tm, sl, auth, tls) - code is contributed GPLv2 or a compatible license for the other components - GPL code is contributed with OpenSSL licensing exception -->
#### Pre-Submission Checklist <!-- Go over all points below, and after creating the PR, tick all the checkboxes that apply --> <!-- All points should be verified, otherwise, read the CONTRIBUTING guidelines from above--> <!-- If you're unsure about any of these, don't hesitate to ask on sr-dev mailing list --> - [x] Commit message has the format required by CONTRIBUTING guide - [x] Commits are split per component (core, individual modules, libs, utils, ...) - [x] Each component has a single commit (if not, squash them into one commit) - [x] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated)
#### Type Of Change - [ ] Small bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality)
#### Checklist: <!-- Go over all points below, and after creating the PR, tick the checkboxes that apply --> - [ ] PR should be backported to stable branches - [x] Tested changes locally - [ ] Related to issue #XXXX (replace XXXX with an open issue number)
#### Description
This commit is introducing JSON format for ACC events. 2 way to output the ACC events are proposed/implemented (syslog and mqueue)
The option to use db_flastore was considered and found less attractive, mainly because the DB abstraction layer is not making much sense, example reformatting the data many times ... and other outputs could benefit from JSON format.
1- syslog (directly from the module, because it may be valuable to run Kamailio using runit or in docker and not "daemionizing" it anymore while still using syslog for acc events)
2- mqueue this way you may be able to control the acc event from Kamailio routing script, and example is provided using the http_client module to publish to NSQD
This seemed to be the best approach to remain as compatible as possible with the ACC module and benefit from it, while introducing the JSON format.
Many parameters are reused like time_mode, log_extra, log_facility, it seemed like reusing was making sense, this may be questionable.
If jansson or mqueue is not available or loaded the module will still build and run disabling the extra features
---------------------------------------------------------- #### Usage example where we use mqueue and process acc events in other worker processes. example also inluded in the doc
``` # example using json_mqueue/http_client to publish to NSQD max_while_loops=100000 modparam("mqueue", "mqueue", "name=acc_events;size=100000") modparam("acc", "json_mqueue", "acc_events") modparam("acc", "json_flag", 2) modparam("acc", "log_extra", "caller_ip_port=$avp(caller_ip_port);") modparam("rtimer", "timer", "name=nsqt;interval=1;mode=1;") modparam("rtimer", "exec", "timer=nsqt;route=RUN_CDR_PUBLISH") modparam("http_client", "keep_connections", 1) modparam("http_client", "httpcon", "nsqd=>http://localhost:4151/pub?topic=acc")
route[RUN_CDR_PUBLISH] { $var(count) = 0; while (mq_fetch("acc_events")) { $var(q_size) = mq_size("acc_events"); $var(count) = $var(count) + 1; xinfo("[RUN_CDR_PUBLISH][$var(q_size)][$var(count)][$mqk(acc_events)][$mqv(acc_events)]\n"); $var(res) = http_connect("nsqd", "", "application/json", $mqv(acc_events), "$var(nsq_res)"); if ($var(res) != "200") { mq_add("acc_events", "acc_key", "$mqv(acc_events)"); return; } } if ($var(count) > 0 ) { xinfo("[RUN_CDR_PUBLISH]done count[$var(count)]\n"); } } ```
#### Load testing Load tests where conducted with both json_syslog and json_mqueue/http to NSQD, latency was very stable it was high because the test server used was on West coast California >> New-Jersey >> California
 average message/response latency graph in ms/sec
---------------------------------------------------------- Using voip_perf to send SIP compliant traffic (INVITE < 100 < 180 < 200 > ACK || BYE < 200) 7M SIP messages calls sent > 800/sec https://github.com/jchavanton/voip_perf ``` Total 1000000 INVITE calls sent in 1037082 ms at rate of 819/sec Total 1000000 responses received in 1037881 ms at rate of 818/sec:
Detailed responses received: - 200 responses: 1000000 (OK) ------ TOTAL responses: 1000000 (rate=818/sec) ```
All the ACC events where queued in NSQD and written to SYSLOG without introducing delay. ``` curl -v http://127.0.0.1:4151/stats
nsqd v1.0.0-compat (built w/go1.8) start_time 2018-02-12T17:10:58Z uptime 33m4.181861175s
Health: OK
[acc ] depth: 3000004 be-depth: 2990004 msgs: 3000004 e2e%: ``` ``` wc -l /var/log/json_acc.log
3000004 /var/log/json_acc.log ``` You can view, comment on, or merge this pull request online at:
https://github.com/kamailio/kamailio/pull/1437
-- Commit Summary --
* acc: new json format with outputs to mqueue or syslog
-- File Changes --
M src/modules/acc/Makefile (12) M src/modules/acc/acc.c (109) M src/modules/acc/acc.h (1) M src/modules/acc/acc_logic.c (27) M src/modules/acc/acc_mod.c (25) M src/modules/acc/acc_mod.h (3) M src/modules/acc/doc/acc.xml (10) M src/modules/acc/doc/acc_admin.xml (121)
-- Patch Links --
https://github.com/kamailio/kamailio/pull/1437.patch https://github.com/kamailio/kamailio/pull/1437.diff
I wonder what is the problem with the CI build, looking at the logs last thing done was building, and then nothing. ``` CC (clang) [M evrexec.so] evrexec_mod.o LD (clang) [M evrexec.so] evrexec.so ```
@jchavanton pushed 1 commit.
6b8fccd acc: Makefile test
Thanks for the contribution!
The option to enable features in a module with compile time defines is not preferred at all. We had in the past the RADIUS (and DIAMETER) accounting as part of acc and it was very inconvenient to maintain or package.
The solution is to implement this as a new module on top of acc module, like `acc_json`, similar to `acc_radius`. In this way the module doesn't have compile time defines and can be packaged separately as part of jansson-modules pkg.
It should be very easy to build the new module by reusing the code you have in this patch. I did the split of acc_radius module.
This makes sense, `acc` would remain the base building block for transaction callbacks and dialogs and other modules could implement transformation and export.
I will do the necessary. Thank you for the review
Closed #1437.
https://github.com/kamailio/kamailio/pull/1440