<h4>Pre-Submission Checklist</h4>
<ul class="contains-task-list">
<li class="task-list-item"><input checked="" class="task-list-item-checkbox" disabled="" id="" type="checkbox"> Commit message has the format required by CONTRIBUTING guide</li>
<li class="task-list-item"><input checked="" class="task-list-item-checkbox" disabled="" id="" type="checkbox"> Commits are split per component (core, individual modules, libs, utils, ...)</li>
<li class="task-list-item"><input checked="" class="task-list-item-checkbox" disabled="" id="" type="checkbox"> Each component has a single commit (if not, squash them into one commit)</li>
<li class="task-list-item"><input checked="" class="task-list-item-checkbox" disabled="" id="" type="checkbox"> No commits to README files for modules (changes must be done to docbook files<br>
in <code>doc/</code> subfolder, the README file is autogenerated)</li>
</ul>
<h4>Type Of Change</h4>
<ul class="contains-task-list">
<li class="task-list-item"><input class="task-list-item-checkbox" disabled="" id="" type="checkbox"> Small bug fix (non-breaking change which fixes an issue)</li>
<li class="task-list-item"><input checked="" class="task-list-item-checkbox" disabled="" id="" type="checkbox"> New feature (non-breaking change which adds new functionality)</li>
<li class="task-list-item"><input class="task-list-item-checkbox" disabled="" id="" type="checkbox"> Breaking change (fix or feature that would change existing functionality)</li>
</ul>
<h4>Checklist:</h4>
<ul class="contains-task-list">
<li class="task-list-item"><input class="task-list-item-checkbox" disabled="" id="" type="checkbox"> PR should be backported to stable branches</li>
<li class="task-list-item"><input checked="" class="task-list-item-checkbox" disabled="" id="" type="checkbox"> Tested changes locally</li>
<li class="task-list-item"><input class="task-list-item-checkbox" disabled="" id="" type="checkbox"> Related to issue #XXXX (replace XXXX with an open issue number)</li>
</ul>
<h4>Description</h4>
<p>This commit is introducing JSON format for ACC events.<br>
2 way to output the ACC events are proposed/implemented (syslog and mqueue)</p>
<p>The option to use db_flastore was considered and found less attractive, mainly because the DB abstraction layer is not making much sense, example reformatting the data many times ...<br>
and other outputs could benefit from JSON format.</p>
<p>1- syslog (directly from the module, because it may be valuable to run Kamailio using runit or in docker and not "daemionizing" it anymore while still using syslog for acc events)</p>
<p>2- mqueue this way you may be able to control the acc event from Kamailio routing script, and example is provided using the http_client module to publish to NSQD</p>
<p>This seemed to be the best approach to remain as compatible as possible with the ACC module and benefit from it, while introducing the JSON format.</p>
<p>Many parameters are reused like time_mode, log_extra, log_facility, it seemed like reusing was making sense, this may be questionable.</p>
<p>If jansson or mqueue is not available or loaded the module will still build and run disabling the extra features</p>
<hr>
<h4>Usage example</h4>
<p>where we use mqueue and process acc events in other worker processes.<br>
example also inluded in the doc</p>
<pre><code># example using json_mqueue/http_client to publish to NSQD
max_while_loops=100000
modparam("mqueue", "mqueue", "name=acc_events;size=100000")
modparam("acc", "json_mqueue", "acc_events")
modparam("acc", "json_flag", 2)
modparam("acc", "log_extra", "caller_ip_port=$avp(caller_ip_port);")
modparam("rtimer", "timer", "name=nsqt;interval=1;mode=1;")
modparam("rtimer", "exec", "timer=nsqt;route=RUN_CDR_PUBLISH")
modparam("http_client", "keep_connections", 1)
modparam("http_client", "httpcon", "nsqd=>http://localhost:4151/pub?topic=acc")
route[RUN_CDR_PUBLISH] {
$var(count) = 0;
while (mq_fetch("acc_events")) {
$var(q_size) = mq_size("acc_events");
$var(count) = $var(count) + 1;
xinfo("[RUN_CDR_PUBLISH][$var(q_size)][$var(count)][$mqk(acc_events)][$mqv(acc_events)]\n");
$var(res) = http_connect("nsqd", "", "application/json", $mqv(acc_events), "$var(nsq_res)");
if ($var(res) != "200") {
mq_add("acc_events", "acc_key", "$mqv(acc_events)");
return;
}
}
if ($var(count) > 0 ) {
xinfo("[RUN_CDR_PUBLISH]done count[$var(count)]\n");
}
}
</code></pre>
<h4>Load testing</h4>
<p>Load tests where conducted with both json_syslog and json_mqueue/http to NSQD, latency was very stable it was high because the test server used was on West coast<br>
California >> New-Jersey >> California</p>
<p><a href="https://user-images.githubusercontent.com/3736014/36117214-a6ebcebc-0fed-11e8-928c-9ff9321bef86.png" target="_blank"><img src="https://user-images.githubusercontent.com/3736014/36117214-a6ebcebc-0fed-11e8-928c-9ff9321bef86.png" alt="image" style="max-width:100%;"></a><br>
average message/response latency graph in ms/sec</p>
<hr>
<p>Using voip_perf to send SIP compliant traffic (INVITE < 100 < 180 < 200 > ACK || BYE < 200)<br>
7M SIP messages calls sent > 800/sec<br>
<a href="https://github.com/jchavanton/voip_perf">https://github.com/jchavanton/voip_perf</a></p>
<pre><code>Total 1000000 INVITE calls sent in 1037082 ms at rate of 819/sec
Total 1000000 responses received in 1037881 ms at rate of 818/sec:
Detailed responses received:
- 200 responses: 1000000 (OK)
------
TOTAL responses: 1000000 (rate=818/sec)
</code></pre>
<p>All the ACC events where queued in NSQD and written to SYSLOG without introducing delay.</p>
<pre><code>curl -v http://127.0.0.1:4151/stats
nsqd v1.0.0-compat (built w/go1.8)
start_time 2018-02-12T17:10:58Z
uptime 33m4.181861175s
Health: OK
[acc ] depth: 3000004 be-depth: 2990004 msgs: 3000004 e2e%:
</code></pre>
<pre><code>wc -l /var/log/json_acc.log
3000004 /var/log/json_acc.log
</code></pre>
<hr>
<h4>You can view, comment on, or merge this pull request online at:</h4>
<p> <a href='https://github.com/kamailio/kamailio/pull/1437'>https://github.com/kamailio/kamailio/pull/1437</a></p>
<h4>Commit Summary</h4>
<ul>
<li>acc: new json format with outputs to mqueue or syslog</li>
</ul>
<h4>File Changes</h4>
<ul>
<li>
<strong>M</strong>
<a href="https://github.com/kamailio/kamailio/pull/1437/files#diff-0">src/modules/acc/Makefile</a>
(12)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/kamailio/kamailio/pull/1437/files#diff-1">src/modules/acc/acc.c</a>
(109)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/kamailio/kamailio/pull/1437/files#diff-2">src/modules/acc/acc.h</a>
(1)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/kamailio/kamailio/pull/1437/files#diff-3">src/modules/acc/acc_logic.c</a>
(27)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/kamailio/kamailio/pull/1437/files#diff-4">src/modules/acc/acc_mod.c</a>
(25)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/kamailio/kamailio/pull/1437/files#diff-5">src/modules/acc/acc_mod.h</a>
(3)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/kamailio/kamailio/pull/1437/files#diff-6">src/modules/acc/doc/acc.xml</a>
(10)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/kamailio/kamailio/pull/1437/files#diff-7">src/modules/acc/doc/acc_admin.xml</a>
(121)
</li>
</ul>
<h4>Patch Links:</h4>
<ul>
<li><a href='https://github.com/kamailio/kamailio/pull/1437.patch'>https://github.com/kamailio/kamailio/pull/1437.patch</a></li>
<li><a href='https://github.com/kamailio/kamailio/pull/1437.diff'>https://github.com/kamailio/kamailio/pull/1437.diff</a></li>
</ul>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/kamailio/kamailio/pull/1437">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AF36ZUzpBFz-dAaK3C0qQ2w2cnnlVA7-ks5tUKThgaJpZM4SCznD">mute the thread</a>.<img alt="" height="1" src="https://github.com/notifications/beacon/AF36ZaFJ7OYs_ow2R8xF3wjjG2G1MH9Xks5tUKThgaJpZM4SCznD.gif" width="1" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
<link itemprop="url" href="https://github.com/kamailio/kamailio/pull/1437"></link>
<meta itemprop="name" content="View Pull Request"></meta>
</div>
<meta itemprop="description" content="View this Pull Request on GitHub"></meta>
</div>
<script type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/kamailio/kamailio","title":"kamailio/kamailio","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/kamailio/kamailio"}},"updates":{"snippets":[{"icon":"DESCRIPTION","message":"acc: new json format with outputs to mqueue or syslog (#1437)"}],"action":{"name":"View Pull Request","url":"https://github.com/kamailio/kamailio/pull/1437"}}}</script>