[SR-Users] RTPEngine recording

[Richard Fuchs]

On 12/08/17 03:49 PM, Alex Balashov wrote:
> Also, the "proc" recording method has an interesting pipeline, involving
> an ephemeral metadata file and a memory sink exposed through /proc, and
> a userspace daemon picking up the data and writing it to disk (if this
> doesn't happen, audio frames going into the sink are discarded).
>
> This pipeline leads to two or three philosophical questions:
>
> 1. Is the purpose of this design to allow for writing audio frames in a
> scatter-gather fashion, so that audio is written serially to storage?

The main idea behind this approach was that it makes it feasible with 
only minimal overhead to enable recording for all calls and then let the 
recording daemon decide which calls to actually record and in what way. 
if the recording daemon is not interested in the call at all, then 
having recording enabled anyway would only have negligible impact on the 
overall performance. And if the recording daemon is interested, then it 
has a variety of options on how to deal with the data: write to pcap, 
decode to playable audio, send off to external system, etc.

It should be noted that the included recording daemon doesn't support 
all of these options (yet?) and should only be seen as a reference 
implementation. The interface should be sufficiently simple that other 
recording solutions can be implemented on top of it.

> 2. Is the other purpose of a RTP sink to allow the possibility of
> real-time call intercept and diversion to live playback? If so, are
> there any plans for the recording daemon to expose an RTSP interface or
> similar to make this easier?

No plans from our side so far, but as I said above, it should be fairly 
simple to create an implementation for such a mechanism.

> 3. What happens if, under high load and I/O wait conditions, the
> userspace recording daemon cannot read frames from the sink fast
> enough, or the CPU encoding workload (-> WAV/MP3) is too high?

You would start to lose network frames (RTP packet data).

> According to the documentation, the depth of the sink is only 10 frames:
>
>     Packet data is held in kernel memory until retrieved by
>     the userspace component, but only a limited number of
>     packets (default 10) per media stream. If packets are not
>     retrieved in time, they will be simply discarded. This makes
>     it possible to flag all calls to be recorded and then leave
>     it to the userspace component to decided whether to use the
>     packet data for any purpose or not.
>
> Is it possible to increase that depth? Or is this not a concern because
> the userspace component is implemented in an asynchronous/threaded
> manner, so frames are retrieved quickly for processing and then enqueued
> into a local blocking queue?

The kernel module supports changing this limit on a per-stream basis 
when recording is enabled (however, such a switch isn't present in the 
rtpengine main daemon as of yet), as well as changing the default value 
of 10 as a module option that can be set when the module is loaded (e.g. 
through modprobe.d config options).

The included recording daemon is multi threaded, but the possibility 
that it cannot keep up with the work load exists nevertheless. If you 
simply queue up everything until it can be processed when there isn't 
enough processing (or I/O) power available, then you would simply run 
out of memory at some point.

Cheers