[sr-dev] Send large responses with tcp_send() results in full write queue

[Alex Hermann]

Hi,

I'm using tcp_send() to send responses to incoming RPC requests over 
TCP. This works fine when responses are small, but tends to fail on 
larger responses on somewhat slower links or clients (it fails whenever 
the links is not on the loopback interface, even on sub-ms RTT links).

The error message thrown in the logs is:

ERROR: <core> [core/tcp_main.c:618]: _wbufq_add(): (32007 bytes): write 
queue full or timeout  (18042, total 18042, last write 0 s ago)

I verified the error is due to a full buffer, not a timeout.

I would like the tcp_send() not to fail on a full send buffer. Instead of 
failing, it should wait until there is enough space in the buffer again 
and send the data. Timeout should still be honoured though.

>From the code this looks like a "feature", but I would classify this as 
a bug. The receiver is still receiving, the timeout is not hit but 
Kamailio just decides not to send any more data because its (by default 
small) buffer is full. If the buffer is full, it should wait until enough 
space becomes free again, not fail before the timeout triggers.

The chunks of data that are sent via tcp_send() are always smaller than 
tcpconn_wq_max, so they should fit in an empty write buffer.
I'd prefer not to increase tcpconn_wq_max as this will be a never ending 
story, the responses will keep growing over time. They're already in the 
10's of MB at this time.

Retrying tcp_send() is not an option. After the first failure, the 
connection is closed and there is no way for the server (Kamailio) to 
open a new connection to the RPC client.

tcp_async is enabled, as that's needed/preferred for the SIP processing.

Is there a way around the (full) write buffer for the caller of 
tcp_send()?
-- 
Alex Hermann