[SR-Users] Huge TLS performance regression migrating from 3.3.7 to 4.2.6

[Tristan Mahé]

Hi Daniel,

I am doing that right now, thank you.

I will keep you posted as told on IRC.

Interesting part to keep everyone in the loop is when I activated the
"latency" logs options, I could see some messages regarding some core
functions taking more than 250ms to complete, or sometimes just a
route(SOMETHING). The other interesting part is that doing some
optimizations on the tls parameters seems to raise the limit where
Kamailio becomes completely unresponsive and starts dropping connections
( where on the 3.3 there was no need ).

I'll be back very soon with more detailed results.

Best,

Tristan.

On 10/19/2015 07:03 AM, Daniel-Constantin Mierla wrote:
> Hello,
>
> can you use benchamrk module to see if any part of config got slower?
> if executing of config from beginning of request_route to save() is
> more or less the same, then setup of connection could be the issue.
>
> Since 3.3., there was some work on usrloc module about gruu/outbound
> and doing unregister if the tcp/tls connection is lost. On tls module,
> there was some work for SNI. In tcp core, some callbacks were added
> for websockets. If I haven't missed something relevant, I can't
> imagine what would create this handling issue between the versions.
>
> Cheers,
> Daniel
>
> On 15/10/15 07:29, Tristan Mahé wrote:
>> Hi Daniel,
>>
>> It is on the exact same server, same system configuration, same
>> configuration trimmed down for fitting 3.3, packages installed from
>> the kamailio debian wheezy repository.
>>
>> I can reproduce it just using sipp and a simple REGISTER loop.
>>
>> Do you want me to run some specific tests ? provide you some specific
>> traces ?
>>
>> Best,
>>
>> Tristan.
>>
>> On 10/14/2015 09:09 PM, Daniel-Constantin Mierla wrote:
>>> Hello,
>>>
>>> is the same operating system and server, or are you using some other
>>> machine for the new version? I can't remember right now any big
>>> change in tcp/tls side, besides sni, which should not have any
>>> relevant impact as described here.
>>>
>>> Also, have you installed from git branch 4.2 or the tarball/packages
>>> of 4.2.6?
>>>
>>> Cheers,
>>> Daniel
>>>
>>> On 15/10/15 03:36, Tristan Mahé wrote:
>>>> Hi List,
>>>>
>>>> I've been reaching a strange thing lately, trying to upgrade to kamailio
>>>> 4.2.6 ( identical config ):
>>>>
>>>> - 35% loss of performance on TLS connections ( Sipp REGISTER scenario,
>>>> easily reproduced ).
>>>> - more TCP workers needed to ease the TCP queues ( 32 workers on 3.3.7,
>>>> at least 64 are needed on 4.2.6 to avoid "queue full" messages in logs ).
>>>> - kamailio 4.2.6 intermittent processing of requests ( Kamailio stops
>>>> accepting new tcp connections, then resume a bit later, then stops
>>>> again,... ) => starts happening when Kamailio is treating more than 20k
>>>> tcp connections, with between 500-1k req/s. During those periods, trying
>>>> to connect to kamailio with an openssl s_client -connect will end up in
>>>> a timeout or with a long delay before the certificate is received ( 03
>>>> CONNECTED ).
>>>> - kamailio 4.2.6 dropping connections, kamailio 3.3.7 keeping those
>>>> connections: after a while, the number of active connections starts to
>>>> decrease on kam4, on kam3 it stays constant.
>>>>
>>>> I was wondering if I was alone in this situation or if someone else is
>>>> impacted, has some advices ?
>>>>
>>>> It does not seem to be a config issue, as it is the same that was
>>>> deployed with both versions.
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
>>>> sr-users at lists.sip-router.org
>>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>>
>>> -- 
>>> Daniel-Constantin Mierla
>>> http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
>>> Book: SIP Routing With Kamailio - http://www.asipto.com
>>>
>>>
>>> _______________________________________________
>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
>>> sr-users at lists.sip-router.org
>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>
>
> -- 
> Daniel-Constantin Mierla
> http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
> Book: SIP Routing With Kamailio - http://www.asipto.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20151020/7d1dec0e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20151020/7d1dec0e/attachment.sig>