[SR-Users] TLS not enough memory issue with git master
Daniel-Constantin Mierla
miconda at gmail.com
Tue Nov 17 21:39:28 CET 2015
Looking at the logs of last commits, I couldn't spot the change that
would add the leak.
What is the exact version you are running (kamailio -v)?
Are you using any of the functions exported by tcpops?
Cheers,
Daniel
On 17/11/15 15:24, Anthony Messina wrote:
> I wish that were the case...
>
> # kamcmd core.tcp_info
> {
> readers: 2
> max_connections: 2048
> max_tls_connections: 2048
> opened_connections: 0
> opened_tls_connections: 0
> write_queued_bytes: 0
> }
>
> # kamcmd tls.info
> {
> max_connections: 2048
> opened_connections: 0
> clear_text_write_queued_bytes: 0
> }
>
>
> On Tuesday, November 17, 2015 03:08:59 PM Daniel-Constantin Mierla wrote:
>> Looks like a lot of connections being open, can you get the output for:
>>
>> kamcmd core.tcp_info
>>
>> kamcmd tls.info
>>
>> Cheers,
>> Daniel
>>
>> On 17/11/15 14:59, Anthony Messina wrote:
>>> Attached. -A
>>>
>>> On Tuesday, November 17, 2015 02:50:21 PM Daniel-Constantin Mierla wrote:
>>>> Can you run the following commands:
>>>>
>>>> kamcmd cfg.set_now_int core memlog 1
>>>> kamcmd corex.shm_summary
>>>>
>>>> Then grab the log messages from syslog related to shared memory summary
>>>> and send them over here.
>>>>
>>>> Cheers,
>>>> Daniel
>>>>
>>>> On 17/11/15 14:31, Anthony Messina wrote:
>>>>> After I reported last night, I restarted Kamailio and even though the 5
>>>>> UACs did nothing but ensure they had a registration overnight, this
>>>>> morning the issue has recurred. The following is the output you
>>>>> requested. Not sure how the memory is being used up by Kamailio.
>>>>>
>>>>> # kamctl stats shmem
>>>>> shmem:fragments = 181
>>>>> shmem:free_size = 8922584
>>>>> shmem:max_used_size = 58243792
>>>>> shmem:real_used_size = 58186280
>>>>> shmem:total_size = 67108864
>>>>> shmem:used_size = 54346088
>>>>>
>>>>> On Tuesday, November 17, 2015 09:03:24 AM Daniel-Constantin Mierla
> wrote:
>>>>>> As you are using the master branch (development), do you run latest
>>>>>> version?
>>>>>>
>>>>>> Can you look at available shared memory?
>>>>>>
>>>>>> kamctl stats shmem
>>>>>>
>>>>>> Check it over time and see if the free memory is decreasing.
>>>>>>
>>>>>> Cheers,
>>>>>> Daniel
>>>>>>
>>>>>> On 17/11/15 00:44, Anthony Messina wrote:
>>>>>>> I have noticed the following issue which began with builds somewhere
>>>>>>> between git master commits bff0a08 and 6173ef7. I did not see this
>>>>>>> issue
>>>>>>> with my previous builds and haven't been able to pin down the problem,
>>>>>>> which is why I haven't formally filed a bug.
>>>>>>>
>>>>>>> Any help or guidance is appreciated, because this has crippled my use
>>>>>>> of
>>>>>>> Kamailio. Only a restart enables it to work again until the issue
>>>>>>> recurs.
>>>>>>>
>>>>>>> ERROR: tls [tls_server.c:189]: tls_complete_init(): tls: ssl bug #1491
>>>>>>> workaround: not enough memory for safe operation: 8870536
>>>>>>> ERROR: <core> [tcp_read.c:1303]: tcp_read_req(): ERROR: tcp_read_req:
>>>>>>> error
>>>>>>> reading
>>>>>>>
>>>>>>> I currently build against and run openssl-1.0.1k-12.fc22.x86_64.
>>>>>>>
>>>>>>> I have a very small operation and the only change on the operational
>>>>>>> side
>>>>>>> is that all 5 of my mobile UACs (yes, that's all) have switched from
>>>>>>> CSipSimple/Android to Zoiper/Android, which doesn't yet have support
>>>>>>> for
>>>>>>> client-side certificates so verify_certificate and require_certificate
>>>>>>> are
>>>>>>> off for both the server and client config.
>>>>>>>
>>>>>>> The server is started with:
>>>>>>> /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -m 64 -M 8
>>>>>>>
>>>>>>> I have tried modifying the shared mem to 128 but the issue still
>>>>>>> occurs.
>>>>>>>
>>>>>>> Even right now, I am seeing the error when only one UAC has
>>>>>>> established
>>>>>>> a
>>>>>>> TLS connection:
>>>>>>>
>>>>>>> # kamcmd tls.list
>>>>>>> {
>>>>>>>
>>>>>>> id: 572
>>>>>>> timeout: 3475
>>>>>>> src_ip: 10.77.79.156
>>>>>>> src_port: 58688
>>>>>>> dst_ip: 10.77.79.3
>>>>>>> dst_port: 5061
>>>>>>> cipher: ECDHE-RSA-RC4-SHA SSLv3 Kx=ECDH Au=RSA Enc=RC4(128)
>>>>>>> Mac=SHA1
>>>>>>> ct_wq_size: 0
>>>>>>> enc_rd_buf: 0
>>>>>>> flags: 2
>>>>>>> state: established
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>> # kamailio.cfg
>>>>>>> enable_tls=yes
>>>>>>> loadmodule "tls.so"
>>>>>>> modparam("tls", "connection_timeout", 60)
>>>>>>> #modparam("tls", "tls_log", 1)
>>>>>>> #modparam("tls", "tls_debug", 1)
>>>>>>> #modparam("tls", "low_mem_threshold1", -1)
>>>>>>> #modparam("tls", "low_mem_threshold2", 0)
>>>>>>> modparam("tls", "session_cache", 1)
>>>>>>>
>>>>>>> # tls.cfg
>>>>>>> [server:default]
>>>>>>> method = TLSv1+
>>>>>>> verify_certificate = no
>>>>>>> require_certificate = no
>>>>>>> private_key = /etc/kamailio/example.org.key.pem
>>>>>>> certificate = /etc/kamailio/example.org.crt.pem
>>>>>>> server_name = example.org
>>>>>>> cipher_list =
>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-
>>>>>>> AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-RC4-SHA:ECDHE-RSA-
>>>>>>> AE
>>>>>>> S
>>>>>>> 256-
>>>>>>> SHA:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-GCM
>>>>>>> -
>>>>>>> SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:RC4
>>>>>>> -
>>>>>>> SHA:AES256-GCM-SHA384:AES256-SHA256:CAMELLIA256-SHA:ECDHE-RSA-AES128-
>>>>>>> SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:CAMELLIA128-
>>>>>>> SHA:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK
>>>>>>>
>>>>>>> [client:default]
>>>>>>> method = TLSv1+
>>>>>>> verify_certificate = no
>>>>>>> require_certificate = no
>>>>>>> private_key = /etc/kamailio/example.org.key.pem
>>>>>>> certificate = /etc/kamailio/example.org.crt.pem
>>>>>>> cipher_list =
>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-
>>>>>>> AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-RC4-SHA:ECDHE-RSA-
>>>>>>> AE
>>>>>>> S
>>>>>>> 256-
>>>>>>> SHA:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-GCM
>>>>>>> -
>>>>>>> SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:RC4
>>>>>>> -
>>>>>>> SHA:AES256-GCM-SHA384:AES256-SHA256:CAMELLIA256-SHA:ECDHE-RSA-AES128-
>>>>>>> SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:CAMELLIA128-
>>>>>>> SHA:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK
>>>>>>>
>>>>>>>
>>>>>>> Thanks. -Anthony
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Book: SIP Routing With Kamailio - http://www.asipto.com
Kamailio Advanced Training, Nov 30-Dec 2, Berlin - http://asipto.com/kat
More information about the sr-users
mailing list