Hello,
No, both server have the same script, the only difference is that they are connected to different DB.
We increase the max TLS connections because we plan to move some customer on this proxy and we want to be ready to accept more than 2K TLS connection .
Laurent
-----Message d'origine----- De : Henning Westerholt hw@kamailio.org Envoyé : mercredi, 24 octobre 2018 22:46 À : sr-users@lists.kamailio.org Cc : Laurent Schweizer laurent.schweizer@peoplefone.com Objet : [JUNK-SPAM] Re: [SR-Users] : hight shared memory usage
Am Mittwoch, 24. Oktober 2018, 15:24:05 CEST schrieb Laurent Schweizer:
I have 2 proxy running and we have a very big difference of memory usage between them, moreover we have more connected users on the proxy with less memory usage.
We have this issue on the first proxy, today we have to increase the shared memory as it was no more possible to setup new TCP connection.
We use the same version of the proxy, script is quite the same, one difference, on the proxy with the issue we increase the number of max TLS connection to 20K
Any idea ? is TLS consuming more memory than TCP or UDP ?
Hello Laurent,
TLS is using more memory than TCP and than UDP, due to the higher workload and overhead for crypto/connection tracking.
But looking to your registered user count this should not make a big difference in the end.
Do you have maybe a different usage pattern on the first proxy comparing to the second one? So that different parts of the cfg or the code are executed?
Why do you increased the number of max TLS connections on the first proxy, you had also some issues here?
First proxy (with high memory usage):
[cycle #: 3; if constant make sure server lives] Kamailio Runtime Details: kamailio 5.0.6 (x86_64/linux) now: Wed Oct 24 15:18:55 2018 up_since: Wed Oct 24 13:21:12 2018 uptime: 7063
Transaction Statistics: tmx:UAS_transactions = 56240 tmx:active_transactions = 4 tmx:UAC_transactions = 0 tmx:inuse_transactions = 18
Stateless Server Statistics: sl:sent_replies = 134317 sl:sent_err_replies = 0
UsrLoc Statistics: usrloc:location_ch-contacts = 1463 usrloc:location_ch-users = 1181 usrloc:location_ch-expires = 1076 usrloc:registered_users = 1181
Core Statistics: core:rcv_requests = 214220 core:rcv_replies = 179145 core:fwd_requests = 5495 core:fwd_replies = 73
Shared Memory Statistics: shmem:fragments = 529 shmem:real_used_size = 15862690496 shmem:free_size = 36566109504 shmem:total_size = 52428800000 shmem:max_used_size = 16198927224 shmem:used_size = 15854961104
cat /proc/net/netstat | cut -d" " -f12-17 ; cat /proc/net/sockstat TW TWRecycled TWKilled PAWSPassive PAWSActive PAWSEstab 339118 0 684 3195 0 4648 InBcastOctets OutBcastOctets InCsumErrors InNoECTPkts InECT1Pkts InECT0Pkts 14620680 0 0 19142166496 9683 133135 sockets: used 1951 TCP: inuse 1029 orphan 0 tw 2 alloc 1036 mem 15 UDP: inuse 540 mem 446 UDPLITE: inuse 0 RAW: inuse 0 FRAG: inuse 0 memory 0
Second proxy (memory is OK) :
[cycle #: 2; if constant make sure server lives] Kamailio Runtime Details: kamailio 5.0.6 (x86_64/linux) now: Wed Oct 24 15:19:39 2018 up_since: Mon Jul 2 00:31:48 2018 uptime: 9902871
Transaction Statistics: tmx:UAS_transactions = 86712048 tmx:active_transactions = 92 tmx:UAC_transactions = 0 tmx:inuse_transactions = 178
Stateless Server Statistics: sl:sent_replies = 641635175 sl:sent_err_replies = 0
UsrLoc Statistics: usrloc:location-contacts = 3971 usrloc:location-users = 3845 usrloc:location-expires = 286685 usrloc:registered_users = 3845
Core Statistics: core:rcv_requests = 752644955 core:rcv_replies = 1181197213 core:fwd_requests = 14482059 core:fwd_replies = 206909
Shared Memory Statistics: shmem:fragments = 28437 shmem:real_used_size = 56447456 shmem:free_size = 2091036192 shmem:total_size = 2147483648 shmem:max_used_size = 82099768 shmem:used_size = 43354160
cat /proc/net/netstat | cut -d" " -f12-17 ; cat /proc/net/sockstat TW TWRecycled TWKilled PAWSPassive PAWSActive PAWSEstab 273625 0 0 0 0 3524 InBcastOctets OutBcastOctets InCsumErrors InNoECTPkts InECT1Pkts InECT0Pkts 2010 0 1 179031265416 81564 21432766 sockets: used 4964 TCP: inuse 576 orphan 0 tw 3 alloc 681 mem 105 UDP: inuse 3908 mem 3117 UDPLITE: inuse 0 RAW: inuse 0 FRAG: inuse 0 memory 0
Best regards,
Henning
-- Henning Westerholt - https://skalatan.de/blog/ Kamailio security assessment - https://skalatan.de/de/assessment
Am Donnerstag, 25. Oktober 2018, 08:53:44 CEST schrieb Laurent Schweizer:
No, both server have the same script, the only difference is that they are connected to different DB.
We increase the max TLS connections because we plan to move some customer on this proxy and we want to be ready to accept more than 2K TLS connection . [..]
Hello Laurent,
I understand, if they are both the same type (e.g. mysql) of database, then it should not make a difference.
How fast is the memory used? Can you do e.g. an evaluation during the day or several days to see the growths rate? What do you configured as pkg and shm memory setting for the server?
Did this error happens just recently or did you observe it since a long time?
One additional comment, there is a page in our wiki: https://www.kamailio.org/wiki/tutorials/troubleshooting/memory
that describes more ideas about memory troubleshooting. As a note: the memory manager debugging technique described is more targeted to a development or test environment.
Best regards,
Henning
-
Henning Westerholt -
Laurent Schweizer