From miconda@gmail.com Thu Jun 29 08:21:53 2017
From: Daniel-Constantin Mierla <miconda@gmail.com>
To: sr-users@lists.kamailio.org
Subject: Re: [SR-Users] Segfault on kamailio 5.0.2. Suddenly stops processing
 traffic, then generates a core when restarting
Date: Thu, 29 Jun 2017 08:20:53 +0200
Message-ID: <1ee7046e-a508-d8d8-473d-f63482706b5f@gmail.com>
In-Reply-To:
 <CAMtXxQn+UT0TTPrWWUrynFHsWfbty75EgfHJcLZGRLTbuyxudw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============0211265133=="

--===============0211265133==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

Hello,

I answered on the github issue tracker and let's continue there.

In short here: from the provided details, it seems to be the case when
all gateways are inactive, the logs provided having:

Jun 28 01:59:10 kam-1 sbc[12075]: ERROR: <script>: No available gateways
- M=INVITE R=sip:+12345678900(a)B.B.B.B:5060 ID=334f3vqDtj

Then a signal 15 (TERM) seems to be received (maybe from a monitoring
app). The segfault happens on shutdown, during modules cleanup.

Cheers,
Daniel


On 29.06.17 03:37, Joel Serrano wrote:
> Hi all, 
>
> I'm running into a weird situation with our newly deployed Kamailio
> servers. These servers are only doing SSL offloading and load
> balancing between another set of servers.
>
> Debian v9.0 + Kamailio v5.0.2 (from kamailio50 repository).
>
> After some time of running, kamailio suddenly stops responding to any
> request but the proccess is still alive. When we restart the service,
> a core is generated.
>
> I have opened an issue in
> GitHub: https://github.com/kamailio/kamailio/issues/1172
>
> I'm attaching the backtrace here too in case anyone can take a look at
> it, unfortunately I'm lost right now. 
>
> In dmesg I saw:
>
> [138953.744199] kamailio[12052]: segfault at 7efaf32f51b8 ip
> 00007efcf3b5df69 sp 00007ffe325dc0c0 error 4 in
> libcrypto.so.1.1[7efcf39f6000+265000]
>
>
> Thanks, 
> Joel.
>
>
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

-- 
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com




--===============0211265133==
Content-Type: text/html
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="attachment.html"
MIME-Version: 1.0

PGh0bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiIGNvbnRl
bnQ9InRleHQvaHRtbDsgY2hhcnNldD11dGYtOCI+CiAgPC9oZWFkPgogIDxib2R5IHRleHQ9IiMw
MDAwMDAiIGJnY29sb3I9IiNGRkZGRkYiPgogICAgPHA+SGVsbG8sPC9wPgogICAgPHA+SSBhbnN3
ZXJlZCBvbiB0aGUgZ2l0aHViIGlzc3VlIHRyYWNrZXIgYW5kIGxldCdzIGNvbnRpbnVlIHRoZXJl
LjwvcD4KICAgIDxwPkluIHNob3J0IGhlcmU6IGZyb20gdGhlIHByb3ZpZGVkIGRldGFpbHMsIGl0
IHNlZW1zIHRvIGJlIHRoZSBjYXNlCiAgICAgIHdoZW4gYWxsIGdhdGV3YXlzIGFyZSBpbmFjdGl2
ZSwgdGhlIGxvZ3MgcHJvdmlkZWQgaGF2aW5nOjwvcD4KICAgIDxwPkp1biAyOCAwMTo1OToxMCBr
YW0tMSBzYmNbMTIwNzVdOiBFUlJPUjogJmx0O3NjcmlwdCZndDs6IE5vCiAgICAgIGF2YWlsYWJs
ZSBnYXRld2F5cyAtIE09SU5WSVRFIFI9PGEgY2xhc3M9Im1vei10eHQtbGluay1mcmVldGV4dCIg
aHJlZj0ic2lwOisxMjM0NTY3ODkwMEBCLkIuQi5COjUwNjAiPnNpcDorMTIzNDU2Nzg5MDBAQi5C
LkIuQjo1MDYwPC9hPgogICAgICBJRD0zMzRmM3ZxRHRqPGJyPgogICAgPC9wPgogICAgPHA+VGhl
biBhIHNpZ25hbCAxNSAoVEVSTSkgc2VlbXMgdG8gYmUgcmVjZWl2ZWQgKG1heWJlIGZyb20gYQog
ICAgICBtb25pdG9yaW5nIGFwcCkuIFRoZSBzZWdmYXVsdCBoYXBwZW5zIG9uIHNodXRkb3duLCBk
dXJpbmcgbW9kdWxlcwogICAgICBjbGVhbnVwLjwvcD4KICAgIDxwPkNoZWVycyw8YnI+CiAgICAg
IERhbmllbDxicj4KICAgIDwvcD4KICAgIDxicj4KICAgIDxkaXYgY2xhc3M9Im1vei1jaXRlLXBy
ZWZpeCI+T24gMjkuMDYuMTcgMDM6MzcsIEpvZWwgU2VycmFubyB3cm90ZTo8YnI+CiAgICA8L2Rp
dj4KICAgIDxibG9ja3F1b3RlIHR5cGU9ImNpdGUiCmNpdGU9Im1pZDpDQU10WHhRbitVVDBUVFBy
V1dVcnluRkhzV2ZidHk3NUVnZkhKY0xaR1JMVGJ1eXh1ZHdAbWFpbC5nbWFpbC5jb20iPgogICAg
ICA8ZGl2IGRpcj0ibHRyIj5IaSBhbGwswqAKICAgICAgICA8ZGl2Pjxicj4KICAgICAgICA8L2Rp
dj4KICAgICAgICA8ZGl2PkknbSBydW5uaW5nIGludG8gYSB3ZWlyZCBzaXR1YXRpb24gd2l0aCBv
dXIgbmV3bHkgZGVwbG95ZWQKICAgICAgICAgIEthbWFpbGlvIHNlcnZlcnMuIFRoZXNlIHNlcnZl
cnMgYXJlIG9ubHkgZG9pbmcgU1NMIG9mZmxvYWRpbmcKICAgICAgICAgIGFuZCBsb2FkIGJhbGFu
Y2luZyBiZXR3ZWVuIGFub3RoZXIgc2V0IG9mIHNlcnZlcnMuPC9kaXY+CiAgICAgICAgPGRpdj48
YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRpdj5EZWJpYW4gdjkuMCArIEthbWFpbGlvIHY1
LjAuMiAoZnJvbSBrYW1haWxpbzUwIHJlcG9zaXRvcnkpLjwvZGl2PgogICAgICAgIDxkaXY+PGJy
PgogICAgICAgIDwvZGl2PgogICAgICAgIDxkaXY+QWZ0ZXIgc29tZSB0aW1lIG9mIHJ1bm5pbmcs
IGthbWFpbGlvIHN1ZGRlbmx5IHN0b3BzCiAgICAgICAgICByZXNwb25kaW5nIHRvIGFueSByZXF1
ZXN0IGJ1dCB0aGUgcHJvY2Nlc3MgaXMgc3RpbGwgYWxpdmUuCiAgICAgICAgICBXaGVuIHdlIHJl
c3RhcnQgdGhlIHNlcnZpY2UsIGEgY29yZSBpcyBnZW5lcmF0ZWQuPC9kaXY+CiAgICAgICAgPGRp
dj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRpdj5JIGhhdmUgb3BlbmVkIGFuIGlzc3Vl
IGluIEdpdEh1YjrCoDxhCiAgICAgICAgICAgIGhyZWY9Imh0dHBzOi8vZ2l0aHViLmNvbS9rYW1h
aWxpby9rYW1haWxpby9pc3N1ZXMvMTE3MiIKICAgICAgICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0
cnVlIj5odHRwczovL2dpdGh1Yi5jb20va2FtYWlsaW8va2FtYWlsaW8vaXNzdWVzLzExNzI8L2E+
PC9kaXY+CiAgICAgICAgPGRpdj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRpdj5JJ20g
YXR0YWNoaW5nIHRoZSBiYWNrdHJhY2UgaGVyZSB0b28gaW4gY2FzZSBhbnlvbmUgY2FuCiAgICAg
ICAgICB0YWtlIGEgbG9vayBhdCBpdCwgdW5mb3J0dW5hdGVseSBJJ20gbG9zdCByaWdodCBub3cu
wqA8L2Rpdj4KICAgICAgICA8ZGl2Pjxicj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8ZGl2Pklu
IGRtZXNnIEkgc2F3OjwvZGl2PgogICAgICAgIDxkaXY+PGJyPgogICAgICAgIDwvZGl2PgogICAg
ICAgIDxkaXY+CiAgICAgICAgICA8ZGl2Pjxmb250IGZhY2U9Im1vbm9zcGFjZSwgbW9ub3NwYWNl
Ij5bMTM4OTUzLjc0NDE5OV0KICAgICAgICAgICAgICBrYW1haWxpb1sxMjA1Ml06IHNlZ2ZhdWx0
IGF0IDdlZmFmMzJmNTFiOCBpcAogICAgICAgICAgICAgIDAwMDA3ZWZjZjNiNWRmNjkgc3AgMDAw
MDdmZmUzMjVkYzBjMCBlcnJvciA0IGluCiAgICAgICAgICAgICAgbGliY3J5cHRvLnNvLjEuMVs3
ZWZjZjM5ZjYwMDArMjY1MDAwXTwvZm9udD48L2Rpdj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8
ZGl2Pjxicj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8ZGl2Pjxicj4KICAgICAgICA8L2Rpdj4K
ICAgICAgICA8ZGl2PlRoYW5rcyzCoDwvZGl2PgogICAgICAgIDxkaXY+Sm9lbC48L2Rpdj4KICAg
ICAgICA8ZGl2Pjxicj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8ZGl2Pjxicj4KICAgICAgICA8
L2Rpdj4KICAgICAgPC9kaXY+CiAgICAgIDxicj4KICAgICAgPGZpZWxkc2V0IGNsYXNzPSJtaW1l
QXR0YWNobWVudEhlYWRlciI+PC9maWVsZHNldD4KICAgICAgPGJyPgogICAgICA8cHJlIHdyYXA9
IiI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KS2FtYWls
aW8gKFNFUikgLSBVc2VycyBNYWlsaW5nIExpc3QKPGEgY2xhc3M9Im1vei10eHQtbGluay1hYmJy
ZXZpYXRlZCIgaHJlZj0ibWFpbHRvOnNyLXVzZXJzQGxpc3RzLmthbWFpbGlvLm9yZyI+c3ItdXNl
cnNAbGlzdHMua2FtYWlsaW8ub3JnPC9hPgo8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZyZWV0ZXh0
IiBocmVmPSJodHRwczovL2xpc3RzLmthbWFpbGlvLm9yZy9jZ2ktYmluL21haWxtYW4vbGlzdGlu
Zm8vc3ItdXNlcnMiPmh0dHBzOi8vbGlzdHMua2FtYWlsaW8ub3JnL2NnaS1iaW4vbWFpbG1hbi9s
aXN0aW5mby9zci11c2VyczwvYT4KPC9wcmU+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8YnI+CiAg
ICA8cHJlIGNsYXNzPSJtb3otc2lnbmF0dXJlIiBjb2xzPSI3MiI+LS0gCkRhbmllbC1Db25zdGFu
dGluIE1pZXJsYQo8YSBjbGFzcz0ibW96LXR4dC1saW5rLWFiYnJldmlhdGVkIiBocmVmPSJodHRw
Oi8vd3d3LnR3aXR0ZXIuY29tL21pY29uZGEiPnd3dy50d2l0dGVyLmNvbS9taWNvbmRhPC9hPiAt
LSA8YSBjbGFzcz0ibW96LXR4dC1saW5rLWFiYnJldmlhdGVkIiBocmVmPSJodHRwOi8vd3d3Lmxp
bmtlZGluLmNvbS9pbi9taWNvbmRhIj53d3cubGlua2VkaW4uY29tL2luL21pY29uZGE8L2E+Ckth
bWFpbGlvIEFkdmFuY2VkIFRyYWluaW5nIC0gPGEgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZp
YXRlZCIgaHJlZj0iaHR0cDovL3d3dy5hc2lwdG8uY29tIj53d3cuYXNpcHRvLmNvbTwvYT4KS2Ft
YWlsaW8gV29ybGQgQ29uZmVyZW5jZSAtIDxhIGNsYXNzPSJtb3otdHh0LWxpbmstYWJicmV2aWF0
ZWQiIGhyZWY9Imh0dHA6Ly93d3cua2FtYWlsaW93b3JsZC5jb20iPnd3dy5rYW1haWxpb3dvcmxk
LmNvbTwvYT48L3ByZT4KICA8L2JvZHk+CjwvaHRtbD4K

--===============0211265133==--