[SR-Users] Coredump while exchanging diameter capabilities

Tsvetan Filev tsvetan.filev at inno-networks.com
Tue Jan 16 09:49:45 CET 2018 

Hi.

I get a coredump which is caused by bad memory handling during the 
diameter capability exchange process.
Here is part of the log file:
===================================
2018-01-16T09:16:39.890992+02:00 linux-o12d kamailio[13298]: 87(13388) 
INFO: cdp [worker.c:332]: worker_process(): [0] Worker process started...
2018-01-16T09:16:40.296393+02:00 linux-o12d systemd-coredump[13196]: 
Process 13193 (kamailio) of user 479 dumped core.
2018-01-16T09:16:40.871483+02:00 linux-o12d kamailio[13298]: 94(13395) 
DEBUG: cdp [peermanager.c:263]: peer_timer(): peer_timer(): taking care 
of peers...
2018-01-16T09:16:40.871545+02:00 linux-o12d kamailio[13298]: 94(13395) 
DEBUG: cdp [peermanager.c:280]: peer_timer(): peer_timer(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org State 0
2018-01-16T09:16:40.871596+02:00 linux-o12d kamailio[13298]: 94(13395) 
DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org State Closed Event Start
2018-01-16T09:16:40.871612+02:00 linux-o12d kamailio[13298]: 94(13395) 
INFO: cdp [peerstatemachine.c:525]: I_Snd_Conn_Req(): I_Snd_Conn_Req(): 
Peer hss.epc.mnc019.mcc425.3gppnetwork.org
2018-01-16T09:16:40.871636+02:00 linux-o12d kamailio[13298]: 94(13395) 
INFO: cdp [receiver.c:869]: peer_connect(): peer_connect(): Trying to 
connect to 10.82.10.85 port 3868
2018-01-16T09:16:40.871782+02:00 linux-o12d kamailio[13298]: 94(13395) 
INFO: cdp [receiver.c:937]: peer_connect(): peer_connect(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org:3868 connected
2018-01-16T09:16:40.871813+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [receiver.c:697]: receive_loop(): select_recv(): There is 
something on the fd exchange pipe
2018-01-16T09:16:40.871828+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [receiver.c:706]: receive_loop(): select_recv(): fd exchange 
pipe says fd [22] for peer 
0x7f0626b91c98:[hss.epc.mnc019.mcc425.3gppnetwork.org]
2018-01-16T09:16:40.871910+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org State Wait_Conn_Ack Event 
I_Rcv_Conn_Ack
2018-01-16T09:16:40.871933+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [diameter_msg.c:184]: AAANewMessage(): AAANewMessage: param 
session received null and it's a request!!
2018-01-16T09:16:40.872011+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [diameter_msg.c:81]: AAABuildMsgBuffer(): 
AAABuildMsgBuffer(): len=204
2018-01-16T09:16:40.872041+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [receiver.c:994]: peer_send_msg(): peer_send_msg(): Pipe push 
[0x7f0626c02548]
2018-01-16T09:16:40.872064+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [receiver.c:751]: receive_loop(): select_recv(): There is 
something on the send pipe
2018-01-16T09:16:40.872084+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [receiver.c:764]: receive_loop(): select_recv(): Send pipe 
says [0x7f0626c02548] 8
2018-01-16T09:16:40.872104+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [diameter_msg.c:410]: AAAFreeMessage(): AAAFreeMessage: 
Freeing message (0x7f0626c02548) 257
2018-01-16T09:16:40.872277+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [receiver.c:574]: do_receive(): receive_loop(): 
[hss.epc.mnc019.mcc425.3gppnetwork.org] Recv Version 1 Length 360
2018-01-16T09:16:40.872322+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [receiver.c:1088]: receive_message(): receive_message(): 
[hss.epc.mnc019.mcc425.3gppnetwork.org] Recv msg 257
2018-01-16T09:16:40.872345+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org State Wait_I_CEA Event I_Rcv_CEA
2018-01-16T09:16:40.872372+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:692]: count_Supported_Vendor_Id_AVPS(): 
Found 4 Supported_Vendor AVPS92(13393) DEBUG: cdp 
[peerstatemachine.c:743]: save_peer_applications(): Found Supported 
Vendor for Application 0: 5535
2018-01-16T09:16:40.872389+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 10415
2018-01-16T09:16:40.872405+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 13019
2018-01-16T09:16:40.872420+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 58637
2018-01-16T09:16:40.872438+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 5535
2018-01-16T09:16:40.872453+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 10415
2018-01-16T09:16:40.872468+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 13019
2018-01-16T09:16:40.872486+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 58637
2018-01-16T09:16:40.872504+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 5535
2018-01-16T09:16:40.872523+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 10415
2018-01-16T09:16:40.872539+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 13019
2018-01-16T09:16:40.872554+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found 
Supported Vendor for Application 0: 58637
2018-01-16T09:16:40.872570+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found 
Supported Vendor for Application 1: 5535
2018-01-16T09:16:40.872586+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found 
Supported Vendor for Application 1: 10415
2018-01-16T09:16:40.872601+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found 
Supported Vendor for Application 1: 13019
2018-01-16T09:16:40.872616+02:00 linux-o12d kamailio[13298]: 92(13393) 
DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found 
Supported Vendor for Application 1: 58637
2018-01-16T09:16:40.872634+02:00 linux-o12d kamailio[13298]: 92(13393) 
CRITICAL: <core> [core/mem/q_malloc.c:145]: qm_debug_frag(): BUG: qm: 
prev. fragm. tail overwritten(28af01000000, 
0)[0x7f0626c038d0:0x7f0626c03908]! Memory allocator was called from cdp: 
diameter_avp.c:365. Fragment marked by cdp: diameter_avp.c:142.
2018-01-16T09:16:41.054292+02:00 linux-o12d kamailio[13298]: 0(13298) 
ALERT: <core> [main.c:746]: handle_sigs(): child process 13393 exited by 
a signal 6
2018-01-16T09:16:41.054403+02:00 linux-o12d kamailio[13298]: 0(13298) 
ALERT: <core> [main.c:749]: handle_sigs(): core was generated
2018-01-16T09:16:41.054422+02:00 linux-o12d kamailio[13298]: 0(13298) 
INFO: <core> [main.c:771]: handle_sigs(): terminating due to SIGCHLD
2018-01-16T09:16:41.054438+02:00 linux-o12d kamailio[13298]: 0(13298) 
DEBUG: <core> [main.c:773]: handle_sigs(): terminating due to SIGCHLD
2018-01-16T09:16:41.054464+02:00 linux-o12d kamailio[13298]: 2(13303) 
INFO: <core> [main.c:826]: sig_usr(): signal 15 received
...
===================================

Attached is wireshark trace.
Here is my DiameterPeer.xml:

===================================
<?xml version="1.0" encoding="UTF-8"?>
<DiameterPeer
         FQDN="ims110-scscf.epc.mnc019.mcc425.3gppnetwork.org"
         Realm="epc.mnc019.mcc425.3gppnetwork.org"
         Vendor_Id="10415"
         Product_Name="CDiameterPeer"
         AcceptUnknownPeers="1"
         DropUnknownOnDisconnect="1"
         Tc="30"
         Workers="4"
         QueueLength="8"
         TransactionTimeout="5"
         SessionsHashSize="128"
         DefaultAuthSessionTimeout="3600"
         MaxAuthSessionTimeout="3600">

         <Peer FQDN="hss.epc.mnc019.mcc425.3gppnetwork.org" 
Realm="epc.mnc019.mcc425.3gppnetwork.org" port="3868" />
         <Acceptor port="3869" bind="10.82.10.56" />
         <Auth id="16777216" vendor="10415" /> <!--3GPP CxDX -->
         <DefaultRoute 
FQDN="ims110-scscf.epc.mnc019.mcc425.3gppnetwork.org" metric="10" />
</DiameterPeer>
===================================

It looks like buffer overflow to me but I'm not sure.

kamailio version is:
===================================
kamailio -v
version: kamailio 5.1.0 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, 
USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, 
F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, 
FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, 
USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, 
MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled on 05:30:36 Jan 15 2018 with gcc 4.8.5
===================================

OS is openSUSE Leap 42.3.
Kernel: Linux linux-o12d 4.4.104-39-default #1 SMP Thu Jan 4 08:11:03 
UTC 2018 (7db1912) x86_64 x86_64 x86_64 GNU/Linux

Do you have any idea what might be wrong ?
It could be bad config but still it should say something in the log 
without a crash.

Regards.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: diameter_trace.pcapng
Type: application/x-pcapng
Size: 3364 bytes
Desc: not available
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20180116/bb112781/attachment.bin>

More information about the sr-users mailing list