[SR-Users] Coredump while exchanging diameter capabilities

Tsvetan Filev tsvetan.filev at inno-networks.com
Fri Jan 19 10:10:34 CET 2018


Carsten,

I pulled all latest changes (including your commit 
72bbf9280879475dcdbda59f00b75a274885abaa), built new RPMS using the spec 
file, upgraded my current packages (5.1.0) with the new ones (5.2.0) and 
restarted the service. I got core dump again.
It seems you have to take care of AVP_Vendor_Specific_Application_Id as 
well.

Log file:

2018-01-19T10:53:14.082895+02:00 linux-o12d systemd-coredump[25526]: 
Process 25523 (kamailio) of user 479 dumped core.
2018-01-19T10:53:14.425810+02:00 linux-o12d kamailio[25613]: 78(25694) 
DEBUG: cdp [peermanager.c:263]: peer_timer(): peer_timer(): taking care 
of peers...
2018-01-19T10:53:14.425875+02:00 linux-o12d kamailio[25613]: 78(25694) 
DEBUG: cdp [peermanager.c:280]: peer_timer(): peer_timer(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org State 0
2018-01-19T10:53:14.425892+02:00 linux-o12d kamailio[25613]: 78(25694) 
DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org State Closed Event Start
2018-01-19T10:53:14.425908+02:00 linux-o12d kamailio[25613]: 78(25694) 
INFO: cdp [peerstatemachine.c:525]: I_Snd_Conn_Req(): I_Snd_Conn_Req(): 
Peer hss.epc.mnc019.mcc425.3gppnetwork.org
2018-01-19T10:53:14.425929+02:00 linux-o12d kamailio[25613]: 78(25694) 
INFO: cdp [receiver.c:869]: peer_connect(): peer_connect(): Trying to 
connect to 10.82.10.85 port 3868
2018-01-19T10:53:14.426048+02:00 linux-o12d kamailio[25613]: 78(25694) 
INFO: cdp [receiver.c:937]: peer_connect(): peer_connect(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org:3868 connected
2018-01-19T10:53:14.426087+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [receiver.c:697]: receive_loop(): select_recv(): There is 
something on the fd exchange pipe
2018-01-19T10:53:14.426106+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [receiver.c:706]: receive_loop(): select_recv(): fd exchange 
pipe says fd [20] for peer 
0x7fe39d89fc98:[hss.epc.mnc019.mcc425.3gppnetwork.org]
2018-01-19T10:53:14.426203+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org State Wait_Conn_Ack Event 
I_Rcv_Conn_Ack
2018-01-19T10:53:14.426230+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [diameter_msg.c:184]: AAANewMessage(): AAANewMessage: param 
session received null and it's a request!!
2018-01-19T10:53:14.426310+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [diameter_msg.c:81]: AAABuildMsgBuffer(): 
AAABuildMsgBuffer(): len=204
2018-01-19T10:53:14.426336+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [receiver.c:994]: peer_send_msg(): peer_send_msg(): Pipe push 
[0x7fe39d907a88]
2018-01-19T10:53:14.426353+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [receiver.c:751]: receive_loop(): select_recv(): There is 
something on the send pipe
2018-01-19T10:53:14.426368+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [receiver.c:764]: receive_loop(): select_recv(): Send pipe 
says [0x7fe39d907a88] 8
2018-01-19T10:53:14.426383+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [diameter_msg.c:410]: AAAFreeMessage(): AAAFreeMessage: 
Freeing message (0x7fe39d907a88) 257
2018-01-19T10:53:14.426610+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [receiver.c:574]: do_receive(): receive_loop(): 
[hss.epc.mnc019.mcc425.3gppnetwork.org] Recv Version 1 Length 360
2018-01-19T10:53:14.426695+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [receiver.c:1088]: receive_message(): receive_message(): 
[hss.epc.mnc019.mcc425.3gppnetwork.org] Recv msg 257
2018-01-19T10:53:14.426722+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer 
hss.epc.mnc019.mcc425.3gppnetwork.org State Wait_I_CEA Event I_Rcv_CEA
2018-01-19T10:53:14.426739+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:698]: count_Supported_Vendor_Id_AVPS(): 
Found 4 Supported_Vendor AVPS76(25692) DEBUG: cdp 
[peerstatemachine.c:681]: add_peer_application(): Application 0 of 
maximum 20
2018-01-19T10:53:14.426755+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 5535
2018-01-19T10:53:14.426770+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
1 of maximum 20
2018-01-19T10:53:14.426785+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 10415
2018-01-19T10:53:14.426799+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
2 of maximum 20
2018-01-19T10:53:14.426814+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 13019
2018-01-19T10:53:14.426830+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
3 of maximum 20
2018-01-19T10:53:14.426845+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 58637
2018-01-19T10:53:14.426860+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
4 of maximum 20
2018-01-19T10:53:14.426875+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
5 of maximum 20
2018-01-19T10:53:14.426890+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 5535
2018-01-19T10:53:14.426904+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
6 of maximum 20
2018-01-19T10:53:14.426919+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 10415
2018-01-19T10:53:14.426934+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
7 of maximum 20
2018-01-19T10:53:14.426949+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 13019
2018-01-19T10:53:14.426974+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
8 of maximum 20
2018-01-19T10:53:14.426990+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 58637
2018-01-19T10:53:14.427005+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
9 of maximum 20
2018-01-19T10:53:14.427020+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
10 of maximum 20
2018-01-19T10:53:14.427035+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 5535
2018-01-19T10:53:14.427049+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
11 of maximum 20
2018-01-19T10:53:14.427064+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 10415
2018-01-19T10:53:14.427079+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
12 of maximum 20
2018-01-19T10:53:14.427093+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 13019
2018-01-19T10:53:14.427108+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
13 of maximum 20
2018-01-19T10:53:14.427122+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:750]: save_peer_applications(): Found 
Supported Vendor for Application 0: 58637
2018-01-19T10:53:14.427137+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
14 of maximum 20
2018-01-19T10:53:14.427152+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
15 of maximum 20
2018-01-19T10:53:14.427166+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:763]: save_peer_applications(): Found 
Supported Vendor for Application 1: 5535
2018-01-19T10:53:14.427181+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
16 of maximum 20
2018-01-19T10:53:14.427196+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:763]: save_peer_applications(): Found 
Supported Vendor for Application 1: 10415
2018-01-19T10:53:14.427210+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
17 of maximum 20
2018-01-19T10:53:14.427225+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:763]: save_peer_applications(): Found 
Supported Vendor for Application 1: 13019
2018-01-19T10:53:14.427244+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
18 of maximum 20
2018-01-19T10:53:14.427259+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:763]: save_peer_applications(): Found 
Supported Vendor for Application 1: 58637
2018-01-19T10:53:14.427274+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
19 of maximum 20
2018-01-19T10:53:14.427289+02:00 linux-o12d kamailio[25613]: 76(25692) 
DEBUG: cdp [peerstatemachine.c:681]: add_peer_application(): Application 
20 of maximum 20
2018-01-19T10:53:14.427304+02:00 linux-o12d kamailio[25613]: 76(25692) 
CRITICAL: <core> [core/mem/q_malloc.c:145]: qm_debug_frag(): BUG: qm: 
prev. fragm. tail overwritten(28af01000000, 
0)[0x7fe39d908e10:0x7fe39d908e48]! Memory allocator was called from cdp: 
diameter_avp.
c:365. Fragment marked by cdp: diameter_avp.c:142.
2018-01-19T10:53:14.480944+02:00 linux-o12d kamailio[25613]: 0(25613) 
ALERT: <core> [main.c:746]: handle_sigs(): child process 25692 exited by 
a signal 6
2018-01-19T10:53:14.481049+02:00 linux-o12d kamailio[25613]: 0(25613) 
ALERT: <core> [main.c:749]: handle_sigs(): core was generated
2018-01-19T10:53:14.481079+02:00 linux-o12d kamailio[25613]: 0(25613) 
INFO: <core> [main.c:771]: handle_sigs(): terminating due to SIGCHLD
2018-01-19T10:53:14.481115+02:00 linux-o12d kamailio[25613]: 0(25613) 
DEBUG: <core> [main.c:773]: handle_sigs(): terminating due to SIGCHLD
2018-01-19T10:53:14.481159+02:00 linux-o12d kamailio[25613]: 1(25617) 
INFO: <core> [main.c:826]: sig_usr(): signal 15 received


This time I have debug symbols:


(gdb) bt full
#0  0x00007feb551c78c7 in __GI_raise (sig=sig at entry=6) at 
../sysdeps/unix/sysv/linux/raise.c:55
         resultvar = 0
         pid = 25344
         selftid = 25344
#1  0x00007feb551c8c9a in __GI_abort () at abort.c:78
         save_stage = 1
         act = {__sigaction_handler = {sa_handler = 0x28af01000000, 
sa_sigaction = 0x28af01000000}, sa_mask = {__val = {140726850332656, 0, 
140648729949568, 6765256, 140648577228624, 140648574046208, 
140648567419080, 140648574276992, 21474836483, 140648574046208,
               9017383233312, 4, 0, 234976, 4, 8193552}}, sa_flags = 
1437865728, sa_restorer = 0x0}
         sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x0000000000673a7b in qm_debug_frag (qm=0x7feb4c4b5000, 
f=0x7feb4c7bde10, file=0x7feb4be6f852 "cdp: diameter_avp.c", line=365) 
at core/mem/q_malloc.c:147
         __FUNCTION__ = "qm_debug_frag"
#3  0x00000000006768bc in qm_free (qmp=0x7feb4c4b5000, p=0x7feb4c7bde48, 
file=0x7feb4be6f852 "cdp: diameter_avp.c", func=0x7feb4be71338 
<__FUNCTION__.7007> "AAAFreeAVP", line=365, mname=0x7feb4be6f680 "cdp") 
at core/mem/q_malloc.c:496
         qm = 0x7feb4c4b5000
         f = 0x7feb4c7bde10
         size = 140726850332944
         next = 0x7ffd85eca9a0
         prev = 0x6756b8 <qm_malloc+1406>
         __FUNCTION__ = "qm_free"
#4  0x0000000000680002 in qm_shm_free (qmp=0x7feb4c4b5000, 
p=0x7feb4c7bde48, file=0x7feb4be6f852 "cdp: diameter_avp.c", 
func=0x7feb4be71338 <__FUNCTION__.7007> "AAAFreeAVP", line=365, 
mname=0x7feb4be6f680 "cdp") at core/mem/q_malloc.c:1255
No locals.
#5  0x00007feb4be4b111 in AAAFreeAVP (avp=0x7ffd85eca9e0) at 
diameter_avp.c:365
         __FUNCTION__ = "AAAFreeAVP"
#6  0x00007feb4be1cbb4 in AAAFreeAVPList (avpList=0x7ffd85ecaa40) at 
diameter_msg.c:396
         avp_t = 0x7feb4c7bde48
         avp = 0x7feb4c7bdee8
#7  0x00007feb4bdea79c in save_peer_applications (p=0x7feb4c754c98, 
msg=0x7feb4c7bca88) at peerstatemachine.c:785
         total_cnt = 20
         supported_vendor_id_avp_cnt = 4
         avp = 0x7feb4c7bdbb0
         avp_vendor = 0x7feb4c7bde48
         avp2 = 0x0
         group = {head = 0x7feb4c7bde48, tail = 0x7feb4c7bdee8}
         id = 16777216
         vendor = 10415
         __FUNCTION__ = "save_peer_applications"
#8  0x00007feb4bdea810 in Process_CEA (p=0x7feb4c754c98, 
cea=0x7feb4c7bca88) at peerstatemachine.c:803
         avp = 0x7feb4c7bd218
#9  0x00007feb4bde39e1 in sm_process (p=0x7feb4c754c98, event=I_Rcv_CEA, 
msg=0x7feb4c7bca88, peer_locked=0, sock=20) at peerstatemachine.c:166
         result_code = 1273380608
         next_event = 32747
         msg_received = 0
         __FUNCTION__ = "sm_process"
#10 0x00007feb4be3b76e in receive_message (msg=0x7feb4c7bca88, 
sp=0x7feb54e43c88) at receiver.c:1128
         avp1 = 0x4be66ee0
         avp2 = 0x7feb4c7bdc50
         __FUNCTION__ = "receive_message"
#11 0x00007feb4be311ca in do_receive (sp=0x7feb54e43c88) at receiver.c:593
         cnt = 340
         n = 340
         version = 1
         dst = 0x7feb4c7bd4f4 ""
         dmsg = 0x7feb4c7bca88
         __FUNCTION__ = "do_receive"
#12 0x00007feb4be34fb1 in receive_loop (original_peer=0x7feb4c754c98) at 
receiver.c:800
         rfds = {__fds_bits = {1048576, 0 <repeats 15 times>}}
         efds = {__fds_bits = {0 <repeats 16 times>}}
         tv = {tv_sec = 0, tv_usec = 999996}
         n = 1
         max = 21
         cnt = 1
         msg = 0x0
         sp = 0x7feb54e43c88
         sp2 = 0x7feb54e43c88
         p = 0x7feb4c754c98
         fd = 20
         fd_exchange_pipe_local = 16
         __FUNCTION__ = "receive_loop"
#13 0x00007feb4be2e6fb in receiver_process (p=0x7feb4c754c98) at 
receiver.c:459
---Type <return> to continue, or q <return> to quit---
         __FUNCTION__ = "receiver_process"
#14 0x00007feb4bddfc4b in diameter_peer_start (blocking=0) at 
diameter_peer.c:289
         pid = 0
         k = -1
         p = 0x7feb4c754c98
         __FUNCTION__ = "diameter_peer_start"
#15 0x00007feb4bdd1e1e in cdp_child_init (rank=0) at cdp_mod.c:243
         __FUNCTION__ = "cdp_child_init"
#16 0x00000000004e6312 in init_mod_child (m=0x7feb54dfc9e8, rank=0) at 
core/sr_module.c:939
         __FUNCTION__ = "init_mod_child"
#17 0x00000000004e5fb4 in init_mod_child (m=0x7feb54dfda10, rank=0) at 
core/sr_module.c:935
         __FUNCTION__ = "init_mod_child"
#18 0x00000000004e5fb4 in init_mod_child (m=0x7feb54dfddd0, rank=0) at 
core/sr_module.c:935
         __FUNCTION__ = "init_mod_child"
#19 0x00000000004e5fb4 in init_mod_child (m=0x7feb54dff050, rank=0) at 
core/sr_module.c:935
         __FUNCTION__ = "init_mod_child"
#20 0x00000000004e5fb4 in init_mod_child (m=0x7feb54dffe70, rank=0) at 
core/sr_module.c:935
         __FUNCTION__ = "init_mod_child"
#21 0x00000000004e66e6 in init_child (rank=0) at core/sr_module.c:966
No locals.
#22 0x0000000000426955 in main_loop () at main.c:1708
         i = 8
         pid = 25334
         si = 0x0
         si_desc = "sctp receiver child=7 sock=192.168.151.1:5060", 
'\000' <repeats 19 times>, 
"\360\270\354\205\375\177\000\000g-\036U\353\177\000\000\000\353v\000\000\000\000\000x\261\343T\353\177\000\000 
\267\354\205\375\177\000\000P\266\354\205\375\177\000\000\004\000\000\000\000\000\000\000\000PKL\353\177\000"
         nrprocs = 8
         woneinit = 1
         __FUNCTION__ = "main_loop"
#23 0x000000000042d24e in main (argc=10, argv=0x7ffd85ecb9d8) at main.c:2646
         cfg_stream = 0x2264010
         c = -1
         r = 0
         tmp = 0x7ffd85eccf01 ""
         tmp_len = 32765
         port = -2048083952
         proto = 32765
         options = 0x74d6d0 
":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
         ret = -1
         seed = 2568094531
         rfd = 4
         debug_save = 0
         debug_flag = 0
         dont_fork_cnt = 2
         n_lst = 0x7feb55b41700
         p = 0x7ffd85ecba30 "\a\317\354\205\375\177"
         st = {st_dev = 20, st_ino = 729, st_nlink = 2, st_mode = 16832, 
st_uid = 479, st_gid = 2, __pad0 = 0, st_rdev = 0, st_size = 40, 
st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1516189324, 
tv_nsec = 279961062}, st_mtim = {tv_sec = 1516351702,
             tv_nsec = 739657044}, st_ctim = {tv_sec = 1516351702, 
tv_nsec = 739657044}, __glibc_reserved = {0, 0, 0}}
         __FUNCTION__ = "main"
(gdb) info locals
cfg_stream = 0x2264010
c = -1
r = 0
tmp = 0x7ffd85eccf01 ""
tmp_len = 32765
port = -2048083952
proto = 32765
options = 0x74d6d0 
":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
ret = -1
seed = 2568094531
rfd = 4
debug_save = 0
debug_flag = 0
dont_fork_cnt = 2
n_lst = 0x7feb55b41700
p = 0x7ffd85ecba30 "\a\317\354\205\375\177"
st = {st_dev = 20, st_ino = 729, st_nlink = 2, st_mode = 16832, st_uid = 
479, st_gid = 2, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 
4096, st_blocks = 0, st_atim = {tv_sec = 1516189324, tv_nsec = 
279961062}, st_mtim = {tv_sec = 1516351702,
     tv_nsec = 739657044}, st_ctim = {tv_sec = 1516351702, tv_nsec = 
739657044}, __glibc_reserved = {0, 0, 0}}
__FUNCTION__ = "main"
(gdb) list
50      in ../sysdeps/unix/sysv/linux/raise.c



Regards.



On 18.01.2018 21:14, Carsten Bock wrote:
> Hi Tsvetan,
>
> i've just pushed a safety check to master. Can you check, if this
> avoids the crash? If yes, can you share the output of the log? If I am
> correct, it's due to a miscalculation of available apps for the peer.
>
> Thanks,
> Carsten
>
> 2018-01-16 9:49 GMT+01:00 Tsvetan Filev <tsvetan.filev at inno-networks.com>:
>> Hi.
>>
>> I get a coredump which is caused by bad memory handling during the diameter
>> capability exchange process.
>> Here is part of the log file:
>> ===================================
>> 2018-01-16T09:16:39.890992+02:00 linux-o12d kamailio[13298]: 87(13388) INFO:
>> cdp [worker.c:332]: worker_process(): [0] Worker process started...
>> 2018-01-16T09:16:40.296393+02:00 linux-o12d systemd-coredump[13196]: Process
>> 13193 (kamailio) of user 479 dumped core.
>> 2018-01-16T09:16:40.871483+02:00 linux-o12d kamailio[13298]: 94(13395)
>> DEBUG: cdp [peermanager.c:263]: peer_timer(): peer_timer(): taking care of
>> peers...
>> 2018-01-16T09:16:40.871545+02:00 linux-o12d kamailio[13298]: 94(13395)
>> DEBUG: cdp [peermanager.c:280]: peer_timer(): peer_timer(): Peer
>> hss.epc.mnc019.mcc425.3gppnetwork.org State 0
>> 2018-01-16T09:16:40.871596+02:00 linux-o12d kamailio[13298]: 94(13395)
>> DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer
>> hss.epc.mnc019.mcc425.3gppnetwork.org State Closed Event Start
>> 2018-01-16T09:16:40.871612+02:00 linux-o12d kamailio[13298]: 94(13395) INFO:
>> cdp [peerstatemachine.c:525]: I_Snd_Conn_Req(): I_Snd_Conn_Req(): Peer
>> hss.epc.mnc019.mcc425.3gppnetwork.org
>> 2018-01-16T09:16:40.871636+02:00 linux-o12d kamailio[13298]: 94(13395) INFO:
>> cdp [receiver.c:869]: peer_connect(): peer_connect(): Trying to connect to
>> 10.82.10.85 port 3868
>> 2018-01-16T09:16:40.871782+02:00 linux-o12d kamailio[13298]: 94(13395) INFO:
>> cdp [receiver.c:937]: peer_connect(): peer_connect(): Peer
>> hss.epc.mnc019.mcc425.3gppnetwork.org:3868 connected
>> 2018-01-16T09:16:40.871813+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [receiver.c:697]: receive_loop(): select_recv(): There is
>> something on the fd exchange pipe
>> 2018-01-16T09:16:40.871828+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [receiver.c:706]: receive_loop(): select_recv(): fd exchange pipe
>> says fd [22] for peer 0x7f0626b91c98:[hss.epc.mnc019.mcc425.3gppnetwork.org]
>> 2018-01-16T09:16:40.871910+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer
>> hss.epc.mnc019.mcc425.3gppnetwork.org State Wait_Conn_Ack Event
>> I_Rcv_Conn_Ack
>> 2018-01-16T09:16:40.871933+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [diameter_msg.c:184]: AAANewMessage(): AAANewMessage: param
>> session received null and it's a request!!
>> 2018-01-16T09:16:40.872011+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [diameter_msg.c:81]: AAABuildMsgBuffer(): AAABuildMsgBuffer():
>> len=204
>> 2018-01-16T09:16:40.872041+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [receiver.c:994]: peer_send_msg(): peer_send_msg(): Pipe push
>> [0x7f0626c02548]
>> 2018-01-16T09:16:40.872064+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [receiver.c:751]: receive_loop(): select_recv(): There is
>> something on the send pipe
>> 2018-01-16T09:16:40.872084+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [receiver.c:764]: receive_loop(): select_recv(): Send pipe says
>> [0x7f0626c02548] 8
>> 2018-01-16T09:16:40.872104+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [diameter_msg.c:410]: AAAFreeMessage(): AAAFreeMessage: Freeing
>> message (0x7f0626c02548) 257
>> 2018-01-16T09:16:40.872277+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [receiver.c:574]: do_receive(): receive_loop():
>> [hss.epc.mnc019.mcc425.3gppnetwork.org] Recv Version 1 Length 360
>> 2018-01-16T09:16:40.872322+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [receiver.c:1088]: receive_message(): receive_message():
>> [hss.epc.mnc019.mcc425.3gppnetwork.org] Recv msg 257
>> 2018-01-16T09:16:40.872345+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer
>> hss.epc.mnc019.mcc425.3gppnetwork.org State Wait_I_CEA Event I_Rcv_CEA
>> 2018-01-16T09:16:40.872372+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:692]: count_Supported_Vendor_Id_AVPS(): Found
>> 4 Supported_Vendor AVPS92(13393) DEBUG: cdp [peerstatemachine.c:743]:
>> save_peer_applications(): Found Supported Vendor for Application 0: 5535
>> 2018-01-16T09:16:40.872389+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 10415
>> 2018-01-16T09:16:40.872405+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 13019
>> 2018-01-16T09:16:40.872420+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 58637
>> 2018-01-16T09:16:40.872438+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 5535
>> 2018-01-16T09:16:40.872453+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 10415
>> 2018-01-16T09:16:40.872468+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 13019
>> 2018-01-16T09:16:40.872486+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 58637
>> 2018-01-16T09:16:40.872504+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 5535
>> 2018-01-16T09:16:40.872523+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 10415
>> 2018-01-16T09:16:40.872539+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 13019
>> 2018-01-16T09:16:40.872554+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found
>> Supported Vendor for Application 0: 58637
>> 2018-01-16T09:16:40.872570+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found
>> Supported Vendor for Application 1: 5535
>> 2018-01-16T09:16:40.872586+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found
>> Supported Vendor for Application 1: 10415
>> 2018-01-16T09:16:40.872601+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found
>> Supported Vendor for Application 1: 13019
>> 2018-01-16T09:16:40.872616+02:00 linux-o12d kamailio[13298]: 92(13393)
>> DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found
>> Supported Vendor for Application 1: 58637
>> 2018-01-16T09:16:40.872634+02:00 linux-o12d kamailio[13298]: 92(13393)
>> CRITICAL: <core> [core/mem/q_malloc.c:145]: qm_debug_frag(): BUG: qm: prev.
>> fragm. tail overwritten(28af01000000, 0)[0x7f0626c038d0:0x7f0626c03908]!
>> Memory allocator was called from cdp: diameter_avp.c:365. Fragment marked by
>> cdp: diameter_avp.c:142.
>> 2018-01-16T09:16:41.054292+02:00 linux-o12d kamailio[13298]: 0(13298) ALERT:
>> <core> [main.c:746]: handle_sigs(): child process 13393 exited by a signal 6
>> 2018-01-16T09:16:41.054403+02:00 linux-o12d kamailio[13298]: 0(13298) ALERT:
>> <core> [main.c:749]: handle_sigs(): core was generated
>> 2018-01-16T09:16:41.054422+02:00 linux-o12d kamailio[13298]: 0(13298) INFO:
>> <core> [main.c:771]: handle_sigs(): terminating due to SIGCHLD
>> 2018-01-16T09:16:41.054438+02:00 linux-o12d kamailio[13298]: 0(13298) DEBUG:
>> <core> [main.c:773]: handle_sigs(): terminating due to SIGCHLD
>> 2018-01-16T09:16:41.054464+02:00 linux-o12d kamailio[13298]: 2(13303) INFO:
>> <core> [main.c:826]: sig_usr(): signal 15 received
>> ...
>> ===================================
>>
>> Attached is wireshark trace.
>> Here is my DiameterPeer.xml:
>>
>> ===================================
>> <?xml version="1.0" encoding="UTF-8"?>
>> <DiameterPeer
>>          FQDN="ims110-scscf.epc.mnc019.mcc425.3gppnetwork.org"
>>          Realm="epc.mnc019.mcc425.3gppnetwork.org"
>>          Vendor_Id="10415"
>>          Product_Name="CDiameterPeer"
>>          AcceptUnknownPeers="1"
>>          DropUnknownOnDisconnect="1"
>>          Tc="30"
>>          Workers="4"
>>          QueueLength="8"
>>          TransactionTimeout="5"
>>          SessionsHashSize="128"
>>          DefaultAuthSessionTimeout="3600"
>>          MaxAuthSessionTimeout="3600">
>>
>>          <Peer FQDN="hss.epc.mnc019.mcc425.3gppnetwork.org"
>> Realm="epc.mnc019.mcc425.3gppnetwork.org" port="3868" />
>>          <Acceptor port="3869" bind="10.82.10.56" />
>>          <Auth id="16777216" vendor="10415" /> <!--3GPP CxDX -->
>>          <DefaultRoute FQDN="ims110-scscf.epc.mnc019.mcc425.3gppnetwork.org"
>> metric="10" />
>> </DiameterPeer>
>> ===================================
>>
>> It looks like buffer overflow to me but I'm not sure.
>>
>> kamailio version is:
>> ===================================
>> kamailio -v
>> version: kamailio 5.1.0 (x86_64/linux)
>> flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE,
>> USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC,
>> TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT,
>> USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST,
>> HAVE_RESOLV_RES
>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
>> MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
>> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
>> id: unknown
>> compiled on 05:30:36 Jan 15 2018 with gcc 4.8.5
>> ===================================
>>
>> OS is openSUSE Leap 42.3.
>> Kernel: Linux linux-o12d 4.4.104-39-default #1 SMP Thu Jan 4 08:11:03 UTC
>> 2018 (7db1912) x86_64 x86_64 x86_64 GNU/Linux
>>
>> Do you have any idea what might be wrong ?
>> It could be bad config but still it should say something in the log without
>> a crash.
>>
>> Regards.
>>
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>
>
>




More information about the sr-users mailing list