[SR-Users] Crash Kamailio 4.1.4

Daniel-Constantin Mierla miconda at gmail.com
Tue Oct 7 08:59:35 CEST 2014


Hello,

I pushed a fix to kamailio master branch that should solve it -- last 
commits to acc module. If you can give it a try and report results, 
would be appreciated.

Cheers,
Daniel

On 03/10/14 17:00, Igor Potjevlesch wrote:
>
> Hello Daniel,
>
> I’m just seeing that 4.2 is scheduled for 15th October.
>
> I can wait until this date.
>
> Regards,
>
> Igor.
>
> *De :*Igor Potjevlesch [mailto:igor.potjevlesch at gmail.com]
> *Envoyé :* jeudi 2 octobre 2014 18:43
> *À :* miconda at gmail.com
> *Cc :* 'Kamailio \(SER\) - Users Mailing List'
> *Objet :* RE: [SR-Users] Crash Kamailio 4.1.4
>
> Hello,
>
> Thank you for your time on this issue.
>
> I understand that I can define “db_extra” like:
>
> modparam("acc", "db_extra",
>
> "src_user=$fU;username=$Au;src_domain=$fd;src_ip=$si;src_pai=*$avp(s:pai)*;"
>
> "dst_ouser=$tU;dst_user=$rU;dst_domain=$rd")
>
> Instead of:
>
> modparam("acc", "db_extra",
>
> "src_user=$fU;username=$Au;src_domain=$fd;src_ip=$si;*src_pai=$ai*;"
>
> "dst_ouser=$tU;dst_user=$rU;dst_domain=$rd")
>
> And, in the request_route
>
> request_route {
>
> *        #Store PAI in AVP for accounting*
>
> *        $avp(s:pai)=$ai;*
>
> # per request initial checks
>
> route(REQINIT);
>
> […]
>
> }
>
> I can put this fix while waiting for 4.2.
>
> Regards,
>
> Igor.
>
> *De :*Daniel-Constantin Mierla [mailto:miconda at gmail.com]
> *Envoyé :* jeudi 2 octobre 2014 10:17
> *À :* Igor Potjevlesch
> *Cc :* 'Kamailio \(SER\) - Users Mailing List'
> *Objet :* Re: [SR-Users] Crash Kamailio 4.1.4
>
> Hello,
>
> not really troubleshooting, as I got traveling, but I thought a bit of 
> it, also from performances point of view.
>
> As it was related to parallel processing of 1xx/2xx sip responses that 
> you are accounting, I think the safest solutions would be:
>
> - clone the message locally to acc if it is the tm (share memory)
> - do synchronized access to tm data inside the acc (use locks there)
>
> Instead of using locks or other workarounds in tm (lower performances 
> in all cases), better do it on the part doing parallel access out of 
> tm. Acc is doing some particular processing -- it uses a callback for 
> a reply to access the response, so it is quite specific to it.
>
> If you want to be safe side immediately, store the PAI in an avp 
> inside request_route and use that avp for accounting.
>
> A solution will be there before 4.2 is out.
>
> Cheers,
> Daniel
>
> On 02/10/14 09:50, Igor Potjevlesch wrote:
>
>     Hello Daniel,
>
>     Have you had the opportunity to look at this?
>
>     Thank you for your feedback.
>
>     Regards,
>
>     Igor.
>
>     *De :*Daniel-Constantin Mierla [mailto:miconda at gmail.com]
>     *Envoyé :* mercredi 24 septembre 2014 09:00
>     *À :* Igor Potjevlesch
>     *Cc :* 'Kamailio \(SER\) - Users Mailing List'
>     *Objet :* Re: [SR-Users] Crash Kamailio 4.1.4
>
>     Hello,
>
>     so it is still related to PAI header -- I will have more time to
>     look at it by end of the week.
>
>     Cheers,
>     Daniel
>
>     On 23/09/14 19:16, Igor Potjevlesch wrote:
>
>         Hello Daniel,
>
>         Patching has been done in the same time than the upgrade to 4.1.5.
>
>         A new crash occurred in pvapi.c (in addition of my other
>         recent post “Crash Kamailio 4.1.5”).
>
>         Here is the result of a “bt full”:
>
>         #0  0x000000000049580e in pv_get_strval (msg=0x7f14efe24ea8,
>         param=0x7f14fb65d058, res=0x7fff6c6b7430, sval=0x22) at
>         pvapi.c:521
>
>         No locals.
>
>         #1  0x00007f14f9677f2b in pv_get_pai (msg=0x7f14efe24ea8,
>         param=0x7f14fb65d058, res=0x7fff6c6b7430) at pv_core.c:1026
>
>                 idxf = 0
>
>                 idx = 0
>
>                 pai_body = 0x7f14fb679d38
>
>                 pai_uri = 0xa
>
>                 i = 0
>
>                 cur_id = 0
>
>                 __FUNCTION__ = "pv_get_pai"
>
>         #2  0x0000000000499594 in pv_get_spec_value
>         (msg=0x7f14efe24ea8, sp=0x7f14fb65d040, value=0x7fff6c6b7430)
>         at pvapi.c:1266
>
>                 ret = 0
>
>                 __FUNCTION__ = "pv_get_spec_value"
>
>         #3  0x00007f14f7d3481d in extra2strar (extra=0x7f14fb65d030,
>         rq=0x7f14efe24ea8, val_arr=0x7f14f7f41e30,
>         int_arr=0x7f14f7f4237c, type_arr=0x7f14f7f424e7
>         "\002\002\002\002") at acc_extra.c:261
>
>                 value = {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}
>
>                 n = 4
>
>                 r = 0
>
>                 __FUNCTION__ = "extra2strar"
>
>         #4  0x00007f14f7d2c3e3 in acc_db_request (rq=0x7f14efe24ea8)
>         at acc.c:474
>
>                 m = 7
>
>                 n = -270381400
>
>                 i = 6
>
>                 t = 0x414cc0
>
>                 __FUNCTION__ = "acc_db_request"
>
>         #5  0x00007f14f7d36bc8 in acc_onreply (t=0x7f14efe525b8,
>         req=0x7f14efe24ea8, reply=0x7f14fb670c48, code=200) at
>         acc_logic.c:471
>
>                 new_uri_bk = {s = 0x7f14efe25590
>         "sip:ABCDEFGHIJ@<IP_GW>oTE sINVITE
>         sip:ABCDEFGHIJ at sip.fqdn.tldSIP/2.0\r\nVia: SIP/2.0/UDP
>         <IP_UAC>:5060;branch=z9hG4bK5f32deec\r\nMax-Forwards:
>         69\r\nFrom: \"KLMNOPQRST\" <sip:KLMNOPQRST at sip.fqdn."..., len
>         = 19}
>
>                 br = 0
>
>                 hdr = 0x7f14f7d3dc20
>
>                 __FUNCTION__ = "acc_onreply"
>
>         #6  0x00007f14f7d3730a in tmcb_func (t=0x7f14efe525b8,
>         type=512, ps=0x7fff6c6b76b0) at acc_logic.c:573
>
>                 __FUNCTION__ = "tmcb_func"
>
>         #7  0x00007f14f9f3146c in run_trans_callbacks_internal
>         (cb_lst=0x7f14efe52628, type=512, trans=0x7f14efe525b8,
>         params=0x7fff6c6b76b0) at t_hooks.c:290
>
>                 cbp = 0x7f14ee4c81b0
>
>                 backup_from = 0x934630
>
>                 backup_to = 0x934638
>
>                 backup_dom_from = 0x934640
>
>                 backup_dom_to = 0x934648
>
>                 backup_uri_from = 0x934620
>
>                 backup_uri_to = 0x934628
>
>                 backup_xavps = 0x934760
>
>                 __FUNCTION__ = "run_trans_callbacks_internal"
>
>         #8  0x00007f14f9f3167e in run_trans_callbacks_with_buf
>         (type=512, rbuf=0x7f14efe52678, req=0x7f14efe24ea8,
>         repl=0x7f14fb670c48, flags=183) at t_hooks.c:336
>
>                 params = {req = 0x7f14efe24ea8, rpl = 0x7f14fb670c48,
>         param = 0x7f14ee4c81c0, code = 200, flags = 183, branch = 0,
>         t_rbuf = 0x7f14efe52678, dst = 0x7f14efe526c8, send_buf = {
>
>                     s = 0x7f14efd7c408 "SIP/2.0 200 OK\r\nVia:
>         SIP/2.0/UDP
>         <IP_UAC>:5060;rport=5060;branch=z9hG4bK5f32deec\r\nCall-ID:
>         7846f7332ce6a7db4484c3d06ce1c387 at sip.fqdn.tld\r\nFrom
>         <mailto:7846f7332ce6a7db4484c3d06ce1c387 at sip.fqdn.tld%5Cr%5CnFrom>:
>         \"KLMNOPQRST\" <sip:KLMNOPQRST at sip.fqdn.tld>
>         <sip:KLMNOPQRST at sip.fqdn.tld>"..., len = 980}}
>
>                 trans = 0x7f14efe525b8
>
>         #9  0x00007f14f9f63bfa in relay_reply (t=0x7f14efe525b8,
>         p_msg=0x7f14fb670c48, branch=0, msg_status=183,
>         cancel_data=0x7fff6c6b7a10, do_put_on_wait=1) at t_reply.c:2001
>
>                 relay = 0
>
>                 save_clone = 0
>
>                 buf = 0x7f14fb67e740 "SIP/2.0 183 Session
>         Progress\r\nVia: SIP/2.0/UDP
>         <IP_UAC>:5060;rport=5060;branch=z9hG4bK5f32deec\r\nCall-ID:
>         7846f7332ce6a7db4484c3d06ce1c387 at sip.fqdn.tld\r\nFrom
>         <mailto:7846f7332ce6a7db4484c3d06ce1c387 at sip.fqdn.tld%5Cr%5CnFrom>:
>         \"KLMNOPQRST\" <sip:KLMNOPQRST at tru"...
>
>                 res_len = 777
>
>                 relayed_code = 183
>
>                 relayed_msg = 0x7f14fb670c48
>
>                 reply_bak = 0x414cc0
>
>                 bm = {to_tag_val = {s = 0x7f14efe53b50 "", len =
>         -79437808}}
>
>                 totag_retr = 0
>
>                 reply_status = RPS_PROVISIONAL
>
>                 uas_rb = 0x7f14efe52678
>
>                 to_tag = 0x0
>
>                 reason = {s = 0x800000001 <Address 0x800000001 out of
>         bounds>, len = 1}
>
>                 onsend_params = {req = 0x200924970, rpl =
>         0x7f14f9f83f90, param = 0x414cc0, code = 1818984640, flags =
>         3, branch = 0, t_rbuf = 0x0, dst = 0x7f14fb670e40, send_buf =
>         {s = 0x7fff6c6b7830 "`xkl\377\177", len = -101469275}}
>
>                 __FUNCTION__ = "relay_reply"
>
>         #10 0x00007f14f9f660ab in reply_received
>         (p_msg=0x7f14fb670c48) at t_reply.c:2499
>
>                 msg_status = 183
>
>                 last_uac_status = 183
>
>                 ack = 0x40 <Address 0x40 out of bounds>
>
>                 ack_len = 0
>
>                 branch = 0
>
>                 reply_status = -77092928
>
>                 onreply_route = 1
>
>                 cancel_data = {cancel_bitmap = 0, reason = {cause = 0,
>         u = {text = {s = 0x0, len = 9586205}, e2e_cancel = 0x0,
>         packed_hdrs = {s = 0x0, len = 9586205}}}}
>
>                 uac = 0x7f14efe52720
>
>                 t = 0x7f14efe525b8
>
>                 lack_dst = {send_sock = 0x7f14fb584420, to = {s =
>         {sa_family = 57360, sa_data =
>         "C\373\024\177\000\000\000F#\000\000\000\000"}, sin =
>         {sin_family = 57360, sin_port = 64323, sin_addr = {s_addr =
>         32532}, sin_zero = "\000F#\000\000\000\000"}, sin6 = {
>
>                       sin6_family = 57360, sin6_port = 64323,
>         sin6_flowinfo = 32532, sin6_addr = {__in6_u = {__u6_addr8 =
>         "\000F#\000\000\000\000\000\020\341C\373\024\177\000",
>         __u6_addr16 = {17920, 35, 0, 0, 57616, 64323, 32532, 0},
>         __u6_addr32 = {2311680, 0, 4215529744,
>
>                             32532}}}, sin6_scope_id = 4215529744}}, id
>         = 32532, proto = 96 '`', send_flags = {f = 64 '@', blst_imask
>         = 103 'g'}}
>
>                 backup_user_from = 0x934630
>
>                 backup_user_to = 0x934638
>
>                 backup_domain_from = 0x934640
>
>                backup_domain_to = 0x934648
>
>                 backup_uri_from = 0x934620
>
>                 backup_uri_to = 0x934628
>
>                 backup_xavps = 0x934760
>
>                 replies_locked = 1
>
>                 branch_ret = 0
>
>                 prev_branch = 1818983120
>
>                 blst_503_timeout = 32767
>
>                 hf = 0x7f14fb670c68
>
>                 onsend_params = {req = 0x7fff6c6b7a90, rpl = 0x550bb0,
>         param = 0x234540, code = 0, flags = 3, branch = 0, t_rbuf =
>         0x7f14fb43e380, dst = 0x7f14fb674030, send_buf = {s =
>         0x7fff6c6b7a90 "`G\223", len = 5538065}}
>
>                 ctx = {rec_lev = 0, run_flags = 0, last_retcode = 0,
>         jmp_env = {{__jmpbuf = {139728093908544, 1898006658898931560,
>         4279488, 140735012372672, 0, 0, 1898006658936680296,
>         -1897762211976106136}, __mask_was_saved = 0, __saved_mask =
>         {__val = {9586373,
>
>                           1365809186688, 124554051613, 9586450,
>         139728093947840, 9587056, 9586211, 361695345073193192,
>         9586309, 9586288, 4217874320, 139728093947840,
>         139728093942016, 139728093908544, 4279488, 140735012372672}}}}}
>
>                 __FUNCTION__ = "reply_received"
>
>         #11 0x000000000045d853 in do_forward_reply
>         (msg=0x7f14fb670c48, mode=0) at forward.c:777
>
>                 new_buf = 0x0
>
>                 dst = {send_sock = 0x0, to = {s = {sa_family = 0,
>         sa_data = '\000' <repeats 13 times>}, sin = {sin_family = 0,
>         sin_port = 0, sin_addr = {s_addr = 0}, sin_zero =
>         "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 0,
>         sin6_port = 0, sin6_flowinfo = 0,
>
>                       sin6_addr = {__in6_u = {__u6_addr8 = '\000'
>         <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0},
>         __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, id = 0,
>         proto = 0 '\000', send_flags = {f = 0 '\000', blst_imask = 0
>         '\000'}}
>
>                 new_len = 32532
>
>                 r = 1
>
>                 s = 0x370fb670c50 <Address 0x370fb670c50 out of bounds>
>
>                 len = 0
>
>                 __FUNCTION__ = "do_forward_reply"
>
>         #12 0x000000000045e114 in forward_reply (msg=0x7f14fb670c48)
>         at forward.c:860
>
>         No locals.
>
>         #13 0x00000000004a5903 in receive_msg (buf=0x924600 "SIP/2.0
>         183 Session Progress\r\nVia: SIP/2.0/UDP
>         <IP_KAMAILIO>;branch=z9hG4bKca48.51c2c569361ea0fedd9c6c70c21b5eed.0;received=<IP_KAMAILIO>\r\nVia:
>         SIP/2.0/UDP <IP_UAC>:5060;rport=5060;branch=z9hG4bK5f32d"...,
>
>             len=880, rcv_info=0x7fff6c6b7d90) at receive.c:273
>
>                 msg = 0x7f14fb670c48
>
>                 ctx = {rec_lev = 8868984, run_flags = 0, last_retcode
>         = 0, jmp_env = {{__jmpbuf = {0, 0, 0, 263853236176, 1, 0,
>         171719254808, 9586112}, __mask_was_saved = 1818983832,
>         __saved_mask = {__val = {139728091862768, 12884901899,
>         139728091862768, 4279488,
>
>                           140735012372672, 140735012371728, 5477982,
>         0, 139727728366976, 50195, 171356018048, 9586112,
>         140735012371856, 140735012371776, 5474817, 4279488}}}}}
>
>                 ret = 32532
>
>                 inb = {s = 0x924600 "SIP/2.0 183 Session
>         Progress\r\nVia: SIP/2.0/UDP
>         <IP_KAMAILIO>;branch=z9hG4bKca48.51c2c569361ea0fedd9c6c70c21b5eed.0;received=<IP_KAMAILIO>\r\nVia:
>         SIP/2.0/UDP <IP_UAC>:5060;rport=5060;branch=z9hG4bK5f32d"...,
>         len = 880}
>
>                 __FUNCTION__ = "receive_msg"
>
>         #14 0x000000000053c9c4 in udp_rcv_loop () at udp_server.c:536
>
>                 len = 880
>
>                 buf = "SIP/2.0 183 Session Progress\r\nVia:
>         SIP/2.0/UDP
>         <IP_KAMAILIO>;branch=z9hG4bKca48.51c2c569361ea0fedd9c6c70c21b5eed.0;received=<IP_KAMAILIO>\r\nVia:
>         SIP/2.0/UDP <IP_UAC>:5060;rport=5060;branch=z9hG4bK5f32d"...
>
>                 tmp = 0x9245c0 "10.143.1.10"
>
>                 from = 0x7f14fb5add70
>
>                 fromlen = 16
>
>                 ri = {src_ip = {af = 2, len = 4, u = {addrl =
>         {403182777, 139728091862768}, addr32 = {403182777, 0,
>         4215789296, 32532}, addr16 = {5305, 6152, 0, 0, 55024, 64327,
>         32532, 0}, addr =
>         "\271\024\b\030\000\000\000\000\360\326G\373\024\177\000"}},
>         dst_ip = {af = 2,
>
>                     len = 4, u = {addrl = {67638457, 0}, addr32 =
>         {67638457, 0, 0, 0}, addr16 = {5305, 1032, 0, 0, 0, 0, 0, 0},
>         addr = "\271\024\b\004", '\000' <repeats 11 times>}}, src_port
>         = 5060, dst_port = 5060, proto_reserved1 = 0, proto_reserved2
>         = 0, src_su = {s = {
>
>                       sa_family = 2, sa_data =
>         "\023Ĺ\024\b\030\000\000\000\000\000\000\000"}, sin =
>         {sin_family = 2, sin_port = 50195, sin_addr = {s_addr =
>         403182777}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 =
>         {sin6_family = 2, sin6_port = 50195,
>
>                       sin6_flowinfo = 403182777, sin6_addr = {__in6_u
>         = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0,
>         0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}},
>         sin6_scope_id = 0}}, bind_address = 0x7f14fb47d588, proto = 1
>         '\001'}
>
>                 __FUNCTION__ = "udp_rcv_loop"
>
>         #15 0x000000000046d447 in main_loop () at main.c:1617
>
>                 i = 13
>
>                 pid = 0
>
>                 si = 0x7f14fb47d588
>
>                 si_desc = "udp receiver child=13
>         sock=<IP_KAMAILIO>:5060\000\373\024\177\000\000\b$P\373\024\177\000\000\036\205^\000\000\000\000\000\000w^\000\000\000\000\000\000\244\303v\000\000\000\000\300LA\000\000\000\000\000\300\200kl\377\177",
>         '\000' <repeats 19 times>,
>         "\177kl\377\177\000\000\020\245K\000\000\000\000"
>
>                 nrprocs = 15
>
>                 __FUNCTION__ = "main_loop"
>
>         #16 0x000000000047054f in main (argc=7, argv=0x7fff6c6b80c8)
>         at main.c:2545
>
>                 cfg_stream = 0xf42010
>
>                 c = -1
>
>                 r = 0
>
>         tmp = 0x7fff6c6b8f70 ""
>
>         tmp_len = 0
>
>         port = 0
>
>         proto = 0
>
>         options = 0x5e0a58
>         ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"
>
>         ret = -1
>
>                 seed = 3572644655
>
>                 rfd = 4
>
>                 debug_save = 0
>
>                 debug_flag = 0
>
>                 dont_fork_cnt = 0
>
>                 n_lst = 0x3d6f60fb88
>
>                 p = 0x5caba0 "H\211l$\330L\211d$\340H\215-O\244*"
>
>                 __FUNCTION__ = "main"
>
>         Let me know if you need further information.
>
>         Regards,
>
>         Igor.
>

  <http://www.linkedin.com/in/miconda>


-- 
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20141007/18547113/attachment.html>


More information about the sr-users mailing list