On Monday 28 February 2011, Andrew O. Zhukov wrote: Hi Andrew, 1.5 still used from a lot of people, including us. Have you tried to reach one of the companies on this list: http://www.kamailio.org/w/business-directory/ There is also a dedicated business list which maybe you get a faster response for your support query, more information can be found here: http://www.kamailio.org/w/business/ Best regards, Henning

On 2/28/11 8:06 AM, Andrew O. Zhukov wrote: In the first place, the problem is you are using very old versions and it is very unlikely someone has a testbed for them. I and many others still have such versions running, but never happened to crash, it has to be something specific, like a not very common module or particular sip request that triggers this one. I tried to help you in the spare time, which didn't happen to be that much lately. Your way of answering the questions was also consuming a lot of such cycles. Normally, yes, we officially support the latest two stable version, those being now 3.0 and 3.1. And it is really advisable to use the latest stable. But as you could see, we don't mind doing it for older version when we can, but that is not always possible we current constraints of time and load. Even if you are willing to get paid support, it is not always possible to get it from a day to the next one, people travel or have other project booked some time ago. Cheers, Daniel -- Daniel-Constantin Mierla http://www.asipto.com

On 28.02.2011 15:49, Iñaki Baz Castillo wrote: He provided detail, unfortunately in another thread which broke continuity. Look for "After upgrade from openser 1.3.4 to kamailio 1.5.5 the same crash set" from 10.2., 7:37. Cheers, --Timo

I someone interested in . It's the old coredumps from 1.3.4. It's really much stabile then 1.5.5 I did degrade today night and going to try opensips. warning: Missing the matching library file: /usr/lib/debug/.build-id/7f/bba7706883599d6df45722d7eb779d8768a80b Core was generated by `openser -P /var/run/openser/openser.pid -m 32 -u openser -g openser -w /var/run'. Program terminated with signal 11, Segmentation fault. #0 fm_malloc (qm=0x7389e0, size=<value optimized out>) at mem/f_malloc.c:267 267 mem/f_malloc.c: No such file or directory. in mem/f_malloc.c (gdb) bt full #0 fm_malloc (qm=0x7389e0, size=<value optimized out>) at mem/f_malloc.c:267 f = (struct fm_frag **) 0x778130 hash = 10 #1 0x00000000004918ad in parse_headers (msg=0x7711b8, flags=256, next=0) at parser/msg_parser.c:308 hf = <value optimized out> itr = (struct hdr_field *) 0xd302e3535666565 tmp = 0x7298b6 "User-Agent: Asterisk PBX\r\nMax-Forwards: 5\r\nContent-Length: 0\r\n\r\n" rest = 0x7298b6 "User-Agent: Asterisk PBX\r\nMax-Forwards: 5\r\nContent-Length: 0\r\n\r\n" end = 0x7298f7 "" orig_flag = 0 __FUNCTION__ = "parse_headers" #2 0x00002aaaab55fc18 in is_maxfwd_present (msg=0x7389e0, foo=0x7fffbf67f400) at mf_funcs.c:61 x = <value optimized out> __FUNCTION__ = "is_maxfwd_present" _c = 57 '9' #3 0x00002aaaab55ed61 in w_process_maxfwd_header (msg=0x7389e0, str1=0xa <Address 0xa out of bounds>, str2=0xd302e3535666565 <Address 0xd302e3535666565 out of bounds>) at maxfwd.c:145 val = <value optimized out> mf_value = {s = 0x2e9 <Address 0x2e9 out of bounds>, len = -1342089735} __FUNCTION__ = "w_process_maxfwd_header" #4 0x000000000040c28f in do_action (a=0x7462d0, msg=0x7711b8) at action.c:821 ret = <value optimized out> v = <value optimized out> to = <value optimized out> p = <value optimized out> tmp = <value optimized out> new_uri = <value optimized out> end = <value optimized out> crt = <value optimized out> len = <value optimized out> user = <value optimized out> uri = {user = {s = 0x7fffbf67f820 "\005", len = 7643232}, passwd = {s = 0x1 <Address 0x1 out of bounds>, len = 7629192}, host = { s = 0x74a0f0 "\006", len = 7803320}, port = {s = 0x2 <Address 0x2 out of bounds>, len = 7803320}, params = {s = 0x7711b8 "�\211\a", len = 4247863}, headers = {s = 0xa8 <Address 0xa8 out of bounds>, len = 1}, port_no = 63760, proto = 48999, type = 32767, transport = {s = 0x70a4a0 "\b�\f��*", len = -1083705312}, ttl = {s = 0x38404748fa "H\205�I\211�tJ\203=_\026.", len = 0}, user_param = {s = 0x86ed10 "�\216\207", len = 1}, maddr = { s = 0x3fb0012c05 "\205������u\035\017�D$\017<It\034<Tt,<Et<�\203\220", len = 67}, method = {s = 0x3840752980 "", len = 9916304}, lr = { s = 0xff96 <Address 0xff96 out of bounds>, len = -1083705072}, r2 = {s = 0x770eb8 "N�\"��*", len = 1}, transport_val = {s = 0x3840475f2c "\203=5", len = 8932}, ttl_val = {s = 0x86f0b8 "`�\206", len = 0}, user_param_val = {s = 0x3fb0014e6a "H\211�H\003S\bH\213\003H\211S\b�\004\020", len = 1}, maddr_val = {s = 0x0, len = 9916304}, method_val = {s = 0x2aaaaacd4030 "I\213~\bH�G0", len = -1083705072}, lr_val = {s = 0x70a4a0 "\b�\f��*", len = 0}, r2_val = {s = 0x417bc1 "H\205�tGH\211���\a", len = 65430}} next_hop = {user = {s = 0x70a4a0 "\b�\f��*", len = -1342088916}, passwd = {s = 0x0, len = 0}, host = { s = 0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>, len = 9935623}, port = { s = 0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>, len = 1081420160}, params = {s = 0xf <Address 0xf out of bounds>, len = 8754704}, headers = {s = 0x4 <Address 0x4 out of bounds>, len = -1083705484}, port_no = 3768, proto = 119, type = ERROR_URI_T, transport = { s = 0x38404748fa "H\205�I\211�tJ\203=_\026.", len = 15}, ttl = {s = 0x384052146b "/etc/localtime", len = 8754704}, user_param = { s = 0x4 <Address 0x4 out of bounds>, len = -1083705484}, maddr = {s = 0x770eb8 "N�\"��*", len = 1}, method = {s = 0x384048cc0e "\213\005\f\221,", len = 5197468}, lr = {s = 0x2aaaaaee2059 "2011-01-27 18:10:30','in','as5a292c8e','')", len = -1427234727}, r2 = {s = 0x384048cf70 "\213\005�\215,", len = 16}, transport_val = {s = 0x38404923d8 "�Y���Hc�L\211��0", len = 0}, ttl_val = {s = 0x200 <Address 0x200 out of bounds>, len = -1}, user_param_val = { s = 0x72971b "193.28.184.13;branch=z9hG4bK558b.e0feef55.0\r\nVia: SIP/2.0/UDP XXX.XX.XXX.13;branch=z9hG4bK558b.d0feef55.0\r\nVia: SIP/2.0/UDP XXX.XX.XXX.13;branch=z9hG4bK558b.c0feef55.0\r\nVia: SIP/2.0/UDP XXX.XX.XXX.70:"..., len = 0}, maddr_val = {s = 0x758c50 "\002", len = 7703472}, method_val = { s = 0x746988 "\016", len = 7655064}, lr_val = {s = 0x7711b8 "�\211\a", len = 1}, r2_val = {s = 0x7711b8 "�\211\a", len = -1083705856}} u = <value optimized out> ---Type <return> to continue, or q <return> to quit--- port = <value optimized out> cmatch = <value optimized out> aitem = <value optimized out> adefault = <value optimized out> spec = <value optimized out> model = <value optimized out> val = {rs = {s = 0x772508 "\n", len = 7807240}, ri = 7807992, flags = 0} __FUNCTION__ = "do_action" #5 0x000000000040e7db in run_action_list (a=<value optimized out>, msg=0x7711b8) at action.c:132 ret = -1 t = (struct action *) 0x7462d0 __FUNCTION__ = "run_action_list" #6 0x000000000045ba94 in eval_expr (e=0x746370, msg=0x7711b8, val=0xd302e3535666565) at route.c:1096 ret = <value optimized out> rec_lev = 3 __FUNCTION__ = "eval_expr" #7 0x000000000045b50d in eval_expr (e=0x7463b8, msg=0x7711b8, val=0x0) at route.c:1403 ret = <value optimized out> rec_lev = 3 __FUNCTION__ = "eval_expr" #8 0x000000000045b4b5 in eval_expr (e=0x746400, msg=0x7711b8, val=0x0) at route.c:1408 ret = <value optimized out> rec_lev = 3 __FUNCTION__ = "eval_expr" #9 0x000000000040bbf3 in do_action (a=0x746988, msg=0x7711b8) at action.c:677 ret = <value optimized out> v = <value optimized out> to = <value optimized out> p = <value optimized out> tmp = <value optimized out> new_uri = <value optimized out> end = <value optimized out> crt = <value optimized out> len = <value optimized out> user = <value optimized out> uri = {user = {s = 0xa0000001e <Address 0xa0000001e out of bounds>, len = 18}, passwd = {s = 0x6f00000000 <Address 0x6f00000000 out of bounds>, len = 4}, host = {s = 0x1d00000000 <Address 0x1d00000000 out of bounds>, len = 7174}, port = {s = 0x7799d8 "\001", len = 7510263}, params = { s = 0x774150 "�", len = 7505986}, headers = { s = 0x728825 "branch=z9hG4bK558b.c4feef55.0\r\nVia: SIP/2.0/UDP XXX.XX.XXX.13;branch=z9hG4bK558b.b4feef55.0\r\nVia: SIP/2.0/UDP XXX.XX.XXX.13;branch=z9hG4bK558b.a4feef55.0\r\nVia: SIP/2.0/UDP XXX.XX.XXX.13;branch=z9hG4bK"..., len = 7803320}, port_no = 39384, proto = 119, type = ERROR_URI_T, transport = { s = 0x7711b8 "�\211\a", len = 7803320}, ttl = {s = 0x40000000d <Address 0x40000000d out of bounds>, len = 9927568}, user_param = {s = 0x0, len = 7852928}, maddr = {s = 0x1f97 <Address 0x1f97 out of bounds>, len = 7823672}, method = {s = 0x1 <Address 0x1 out of bounds>, len = 7505931}, lr = {s = 0x7298f7 "", len = 7838168}, r2 = {s = 0x490b5c "H\211D$PA\203?�\017\204�\t", len = 7811592}, transport_val = { s = 0x727960 "CANCEL sip:0015554189310@XXXXX.com SIP/2.0\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ftag=as5a292c8e>\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ftag=as5a292c8e>\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ft"..., len = 7803320}, ttl_val = {s = 0x0, len = 7803272}, user_param_val = { s = 0x38404d1600 "H\201��", len = 32}, maddr_val = {s = 0x7fffbf6800d0 "", len = -1083703280}, method_val = { s = 0x727960 "CANCEL sip:0015554189310@XXXXXX.com SIP/2.0\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ftag=as5a292c8e>\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ftag=as5a292c8e>\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ft"..., len = 7505988}, lr_val = {s = 0x7711b8 "�\211\a", len = 5149600}, r2_val = { s = 0x773208 "XXXXX.com", len = 144}} next_hop = {user = {s = 0x2010 <Address 0x2010 out of bounds>, len = 9927568}, passwd = {s = 0x977b80 "", len = 1078411840}, host = { s = 0x3840752980 "", len = 113}, port = {s = 0x7fffbf67fe94 "", len = 21}, params = {s = 0x3000000010 <Address 0x3000000010 out of bounds>, len = -1083703088}, headers = {s = 0x7fffbf680010 "D\210r", len = 9927568}, port_no = 96, proto = 0, type = ERROR_URI_T, transport = { s = 0x3840752980 "", len = 0}, ttl = {s = 0x1 <Address 0x1 out of bounds>, len = 5149128}, user_param = {s = 0x384047625f "H\211�\203=��-", ---Type <return> to continue, or q <return> to quit--- len = 8754528}, maddr = {s = 0x10 <Address 0x10 out of bounds>, len = 5897}, method = {s = 0x2010 <Address 0x2010 out of bounds>, len = 6}, lr = { s = 0x3840752980 "", len = 9926960}, r2 = {s = 0x0, len = 0}, transport_val = {s = 0x3840752980 "", len = 9926960}, ttl_val = { s = 0x15 <Address 0x15 out of bounds>, len = 0}, user_param_val = {s = 0x7fffbf67fff0 " ", len = 5149128}, maddr_val = {s = 0x3840475f2c "\203=5", len = 7}, method_val = {s = 0x15 <Address 0x15 out of bounds>, len = 21}, lr_val = { s = 0x977930 "IP/2.0/UDP 217.76.199.70:5060;branch=z9hG4bK71f1bec6;rport=5060\\\\015\\\\012From: \"1121\" <sip:XXXXXX@XXXXX.com:5060>;tag=as5a292c8e\\\\015\\\\012To: <sip:0015554189310@XXXXXX.com>\\\\015\\\\012Call-ID: 740fd846"..., len = 21}, r2_val = {s = 0x15 <Address 0x15 out of bounds>, len = 9926960}} u = <value optimized out> port = <value optimized out> cmatch = <value optimized out> aitem = <value optimized out> adefault = <value optimized out> spec = <value optimized out> model = <value optimized out> val = {rs = {s = 0xa <Address 0xa out of bounds>, len = 7502176}, ri = 7823672, flags = 0} __FUNCTION__ = "do_action" #10 0x000000000040e7db in run_action_list (a=<value optimized out>, msg=0x7711b8) at action.c:132 ret = -1 t = (struct action *) 0x746988 __FUNCTION__ = "run_action_list" #11 0x000000000040eb38 in run_top_route (a=0x746988, msg=0x7711b8) at action.c:112 bk_action_flags = 0 bk_rec_lev = 0 #12 0x000000000044c95f in receive_msg ( buf=0x727960 "CANCEL sip:0015554189310@XXXXXX.com SIP/2.0\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ftag=as5a292c8e>\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ftag=as5a292c8e>\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ft"..., len=8087, rcv_info=0x7fffbf680170) at receive.c:156 msg = (struct sip_msg *) 0x48 __FUNCTION__ = "receive_msg" #13 0x000000000048959e in udp_rcv_loop () at udp_server.c:438 len = 8087 tmp = <value optimized out> from = <value optimized out> fromlen = 16 ri = {src_ip = {af = 2, len = 4, u = {addrl = {230169793, 4}, addr32 = {230169793, 0, 4, 0}, addr16 = {7361, 3512, 0, 0, 4, 0, 0, 0}, addr = { XXX '�', XX '\034', XXX '�', 13 '\r', 0 '\0', 0 '\0', 0 '\0', 0 '\0', 4 '\004', 0 '\0', 0 '\0', 0 '\0', 0 '\0', 0 '\0', 0 '\0', 0 '\0'}}}, dst_ip = {af = 2, len = 4, u = {addrl = {230169793, 0}, addr32 = {230169793, 0, 0, 0}, addr16 = {7361, 3512, 0, 0, 0, 0, 0, 0}, addr = {XXX '�', XX '\034', XXX '�', 13 '\r', 0 '\0' <repeats 12 times>}}}, src_port = 5060, dst_port = 5060, proto = 1, proto_reserved1 = 0, proto_reserved2 = 0, src_su = {s = {sa_family = 2, sa_data = "\023��\034�\r\000\000\000\000\000\000\000"}, sin = {sin_family = 2, sin_port = 50195, sin_addr = { s_addr = 230169793}, sin_zero = {0 '\0', 0 '\0', 0 '\0', 0 '\0', 0 '\0', 0 '\0', 0 '\0', 0 '\0'}}, sin6 = {sin6_family = 2, sin6_port = 50195, sin6_flowinfo = 230169793, sin6_addr = {in6_u = {u6_addr8 = {0 '\0' <repeats 16 times>}, u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, bind_address = 0x741688} buf = "CANCEL sip:0015554189310@XXXXX.com SIP/2.0\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ftag=as5a292c8e>\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ftag=as5a292c8e>\r\nRecord-Route: <sip:XXX.XX.XXX.13;lr=on;ft"... __FUNCTION__ = "udp_rcv_loop" #14 0x00000000004257f3 in main (argc=11, argv=0x7fffbf680388) at main.c:834 cfg_log_stderr = <value optimized out> cfg_stream = (FILE *) 0x839010 c = <value optimized out> r = <value optimized out> tmp = 0x7fffbf680f16 "" tmp_len = <value optimized out> port = 32767 proto = <value optimized out> ret = <value optimized out> ---Type <return> to continue, or q <return> to quit--- seed = 1908031688 rfd = <value optimized out> __FUNCTION__ = "main"

On 03/01/2011 10:49 AM, Daniel-Constantin Mierla wrote: Daniel, I sent you my config. How can I do it on a hi usage production server for a one night. The lot of fixes for a different buggy customers SIP and NAT devices which is impossible to retest again. I'll try opensips and possible will be back.

On 03/01/2011 11:02 AM, Andrew O. Zhukov wrote: Hello, I had been checking the coredumps you provided for a while. I don't think they are very useful because for me this looks like an Heisenbug. The coredumps only show the result of the memory corruption and not the cause. Daniel has asked you for some input when compiling with memory debug on (to see if canary values where overwritten by what operations). I have not seen this output yet (don't know if you send it privately to him, or I have missed it on the list). Try to compile again with debug memory support, and set logging to a apropriate level. An idea is to set a special server with a special version of Kamailio. Minimize the number of children (use only one worker) and use just part of the traffic so you have a easier debugging. The bug affects private memory so the number of children should not be an impact. Also you might want to increase the PKG_MEM_SIZE from the default value of 4 MB to something bigger (try 10-15 MB or more). See if this has an impact on the bug (it might be caused by fragmentation in this case a bigger pool might help). If all else fail, I strongly suggest dropping pkg_malloc all together, and using libc's Malloc() instead. This is done at compile time by removing the -DPKG_MALLOC from Makefile.defs and recompiling.. This should fix your bug Marius