We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
-Jonathan
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660 #4 0x08094695 in receive_msg ( buf=0x8168220 "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"..., len=1271, rcv_info=0xbf82f844) at receive.c:206 #5 0x080d3e0d in udp_rcv_loop () at udp_server.c:438 #6 0x0806c8f5 in main (argc=3, argv=0xbf82fa34) at main.c:834 (gdb) backtrace full #0 free_to (tb=0x817cf10) at parser/parse_to.c:75 No locals. #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 __FUNCTION__ = "clean_hdr_field" #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 No locals. #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660 No locals. #4 0x08094695 in receive_msg ( buf=0x8168220 "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"..., len=1271, rcv_info=0xbf82f844) at receive.c:206 msg = (struct sip_msg *) 0x8193030 __FUNCTION__ = "receive_msg" #5 0x080d3e0d in udp_rcv_loop () at udp_server.c:438 len = 1271 buf = "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"... tmp = <value optimized out> from = (union sockaddr_union *) 0x8a91318 fromlen = 16 ri = {src_ip = {af = 2, len = 4, u = {addrl = {2850040017, 135426432, 3213031528, 3085483479}, addr32 = {2850040017, 135426432, 3213031528, 3085483479}, addr16 = {10449, 43488, 29056, 2066, 63592, 49026, 48599, 47080}, addr = "Ñ(à©\200q\022\bhø\202¿×½è·"}}, dst_ip = {af = 2, len = 4, u = {addrl = {509300942, 0, 0, 0}, addr32 = {509300942, 0, 0, 0}, addr16 = {20686, 7771, 0, 0, 0, 0, 0, 0}, addr = "ÎP[\036", '\0' <repeats 11 times>}}, src_port = 5060, dst_port = 5060, proto = 1, proto_reserved1 = 0, proto_reserved2 = 0, src_su = {s = {sa_family = 2, sa_data = "\023ÄÑ(à©\000\000\000\000\000\000\000"}, sin = {sin_family = 2, sin_port = 50195, sin_addr = {s_addr = 2850040017}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2, sin6_port = 50195, sin6_flowinfo = 2850040017, sin6_addr = {in6_u = {u6_addr8 = '\0' <repeats 15 times>, u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, bind_address = 0x818dd90} __FUNCTION__ = "udp_rcv_loop" #6 0x0806c8f5 in main (argc=3, argv=0xbf82fa34) at main.c:834 cfg_log_stderr = 0 cfg_stream = (FILE *) 0xa179008 c = <value optimized out> r = <value optimized out> tmp = 0x8148b9c "ÿÿÿÿ" tmp_len = <value optimized out> port = <value optimized out> proto = <value optimized out> ret = <value optimized out> seed = 639236262 rfd = 4 __FUNCTION__ = "main"
The info that you provided is not enough. Please provide the S/W version, platform, etc. 'openser -V' or 'kamailio -V' should provide the compile flags and the S/W version that you are running. We are using carrierroute with 1,000,000 routes and we never had a crash. Your crash doesn't seem to be related to carrierroute.
Regards, Ovidiu Sas
On Fri, Aug 22, 2008 at 6:00 PM, Jonathan K. Creasy JCreasy@voxitas.com wrote:
We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
-Jonathan
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660 #4 0x08094695 in receive_msg ( buf=0x8168220 "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"..., len=1271, rcv_info=0xbf82f844) at receive.c:206 #5 0x080d3e0d in udp_rcv_loop () at udp_server.c:438 #6 0x0806c8f5 in main (argc=3, argv=0xbf82fa34) at main.c:834 (gdb) backtrace full #0 free_to (tb=0x817cf10) at parser/parse_to.c:75 No locals. #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 __FUNCTION__ = "clean_hdr_field" #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 No locals. #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660 No locals. #4 0x08094695 in receive_msg ( buf=0x8168220 "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"..., len=1271, rcv_info=0xbf82f844) at receive.c:206 msg = (struct sip_msg *) 0x8193030 __FUNCTION__ = "receive_msg" #5 0x080d3e0d in udp_rcv_loop () at udp_server.c:438 len = 1271 buf = "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"... tmp = <value optimized out> from = (union sockaddr_union *) 0x8a91318 fromlen = 16 ri = {src_ip = {af = 2, len = 4, u = {addrl = {2850040017, 135426432, 3213031528, 3085483479}, addr32 = {2850040017, 135426432, 3213031528, 3085483479}, addr16 = {10449, 43488, 29056, 2066, 63592, 49026, 48599, 47080}, addr = "Ñ(à(c)\200q\022\bhø\202¿×½è·"}}, dst_ip = {af = 2, len = 4, u = {addrl = {509300942, 0, 0, 0}, addr32 = {509300942, 0, 0, 0}, addr16 = {20686, 7771, 0, 0, 0, 0, 0, 0}, addr = "ÎP[\036", '\0' <repeats 11 times>}}, src_port = 5060, dst_port = 5060, proto = 1, proto_reserved1 = 0, proto_reserved2 = 0, src_su = {s = {sa_family = 2, sa_data = "\023ÄÑ(à(c)\000\000\000\000\000\000\000"}, sin = {sin_family = 2, sin_port = 50195, sin_addr = {s_addr = 2850040017}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2, sin6_port = 50195, sin6_flowinfo = 2850040017, sin6_addr = {in6_u = {u6_addr8 = '\0' <repeats 15 times>, u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, bind_address = 0x818dd90} __FUNCTION__ = "udp_rcv_loop" #6 0x0806c8f5 in main (argc=3, argv=0xbf82fa34) at main.c:834 cfg_log_stderr = 0 cfg_stream = (FILE *) 0xa179008 c = <value optimized out> r = <value optimized out> tmp = 0x8148b9c "ÿÿÿÿ" tmp_len = <value optimized out> port = <value optimized out> proto = <value optimized out> ret = <value optimized out> seed = 639236262 rfd = 4 __FUNCTION__ = "main"
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
Linux 2.6.17.11-netlogic.1.3 #1 SMP Wed Oct 3 14:36:23 CDT 2007 i686 GNU/Linux
version: openser 1.3.2-notls (i386/linux) flags: STATS: Off, USE_IPV6, USE_TCP, DISABLE_NAGLE, USE_MCAST, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535 poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. svnrevision: unknown @(#) $Id: main.c 3590 2008-01-28 17:46:56Z bogdan_iancu $ main.c compiled on 21:56:00 Aug 20 2008 with gcc 4.0.3
Modules:
/* uncomment next line for MySQL DB support */ loadmodule "mysql.so" loadmodule "sl.so" loadmodule "avpops.so" loadmodule "tm.so" loadmodule "rr.so" loadmodule "maxfwd.so" loadmodule "usrloc.so" loadmodule "registrar.so" loadmodule "mangler.so" loadmodule "textops.so" loadmodule "mi_fifo.so" loadmodule "uri_db.so" loadmodule "uri.so" loadmodule "xlog.so" loadmodule "acc.so" loadmodule "carrierroute.so" loadmodule "nathelper.so" loadmodule "permissions.so"
Thanks for your help. Would you be willing to share your configuration with me?
-Jonathan
Jonathan Creasy VOIP Engineer Voxitas.com 314-266-4000 x109
-----Original Message----- From: sip.nslu@gmail.com [mailto:sip.nslu@gmail.com] On Behalf Of Ovidiu Sas Sent: Friday, August 22, 2008 5:16 PM To: Jonathan K. Creasy Cc: users@lists.kamailio.org Subject: Re: [Kamailio-Users] crashing problem
The info that you provided is not enough. Please provide the S/W version, platform, etc. 'openser -V' or 'kamailio -V' should provide the compile flags and the S/W version that you are running. We are using carrierroute with 1,000,000 routes and we never had a crash. Your crash doesn't seem to be related to carrierroute.
Regards, Ovidiu Sas
On Fri, Aug 22, 2008 at 6:00 PM, Jonathan K. Creasy JCreasy@voxitas.com wrote:
We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
-Jonathan
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660 #4 0x08094695 in receive_msg ( buf=0x8168220 "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"..., len=1271, rcv_info=0xbf82f844) at receive.c:206 #5 0x080d3e0d in udp_rcv_loop () at udp_server.c:438 #6 0x0806c8f5 in main (argc=3, argv=0xbf82fa34) at main.c:834 (gdb) backtrace full #0 free_to (tb=0x817cf10) at parser/parse_to.c:75 No locals. #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 __FUNCTION__ = "clean_hdr_field" #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 No locals. #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660 No locals. #4 0x08094695 in receive_msg ( buf=0x8168220 "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"..., len=1271, rcv_info=0xbf82f844) at receive.c:206 msg = (struct sip_msg *) 0x8193030 __FUNCTION__ = "receive_msg" #5 0x080d3e0d in udp_rcv_loop () at udp_server.c:438 len = 1271 buf = "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"... tmp = <value optimized out> from = (union sockaddr_union *) 0x8a91318 fromlen = 16 ri = {src_ip = {af = 2, len = 4, u = {addrl = {2850040017, 135426432, 3213031528, 3085483479}, addr32 = {2850040017, 135426432, 3213031528, 3085483479}, addr16 = {10449, 43488, 29056, 2066, 63592, 49026, 48599, 47080}, addr = "Ñ(à(c)\200q\022\bhø\202¿×½è·"}}, dst_ip = {af = 2, len = 4, u = {addrl = {509300942, 0, 0, 0}, addr32 = {509300942, 0, 0, 0}, addr16 = {20686, 7771, 0, 0, 0, 0, 0, 0}, addr = "ÎP[\036", '\0' <repeats 11 times>}}, src_port = 5060, dst_port = 5060, proto = 1, proto_reserved1 = 0, proto_reserved2 = 0, src_su = {s = {sa_family = 2, sa_data = "\023ÄÑ(à(c)\000\000\000\000\000\000\000"}, sin = {sin_family = 2, sin_port = 50195, sin_addr = {s_addr = 2850040017}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2, sin6_port = 50195, sin6_flowinfo = 2850040017, sin6_addr = {in6_u = {u6_addr8 = '\0' <repeats 15 times>, u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, bind_address = 0x818dd90} __FUNCTION__ = "udp_rcv_loop" #6 0x0806c8f5 in main (argc=3, argv=0xbf82fa34) at main.c:834 cfg_log_stderr = 0 cfg_stream = (FILE *) 0xa179008 c = <value optimized out> r = <value optimized out> tmp = 0x8148b9c "ÿÿÿÿ" tmp_len = <value optimized out> port = <value optimized out> proto = <value optimized out> ret = <value optimized out> seed = 639236262 rfd = 4 __FUNCTION__ = "main"
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
Here's the basic config for carrierroute. You can start building on top of it. It matches the version of openser that you are using. http://voipembedded.com/resources/openser_cr.cfg
Regards, Ovidiu Sas
On Fri, Aug 22, 2008 at 10:57 PM, Jonathan K. Creasy JCreasy@voxitas.com wrote:
Linux 2.6.17.11-netlogic.1.3 #1 SMP Wed Oct 3 14:36:23 CDT 2007 i686 GNU/Linux
version: openser 1.3.2-notls (i386/linux) flags: STATS: Off, USE_IPV6, USE_TCP, DISABLE_NAGLE, USE_MCAST, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535 poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. svnrevision: unknown @(#) $Id: main.c 3590 2008-01-28 17:46:56Z bogdan_iancu $ main.c compiled on 21:56:00 Aug 20 2008 with gcc 4.0.3
Modules:
/* uncomment next line for MySQL DB support */ loadmodule "mysql.so" loadmodule "sl.so" loadmodule "avpops.so" loadmodule "tm.so" loadmodule "rr.so" loadmodule "maxfwd.so" loadmodule "usrloc.so" loadmodule "registrar.so" loadmodule "mangler.so" loadmodule "textops.so" loadmodule "mi_fifo.so" loadmodule "uri_db.so" loadmodule "uri.so" loadmodule "xlog.so" loadmodule "acc.so" loadmodule "carrierroute.so" loadmodule "nathelper.so" loadmodule "permissions.so"
Thanks for your help. Would you be willing to share your configuration with me?
-Jonathan
Jonathan Creasy VOIP Engineer Voxitas.com 314-266-4000 x109
-----Original Message----- From: sip.nslu@gmail.com [mailto:sip.nslu@gmail.com] On Behalf Of Ovidiu Sas Sent: Friday, August 22, 2008 5:16 PM To: Jonathan K. Creasy Cc: users@lists.kamailio.org Subject: Re: [Kamailio-Users] crashing problem
The info that you provided is not enough. Please provide the S/W version, platform, etc. 'openser -V' or 'kamailio -V' should provide the compile flags and the S/W version that you are running. We are using carrierroute with 1,000,000 routes and we never had a crash. Your crash doesn't seem to be related to carrierroute.
Regards, Ovidiu Sas
On Fri, Aug 22, 2008 at 6:00 PM, Jonathan K. Creasy JCreasy@voxitas.com wrote:
We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
-Jonathan
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660 #4 0x08094695 in receive_msg ( buf=0x8168220 "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"..., len=1271, rcv_info=0xbf82f844) at receive.c:206 #5 0x080d3e0d in udp_rcv_loop () at udp_server.c:438 #6 0x0806c8f5 in main (argc=3, argv=0xbf82fa34) at main.c:834 (gdb) backtrace full #0 free_to (tb=0x817cf10) at parser/parse_to.c:75 No locals. #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 __FUNCTION__ = "clean_hdr_field" #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 No locals. #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660 No locals. #4 0x08094695 in receive_msg ( buf=0x8168220 "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"..., len=1271, rcv_info=0xbf82f844) at receive.c:206 msg = (struct sip_msg *) 0x8193030 __FUNCTION__ = "receive_msg" #5 0x080d3e0d in udp_rcv_loop () at udp_server.c:438 len = 1271 buf = "SIP/2.0 183 Session Progress\r\nf: "16512552900"sip:16512552900@206.80.65.90;tag=as781b438d\r\nt: sip:+16513401989@lefty.netlogic.net;tag=c7ad5f5020080822192510\r\ni: 396fb801653abd6f605a12153e0e3e72@20"... tmp = <value optimized out> from = (union sockaddr_union *) 0x8a91318 fromlen = 16 ri = {src_ip = {af = 2, len = 4, u = {addrl = {2850040017, 135426432, 3213031528, 3085483479}, addr32 = {2850040017, 135426432, 3213031528, 3085483479}, addr16 = {10449, 43488, 29056, 2066, 63592, 49026, 48599, 47080}, addr = "Ñ(à(c)\200q\022\bhø\202¿×½è·"}}, dst_ip = {af = 2, len = 4, u = {addrl = {509300942, 0, 0, 0}, addr32 = {509300942, 0, 0, 0}, addr16 = {20686, 7771, 0, 0, 0, 0, 0, 0}, addr = "ÎP[\036", '\0' <repeats 11 times>}}, src_port = 5060, dst_port = 5060, proto = 1, proto_reserved1 = 0, proto_reserved2 = 0, src_su = {s = {sa_family = 2, sa_data = "\023ÄÑ(à(c)\000\000\000\000\000\000\000"}, sin = {sin_family = 2, sin_port = 50195, sin_addr = {s_addr = 2850040017}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2, sin6_port = 50195, sin6_flowinfo = 2850040017, sin6_addr = {in6_u = {u6_addr8 = '\0' <repeats 15 times>, u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, bind_address = 0x818dd90} __FUNCTION__ = "udp_rcv_loop" #6 0x0806c8f5 in main (argc=3, argv=0xbf82fa34) at main.c:834 cfg_log_stderr = 0 cfg_stream = (FILE *) 0xa179008 c = <value optimized out> r = <value optimized out> tmp = 0x8148b9c "ÿÿÿÿ" tmp_len = <value optimized out> port = <value optimized out> proto = <value optimized out> ret = <value optimized out> seed = 639236262 rfd = 4 __FUNCTION__ = "main"
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
On Saturday 23 August 2008, Jonathan K. Creasy wrote:
We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660
Hi Jonathan,
this seems not related to carrierroute, as you can see from the backtrace the crash is in the parser code. Would be interesting if this is related to some invalid message, or an unrelated error. Have you tried to replay the message in your lab that causes the crash, after isolating it from the backtrace? You should also take a look into the "tp" pointer with the debugger, try to investigate why it has an apparently invalid value here.
Cheers,
Henning
Hello,
as it seems to happen in a place that look safe from this point of view, might be the case of memory corruption (overwrite, double free, ...).
Needs to compile in memory debug mode, guidelines at: http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory
When run in debug mode, a double free or overwrite is reported in the log.
Cheers, Daniel
On 08/25/08 11:42, Henning Westerholt wrote:
On Saturday 23 August 2008, Jonathan K. Creasy wrote:
We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660
Hi Jonathan,
this seems not related to carrierroute, as you can see from the backtrace the crash is in the parser code. Would be interesting if this is related to some invalid message, or an unrelated error. Have you tried to replay the message in your lab that causes the crash, after isolating it from the backtrace? You should also take a look into the "tp" pointer with the debugger, try to investigate why it has an apparently invalid value here.
Cheers,
Henning
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
Thanks for the assistance. So far I have not been able to re-create the problem in the lab but have had the crash with identical back traces (same location) on 7 different border servers.
More RAM and less children seems to have led to stability but I will still be trying to recreate the problem and fix the root cause.
-Jonathan
-----Original Message----- From: Daniel-Constantin Mierla [mailto:miconda@gmail.com] Sent: Monday, August 25, 2008 3:56 AM To: Jonathan K. Creasy Cc: Henning Westerholt; users@lists.kamailio.org Subject: Re: [Kamailio-Users] crashing problem
Hello,
as it seems to happen in a place that look safe from this point of view, might be the case of memory corruption (overwrite, double free, ...).
Needs to compile in memory debug mode, guidelines at: http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory
When run in debug mode, a double free or overwrite is reported in the log.
Cheers, Daniel
On 08/25/08 11:42, Henning Westerholt wrote:
On Saturday 23 August 2008, Jonathan K. Creasy wrote:
We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660
Hi Jonathan,
this seems not related to carrierroute, as you can see from the backtrace the crash is in the parser code. Would be interesting if this is related to some invalid message, or an unrelated error. Have you tried to replay the message in your lab that causes the crash, after isolating it from the backtrace? You should also take a look into the "tp" pointer with the debugger, try to investigate why it has an apparently invalid value here.
Cheers,
Henning
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
-- Daniel-Constantin Mierla http://www.asipto.com
How many calls per second are you routing? Are you doing something specific beside just routing calls via carrierroute? Are you running any private modules?
Regards, Ovidiu Sas
On Mon, Aug 25, 2008 at 10:55 AM, Jonathan K. Creasy JCreasy@voxitas.com wrote:
Thanks for the assistance. So far I have not been able to re-create the problem in the lab but have had the crash with identical back traces (same location) on 7 different border servers.
More RAM and less children seems to have led to stability but I will still be trying to recreate the problem and fix the root cause.
-Jonathan
-----Original Message----- From: Daniel-Constantin Mierla [mailto:miconda@gmail.com] Sent: Monday, August 25, 2008 3:56 AM To: Jonathan K. Creasy Cc: Henning Westerholt; users@lists.kamailio.org Subject: Re: [Kamailio-Users] crashing problem
Hello,
as it seems to happen in a place that look safe from this point of view, might be the case of memory corruption (overwrite, double free, ...).
Needs to compile in memory debug mode, guidelines at: http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory
When run in debug mode, a double free or overwrite is reported in the log.
Cheers, Daniel
On 08/25/08 11:42, Henning Westerholt wrote:
On Saturday 23 August 2008, Jonathan K. Creasy wrote:
We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660
Hi Jonathan,
this seems not related to carrierroute, as you can see from the backtrace the crash is in the parser code. Would be interesting if this is related to some invalid message, or an unrelated error. Have you tried to replay the message in your lab that causes the crash, after isolating it from the backtrace? You should also take a look into the "tp" pointer with the debugger, try to investigate why it has an apparently invalid value here.
Cheers,
Henning
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
-- Daniel-Constantin Mierla http://www.asipto.com
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
5-15 cps throughout the day, peaks around 60-70, crashes at non-peak times
Nope, our config is pretty vanilla
No private modules
-----Original Message----- From: sip.nslu@gmail.com [mailto:sip.nslu@gmail.com] On Behalf Of Ovidiu Sas Sent: Monday, August 25, 2008 10:23 AM To: Jonathan K. Creasy Cc: miconda@gmail.com; users@lists.kamailio.org Subject: Re: [Kamailio-Users] crashing problem
How many calls per second are you routing? Are you doing something specific beside just routing calls via carrierroute? Are you running any private modules?
Regards, Ovidiu Sas
On Mon, Aug 25, 2008 at 10:55 AM, Jonathan K. Creasy JCreasy@voxitas.com wrote:
Thanks for the assistance. So far I have not been able to re-create the problem in the lab but have had the crash with identical back traces (same location) on 7 different border servers.
More RAM and less children seems to have led to stability but I will still be trying to recreate the problem and fix the root cause.
-Jonathan
-----Original Message----- From: Daniel-Constantin Mierla [mailto:miconda@gmail.com] Sent: Monday, August 25, 2008 3:56 AM To: Jonathan K. Creasy Cc: Henning Westerholt; users@lists.kamailio.org Subject: Re: [Kamailio-Users] crashing problem
Hello,
as it seems to happen in a place that look safe from this point of view, might be the case of memory corruption (overwrite, double free, ...).
Needs to compile in memory debug mode, guidelines at: http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory
When run in debug mode, a double free or overwrite is reported in the log.
Cheers, Daniel
On 08/25/08 11:42, Henning Westerholt wrote:
On Saturday 23 August 2008, Jonathan K. Creasy wrote:
We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660
Hi Jonathan,
this seems not related to carrierroute, as you can see from the backtrace the crash is in the parser code. Would be interesting if this is related to some invalid message, or an unrelated error. Have you tried to replay the message in your lab that causes the crash, after isolating it from the backtrace? You should also take a look into the "tp" pointer with the debugger, try to investigate why it has an apparently invalid value here.
Cheers,
Henning
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
-- Daniel-Constantin Mierla http://www.asipto.com
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
Hmm ... Can you post your config file here (or can you send it to me privately)? I will take a look to see if I can found something there.
Regards, Ovidiu Sas
On Mon, Aug 25, 2008 at 11:50 AM, Jonathan K. Creasy JCreasy@voxitas.com wrote:
5-15 cps throughout the day, peaks around 60-70, crashes at non-peak times
Nope, our config is pretty vanilla
No private modules
-----Original Message----- From: sip.nslu@gmail.com [mailto:sip.nslu@gmail.com] On Behalf Of Ovidiu Sas Sent: Monday, August 25, 2008 10:23 AM To: Jonathan K. Creasy Cc: miconda@gmail.com; users@lists.kamailio.org Subject: Re: [Kamailio-Users] crashing problem
How many calls per second are you routing? Are you doing something specific beside just routing calls via carrierroute? Are you running any private modules?
Regards, Ovidiu Sas
On Mon, Aug 25, 2008 at 10:55 AM, Jonathan K. Creasy JCreasy@voxitas.com wrote:
Thanks for the assistance. So far I have not been able to re-create the problem in the lab but have had the crash with identical back traces (same location) on 7 different border servers.
More RAM and less children seems to have led to stability but I will still be trying to recreate the problem and fix the root cause.
-Jonathan
-----Original Message----- From: Daniel-Constantin Mierla [mailto:miconda@gmail.com] Sent: Monday, August 25, 2008 3:56 AM To: Jonathan K. Creasy Cc: Henning Westerholt; users@lists.kamailio.org Subject: Re: [Kamailio-Users] crashing problem
Hello,
as it seems to happen in a place that look safe from this point of view, might be the case of memory corruption (overwrite, double free, ...).
Needs to compile in memory debug mode, guidelines at: http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory
When run in debug mode, a double free or overwrite is reported in the log.
Cheers, Daniel
On 08/25/08 11:42, Henning Westerholt wrote:
On Saturday 23 August 2008, Jonathan K. Creasy wrote:
We're using Carrier route with about 53,000 routes stored in a mysql databse. We're getting this crash in production that we never had in the lab.
The bt is below. Does anyone have any suggestions?
nline static void free_to_params(struct to_body* tb) { struct to_param *tp=tb->param_lst; struct to_param *foo; while (tp){ foo = tp->next; <-- **** Line 75 ***** pkg_free(tp); tp=foo; } }
#0 free_to (tb=0x817cf10) at parser/parse_to.c:75 #1 0x080daf70 in clean_hdr_field (hf=0x8a8ee18) at parser/hf.c:182 #2 0x080db05b in free_hdr_field_lst (hf=0x8a905b8) at parser/hf.c:209 #3 0x080db12d in free_sip_msg (msg=0x8193030) at parser/msg_parser.c:660
Hi Jonathan,
this seems not related to carrierroute, as you can see from the backtrace the crash is in the parser code. Would be interesting if this is related to some invalid message, or an unrelated error. Have you tried to replay the message in your lab that causes the crash, after isolating it from the backtrace? You should also take a look into the "tp" pointer with the debugger, try to investigate why it has an apparently invalid value here.
Cheers,
Henning
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
-- Daniel-Constantin Mierla http://www.asipto.com
Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
-
Daniel-Constantin Mierla -
Henning Westerholt -
Jonathan K. Creasy -
Ovidiu Sas