Hey guys,
I'm using kamailio-1.4.3-notls with ACC module on mysql.
I'm trying to benchmark it installed on a virtual server (Xen on CentOS) and I have everything going OK for my basic configuration but when I turn on ACC (by flagging transactions) I get the following lines on the log:
Feb 19 08:48:57 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: child process 4899 exited by a signal 11 Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: core was generated Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: terminating due to SIGCHLD Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4902]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4903]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4901]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4897]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4904]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4898]: INFO:core:sig_usr: signal 15 received
This issue only happens when i account messages using the following lines: if (is_method("INVITE|BYE|CANCEL")) { setflag(1); setflag(2); }
and the configuration for the modules are ass follows: # ----- acc params ----- /* what sepcial events should be accounted ? */ modparam("acc", "early_media", 1) modparam("acc", "report_ack", 1) modparam("acc", "report_cancels", 1) /* by default ww do not adjust the direct of the sequential requests. if you enable this parameter, be sure the enable "append_fromtag" in "rr" module */ modparam("acc", "detect_direction", 0) /* account triggers (flags) */ modparam("acc", "failed_transaction_flag", 3) modparam("acc", "log_flag", 1) modparam("acc", "log_missed_flag", 2) /* uncomment the following lines to enable DB accounting also */ modparam("acc", "db_flag", 1) modparam("acc", "db_missed_flag", 2) # Descomentar para guardar parametros adicionales # modparam("acc", "log_extra", "src_user=$fU;src_domain=$fd;dst_user=$rU;dst_domain=$rd") modparam("acc", "db_table_acc", "acc") modparam("acc", "db_table_missed_calls", "missed_calls") modparam("acc", "db_url", "mysql://kamailio:kamailiorw@localhost/kamailio") modparam("acc", "acc_method_column", "method") modparam("acc", "acc_from_tag_column", "from_tag") modparam("acc", "acc_to_tag_column", "to_tag") modparam("acc", "acc_callid_column", "callid") modparam("acc", "acc_sip_code_column", "sip_code") modparam("acc", "acc_sip_reason_column", "sip_reason") modparam("acc", "acc_time_column", "time") modparam("acc", "db_extra", "cseq=$cs;privacy=$dip;reason=$dir;from_uri=$fu;from_username=$fU;from_displayname=$fn;from_domain=$fd;req_username=$rU;req_domain=$rd;rpid=$re;source_ip_addr=$si;source_port=$sp;to_uri=$tu;to_username=$tU;to_displayname=$tn;to_domain=$td;original_uri=$ou;original_uri_username=$oU;original_uri_domain=$od;unix_timestamp=$Ts")
Do you think there is something I'm missing on the configuration? I tried to modify parameters on mysql but get the same errors all the time?
Thanks in advance! Uriel
An update, I removed the rpid=$re from db_extra parameter and now calls are flowing fine. Can that be the whole issue?
On Thu, Feb 19, 2009 at 3:09 PM, Uriel Rozenbaum uriel.rozenbaum@gmail.comwrote:
Hey guys,
I'm using kamailio-1.4.3-notls with ACC module on mysql.
I'm trying to benchmark it installed on a virtual server (Xen on CentOS) and I have everything going OK for my basic configuration but when I turn on ACC (by flagging transactions) I get the following lines on the log:
Feb 19 08:48:57 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: child process 4899 exited by a signal 11 Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: core was generated Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: terminating due to SIGCHLD Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4902]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4903]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4901]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4897]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4904]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4898]: INFO:core:sig_usr: signal 15 received
This issue only happens when i account messages using the following lines: if (is_method("INVITE|BYE|CANCEL")) { setflag(1); setflag(2); }
and the configuration for the modules are ass follows: # ----- acc params ----- /* what sepcial events should be accounted ? */ modparam("acc", "early_media", 1) modparam("acc", "report_ack", 1) modparam("acc", "report_cancels", 1) /* by default ww do not adjust the direct of the sequential requests. if you enable this parameter, be sure the enable "append_fromtag" in "rr" module */ modparam("acc", "detect_direction", 0) /* account triggers (flags) */ modparam("acc", "failed_transaction_flag", 3) modparam("acc", "log_flag", 1) modparam("acc", "log_missed_flag", 2) /* uncomment the following lines to enable DB accounting also */ modparam("acc", "db_flag", 1) modparam("acc", "db_missed_flag", 2) # Descomentar para guardar parametros adicionales # modparam("acc", "log_extra", "src_user=$fU;src_domain=$fd;dst_user=$rU;dst_domain=$rd") modparam("acc", "db_table_acc", "acc") modparam("acc", "db_table_missed_calls", "missed_calls") modparam("acc", "db_url", "mysql://kamailio:kamailiorw@localhost/kamailio") modparam("acc", "acc_method_column", "method") modparam("acc", "acc_from_tag_column", "from_tag") modparam("acc", "acc_to_tag_column", "to_tag") modparam("acc", "acc_callid_column", "callid") modparam("acc", "acc_sip_code_column", "sip_code") modparam("acc", "acc_sip_reason_column", "sip_reason") modparam("acc", "acc_time_column", "time") modparam("acc", "db_extra",
"cseq=$cs;privacy=$dip;reason=$dir;from_uri=$fu;from_username=$fU;from_displayname=$fn;from_domain=$fd;req_username=$rU;req_domain=$rd;rpid=$re;source_ip_addr=$si;source_port=$sp;to_uri=$tu;to_username=$tU;to_displayname=$tn;to_domain=$td;original_uri=$ou;original_uri_username=$oU;original_uri_domain=$od;unix_timestamp=$Ts")
Do you think there is something I'm missing on the configuration? I tried to modify parameters on mysql but get the same errors all the time?
Thanks in advance! Uriel
On 02/19/2009 10:01 PM, Uriel Rozenbaum wrote:
An update, I removed the rpid=$re from db_extra parameter and now calls are flowing fine. Can that be the whole issue?
There were some similar issues in the past, but they should have been fixed long time ago. I see you got a core dump, can you send the backtrace, it will help to troubleshoot.
Thanks, Daniel
On Thu, Feb 19, 2009 at 3:09 PM, Uriel Rozenbaum <uriel.rozenbaum@gmail.com mailto:uriel.rozenbaum@gmail.com> wrote:
Hey guys, I'm using kamailio-1.4.3-notls with ACC module on mysql. I'm trying to benchmark it installed on a virtual server (Xen on CentOS) and I have everything going OK for my basic configuration but when I turn on ACC (by flagging transactions) I get the following lines on the log: Feb 19 08:48:57 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: child process 4899 exited by a signal 11 Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: core was generated Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: terminating due to SIGCHLD Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4902]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4903]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4901]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4897]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4904]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4898]: INFO:core:sig_usr: signal 15 received This issue only happens when i account messages using the following lines: if (is_method("INVITE|BYE|CANCEL")) { setflag(1); setflag(2); } and the configuration for the modules are ass follows: # ----- acc params ----- /* what sepcial events should be accounted ? */ modparam("acc", "early_media", 1) modparam("acc", "report_ack", 1) modparam("acc", "report_cancels", 1) /* by default ww do not adjust the direct of the sequential requests. if you enable this parameter, be sure the enable "append_fromtag" in "rr" module */ modparam("acc", "detect_direction", 0) /* account triggers (flags) */ modparam("acc", "failed_transaction_flag", 3) modparam("acc", "log_flag", 1) modparam("acc", "log_missed_flag", 2) /* uncomment the following lines to enable DB accounting also */ modparam("acc", "db_flag", 1) modparam("acc", "db_missed_flag", 2) # Descomentar para guardar parametros adicionales # modparam("acc", "log_extra", "src_user=$fU;src_domain=$fd;dst_user=$rU;dst_domain=$rd") modparam("acc", "db_table_acc", "acc") modparam("acc", "db_table_missed_calls", "missed_calls") modparam("acc", "db_url", "mysql://kamailio:kamailiorw@localhost/kamailio") modparam("acc", "acc_method_column", "method") modparam("acc", "acc_from_tag_column", "from_tag") modparam("acc", "acc_to_tag_column", "to_tag") modparam("acc", "acc_callid_column", "callid") modparam("acc", "acc_sip_code_column", "sip_code") modparam("acc", "acc_sip_reason_column", "sip_reason") modparam("acc", "acc_time_column", "time") modparam("acc", "db_extra", "cseq=$cs;privacy=$dip;reason=$dir;from_uri=$fu;from_username=$fU;from_displayname=$fn;from_domain=$fd;req_username=$rU;req_domain=$rd;rpid=$re;source_ip_addr=$si;source_port=$sp;to_uri=$tu;to_username=$tU;to_displayname=$tn;to_domain=$td;original_uri=$ou;original_uri_username=$oU;original_uri_domain=$od;unix_timestamp=$Ts") Do you think there is something I'm missing on the configuration? I tried to modify parameters on mysql but get the same errors all the time? Thanks in advance! Uriel
Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
Sure, but I'll need some guidance on obtaining it; I'm just checking with the syslog messages with debugging level in 3. Can you show me how to do it?
Thanks, Uriel
On Thu, Feb 19, 2009 at 6:38 PM, Daniel-Constantin Mierla <miconda@gmail.com
wrote:
On 02/19/2009 10:01 PM, Uriel Rozenbaum wrote:
An update, I removed the rpid=$re from db_extra parameter and now calls are flowing fine. Can that be the whole issue?
There were some similar issues in the past, but they should have been fixed long time ago. I see you got a core dump, can you send the backtrace, it will help to troubleshoot.
Thanks, Daniel
On Thu, Feb 19, 2009 at 3:09 PM, Uriel Rozenbaum < uriel.rozenbaum@gmail.com mailto:uriel.rozenbaum@gmail.com> wrote:
Hey guys,
I'm using kamailio-1.4.3-notls with ACC module on mysql.
I'm trying to benchmark it installed on a virtual server (Xen on CentOS) and I have everything going OK for my basic configuration but when I turn on ACC (by flagging transactions) I get the following lines on the log:
Feb 19 08:48:57 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: child process 4899 exited by a signal 11 Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: core was generated Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: terminating due to SIGCHLD Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4902]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4903]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4901]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4897]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4904]: INFO:core:sig_usr: signal 15 received Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4898]: INFO:core:sig_usr: signal 15 received
This issue only happens when i account messages using the following lines: if (is_method("INVITE|BYE|CANCEL")) { setflag(1); setflag(2); }
and the configuration for the modules are ass follows: # ----- acc params ----- /* what sepcial events should be accounted ? */ modparam("acc", "early_media", 1) modparam("acc", "report_ack", 1) modparam("acc", "report_cancels", 1) /* by default ww do not adjust the direct of the sequential requests. if you enable this parameter, be sure the enable "append_fromtag" in "rr" module */ modparam("acc", "detect_direction", 0) /* account triggers (flags) */ modparam("acc", "failed_transaction_flag", 3) modparam("acc", "log_flag", 1) modparam("acc", "log_missed_flag", 2) /* uncomment the following lines to enable DB accounting also */ modparam("acc", "db_flag", 1) modparam("acc", "db_missed_flag", 2) # Descomentar para guardar parametros adicionales # modparam("acc", "log_extra", "src_user=$fU;src_domain=$fd;dst_user=$rU;dst_domain=$rd") modparam("acc", "db_table_acc", "acc") modparam("acc", "db_table_missed_calls", "missed_calls") modparam("acc", "db_url", "mysql://kamailio:kamailiorw@localhost/kamailio") modparam("acc", "acc_method_column", "method") modparam("acc", "acc_from_tag_column", "from_tag") modparam("acc", "acc_to_tag_column", "to_tag") modparam("acc", "acc_callid_column", "callid") modparam("acc", "acc_sip_code_column", "sip_code") modparam("acc", "acc_sip_reason_column", "sip_reason") modparam("acc", "acc_time_column", "time") modparam("acc", "db_extra",
"cseq=$cs;privacy=$dip;reason=$dir;from_uri=$fu;from_username=$fU;from_displayname=$fn;from_domain=$fd;req_username=$rU;req_domain=$rd;rpid=$re;source_ip_addr=$si;source_port=$sp;to_uri=$tu;to_username=$tU;to_displayname=$tn;to_domain=$td;original_uri=$ou;original_uri_username=$oU;original_uri_domain=$od;unix_timestamp=$Ts")
Do you think there is something I'm missing on the configuration? I tried to modify parameters on mysql but get the same errors all the time?
Thanks in advance! Uriel
Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
-- Daniel-Constantin Mierla http://www.asipto.com
On Thursday 19 February 2009, Uriel Rozenbaum wrote:
Sure, but I'll need some guidance on obtaining it; I'm just checking with the syslog messages with debugging level in 3. Can you show me how to do it?
Hi Uriel,
the core was already generated, as you can see in the logs:
Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: core was generated
You should find this file in the working directory of the server.
Cheers,
Henning
Hi Henning, here's the core file zipped.
On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt < henning.westerholt@1und1.de> wrote:
On Thursday 19 February 2009, Uriel Rozenbaum wrote:
Sure, but I'll need some guidance on obtaining it; I'm just checking with the syslog messages with debugging level in 3. Can you show me how to do it?
Hi Uriel,
the core was already generated, as you can see in the logs:
Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: core was generated
You should find this file in the working directory of the server.
Cheers,
Henning
First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use:
gdb /usr/local/sbin/kamailio /path/to/corefile
The do "bt" there and send the output here.
Thanks, Daniel
On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote:
Hi Henning, here's the core file zipped.
On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de mailto:henning.westerholt@1und1.de> wrote:
On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning
Here it is:
[root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".
warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from /usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from /usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63sip%3A8888860911167@192.168.200.63SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: sip:192.168.200.9;lr=on\r\nFrom: "7001" <sip:60911100@ops3.cpsnet.com.arsip%3A60911100@ops3.cpsnet.com.ar>;tag=as"..., len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780
On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla <miconda@gmail.com
wrote:
First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use:
gdb /usr/local/sbin/kamailio /path/to/corefile
The do "bt" there and send the output here.
Thanks, Daniel
On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote:
Hi Henning, here's the core file zipped.
On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt < henning.westerholt@1und1.de mailto:henning.westerholt@1und1.de> wrote:
On Thursday 19 February 2009, Uriel Rozenbaum wrote:
Sure, but I'll need some guidance on obtaining it; I'm just
checking with
the syslog messages with debugging level in 3. Can you show me
how to do
it?
Hi Uriel,
the core was already generated, as you can see in the logs:
Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: INFO:core:handle_sigs: core was generated
You should find this file in the working directory of the server.
Cheers,
Henning
-- Daniel-Constantin Mierla http://www.asipto.com
Hello,
thanks. Do you have report ack set for acc module?
The crash happens to an ACK and relates to building outgoing request and processing lumps. I will investigate and try to reproduce. Just asking for now, would be any chance to get the ACK and eventually your config in case it is needed for troubleshooting?
Thanks, Daniel
On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote:
Here it is:
[root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".
warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from /usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from /usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63 mailto:sip%3A8888860911167@192.168.200.63 SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: sip:192.168.200.9;lr=on\r\nFrom: "7001" <sip:60911100@ops3.cpsnet.com.ar mailto:sip%3A60911100@ops3.cpsnet.com.ar>;tag=as"..., len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780
On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:
First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use: gdb /usr/local/sbin/kamailio /path/to/corefile The do "bt" there and send the output here. Thanks, Daniel On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote: Hi Henning, here's the core file zipped. On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>> wrote: On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning -- Daniel-Constantin Mierla http://www.asipto.com
Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
Sure, I can turn of report ack to check; anyway i substracter $re from db_extra and now I'm not seeing any crashes. But maybe its useful for you to clear the bug in there.
Let me know how you want to proceed and I'll send you everithing.
-- Uriel
On Mon, Feb 23, 2009 at 10:18 AM, Daniel-Constantin Mierla < miconda@gmail.com> wrote:
Hello,
thanks. Do you have report ack set for acc module?
The crash happens to an ACK and relates to building outgoing request and processing lumps. I will investigate and try to reproduce. Just asking for now, would be any chance to get the ACK and eventually your config in case it is needed for troubleshooting?
Thanks, Daniel
On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote:
Here it is:
[root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".
warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from /usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from /usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63sip%3A8888860911167@192.168.200.63<mailto: sip%3A8888860911167@192.168.200.63 sip%253A8888860911167@192.168.200.63> SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: sip:192.168.200.9;lr=on\r\nFrom: "7001" < sip:60911100@ops3.cpsnet.com.ar sip%3A60911100@ops3.cpsnet.com.ar<mailto: sip%3A60911100@ops3.cpsnet.com.ar sip%253A60911100@ops3.cpsnet.com.ar>>;tag=as"..., len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780
On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla < miconda@gmail.com mailto:miconda@gmail.com> wrote:
First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use:
gdb /usr/local/sbin/kamailio /path/to/corefile
The do "bt" there and send the output here.
Thanks, Daniel
On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote:
Hi Henning, here's the core file zipped. On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>> wrote: On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning-- Daniel-Constantin Mierla http://www.asipto.com
Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
-- Daniel-Constantin Mierla http://www.asipto.com
On 02/23/2009 02:25 PM, Uriel Rozenbaum wrote:
Sure, I can turn of report ack to check; anyway i substracter $re from db_extra and now I'm not seeing any crashes. But maybe its useful for you to clear the bug in there.
yes, this is the goal. This issue has to be fixed. You can keep the report ack on, I just wanted to get your confirmation.
I will try to reproduce first by myself, and then ask you for more. Just in case you have such ACK at your hand, send it to me, I will use it in tests. Does it have a RPID header?
Thanks, Daniel
Let me know how you want to proceed and I'll send you everithing.
-- Uriel
On Mon, Feb 23, 2009 at 10:18 AM, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:
Hello, thanks. Do you have report ack set for acc module? The crash happens to an ACK and relates to building outgoing request and processing lumps. I will investigate and try to reproduce. Just asking for now, would be any chance to get the ACK and eventually your config in case it is needed for troubleshooting? Thanks, Daniel On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote: Here it is: [root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1". warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from /usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from /usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63 <mailto:sip%3A8888860911167@192.168.200.63> <mailto:sip%3A8888860911167@192.168.200.63 <mailto:sip%253A8888860911167@192.168.200.63>> SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: <sip:192.168.200.9;lr=on>\r\nFrom: \"7001\" <sip:60911100@ops3.cpsnet.com.ar <mailto:sip%3A60911100@ops3.cpsnet.com.ar> <mailto:sip%3A60911100@ops3.cpsnet.com.ar <mailto:sip%253A60911100@ops3.cpsnet.com.ar>>>;tag=as"..., len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780 On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>> wrote: First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use: gdb /usr/local/sbin/kamailio /path/to/corefile The do "bt" there and send the output here. Thanks, Daniel On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote: Hi Henning, here's the core file zipped. On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>>> wrote: On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning -- Daniel-Constantin Mierla http://www.asipto.com ------------------------------------------------------------------------ _______________________________________________ Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org <mailto:Users@lists.kamailio.org> http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users -- Daniel-Constantin Mierla http://www.asipto.com
I didn't really capture the signaling for the issue, I'll get a trace when I get the chance and send it to you.
Rgds, Uriel
On Mon, Feb 23, 2009 at 12:02 PM, Daniel-Constantin Mierla < miconda@gmail.com> wrote:
On 02/23/2009 02:25 PM, Uriel Rozenbaum wrote:
Sure, I can turn of report ack to check; anyway i substracter $re from db_extra and now I'm not seeing any crashes. But maybe its useful for you to clear the bug in there.
yes, this is the goal. This issue has to be fixed. You can keep the report ack on, I just wanted to get your confirmation.
I will try to reproduce first by myself, and then ask you for more. Just in case you have such ACK at your hand, send it to me, I will use it in tests. Does it have a RPID header?
Thanks, Daniel
Let me know how you want to proceed and I'll send you everithing.
-- Uriel
On Mon, Feb 23, 2009 at 10:18 AM, Daniel-Constantin Mierla < miconda@gmail.com mailto:miconda@gmail.com> wrote:
Hello,
thanks. Do you have report ack set for acc module?
The crash happens to an ACK and relates to building outgoing request and processing lumps. I will investigate and try to reproduce. Just asking for now, would be any chance to get the ACK and eventually your config in case it is needed for troubleshooting?
Thanks, Daniel
On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote:
Here it is: [root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1". warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from /usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from /usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63<sip%3A8888860911167@192.168.200.63> <mailto:sip%3A8888860911167@192.168.200.63<sip%253A8888860911167@192.168.200.63><mailto:sip%3A8888860911167@192.168.200.63<sip%253A8888860911167@192.168.200.63> <mailto:sip%253A8888860911167@192.168.200.63<sip%25253A8888860911167@192.168.200.63>>>SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: sip:192.168.200.9;lr=on\r\nFrom: "7001" <sip:60911100@ops3.cpsnet.com.arsip%3A60911100@ops3.cpsnet.com.ar <mailto:sip%3A60911100@ops3.cpsnet.com.arsip%253A60911100@ops3.cpsnet.com.ar
<mailto:sip%3A60911100@ops3.cpsnet.com.ar<sip%253A60911100@ops3.cpsnet.com.ar> <mailto:sip%253A60911100@ops3.cpsnet.com.ar<sip%25253A60911100@ops3.cpsnet.com.ar>;tag=as"...,
len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780 On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>> wrote: First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use: gdb /usr/local/sbin/kamailio /path/to/corefile The do "bt" there and send the output here. Thanks, Daniel On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote: Hi Henning, here's the core file zipped. On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>>> wrote: On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning -- Daniel-Constantin Mierla http://www.asipto.com
_______________________________________________ Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org <mailto:Users@lists.kamailio.org> http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users-- Daniel-Constantin Mierla http://www.asipto.com
-- Daniel-Constantin Mierla http://www.asipto.com
Daniel, I've been trying a litte longer with the server and I captured one ACK that definetely has RPID:
*ACK sip:1054111556446967@cc1int.x.com.arsip%3A1054111556446967@cc1int.x.com.arSIP/2.0 Via: SIP/2.0/UDP 192.168.200.11:5060;branch=z9hG4bK31a69f48;rport From: "541160911100" <sip:541160911100@192.168.200.11sip%3A541160911100@192.168.200.11
;tag=as799b3334
To: <sip:1054111556446967@cc1int.cpsnet.com.arsip%3A1054111556446967@cc1int.cpsnet.com.ar
;tag=f8f2ab2c1295e90ed7dbb499b30f44b2.90f4
Contact: <sip:541160911100@192.168.200.11sip%3A541160911100@192.168.200.11
Call-ID: 5d7adcce1399bafd45c194b049479a1b@192.168.200.11 CSeq: 102 ACK User-Agent: Metrotel Max-Forwards: 70 Remote-Party-ID: "541160911100" <sip:541160911100@192.168.200.11sip%3A541160911100@192.168.200.11
;privacy=off;screen=no
Content-Length: 0*
Hope it helps, Uriel
On Mon, Feb 23, 2009 at 12:08 PM, Uriel Rozenbaum <uriel.rozenbaum@gmail.com
wrote:
I didn't really capture the signaling for the issue, I'll get a trace when I get the chance and send it to you.
Rgds, Uriel
On Mon, Feb 23, 2009 at 12:02 PM, Daniel-Constantin Mierla < miconda@gmail.com> wrote:
On 02/23/2009 02:25 PM, Uriel Rozenbaum wrote:
Sure, I can turn of report ack to check; anyway i substracter $re from db_extra and now I'm not seeing any crashes. But maybe its useful for you to clear the bug in there.
yes, this is the goal. This issue has to be fixed. You can keep the report ack on, I just wanted to get your confirmation.
I will try to reproduce first by myself, and then ask you for more. Just in case you have such ACK at your hand, send it to me, I will use it in tests. Does it have a RPID header?
Thanks, Daniel
Let me know how you want to proceed and I'll send you everithing.
-- Uriel
On Mon, Feb 23, 2009 at 10:18 AM, Daniel-Constantin Mierla < miconda@gmail.com mailto:miconda@gmail.com> wrote:
Hello,
thanks. Do you have report ack set for acc module?
The crash happens to an ACK and relates to building outgoing request and processing lumps. I will investigate and try to reproduce. Just asking for now, would be any chance to get the ACK and eventually your config in case it is needed for troubleshooting?
Thanks, Daniel
On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote:
Here it is: [root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1". warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from /usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from /usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63<sip%3A8888860911167@192.168.200.63> <mailto:sip%3A8888860911167@192.168.200.63<sip%253A8888860911167@192.168.200.63><mailto:sip%3A8888860911167@192.168.200.63<sip%253A8888860911167@192.168.200.63> <mailto:sip%253A8888860911167@192.168.200.63<sip%25253A8888860911167@192.168.200.63>>>SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: sip:192.168.200.9;lr=on\r\nFrom: "7001" <sip:60911100@ops3.cpsnet.com.arsip%3A60911100@ops3.cpsnet.com.ar <mailto:sip%3A60911100@ops3.cpsnet.com.arsip%253A60911100@ops3.cpsnet.com.ar
<mailto:sip%3A60911100@ops3.cpsnet.com.ar<sip%253A60911100@ops3.cpsnet.com.ar> <mailto:sip%253A60911100@ops3.cpsnet.com.ar<sip%25253A60911100@ops3.cpsnet.com.ar>;tag=as"...,
len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780 On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>> wrote: First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use: gdb /usr/local/sbin/kamailio /path/to/corefile The do "bt" there and send the output here. Thanks, Daniel On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote: Hi Henning, here's the core file zipped. On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>>> wrote: On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning -- Daniel-Constantin Mierla http://www.asipto.com
_______________________________________________ Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org <mailto:Users@lists.kamailio.org> http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users-- Daniel-Constantin Mierla http://www.asipto.com
-- Daniel-Constantin Mierla http://www.asipto.com
On 02/24/2009 10:05 PM, Uriel Rozenbaum wrote:
Daniel, I've been trying a litte longer with the server and I captured one ACK that definetely has RPID:
/ACK sip:1054111556446967@cc1int.x.com.ar mailto:sip%3A1054111556446967@cc1int.x.com.ar SIP/2.0 Via: SIP/2.0/UDP 192.168.200.11:5060;branch=z9hG4bK31a69f48;rport From: "541160911100" <sip:541160911100@192.168.200.11 mailto:sip%3A541160911100@192.168.200.11>;tag=as799b3334 To: <sip:1054111556446967@cc1int.cpsnet.com.ar mailto:sip%3A1054111556446967@cc1int.cpsnet.com.ar>;tag=f8f2ab2c1295e90ed7dbb499b30f44b2.90f4 Contact: <sip:541160911100@192.168.200.11 mailto:sip%3A541160911100@192.168.200.11> Call-ID: 5d7adcce1399bafd45c194b049479a1b@192.168.200.11 mailto:5d7adcce1399bafd45c194b049479a1b@192.168.200.11 CSeq: 102 ACK User-Agent: Metrotel Max-Forwards: 70 Remote-Party-ID: "541160911100" <sip:541160911100@192.168.200.11 mailto:sip%3A541160911100@192.168.200.11>;privacy=off;screen=no Content-Length: 0/
Hope it helps,
thanks, I was not that much available last days. I will start testing soon.
Cheers, Daniel
Uriel
On Mon, Feb 23, 2009 at 12:08 PM, Uriel Rozenbaum <uriel.rozenbaum@gmail.com mailto:uriel.rozenbaum@gmail.com> wrote:
I didn't really capture the signaling for the issue, I'll get a trace when I get the chance and send it to you. Rgds, Uriel On Mon, Feb 23, 2009 at 12:02 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> wrote: On 02/23/2009 02:25 PM, Uriel Rozenbaum wrote: Sure, I can turn of report ack to check; anyway i substracter $re from db_extra and now I'm not seeing any crashes. But maybe its useful for you to clear the bug in there. yes, this is the goal. This issue has to be fixed. You can keep the report ack on, I just wanted to get your confirmation. I will try to reproduce first by myself, and then ask you for more. Just in case you have such ACK at your hand, send it to me, I will use it in tests. Does it have a RPID header? Thanks, Daniel Let me know how you want to proceed and I'll send you everithing. -- Uriel On Mon, Feb 23, 2009 at 10:18 AM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>> wrote: Hello, thanks. Do you have report ack set for acc module? The crash happens to an ACK and relates to building outgoing request and processing lumps. I will investigate and try to reproduce. Just asking for now, would be any chance to get the ACK and eventually your config in case it is needed for troubleshooting? Thanks, Daniel On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote: Here it is: [root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1". warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from /usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from /usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63 <mailto:sip%3A8888860911167@192.168.200.63> <mailto:sip%3A8888860911167@192.168.200.63 <mailto:sip%253A8888860911167@192.168.200.63>> <mailto:sip%3A8888860911167@192.168.200.63 <mailto:sip%253A8888860911167@192.168.200.63> <mailto:sip%253A8888860911167@192.168.200.63 <mailto:sip%25253A8888860911167@192.168.200.63>>> SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: <sip:192.168.200.9;lr=on>\r\nFrom: \"7001\" <sip:60911100@ops3.cpsnet.com.ar <mailto:sip%3A60911100@ops3.cpsnet.com.ar> <mailto:sip%3A60911100@ops3.cpsnet.com.ar <mailto:sip%253A60911100@ops3.cpsnet.com.ar>> <mailto:sip%3A60911100@ops3.cpsnet.com.ar <mailto:sip%253A60911100@ops3.cpsnet.com.ar> <mailto:sip%253A60911100@ops3.cpsnet.com.ar <mailto:sip%25253A60911100@ops3.cpsnet.com.ar>>>>;tag=as"..., len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780 On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>> <mailto:miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>>> wrote: First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use: gdb /usr/local/sbin/kamailio /path/to/corefile The do "bt" there and send the output here. Thanks, Daniel On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote: Hi Henning, here's the core file zipped. On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>>>> wrote: On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning -- Daniel-Constantin Mierla http://www.asipto.com ------------------------------------------------------------------------ _______________________________________________ Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org <mailto:Users@lists.kamailio.org> <mailto:Users@lists.kamailio.org <mailto:Users@lists.kamailio.org>> http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users -- Daniel-Constantin Mierla http://www.asipto.com -- Daniel-Constantin Mierla http://www.asipto.com
Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
Hello,
is this of an ACK due to 200ok? Are you doing record-routing? Because Route header is missing.
If you don't do record-routing, then the ACK should be end-to-end between caller and callee and you cannot account.
Anyway, the reported issue should be fixed.
Cheers, Daniel
On 02/24/2009 10:05 PM, Uriel Rozenbaum wrote:
Daniel, I've been trying a litte longer with the server and I captured one ACK that definetely has RPID:
/ACK sip:1054111556446967@cc1int.x.com.ar mailto:sip%3A1054111556446967@cc1int.x.com.ar SIP/2.0 Via: SIP/2.0/UDP 192.168.200.11:5060;branch=z9hG4bK31a69f48;rport From: "541160911100" <sip:541160911100@192.168.200.11 mailto:sip%3A541160911100@192.168.200.11>;tag=as799b3334 To: <sip:1054111556446967@cc1int.cpsnet.com.ar mailto:sip%3A1054111556446967@cc1int.cpsnet.com.ar>;tag=f8f2ab2c1295e90ed7dbb499b30f44b2.90f4 Contact: <sip:541160911100@192.168.200.11 mailto:sip%3A541160911100@192.168.200.11> Call-ID: 5d7adcce1399bafd45c194b049479a1b@192.168.200.11 mailto:5d7adcce1399bafd45c194b049479a1b@192.168.200.11 CSeq: 102 ACK User-Agent: Metrotel Max-Forwards: 70 Remote-Party-ID: "541160911100" <sip:541160911100@192.168.200.11 mailto:sip%3A541160911100@192.168.200.11>;privacy=off;screen=no Content-Length: 0/
Hope it helps, Uriel
On Mon, Feb 23, 2009 at 12:08 PM, Uriel Rozenbaum <uriel.rozenbaum@gmail.com mailto:uriel.rozenbaum@gmail.com> wrote:
I didn't really capture the signaling for the issue, I'll get a trace when I get the chance and send it to you. Rgds, Uriel On Mon, Feb 23, 2009 at 12:02 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> wrote: On 02/23/2009 02:25 PM, Uriel Rozenbaum wrote: Sure, I can turn of report ack to check; anyway i substracter $re from db_extra and now I'm not seeing any crashes. But maybe its useful for you to clear the bug in there. yes, this is the goal. This issue has to be fixed. You can keep the report ack on, I just wanted to get your confirmation. I will try to reproduce first by myself, and then ask you for more. Just in case you have such ACK at your hand, send it to me, I will use it in tests. Does it have a RPID header? Thanks, Daniel Let me know how you want to proceed and I'll send you everithing. -- Uriel On Mon, Feb 23, 2009 at 10:18 AM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>> wrote: Hello, thanks. Do you have report ack set for acc module? The crash happens to an ACK and relates to building outgoing request and processing lumps. I will investigate and try to reproduce. Just asking for now, would be any chance to get the ACK and eventually your config in case it is needed for troubleshooting? Thanks, Daniel On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote: Here it is: [root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1". warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from /usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from /usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63 <mailto:sip%3A8888860911167@192.168.200.63> <mailto:sip%3A8888860911167@192.168.200.63 <mailto:sip%253A8888860911167@192.168.200.63>> <mailto:sip%3A8888860911167@192.168.200.63 <mailto:sip%253A8888860911167@192.168.200.63> <mailto:sip%253A8888860911167@192.168.200.63 <mailto:sip%25253A8888860911167@192.168.200.63>>> SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: <sip:192.168.200.9;lr=on>\r\nFrom: \"7001\" <sip:60911100@ops3.cpsnet.com.ar <mailto:sip%3A60911100@ops3.cpsnet.com.ar> <mailto:sip%3A60911100@ops3.cpsnet.com.ar <mailto:sip%253A60911100@ops3.cpsnet.com.ar>> <mailto:sip%3A60911100@ops3.cpsnet.com.ar <mailto:sip%253A60911100@ops3.cpsnet.com.ar> <mailto:sip%253A60911100@ops3.cpsnet.com.ar <mailto:sip%25253A60911100@ops3.cpsnet.com.ar>>>>;tag=as"..., len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780 On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>> <mailto:miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>>> wrote: First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use: gdb /usr/local/sbin/kamailio /path/to/corefile The do "bt" there and send the output here. Thanks, Daniel On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote: Hi Henning, here's the core file zipped. On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>>>> wrote: On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning -- Daniel-Constantin Mierla http://www.asipto.com ------------------------------------------------------------------------ _______________________________________________ Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org <mailto:Users@lists.kamailio.org> <mailto:Users@lists.kamailio.org <mailto:Users@lists.kamailio.org>> http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users -- Daniel-Constantin Mierla http://www.asipto.com -- Daniel-Constantin Mierla http://www.asipto.com
I tried to reproduce, but no success. The crash is indicated when processing the lumps, a very old piece of code, so I guess there is somewhere a buffer overlow.
Are you having some private modules that are you using the config file?
Thanks, Daniel
On 02/27/2009 11:39 AM, Daniel-Constantin Mierla wrote:
Hello,
is this of an ACK due to 200ok? Are you doing record-routing? Because Route header is missing.
If you don't do record-routing, then the ACK should be end-to-end between caller and callee and you cannot account.
Anyway, the reported issue should be fixed.
Cheers, Daniel
On 02/24/2009 10:05 PM, Uriel Rozenbaum wrote:
Daniel, I've been trying a litte longer with the server and I captured one ACK that definetely has RPID:
/ACK sip:1054111556446967@cc1int.x.com.ar mailto:sip%3A1054111556446967@cc1int.x.com.ar SIP/2.0 Via: SIP/2.0/UDP 192.168.200.11:5060;branch=z9hG4bK31a69f48;rport From: "541160911100" <sip:541160911100@192.168.200.11 mailto:sip%3A541160911100@192.168.200.11>;tag=as799b3334 To: <sip:1054111556446967@cc1int.cpsnet.com.ar mailto:sip%3A1054111556446967@cc1int.cpsnet.com.ar>;tag=f8f2ab2c1295e90ed7dbb499b30f44b2.90f4
Contact: <sip:541160911100@192.168.200.11 mailto:sip%3A541160911100@192.168.200.11> Call-ID: 5d7adcce1399bafd45c194b049479a1b@192.168.200.11 mailto:5d7adcce1399bafd45c194b049479a1b@192.168.200.11 CSeq: 102 ACK User-Agent: Metrotel Max-Forwards: 70 Remote-Party-ID: "541160911100" <sip:541160911100@192.168.200.11 mailto:sip%3A541160911100@192.168.200.11>;privacy=off;screen=no Content-Length: 0/
Hope it helps, Uriel
On Mon, Feb 23, 2009 at 12:08 PM, Uriel Rozenbaum <uriel.rozenbaum@gmail.com mailto:uriel.rozenbaum@gmail.com> wrote:
I didn't really capture the signaling for the issue, I'll get a trace when I get the chance and send it to you. Rgds, Uriel On Mon, Feb 23, 2009 at 12:02 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> wrote: On 02/23/2009 02:25 PM, Uriel Rozenbaum wrote: Sure, I can turn of report ack to check; anyway i substracter $re from db_extra and now I'm not seeing any crashes. But maybe its useful for you to clear the bug in there. yes, this is the goal. This issue has to be fixed. You can keep the report ack on, I just wanted to get your confirmation. I will try to reproduce first by myself, and then ask you for more. Just in case you have such ACK at your hand, send it to me, I will use it in tests. Does it have a RPID header? Thanks, Daniel Let me know how you want to proceed and I'll send you everithing. -- Uriel On Mon, Feb 23, 2009 at 10:18 AM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>>wrote:
Hello, thanks. Do you have report ack set for acc module? The crash happens to an ACK and relates to building outgoing request and processing lumps. I will investigate andtry to reproduce. Just asking for now, would be any chance to get the ACK and eventually your config in case it is needed for troubleshooting?
Thanks, Daniel On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote: Here it is: [root@ops3 ~]# gdb /usr/local/sbin/kamailio/core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".
warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for/usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from
/usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from
/usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63 mailto:sip%3A8888860911167@192.168.200.63 <mailto:sip%3A8888860911167@192.168.200.63 mailto:sip%253A8888860911167@192.168.200.63> <mailto:sip%3A8888860911167@192.168.200.63 mailto:sip%253A8888860911167@192.168.200.63 <mailto:sip%253A8888860911167@192.168.200.63 mailto:sip%25253A8888860911167@192.168.200.63>> SIP/2.0\r\nVia:
SIP/2.0/UDP192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: sip:192.168.200.9;lr=on\r\nFrom: "7001" <sip:60911100@ops3.cpsnet.com.ar mailto:sip%3A60911100@ops3.cpsnet.com.ar <mailto:sip%3A60911100@ops3.cpsnet.com.ar mailto:sip%253A60911100@ops3.cpsnet.com.ar> <mailto:sip%3A60911100@ops3.cpsnet.com.ar mailto:sip%253A60911100@ops3.cpsnet.com.ar <mailto:sip%253A60911100@ops3.cpsnet.com.ar
mailto:sip%25253A60911100@ops3.cpsnet.com.ar>>>;tag=as"...,
len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780 On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>> <mailto:miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>>> wrote: First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use: gdb /usr/local/sbin/kamailio /path/to/corefile The do "bt" there and send the output here. Thanks, Daniel On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote: Hi Henning, here's the core file zipped. On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>>>> wrote: On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning -- Daniel-Constantin Mierla http://www.asipto.com
_______________________________________________ Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org <mailto:Users@lists.kamailio.org> <mailto:Users@lists.kamailio.org <mailto:Users@lists.kamailio.org>>http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
-- Daniel-Constantin Mierla http://www.asipto.com -- Daniel-Constantin Mierla http://www.asipto.com
I´ve checked the captures again and all ACK messages have the RPID, for 487 or 200 messages. I'm just using a plain build of kamailio downloaded from the site including carrerroute module.
Would it be useful if I sent the .cfg?
On Fri, Feb 27, 2009 at 8:34 AM, Daniel-Constantin Mierla <miconda@gmail.com
wrote:
I tried to reproduce, but no success. The crash is indicated when processing the lumps, a very old piece of code, so I guess there is somewhere a buffer overlow.
Are you having some private modules that are you using the config file?
Thanks, Daniel
On 02/27/2009 11:39 AM, Daniel-Constantin Mierla wrote:
Hello,
is this of an ACK due to 200ok? Are you doing record-routing? Because Route header is missing.
If you don't do record-routing, then the ACK should be end-to-end between caller and callee and you cannot account.
Anyway, the reported issue should be fixed.
Cheers, Daniel
On 02/24/2009 10:05 PM, Uriel Rozenbaum wrote:
Daniel, I've been trying a litte longer with the server and I captured one ACK that definetely has RPID:
/ACK sip:1054111556446967@cc1int.x.com.arsip%3A1054111556446967@cc1int.x.com.ar<mailto: sip%3A1054111556446967@cc1int.x.com.arsip%253A1054111556446967@cc1int.x.com.ar> SIP/2.0 Via: SIP/2.0/UDP 192.168.200.11:5060;branch=z9hG4bK31a69f48;rport From: "541160911100" <sip:541160911100@192.168.200.11sip%3A541160911100@192.168.200.11<mailto: sip%3A541160911100@192.168.200.11 sip%253A541160911100@192.168.200.11
;tag=as799b3334
To: <sip:1054111556446967@cc1int.cpsnet.com.arsip%3A1054111556446967@cc1int.cpsnet.com.ar<mailto: sip%3A1054111556446967@cc1int.cpsnet.com.arsip%253A1054111556446967@cc1int.cpsnet.com.ar>>;tag=f8f2ab2c1295e90ed7dbb499b30f44b2.90f4
Contact: <sip:541160911100@192.168.200.11sip%3A541160911100@192.168.200.11<mailto: sip%3A541160911100@192.168.200.11 sip%253A541160911100@192.168.200.11
Call-ID: 5d7adcce1399bafd45c194b049479a1b@192.168.200.11 mailto: 5d7adcce1399bafd45c194b049479a1b@192.168.200.11 CSeq: 102 ACK User-Agent: Metrotel Max-Forwards: 70 Remote-Party-ID: "541160911100" <sip:541160911100@192.168.200.11sip%3A541160911100@192.168.200.11<mailto: sip%3A541160911100@192.168.200.11 sip%253A541160911100@192.168.200.11
;privacy=off;screen=no
Content-Length: 0/
Hope it helps, Uriel
On Mon, Feb 23, 2009 at 12:08 PM, Uriel Rozenbaum < uriel.rozenbaum@gmail.com mailto:uriel.rozenbaum@gmail.com> wrote:
I didn't really capture the signaling for the issue, I'll get a trace when I get the chance and send it to you.
Rgds, Uriel
On Mon, Feb 23, 2009 at 12:02 PM, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:
On 02/23/2009 02:25 PM, Uriel Rozenbaum wrote: Sure, I can turn of report ack to check; anyway i substracter $re from db_extra and now I'm not seeing any crashes. But maybe its useful for you to clear the bug in there. yes, this is the goal. This issue has to be fixed. You can keep the report ack on, I just wanted to get your confirmation. I will try to reproduce first by myself, and then ask you for more. Just in case you have such ACK at your hand, send it to me, I will use it in tests. Does it have a RPID header? Thanks, Daniel Let me know how you want to proceed and I'll send you everithing. -- Uriel On Mon, Feb 23, 2009 at 10:18 AM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>> wrote: Hello, thanks. Do you have report ack set for acc module? The crash happens to an ACK and relates to building outgoing request and processing lumps. I will investigate and try to reproduce. Just asking for now, would be any chance to get the ACK and eventually your config in case it is needed for troubleshooting? Thanks, Daniel On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote: Here it is: [root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899 GNU gdb Red Hat Linux (6.5-37.el5_2.2rh) Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1". warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/local/lib/kamailio/modules/db_mysql.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done. Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libm.so.6...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libssl.so.6...done. Loaded symbols for /lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /usr/lib/libkrb5.so.3...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libk5crypto.so.3...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libselinux.so.1...done. Loaded symbols for /lib/libselinux.so.1 Reading symbols from /lib/libsepol.so.1...done. Loaded symbols for /lib/libsepol.so.1 Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/sl.so Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/tm.so Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/rr.so Reading symbols from /usr/local/lib/kamailio/modules/maxfwd.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so Reading symbols from /usr/local/lib/kamailio/modules/usrloc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so Reading symbols from /usr/local/lib/kamailio/modules/registrar.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so Reading symbols from /usr/local/lib/kamailio/modules/textops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/textops.so Reading symbols from /usr/local/lib/kamailio/modules/mi_fifo.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so Reading symbols from /usr/local/lib/kamailio/modules/uri_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so Reading symbols from /usr/local/lib/kamailio/modules/uri.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/uri.so Reading symbols from /usr/local/lib/kamailio/modules/xlog.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so Reading symbols from /usr/local/lib/kamailio/modules/acc.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/acc.so Reading symbols from /usr/local/lib/kamailio/modules/auth.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth.so Reading symbols from /usr/local/lib/kamailio/modules/auth_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so Reading symbols from /usr/local/lib/kamailio/modules/alias_db.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so Reading symbols from /usr/local/lib/kamailio/modules/domain.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/domain.so Reading symbols from /usr/local/lib/kamailio/modules/lcr.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so Reading symbols from /usr/local/lib/kamailio/modules/avpops.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so Reading symbols from /usr/local/lib/kamailio/modules/permissions.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so Reading symbols from /usr/local/lib/kamailio/modules/carrierroute.so...done. Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid'. Program terminated with signal 11, Segmentation fault. #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 665 switch(r->op){ (gdb) bt #0 lumps_len (msg=0x81b3450, lumps=<value optimized out>, send_sock=0x81afe28) at msg_translator.c:665 #1 0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450, returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1, flags=<value optimized out>) at msg_translator.c:1456 #2 0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8) at forward.c:362 #3 0x00157a28 in t_relay_to (p_msg=0x81b3450, proxy=0x81b2dc0, flags=0) at t_funcs.c:238 #4 0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0, flags=0x0) at tm.c:977 #5 0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at action.c:850 #6 0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450) at action.c:138 #7 0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450, val=0x0) at route.c:1116 #8 0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450, val=0x0) at route.c:1429 #9 0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450, val=0x0) at route.c:1434 #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at action.c:705 #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450) at action.c:138 #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at action.c:118 #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450) at action.c:138 #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at action.c:722 #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450) at action.c:138 #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450) at action.c:118 #17 0x08094e29 in receive_msg ( buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63<sip%3A8888860911167@192.168.200.63> <mailto:sip%3A8888860911167@192.168.200.63<sip%253A8888860911167@192.168.200.63><mailto:sip%3A8888860911167@192.168.200.63<sip%253A8888860911167@192.168.200.63> <mailto:sip%253A8888860911167@192.168.200.63<sip%25253A8888860911167@192.168.200.63><mailto:sip%3A8888860911167@192.168.200.63<sip%253A8888860911167@192.168.200.63> <mailto:sip%253A8888860911167@192.168.200.63<sip%25253A8888860911167@192.168.200.63><mailto:sip%253A8888860911167@192.168.200.63<sip%25253A8888860911167@192.168.200.63> <mailto:sip%25253A8888860911167@192.168.200.63<sip%2525253A8888860911167@192.168.200.63>SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute: sip:192.168.200.9;lr=on\r\nFrom: "7001" <sip:60911100@ops3.cpsnet.com.arsip%3A60911100@ops3.cpsnet.com.ar <mailto:sip%3A60911100@ops3.cpsnet.com.arsip%253A60911100@ops3.cpsnet.com.ar
<mailto:sip%3A60911100@ops3.cpsnet.com.ar<sip%253A60911100@ops3.cpsnet.com.ar> <mailto:sip%253A60911100@ops3.cpsnet.com.ar<sip%25253A60911100@ops3.cpsnet.com.ar><mailto:sip%3A60911100@ops3.cpsnet.com.ar<sip%253A60911100@ops3.cpsnet.com.ar> <mailto:sip%253A60911100@ops3.cpsnet.com.ar<sip%25253A60911100@ops3.cpsnet.com.ar><mailto:sip%253A60911100@ops3.cpsnet.com.ar<sip%25253A60911100@ops3.cpsnet.com.ar> <mailto:sip%25253A60911100@ops3.cpsnet.com.ar<sip%2525253A60911100@ops3.cpsnet.com.ar>>;tag=as"...,
len=523, rcv_info=0xbffd3904) at receive.c:165 #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449 #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780 On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>> <mailto:miconda@gmail.com <mailto:miconda@gmail.com> <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>>> wrote: First send the backtrace, the core without binaries is not much helpful as the symbols can differ to a local compiled version. Use: gdb /usr/local/sbin/kamailio /path/to/corefile The do "bt" there and send the output here. Thanks, Daniel On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote: Hi Henning, here's the core file zipped. On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt <henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de> <mailto:henning.westerholt@1und1.de <mailto:henning.westerholt@1und1.de>>>>> wrote: On Thursday 19 February 2009, Uriel Rozenbaum wrote: > Sure, but I'll need some guidance on obtaining it; I'm just checking with > the syslog messages with debugging level in 3. Can you show me how to do > it? Hi Uriel, the core was already generated, as you can see in the logs: >> Feb 19 08:48:58 ops3 /usr/local/sbin/kamailio[4895]: >> INFO:core:handle_sigs: core was generated You should find this file in the working directory of the server. Cheers, Henning -- Daniel-Constantin Mierla http://www.asipto.com
_______________________________________________ Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org <mailto:Users@lists.kamailio.org> <mailto:Users@lists.kamailio.org <mailto:Users@lists.kamailio.org>>http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
-- Daniel-Constantin Mierla http://www.asipto.com -- Daniel-Constantin Mierla http://www.asipto.com-- Daniel-Constantin Mierla http://www.asipto.com
On 02/27/2009 12:57 PM, Uriel Rozenbaum wrote:
I´ve checked the captures again and all ACK messages have the RPID, for 487 or 200 messages.
ok, but is the ack you sent causing the trouble? It does not look as an ack for 200ok, because it does not have a Route header. Non-200ok ACK do not get accounted.
I'm just using a plain build of kamailio downloaded from the site including carrerroute module.
Would it be useful if I sent the .cfg?
Could help, yes.
Daniel
On Fri, Feb 27, 2009 at 8:34 AM, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:
I tried to reproduce, but no success. The crash is indicated when processing the lumps, a very old piece of code, so I guess there is somewhere a buffer overlow. Are you having some private modules that are you using the config file? Thanks, Daniel On 02/27/2009 11:39 AM, Daniel-Constantin Mierla wrote: Hello, is this of an ACK due to 200ok? Are you doing record-routing? Because Route header is missing. If you don't do record-routing, then the ACK should be end-to-end between caller and callee and you cannot account. Anyway, the reported issue should be fixed. Cheers, Daniel On 02/24/2009 10:05 PM, Uriel Rozenbaum wrote: Daniel, I've been trying a litte longer with the server and I captured one ACK that definetely has RPID: /ACK sip:1054111556446967@cc1int.x.com.ar <mailto:sip%3A1054111556446967@cc1int.x.com.ar> <mailto:sip%3A1054111556446967@cc1int.x.com.ar <mailto:sip%253A1054111556446967@cc1int.x.com.ar>> SIP/2.0 Via: SIP/2.0/UDP 192.168.200.11:5060;branch=z9hG4bK31a69f48;rport From: "541160911100" <sip:541160911100@192.168.200.11 <mailto:sip%3A541160911100@192.168.200.11> <mailto:sip%3A541160911100@192.168.200.11 <mailto:sip%253A541160911100@192.168.200.11>>>;tag=as799b3334 To: <sip:1054111556446967@cc1int.cpsnet.com.ar <mailto:sip%3A1054111556446967@cc1int.cpsnet.com.ar> <mailto:sip%3A1054111556446967@cc1int.cpsnet.com.ar <mailto:sip%253A1054111556446967@cc1int.cpsnet.com.ar>>>;tag=f8f2ab2c1295e90ed7dbb499b30f44b2.90f4 Contact: <sip:541160911100@192.168.200.11 <mailto:sip%3A541160911100@192.168.200.11> <mailto:sip%3A541160911100@192.168.200.11 <mailto:sip%253A541160911100@192.168.200.11>>> Call-ID: 5d7adcce1399bafd45c194b049479a1b@192.168.200.11 <mailto:5d7adcce1399bafd45c194b049479a1b@192.168.200.11> <mailto:5d7adcce1399bafd45c194b049479a1b@192.168.200.11 <mailto:5d7adcce1399bafd45c194b049479a1b@192.168.200.11>> CSeq: 102 ACK User-Agent: Metrotel Max-Forwards: 70 Remote-Party-ID: "541160911100" <sip:541160911100@192.168.200.11 <mailto:sip%3A541160911100@192.168.200.11> <mailto:sip%3A541160911100@192.168.200.11 <mailto:sip%253A541160911100@192.168.200.11>>>;privacy=off;screen=no Content-Length: 0/ Hope it helps, Uriel
-
Daniel-Constantin Mierla -
Henning Westerholt -
Uriel Rozenbaum