I have 2 servers running Solaris and Kamailio 3.2.3 where on one Kamailio is terminating when it tries to save the location for a register request and the other is producing a core dump when processing an Option request. I have one server handling Register request while the other sip server forwards the register requests and handles the other requests. I have included the backtraces from the core dumps and the output from the log for the registrar server as well as the command that is causing kamailio to terminate: if (!save("location")) sl_reply_error(); 4(3364) ERROR: *** cfgtrace: c=[/opt/kamailio-3.2/etc/kamailio/kamailio.cfg] l=714 a=17 n=if 4(3364) ERROR: *** cfgtrace: c=[/opt/kamailio-3.2/etc/kamailio/kamailio.cfg] l=711 a=26 n=save 14(3374) : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 15 14(3374) DEBUG: <core> [tcp_main.c:3555]: DBG: handle_ser_child: dead child 4, pid 3364 (shutting down?) 14(3374) DEBUG: <core> [io_wait.h:617]: DBG: io_watch_del (1003743d8, 15, 0, 0x0) fd_no=18 called 0(3360) ALERT: <core> [main.c:751]: child process 3364 exited by a signal 10 0(3360) ALERT: <core> [main.c:754]: core was not generated 0(3360) INFO: <core> [main.c:766]: INFO: terminating due to SIGCHLD 6(3366) INFO: <core> [main.c:817]: INFO: signal 15 received 1(3361) INFO: <core> [main.c:817]: INFO: signal 15 received 2(3362) INFO: <core> [main.c:817]: INFO: signal 15 received 3(3363) INFO: <core> [main.c:817]: INFO: signal 15 received 5(3365) INFO: <core> [main.c:817]: INFO: signal 15 received 7(3367) INFO: <core> [main.c:817]: INFO: signal 15 received 8(3368) INFO: <core> [main.c:817]: INFO: signal 15 received 9(3369) INFO: <core> [main.c:817]: INFO: signal 15 received 10(3370) INFO: <core> [main.c:817]: INFO: signal 15 received 11(3371) INFO: <core> [main.c:817]: INFO: signal 15 received 12(3372) INFO: <core> [main.c:817]: INFO: signal 15 received 13(3373) INFO: <core> [main.c:817]: INFO: signal 15 received 14(3374) INFO: <core> [main.c:817]: INFO: signal 15 received 0(3360) DEBUG: presence_xml [presence_xml.c:347]: start 0(3360) ERROR: ctl [ctl.c:379]: ERROR: ctl: could not delete unix socket /tmp/kamailio_ctl: Permission denied (13) 0(3360) DEBUG: <core> [db_pool.c:102]: removing connection from the pool 0(3360) DEBUG: db_postgres [km_pg_con.c:122]: PQfinish(100842470) 0(3360) DEBUG: db_postgres [km_pg_con.c:126]: pkg_free(1004c1f30) 0(3360) DEBUG: tm [t_funcs.c:122]: DEBUG: tm_shutdown : start 0(3360) DEBUG: tm [t_funcs.c:125]: DEBUG: tm_shutdown : emptying hash table 0(3360) DEBUG: tm [t_funcs.c:127]: DEBUG: tm_shutdown : removing semaphores 0(3360) DEBUG: tm [t_funcs.c:129]: DEBUG: tm_shutdown : destroying tmcb lists 0(3360) DEBUG: tm [t_funcs.c:132]: DEBUG: tm_shutdown : done Thanks Nathaniel _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

No, but I do have an alias defined. alias="mydomain.com:5080" Thanks Nathaniel On 4/26/2012 3:41 AM, Daniel-Constantin Mierla wrote: _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users ----- No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.1913 / Virus Database: 2411/4960 - Release Date: 04/26/12

No, but I do have an alias defined. alias="mydomain.com:5080" Thanks Nathaniel On 4/26/2012 3:41 AM, Daniel-Constantin Mierla wrote: _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users ----- No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.1913 / Virus Database: 2411/4960 - Release Date: 04/26/12

I made the change and still have the problem. I have included the output from stepping thru via gdb and the block of code where I made the change and that was executed. 98 if (reg_use_domain) { 99 if (user_len) 100 aor_buf[_a->len++] = '@'; 101 /* strip prefix (if defined) */ 102 realm_prefix.s = cfg_get(registrar, registrar_cfg, realm_pref).s; 103 realm_prefix.len = cfg_get(registrar, registrar_cfg, realm_pref).len; 104 if (realm_prefix.len>0 && realm_prefix.len<puri.host.len && 105 (memcmp(realm_prefix.s, puri.host.s, realm_prefix.len)==0) ) { 106 memcpy(aor_buf + _a->len, puri.host.s + realm_prefix.len, 107 puri.host.len - realm_prefix.len); 108 _a->len += puri.host.len - realm_prefix.len; 109 } else { 110 memcpy(aor_buf + _a->len, puri.host.s, puri.host.len); 111 _a->len += puri.host.len; 112 } 113 } Thanks Nathaniel On 5/4/2012 2:55 AM, Daniel-Constantin Mierla wrote: > Hello, > > interesting, it seems to crash at the evaluation of > 'realm_prefix.len' - because it is 0, the IF condition should stop > evaluation of the rest of the expression. > > Can you try to change the first part of the condition at line 104 to: > > if (realm_prefix.len>0 && ... > > Cheers, > Daniel > > On 5/4/12 4:42 AM, Akan wrote: >> I was able to step thru via gdb to the point where Kamailio took a >> segment fault. I have included a backtrace as well as the output >> from me stepping thru common.c to the point where it failed. Hope >> this will help. >> >> Thanks >> >> Nathaniel >> >> On 4/29/2012 4:37 AM, Daniel-Constantin Mierla wrote: >>> Hello, >>> >>> the issue was a sig bus this time, which more enforced in >>> sparc/solaris, not common in linux. I was looked at the structure >>> and seems aligned ok, mis-alignment being the most often used to >>> rise sigbus. What I could think as next reason was the empty >>> string default value which may make solaris think is not >>> accessible anymore at runtime via the cfg structure, so I changed >>> the field for prefix to str and initialized to null to force >>> allocation in any case. >>> >>> Can you try with the attached patch and tell if works fine? >>> >>> Cheers, >>> Daniel >>> >>> On 4/28/12 6:49 PM, Akan wrote: >>>> I tried adding the realm_prefix and still got the same problem. >>>> I ran kamailio thru gdb to try and step thru and get more >>>> information and have included the output in the attached text >>>> file. Hope this helps. >>>> >>>> Thanks >>>> >>>> Nathaniel L Keeling >>>> >>>> On 4/26/2012 2:42 PM, Akan wrote: >>>>> No, but I do have an alias defined. >>>>> >>>>> alias="mydomain.com:5080" >>>>> >>>>> Thanks >>>>> >>>>> Nathaniel >>>>> >>>>> On 4/26/2012 3:41 AM, Daniel-Constantin Mierla wrote: >>>>>> Hello, >>>>>> >>>>>> do you set the realm_prefix parameter of registrar module? >>>>>> >>>>>> http://kamailio.org/docs/modules/stable/modules_k/registrar.html#id2495082 >>>>>> >>>>>> >>>>>> If yes, can you paste it here? >>>>>> >>>>>> Cheers, >>>>>> Daniel >>>>>> >>>>>> On 4/25/12 9:42 PM, Akan wrote: >>>>>>> I have 2 servers running Solaris and Kamailio 3.2.3 where on >>>>>>> one Kamailio is terminating when it tries to save the >>>>>>> location for a register request and the other is producing a >>>>>>> core dump when processing an Option request. I have one >>>>>>> server handling Register request while the other sip server >>>>>>> forwards the register requests and handles the other >>>>>>> requests. I have included the backtraces from the core dumps >>>>>>> and the output from the log for the registrar server as well >>>>>>> as the command that is causing kamailio to terminate: >>>>>>> >>>>>>> if (!save("location")) >>>>>>> sl_reply_error(); >>>>>>> >>>>>>> 4(3364) ERROR: *** cfgtrace: >>>>>>> c=[/opt/kamailio-3.2/etc/kamailio/kamailio.cfg] l=714 a=17 n=if >>>>>>> 4(3364) ERROR: *** cfgtrace: >>>>>>> c=[/opt/kamailio-3.2/etc/kamailio/kamailio.cfg] l=711 a=26 >>>>>>> n=save >>>>>>> 14(3374) : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 15 >>>>>>> 14(3374) DEBUG: <core> [tcp_main.c:3555]: DBG: >>>>>>> handle_ser_child: dead child 4, pid 3364 (shutting down?) >>>>>>> 14(3374) DEBUG: <core> [io_wait.h:617]: DBG: io_watch_del >>>>>>> (1003743d8, 15, 0, 0x0) fd_no=18 called >>>>>>> 0(3360) ALERT: <core> [main.c:751]: child process 3364 >>>>>>> exited by a signal 10 >>>>>>> 0(3360) ALERT: <core> [main.c:754]: core was not generated >>>>>>> 0(3360) INFO: <core> [main.c:766]: INFO: terminating due to >>>>>>> SIGCHLD >>>>>>> 6(3366) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 1(3361) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 2(3362) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 3(3363) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 5(3365) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 7(3367) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 8(3368) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 9(3369) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 10(3370) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 11(3371) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 12(3372) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 13(3373) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 14(3374) INFO: <core> [main.c:817]: INFO: signal 15 received >>>>>>> 0(3360) DEBUG: presence_xml [presence_xml.c:347]: start >>>>>>> 0(3360) ERROR: ctl [ctl.c:379]: ERROR: ctl: could not delete >>>>>>> unix socket /tmp/kamailio_ctl: Permission denied (13) >>>>>>> 0(3360) DEBUG: <core> [db_pool.c:102]: removing connection >>>>>>> from the pool >>>>>>> 0(3360) DEBUG: db_postgres [km_pg_con.c:122]: >>>>>>> PQfinish(100842470) >>>>>>> 0(3360) DEBUG: db_postgres [km_pg_con.c:126]: >>>>>>> pkg_free(1004c1f30) >>>>>>> 0(3360) DEBUG: tm [t_funcs.c:122]: DEBUG: tm_shutdown : start >>>>>>> 0(3360) DEBUG: tm [t_funcs.c:125]: DEBUG: tm_shutdown : >>>>>>> emptying hash table >>>>>>> 0(3360) DEBUG: tm [t_funcs.c:127]: DEBUG: tm_shutdown : >>>>>>> removing semaphores >>>>>>> 0(3360) DEBUG: tm [t_funcs.c:129]: DEBUG: tm_shutdown : >>>>>>> destroying tmcb lists >>>>>>> 0(3360) DEBUG: tm [t_funcs.c:132]: DEBUG: tm_shutdown : done >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Nathaniel >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users >>>>>>> mailing list >>>>>>> sr-users(a)lists.sip-router.org >>>>>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>>>>> >>>>>> -- >>>>>> Daniel-Constantin Mierla -http://www.asipto.com >>>>>> http://twitter.com/#!/miconda >>>>>> -http://www.linkedin.com/in/miconda >>>>>> >>>>>> No virus found in this message. >>>>>> Checked by AVG - www.avg.com <http://www.avg.com> >>>>>> Version: 2012.0.1913 / Virus Database: 2411/4959 - Release >>>>>> Date: 04/25/12 >>>>>> >>>>> >>>>> _______________________________________________ >>>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users >>>>> mailing list >>>>> sr-users(a)lists.sip-router.org >>>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>>>> >>>>> >>>>> >>>>> ----- >>>>> No virus found in this message. >>>>> Checked by AVG - www.avg.com >>>>> Version: 2012.0.1913 / Virus Database: 2411/4960 - Release >>>>> Date: 04/26/12 >>>>> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users >>>> mailing list >>>> sr-users(a)lists.sip-router.org >>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>> >>> -- >>> Daniel-Constantin Mierla -http://www.asipto.com >>> http://twitter.com/#!/miconda -http://www.linkedin.com/in/miconda >>> >>> >>> No virus found in this message. >>> Checked by AVG - www.avg.com <http://www.avg.com> >>> Version: 2012.0.1913 / Virus Database: 2411/4965 - Release Date: >>> 04/28/12 >>> >

Hello, On 5/30/12 9:10 PM, Akan wrote: do not checkout branch 3.2, just try with the master branch, as said, not all patches are backported to stable 3.2. For sake of completeness, here are some guidelines: http://www.kamailio.org/wiki/install/devel/git Cheers, Daniel

Hello, still strange with the crash on that if condition. One of your previous reports showed a different line, when assigning a pointer. Also, I kind of understood that save() was ok, but lookup() not. Is not the case anymore as the backtrace shows. SIGBUS can occur because of alignment in memory which is thrown in solaris, does not happen in linux flavours. I am thinking of putting the prefix field first so the access to it is at beginning of the straucture, just to see if that is the problem, although is should not be. I will add a patch soon, otherwise I am out of ideas for the moment with the SIGBUS, I would need access to the machine to do some testing by myself. Cheers, Daniel On 6/2/12 10:36 PM, Akan wrote: -- Daniel-Constantin Mierla -http://www.asipto.com http://twitter.com/#!/miconda -http://www.linkedin.com/in/miconda Kamailio Advanced Training, Seattle, USA, Sep 23-26, 2012 -http://asipto.com/u/katu Kamailio Practical Workshop, Netherlands, Sep 10-12, 2012 -http://asipto.com/u/kpw

Hello, I committed a patch that should make it work when realm_prefix is not set for registrar module: http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=d42379d… For the moment I have no idea why SIGBUS is triggered, i tried to access the structure directly, not via defines, and there is still a problem when realm_prefix is set. It's ok accessing the .len field but not .s of the same structure. The field .s is the first, making me think it's not alignment issue, or it is something very specific to this system. Overall, I am not familiar with sparc/solaris systems, maybe if you send here full details about OS version, architecture and CPU type, others using similar systems can give hints. Jason reported that on solaris with x86, this problem was not encountered. Cheers, Daniel On 6/4/12 8:57 AM, Daniel-Constantin Mierla wrote: -- Daniel-Constantin Mierla -http://www.asipto.com http://twitter.com/#!/miconda -http://www.linkedin.com/in/miconda Kamailio Advanced Training, Seattle, USA, Sep 23-26, 2012 -http://asipto.com/u/katu Kamailio Practical Workshop, Netherlands, Sep 10-12, 2012 -http://asipto.com/u/kpw No virus found in this message. Checked by AVG - www.avg.com <http://www.avg.com> Version: 2012.0.2178 / Virus Database: 2433/5061 - Release Date: 06/10/12

Hello, is it 64bit architecture? Cheers, Daniel On 6/11/12 9:12 PM, Akan wrote:

Hello, thanks for troubleshooting further -- as I expected, it is a memalign problem, but some confusing reports by not using always the patches I made to registrar module to align the structure, made thinking is something else. Now is no longer in registrar, but in another module. There might be many of these, can you try to compile first without strict alignment to 8 bytes? Checking quickly to gcc, the option should be |-mno-faster-structs So try: make proper make FLAVOUR=kamailio cfg| make Q=0 CC_EXTRA_OPTS=|-mno-faster-structs all| ... By providing Q=0, you will see all compile flags, verify that |-mno-faster-structs is there. Cheers, Daniel | On 6/22/12 3:26 PM, Akan wrote: -- Daniel-Constantin Mierla -http://www.asipto.com http://twitter.com/#!/miconda -http://www.linkedin.com/in/miconda Kamailio Advanced Training, Seattle, USA, Sep 23-26, 2012 -http://asipto.com/u/katu Kamailio Practical Workshop, Netherlands, Sep 10-12, 2012 -http://asipto.com/u/kpw No virus found in this message. Checked by AVG - www.avg.com <http://www.avg.com> Version: 2012.0.2180 / Virus Database: 2437/5091 - Release Date: 06/24/12

Just to let you know I made a lot of tests using different modules including carrierroute, I do have the bus error also after the first call and core dump. thank you O On Jun 27, 2012, at 11:45 PM, Akan wrote: _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users