[SR-Users] Kamailio 3.1.5 core dump from the snmpstats module

[Klaus Feichtinger]

Hello Daniel,

today I've tested your commit and the result was fine. No more core dumps
have been created. When I did a downgrade to the official version 3.1.5
the core dumps became active again.

So I can commit: your solution is working fine.

Regards,
Klaus

> Hello,
>
> I pushed a commit in master branch to deal with the case of unsuccessful
>  startup which was the cause for this issue.
>
> The problem seemed to be some pointers which were not initialized yet,
> and because startup failed, the destroy function in snmstats module was
> called and accessed these pointers.
>
> Here is the link to commit, you can cherry pick it by hash id to 3.1
> branch via git:
> http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=06e71ad96e8f13bafac1fa5d968538f98bd08df5
>
> I had no meanings to test it, so a report of results will be
> appreciated. If all goes fine, then I will backport.
>
> Cheers,
> Daniel
>
>
>
> On 1/31/12 4:25 PM, Klaus Feichtinger wrote:
>> Hello list,
>> I`ve found a negative phenomenon during startup of kamailio 3.1.x
>> (orgininally found in version 3.1.5) that results in a core dump,
>> triggered by the snmpstats module.
>>
>> During startup of the SIP proxy server machine the linux service
>> "heartbeat" is responsible for starting required ressources like
>> mysql, kamailio and snmpd. These services are monitored by the tool
>> "monit". Monit is supervising the state of the named services and if
>> kamailio is not up and running, monit is trying to restart the
>> service. During startup the mysql database takes some time until it is
>> ready to use and therefore kamailio can`t connect to the database.
>> Monit is detecting this and restarting kamailio. This is done so often
>> until kamailio is up and running.
>>
>> Finally all services are up and running and the machine is working
>> fine. However, I do not like the core dump that was created during
>> this "restart phase". As soon as the module "snmpstats" is not used,
>> no core dump is created any more.
>>
>> Information about the core dump looks as follows:
>>
>> (1) output of kamailio log files:
>> <prefix>  kamailio: INFO:<core>  [tcp_main.c:4726]: init_tcp: using
>> epoll_lt as the io watch method (auto detected)<prefix>
>> /usr/sbin/kamailio[3027]: ERROR: db_mysql [km_my_con.c:109]: driver
>> error: Can't connect to local MySQL server through socket
>> '/var/run/mysqld/mysqld.sock' (2)<prefix>  /usr/sbin/kamailio[3027]:
>> ERROR:<core>  [db.c:289]: could not add connection to the pool<prefix>
>> /usr/sbin/kamailio[3027]: ERROR: lcr [lcr_mod.c:359]: unable to
>> connect to the database<prefix>  /usr/sbin/kamailio[3027]: ERROR: lcr
>> [lcr_mod.c:571]: unable to open database connection<prefix>
>> /usr/sbin/kamailio[3027]: ERROR:<core>  [sr_module.c:875]: init_mod():
>> Error while initializing module lcr (/usr/lib/kamailio/modules/lcr.so)
>> <prefix>  /usr/sbin/kamailio[3027]: INFO: snmpstats [snmpstats.c:387]:
>> The SNMPStats module got the kill signal<prefix>  kamailio: ERROR:
>> <core>  [daemonize.c:307]: Main process exited before writing to pipe
>> <prefix>  kamailio: INFO:<core>  [tcp_main.c:4726]: init_tcp: using
>> epoll_lt as the io watch method (auto detected)<prefix>  kamailio:
>> WARNING:<core>  [daemonize.c:352]: pid file contains old pid,
>> replacing pid<prefix>  /usr/sbin/kamailio[3322]: INFO: usrloc
>> [hslot.c:53]: locks array size 512<prefix>  /usr/sbin/kamailio[3322]:
>> INFO: auth
>> [auth_mod.c:312]: auth: qop set, but nonce-count (nc_enabled) support
>> disabled<prefix>  /usr/sbin/kamailio[3322]: INFO: pua [pua.c:360]: the
>> query returned no result<prefix>  /usr/sbin/kamailio[3322]: INFO: pike
>> [ip_tree.c:88]: probing 256 set size<prefix>
>> /usr/sbin/kamailio[3322]: INFO:<core>  [udp_server.c:178]: INFO:
>> udp_init: SO_RCVBUF is initially 111616<prefix>
>> /usr/sbin/kamailio[3322]: INFO:<core>
>> [udp_server.c:229]: INFO: udp_init: SO_RCVBUF is finally
>> 262142<prefix> /usr/sbin/kamailio[3329]: INFO: mi_datagram
>> [mi_datagram.c:326]: a new child 0/3329
>>
>> (2) output of GDB / core file:
>> Core was generated by `/usr/sbin/kamailio -P
>> /var/run/kamailio/kamailio.pid -m 1024 -u kamailio -g kama'. Program
>> terminated with signal 11, Segmentation fault.
>> [New process 3027]
>> #0  0xb775539c in freeInterprocessBuffer () at
>> interprocess_buffer.c:354 354     interprocess_buffer.c: No such file
>> or directory.
>>          in interprocess_buffer.c
>> (gdb)
>>
>>
>> I know that it is not a fine method in killing kamailio during
>> initialisation. But I have not found a better way yet. The other
>> modules like LCR have no problem with the kill signal during
>> initialisation. My main question is now: how could these core dumps
>> being avoided? Did anybody have the same experience as me?
>>
>> Thanks in advance,
>> Klaus
>>
>>
>>
>>
>> _______________________________________________
>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing
>> list sr-users at lists.sip-router.org
>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>
> --
> Daniel-Constantin Mierla -- http://www.asipto.com
> http://linkedin.com/in/miconda -- http://twitter.com/miconda