[sr-dev] cdp module - timeouts

Daniel Ciprus daniel.ciprus at acision.com
Wed Jan 29 23:11:25 CET 2014


Jason,

this is bt generated from running process .. I think my scscf is in this stage now. Obviously hss has listener up and running and I can telnet to the port. "Lately" means that nobody paid attention to this issue before :-), it was discovered just recently.

As I said: nothing's in capture :

[root at kamailio-4-1-test kamailio]# date ; tcpdump -i any -s0 -w /tmp/crap.pcap -v port 3868  -n ; date
Wed Jan 29 16:45:15 EST 2014
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
^C0 packets captured
0 packets received by filter
0 packets dropped by kernel
Wed Jan 29 17:09:08 EST 2014
[root at kamailio-4-1-test kamailio]#

Once I restart scscf/icscf everything will be OK ... so there is likely no way to get logging out of the running scscf, is it ?

On 01/29/2014 09:45 AM, Jason Penton wrote:
Hi Daniel,

This looks like a bootup of kamailio, where the peer process is trying to start? Is your HSS up. Can you telnet to your HSS:3868?

Also, your first message says you are having trouble with CDP "lately". What does this mean? There have been no changes to CDP code for some time now...

Cheers
jason


On Wed, Jan 29, 2014 at 4:30 PM, Daniel Ciprus <daniel.ciprus at acision.com<mailto:daniel.ciprus at acision.com>> wrote:
(gdb) bt
#0  0x00007f5b564f7900 in __nanosleep_nocancel () from /lib64/libc.so.6
#1  0x00007f5b564f7790 in sleep () from /lib64/libc.so.6
#2  0x00007f5b5095cb71 in timer_loop () at timer.c:120
#3  0x00007f5b5095d5a4 in timer_process (returns=0) at timer.c:207
#4  0x00007f5b5093f891 in diameter_peer_start (blocking=0) at diameter_peer.c:396
#5  0x00007f5b5094102b in cdp_child_init (rank=0) at mod.c:237
#6  0x00000000004f7ec2 in init_mod_child (m=0x7f5b5609a670, rank=0) at sr_module.c:924
#7  0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b218, rank=0) at sr_module.c:921
#8  0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b5c0, rank=0) at sr_module.c:921
#9  0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b970, rank=0) at sr_module.c:921
#10 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c000, rank=0) at sr_module.c:921
#11 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c478, rank=0) at sr_module.c:921
#12 0x00000000004f8048 in init_child (rank=0) at sr_module.c:948
#13 0x000000000046d57c in main_loop () at main.c:1694
#14 0x000000000047030b in main (argc=13, argv=0x7ffff3e21128) at main.c:2533
(gdb) bt full
#0  0x00007f5b564f7900 in __nanosleep_nocancel () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007f5b564f7790 in sleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007f5b5095cb71 in timer_loop () at timer.c:120
        now = 1390934103
        i = 0x0
        cb = 0
        ptr = 0x0
        interval = 1
        __FUNCTION__ = "timer_loop"
#3  0x00007f5b5095d5a4 in timer_process (returns=0) at timer.c:207
        __FUNCTION__ = "timer_process"
#4  0x00007f5b5093f891 in diameter_peer_start (blocking=0) at diameter_peer.c:396
        pid = 0
        k = -2
        p = 0x0
        __FUNCTION__ = "diameter_peer_start"
#5  0x00007f5b5094102b in cdp_child_init (rank=0) at mod.c:237
        __FUNCTION__ = "cdp_child_init"
#6  0x00000000004f7ec2 in init_mod_child (m=0x7f5b5609a670, rank=0) at sr_module.c:924
        __FUNCTION__ = "init_mod_child"
#7  0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b218, rank=0) at sr_module.c:921
        __FUNCTION__ = "init_mod_child"
#8  0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b5c0, rank=0) at sr_module.c:921
        __FUNCTION__ = "init_mod_child"
#9  0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b970, rank=0) at sr_module.c:921
        __FUNCTION__ = "init_mod_child"
#10 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c000, rank=0) at sr_module.c:921
        __FUNCTION__ = "init_mod_child"
#11 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c478, rank=0) at sr_module.c:921
        __FUNCTION__ = "init_mod_child"
#12 0x00000000004f8048 in init_child (rank=0) at sr_module.c:948
No locals.
#13 0x000000000046d57c in main_loop () at main.c:1694
        i = 8
        pid = 19298
        si = 0x0
        si_desc = "udp receiver child=7 sock=10.96.173.17:6060<http://10.96.173.17:6060>\000[\177\000\000\001\000\000\000z\000\000\000~}^\000\000\000\000\000`o^\000\000\000\000\000\254\305\006m\000\000\000\000\300LA\000\000\000\000\000 \021\342\363\377\177", '\000' <repeats 18 times>, "`\017\342\363\377\177\000\000.\241K\000\000\000\000"
        nrprocs = 8
        __FUNCTION__ = "main_loop"
#14 0x000000000047030b in main (argc=13, argv=0x7ffff3e21128) at main.c:2533
        cfg_stream = 0x1a7f010
        c = -1
        r = 0
        tmp = 0x7ffff3e22f40 ""
        tmp_len = 0
        port = 0
        proto = 0
        options = 0x5e02c0 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"
        ret = -1
        seed = 2714878883
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 0
---Type <return> to continue, or q <return> to quit---
        n_lst = 0x7f5b5645bb88
        p = 0x5ca3e0 "H\211l$\330L\211d$\340H\215-\017\234*"
        __FUNCTION__ = "main"
(gdb)
On 01/29/2014 08:45 AM, Daniel Ciprus wrote:
Unfortunately the only thing I have is core file and bt. But I can assure you that there was absolutely no traffic between diameter interfaces. Looks like I will need to reproduce the problem to get right input.

thanks
Dan


On 01/29/2014 02:39 AM, Jason Penton wrote:
Hi,

can you send us a pcap as well as a log file? I can't debug much with what you have given so far.

Cheers
Jason


On Tue, Jan 28, 2014 at 8:42 PM, Daniel Ciprus <daniel.ciprus at acision.com<mailto:daniel.ciprus at acision.com>> wrote:
Hi,

seems like CDP module is giving us hard time in last few days. For some
reason s-cscf and i-cscf is loosing connection to HSS and is not even
trying to close socket nor trying to reconnect back. Here is what I see
on the kamailio node:

Name        : kamailio                     Relocations: (not relocatable)
Version     : 4.2.0                             Vendor: kamailio.org<http://kamailio.org>
Release     : dev0.0.el6                    Build Date: Wed Dec 18
12:46:11 2013

[root at kamailio-4-1-staging kamailio]# netstat -apn | grep 3868
tcp        0      0 10.96.173.17:34915<http://10.96.173.17:34915> 10.96.173.18:3868<http://10.96.173.18:3868>
CLOSE_WAIT  10087/kamailio
tcp        0      0 10.96.173.17:35361<http://10.96.173.17:35361> 10.96.173.18:3868<http://10.96.173.18:3868>
CLOSE_WAIT  19308/kamailio
[root at kamailio-4-1-staging kamailio]#

and HSS:

[root at hss-config-staging phss-sop]# netstat -apn | grep 3868
tcp        0      0 10.96.173.18:3868<http://10.96.173.18:3868> 0.0.0.0:*
LISTEN      10549/ugc_server
[root at hss-config-staging phss-sop]#

I have core files (created by gcore) if this helps.

thanks
Dan



This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.



_______________________________________________
sr-dev mailing list
sr-dev at lists.sip-router.org<mailto:sr-dev at lists.sip-router.org>
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev


--
Daniel Ciprus
Integration engineer
http://www.acision.com

9954 Mayland Dr
Suite 3100
Richmond, VA 23233
USA
T: +1 804 762 5601<tel:%2B1%20804%20762%205601>
E: daniel.ciprus at acision.com<mailto:daniel.ciprus at acision.com>

________________________________
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.


--
Daniel Ciprus
Integration engineer
http://www.acision.com

9954 Mayland Dr
Suite 3100
Richmond, VA 23233
USA
T: +1 804 762 5601<tel:%2B1%20804%20762%205601>
E: daniel.ciprus at acision.com<mailto:daniel.ciprus at acision.com>

________________________________
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.



--
Daniel Ciprus
Integration engineer
http://www.acision.com

9954 Mayland Dr
Suite 3100
Richmond, VA 23233
USA
T: +1 804 762 5601
E: daniel.ciprus at acision.com<mailto:daniel.ciprus at acision.com>

________________________________
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-dev/attachments/20140129/dcbae018/attachment-0001.html>


More information about the sr-dev mailing list