[sr-dev] cdp module - timeouts
Jason Penton
jason.penton at gmail.com
Thu Jan 30 08:29:44 CET 2014
Hi Daniel,
That bt you sent is normal. That is the CDP timer process so I don't see
any issues there. What you can do is send me bt's on the "CDP worker
childs", the "cdp_receiver_peer" for your HSS connection, and the "cdp
acceptor".
to get the processes, use sercmd ps. For example, mine looks like this:
8633 cdp worker child=0
8634 cdp worker child=1
8635 cdp worker child=2
8636 cdp worker child=3
8637 cdp worker child=4
8638 cdp worker child=5
8639 cdp worker child=6
8640 cdp worker child=7
8641 cdp worker child=8
8642 cdp worker child=9
8643 cdp receiver peer unknown
8644 cdp_receiver_peer=ZAUbuntu005.it.za.smilecoms.com
8645 cdp_acceptor
8646 cdp_timer
re. logs, you can turn on debugging using sercmd/kamcmd.
cheers
Jason
Cheers
Jason
On Thu, Jan 30, 2014 at 12:11 AM, Daniel Ciprus
<daniel.ciprus at acision.com>wrote:
> Jason,
>
> this is bt generated from running process .. I think my scscf is in this
> stage now. Obviously hss has listener up and running and I can telnet to
> the port. "Lately" means that nobody paid attention to this issue before
> :-), it was discovered just recently.
>
> As I said: nothing's in capture :
>
> [root at kamailio-4-1-test kamailio]# date ; tcpdump -i any -s0 -w
> /tmp/crap.pcap -v port 3868 -n ; date
> Wed Jan 29 16:45:15 EST 2014
> tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture
> size 65535 bytes
> ^C0 packets captured
> 0 packets received by filter
> 0 packets dropped by kernel
> Wed Jan 29 17:09:08 EST 2014
> [root at kamailio-4-1-test kamailio]#
>
> Once I restart scscf/icscf everything will be OK ... so there is likely no
> way to get logging out of the running scscf, is it ?
>
>
> On 01/29/2014 09:45 AM, Jason Penton wrote:
>
> Hi Daniel,
>
> This looks like a bootup of kamailio, where the peer process is trying
> to start? Is your HSS up. Can you telnet to your HSS:3868?
>
> Also, your first message says you are having trouble with CDP "lately".
> What does this mean? There have been no changes to CDP code for some time
> now...
>
> Cheers
> jason
>
>
> On Wed, Jan 29, 2014 at 4:30 PM, Daniel Ciprus <daniel.ciprus at acision.com>wrote:
>
>> (gdb) bt
>> #0 0x00007f5b564f7900 in __nanosleep_nocancel () from /lib64/libc.so.6
>> #1 0x00007f5b564f7790 in sleep () from /lib64/libc.so.6
>> #2 0x00007f5b5095cb71 in timer_loop () at timer.c:120
>> #3 0x00007f5b5095d5a4 in timer_process (returns=0) at timer.c:207
>> #4 0x00007f5b5093f891 in diameter_peer_start (blocking=0) at
>> diameter_peer.c:396
>> #5 0x00007f5b5094102b in cdp_child_init (rank=0) at mod.c:237
>> #6 0x00000000004f7ec2 in init_mod_child (m=0x7f5b5609a670, rank=0) at
>> sr_module.c:924
>> #7 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b218, rank=0) at
>> sr_module.c:921
>> #8 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b5c0, rank=0) at
>> sr_module.c:921
>> #9 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b970, rank=0) at
>> sr_module.c:921
>> #10 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c000, rank=0) at
>> sr_module.c:921
>> #11 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c478, rank=0) at
>> sr_module.c:921
>> #12 0x00000000004f8048 in init_child (rank=0) at sr_module.c:948
>> #13 0x000000000046d57c in main_loop () at main.c:1694
>> #14 0x000000000047030b in main (argc=13, argv=0x7ffff3e21128) at
>> main.c:2533
>> (gdb) bt full
>> #0 0x00007f5b564f7900 in __nanosleep_nocancel () from /lib64/libc.so.6
>> No symbol table info available.
>> #1 0x00007f5b564f7790 in sleep () from /lib64/libc.so.6
>> No symbol table info available.
>> #2 0x00007f5b5095cb71 in timer_loop () at timer.c:120
>> now = 1390934103
>> i = 0x0
>> cb = 0
>> ptr = 0x0
>> interval = 1
>> __FUNCTION__ = "timer_loop"
>> #3 0x00007f5b5095d5a4 in timer_process (returns=0) at timer.c:207
>> __FUNCTION__ = "timer_process"
>> #4 0x00007f5b5093f891 in diameter_peer_start (blocking=0) at
>> diameter_peer.c:396
>> pid = 0
>> k = -2
>> p = 0x0
>> __FUNCTION__ = "diameter_peer_start"
>> #5 0x00007f5b5094102b in cdp_child_init (rank=0) at mod.c:237
>> __FUNCTION__ = "cdp_child_init"
>> #6 0x00000000004f7ec2 in init_mod_child (m=0x7f5b5609a670, rank=0) at
>> sr_module.c:924
>> __FUNCTION__ = "init_mod_child"
>> #7 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b218, rank=0) at
>> sr_module.c:921
>> __FUNCTION__ = "init_mod_child"
>> #8 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b5c0, rank=0) at
>> sr_module.c:921
>> __FUNCTION__ = "init_mod_child"
>> #9 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609b970, rank=0) at
>> sr_module.c:921
>> __FUNCTION__ = "init_mod_child"
>> #10 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c000, rank=0) at
>> sr_module.c:921
>> __FUNCTION__ = "init_mod_child"
>> #11 0x00000000004f7d65 in init_mod_child (m=0x7f5b5609c478, rank=0) at
>> sr_module.c:921
>> __FUNCTION__ = "init_mod_child"
>> #12 0x00000000004f8048 in init_child (rank=0) at sr_module.c:948
>> No locals.
>> #13 0x000000000046d57c in main_loop () at main.c:1694
>> i = 8
>> pid = 19298
>> si = 0x0
>> si_desc = "udp receiver child=7 sock=10.96.173.17:6060\000[\177\000\000\001\000\000\000z\000\000\000~}^\000\000\000\000\000`o^\000\000\000\000\000\254\305\006m\000\000\000\000\300LA\000\000\000\000\000
>> \021\342\363\377\177", '\000' <repeats 18 times>,
>> "`\017\342\363\377\177\000\000.\241K\000\000\000\000"
>> nrprocs = 8
>> __FUNCTION__ = "main_loop"
>> #14 0x000000000047030b in main (argc=13, argv=0x7ffff3e21128) at
>> main.c:2533
>> cfg_stream = 0x1a7f010
>> c = -1
>> r = 0
>> tmp = 0x7ffff3e22f40 ""
>> tmp_len = 0
>> port = 0
>> proto = 0
>> options = 0x5e02c0
>> ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"
>> ret = -1
>> seed = 2714878883
>> rfd = 4
>> debug_save = 0
>> debug_flag = 0
>> dont_fork_cnt = 0
>> ---Type <return> to continue, or q <return> to quit---
>> n_lst = 0x7f5b5645bb88
>> p = 0x5ca3e0 "H\211l$\330L\211d$\340H\215-\017\234*"
>> __FUNCTION__ = "main"
>> (gdb)
>> On 01/29/2014 08:45 AM, Daniel Ciprus wrote:
>>
>> Unfortunately the only thing I have is core file and bt. But I can assure
>> you that there was absolutely no traffic between diameter interfaces. Looks
>> like I will need to reproduce the problem to get right input.
>>
>> thanks
>> Dan
>>
>>
>> On 01/29/2014 02:39 AM, Jason Penton wrote:
>>
>> Hi,
>>
>> can you send us a pcap as well as a log file? I can't debug much with
>> what you have given so far.
>>
>> Cheers
>> Jason
>>
>>
>> On Tue, Jan 28, 2014 at 8:42 PM, Daniel Ciprus <daniel.ciprus at acision.com
>> > wrote:
>>
>>> Hi,
>>>
>>> seems like CDP module is giving us hard time in last few days. For some
>>> reason s-cscf and i-cscf is loosing connection to HSS and is not even
>>> trying to close socket nor trying to reconnect back. Here is what I see
>>> on the kamailio node:
>>>
>>> Name : kamailio Relocations: (not relocatable)
>>> Version : 4.2.0 Vendor: kamailio.org
>>> Release : dev0.0.el6 Build Date: Wed Dec 18
>>> 12:46:11 2013
>>>
>>> [root at kamailio-4-1-staging kamailio]# netstat -apn | grep 3868
>>> tcp 0 0 10.96.173.17:34915 10.96.173.18:3868
>>> CLOSE_WAIT 10087/kamailio
>>> tcp 0 0 10.96.173.17:35361 10.96.173.18:3868
>>> CLOSE_WAIT 19308/kamailio
>>> [root at kamailio-4-1-staging kamailio]#
>>>
>>> and HSS:
>>>
>>> [root at hss-config-staging phss-sop]# netstat -apn | grep 3868
>>> tcp 0 0 10.96.173.18:3868 0.0.0.0:*
>>> LISTEN 10549/ugc_server
>>> [root at hss-config-staging phss-sop]#
>>>
>>> I have core files (created by gcore) if this helps.
>>>
>>> thanks
>>> Dan
>>>
>>>
>>>
>>> This e-mail and any attachment is for authorised use by the intended
>>> recipient(s) only. It may contain proprietary material, confidential
>>> information and/or be subject to legal privilege. It should not be copied,
>>> disclosed to, retained or used by, any other party. If you are not an
>>> intended recipient then please promptly delete this e-mail and any
>>> attachment and all copies and inform the sender. Thank you for
>>> understanding.
>>>
>>>
>>>
>>> _______________________________________________
>>> sr-dev mailing list
>>> sr-dev at lists.sip-router.org
>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
>>>
>>
>>
>> --
>> *Daniel Ciprus*
>> Integration engineer
>> http://www.acision.com
>>
>> 9954 Mayland Dr
>> Suite 3100
>> Richmond, VA 23233
>> USA
>> T: +1 804 762 5601
>> E: daniel.ciprus at acision.com
>>
>> ------------------------------
>> This e-mail and any attachment is for authorised use by the intended
>> recipient(s) only. It may contain proprietary material, confidential
>> information and/or be subject to legal privilege. It should not be copied,
>> disclosed to, retained or used by, any other party. If you are not an
>> intended recipient then please promptly delete this e-mail and any
>> attachment and all copies and inform the sender. Thank you for
>> understanding.
>>
>>
>> --
>> *Daniel Ciprus*
>> Integration engineer
>> http://www.acision.com
>>
>> 9954 Mayland Dr
>> Suite 3100
>> Richmond, VA 23233
>> USA
>> T: +1 804 762 5601
>> E: daniel.ciprus at acision.com
>>
>> ------------------------------
>> This e-mail and any attachment is for authorised use by the intended
>> recipient(s) only. It may contain proprietary material, confidential
>> information and/or be subject to legal privilege. It should not be copied,
>> disclosed to, retained or used by, any other party. If you are not an
>> intended recipient then please promptly delete this e-mail and any
>> attachment and all copies and inform the sender. Thank you for
>> understanding.
>>
>>
>
> --
> *Daniel Ciprus*
> Integration engineer
> http://www.acision.com
>
> 9954 Mayland Dr
> Suite 3100
> Richmond, VA 23233
> USA
> T: +1 804 762 5601
> E: daniel.ciprus at acision.com
>
> ------------------------------
> This e-mail and any attachment is for authorised use by the intended
> recipient(s) only. It may contain proprietary material, confidential
> information and/or be subject to legal privilege. It should not be copied,
> disclosed to, retained or used by, any other party. If you are not an
> intended recipient then please promptly delete this e-mail and any
> attachment and all copies and inform the sender. Thank you for
> understanding.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-dev/attachments/20140130/7dd3fa9b/attachment.html>
More information about the sr-dev
mailing list