[SR-Users] Kamailio on AWS Kubernetes with NLB

Cedric Thiebault cedric.thiebault at sensefly.com
Mon Oct 18 10:07:58 CEST 2021


Here is sipgrep output:

root at ip-172-31-11-79:/# sipgrep
interface: eth0 (172.31.0.0/255.255.240.0)
filter: (ip or ip6) and ( portrange 5060-5061) or (udp and ip[6:2] & 0x3fff != 0)

T 2021/10/18 08:04:53.496957 194.209.160.181:41733 -> 172.31.11.79:5060
 [AP]
REGISTER sip:test-sip.sensefly.io:5060;transport=TCP SIP/2.0.
Via: SIP/2.0/TCP 194.209.160.181:36703;branch=z9hG4bK-524287-1---0bc0be77eac080a1;rport.
Max-Forwards: 70.
Contact: <sip:cedric at 194.209.160.181:36703;rinstance=010fcab4c692c129;transport=tcp>.
To: <sip:cedric at test-sip.sensefly.io:5060;transport=TCP>.
From: <sip:cedric at test-sip.sensefly.io:5060;transport=TCP>;tag=f8166d78.
Call-ID: 9PbwQuHQ1PQQsE0-ZIrNEQ...
CSeq: 1 REGISTER.
Expires: 70.
Allow: INVITE, ACK, CANCEL, BYE, NOTIFY, REFER, MESSAGE, OPTIONS, INFO, SUBSCRIBE.
Supported: replaces, norefersub, extended-refer, timer, outbound, path, X-cisco-serviceuri.
User-Agent: Z 5.5.6 v2.10.15.1-141-g4fc603c55.
Allow-Events: presence, kpml, talk.
Content-Length: 0.
.


It seems right no?
________________________________
From: sr-users <sr-users-bounces at lists.kamailio.org> on behalf of Cedric Thiebault <cedric.thiebault at sensefly.com>
Sent: Monday, October 18, 2021 9:28 AM
To: Henning Westerholt <hw at skalatan.de>; Kamailio (SER) - Users Mailing List <sr-users at lists.kamailio.org>
Subject: Re: [SR-Users] Kamailio on AWS Kubernetes with NLB

Hello,

Thanks for your help :-)

Here are my logs with debug loglevel:

43(48) DEBUG: <core> [core/tcp_read.c:1536]: tcp_read_req(): content-length=0
43(48) INFO: <core> [core/parser/parse_fline.c:147]: parse_first_line(): method not followed by SP
43(48) ERROR: <core> [core/parser/parse_fline.c:271]: parse_first_line(): parse_first_line: bad message (offset: 0)
43(48) DEBUG: <core> [core/parser/msg_parser.c:673]: parse_msg(): invalid message
43(48) ERROR: <core> [core/parser/msg_parser.c:747]: parse_msg(): ERROR: parse_msg: message=<>
43(48) ERROR: <core> [core/receive.c:374]: receive_msg(): core parsing of SIP message failed (194.209.160.181:51603/2)
43(48) DEBUG: <core> [core/usr_avp.c:637]: destroy_avp_list(): destroying list (nil)
43(48) DEBUG: <core> [core/usr_avp.c:637]: destroy_avp_list(): destroying list (nil)
43(48) DEBUG: <core> [core/usr_avp.c:637]: destroy_avp_list(): destroying list (nil)
43(48) DEBUG: <core> [core/usr_avp.c:637]: destroy_avp_list(): destroying list (nil)
43(48) DEBUG: <core> [core/usr_avp.c:637]: destroy_avp_list(): destroying list (nil)
43(48) DEBUG: <core> [core/usr_avp.c:637]: destroy_avp_list(): destroying list (nil)
43(48) DEBUG: <core> [core/xavp.c:541]: xavp_destroy_list(): destroying xavp list (nil)
43(48) DEBUG: <core> [core/xavp.c:541]: xavp_destroy_list(): destroying xavp list (nil)
43(48) DEBUG: <core> [core/xavp.c:541]: xavp_destroy_list(): destroying xavp list (nil)
43(48) DEBUG: <core> [core/tcp_read.c:1656]: release_tcpconn(): releasing con 0x7f7bcb670558, state -2, fd=7, id=68 ([194.209.160.181]:51603 -> [194.209.160.181]:5060)
43(48) DEBUG: <core> [core/tcp_read.c:1660]: release_tcpconn(): extra_data (nil)
48(53) DEBUG: <core> [core/tcp_main.c:3559]: handle_tcp_child(): reader response= 7f7bcb670558, -2 from 3
48(53) DEBUG: <core> [core/ip_addr.c:571]: print_ip(): tcpconn_new: new tcp connection: 172.31.4.8
48(53) DEBUG: <core> [core/tcp_main.c:1175]: tcpconn_new(): on port 25099, type 2, socket 71
48(53) DEBUG: <core> [core/tcp_main.c:1494]: tcpconn_add(): hashes: 342:3855:3308, 69
48(53) DEBUG: <core> [core/io_wait.h:374]: io_watch_add(): DBG: io_watch_add(0x559718b4f840, 71, 2, 0x7f7bcb670558), fd_no=59
48(53) DEBUG: <core> [core/io_wait.h:599]: io_watch_del(): DBG: io_watch_del (0x559718b4f840, 71, -1, 0x0) fd_no=60 called
48(53) DEBUG: <core> [core/tcp_main.c:4457]: handle_tcpconn_ev(): sending to child, events 2001
48(53) DEBUG: <core> [core/tcp_main.c:4127]: send2child(): selected tcp worker idx:4 proc:44 pid:49 for activity on [tcp:172.31.11.79:5060], 0x7f7bcb670558
44(49) DEBUG: <core> [core/tcp_read.c:1737]: handle_io(): received n=8 con=0x7f7bcb670558, fd=7
44(49) DEBUG: <core> [core/tcp_read.c:300]: tcp_read_data(): EOF on connection 0x7f7bcb670558 (state: 3, flags: 4118) - FD 7, bytes 23, rd-flags 10000 ([172.31.4.8]:25099 -> [172.31.4.8]:5060)
44(49) DEBUG: <core> [core/tcp_read.c:184]: tcp_emit_closed_event(): TCP closed event creation triggered (reason: 0)
44(49) DEBUG: <core> [core/tcp_read.c:192]: tcp_emit_closed_event(): no callback registering for handling TCP closed event
44(49) DEBUG: <core> [core/tcp_read.c:1536]: tcp_read_req(): content-length=0
44(49) DEBUG: <core> [core/tcp_main.c:1590]: _tcpconn_find(): found connection by id: 69
44(49) DEBUG: <core> [core/tcp_main.c:2471]: tcpconn_send_put(): send from reader (49 (44)), reusing fd
44(49) DEBUG: <core> [core/tcp_main.c:2706]: tcpconn_do_send(): sending...
44(49) DEBUG: <core> [core/tcp_main.c:2739]: tcpconn_do_send(): after real write: c= 0x7f7bcb670558 n=2 fd=7
44(49) DEBUG: <core> [core/tcp_main.c:2740]: tcpconn_do_send(): buf=

I'm going to investigate SIP requests with ngrep now...

Cedric

________________________________
From: Henning Westerholt <hw at skalatan.de>
Sent: Sunday, October 17, 2021 1:20 PM
To: Kamailio (SER) - Users Mailing List <sr-users at lists.kamailio.org>
Cc: Cedric Thiebault <cedric.thiebault at sensefly.com>
Subject: RE: Kamailio on AWS Kubernetes with NLB


Hello,



I would try to connect to the kamailio container and increase debug level to see what is happening on a lower level. You can also use tools like ngrep/sngrep to see if the SIP request is routed to the instance.



Cheers,



Henning

--

Henning Westerholt – https://skalatan.de/blog/

Kamailio services – https://gilawa.com<https://gilawa.com/>



From: sr-users <sr-users-bounces at lists.kamailio.org> On Behalf Of Cedric Thiebault
Sent: Friday, October 15, 2021 3:37 PM
To: Kamailio (SER) - Users Mailing List <sr-users at lists.kamailio.org>
Subject: Re: [SR-Users] Kamailio on AWS Kubernetes with NLB



Adding a log in kamailio.cfg in request_route method:

xlog("L_INFO", "INFO - request_route - $rm ($fu ($si:$sp) to $tu, $ci)\n");



Using docker-compose I get:

24(30) INFO: {1 1 REGISTER Hotel7dIcm7DCPtmR3jACw..} <script>: INFO - request_route - REGISTER (sip:cedric at localhost:5060;transport=TCP (172.19.0.1:43696) to sip:cedric at localhost:5060;transport=TCP, Hotel7dIcm7DCPtmR3jACw..)



But I don't see any log message on AWS :-(



Cedric

________________________________

From: sr-users <sr-users-bounces at lists.kamailio.org<mailto:sr-users-bounces at lists.kamailio.org>> on behalf of Cedric Thiebault <cedric.thiebault at sensefly.com<mailto:cedric.thiebault at sensefly.com>>
Sent: Friday, October 15, 2021 11:52 AM
To: sr-users at lists.kamailio.org<mailto:sr-users at lists.kamailio.org> <sr-users at lists.kamailio.org<mailto:sr-users at lists.kamailio.org>>
Subject: [SR-Users] Kamailio on AWS Kubernetes with NLB



Hello,



I'm trying to deploy Kamailio on Kubernetes (AWS EKS with Network Load Balancer) but I'm not able to register using TCP.

I'm using official Docker image 5.5.0-buster.

I did some tests with Docker Compose and it worked fine, but on K8S it does not :-(



Here are my logs:



Listening on

             udp: 127.0.0.1:5060

             udp: 172.31.4.29:5060

             udp: 172.31.0.221:5060

             tcp: 127.0.0.1:5060

             tcp: 172.31.4.29:5060

             tcp: 172.31.0.221:5060

Aliases:

             tcp: ip-172-31-0-221.eu-central-1.compute.internal:5060

             tcp: 172-31-4-29.kube-prometheus-stack-prometheus-node-exporter.infra-prometheus.svc.cluster.local:5060

             tcp: localhost4.localdomain4:5060

             tcp: localhost4:5060

             tcp: localhost.localdomain:5060

             tcp: localhost:5060

             udp: ip-172-31-0-221.eu-central-1.compute.internal:5060

             udp: 172-31-4-29.kube-prometheus-stack-kubelet.kube-system.svc.cluster.local:5060

             udp: localhost4.localdomain4:5060

             udp: localhost4:5060

             udp: localhost.localdomain:5060

             udp: localhost:5060



 0(1) INFO: <core> [core/tcp_main.c:4983]: init_tcp(): using epoll_lt as the io watch method (auto detected)

 0(1) INFO: rr [../outbound/api.h:52]: ob_load_api(): unable to import bind_ob - maybe module is not loaded

 0(1) INFO: rr [rr_mod.c:188]: mod_init(): outbound module not available

 0(1) INFO: <core> [main.c:3030]: main(): processes (at least): 41 - shm size: 67108864 - pkg size: 8388608

 0(1) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992

 0(1) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984

 0(1) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992

 0(1) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984

 0(1) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992

 0(1) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984

29(35) INFO: jsonrpcs [jsonrpcs_sock.c:443]: jsonrpc_dgram_process(): a new child 0/35

31(37) INFO: ctl [io_listener.c:213]: io_listen_loop(): io_listen_loop:  using epoll_lt io watch method (config)

35(41) INFO: <core> [core/parser/parse_fline.c:147]: parse_first_line(): method not followed by SP

35(41) ERROR: <core> [core/parser/parse_fline.c:271]: parse_first_line(): parse_first_line: bad message (offset: 0)

35(41) ERROR: <core> [core/parser/msg_parser.c:747]: parse_msg(): ERROR: parse_msg: message=<>

35(41) ERROR: <core> [core/receive.c:374]: receive_msg(): core parsing of SIP message failed (194.230.146.66:9055/2)





And my service:



apiVersion: v1

kind: Service

metadata:

  name: kamailio-tcp

  annotations:

    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip

    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'

    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing

    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: preserve_client_ip.enabled=true

    service.beta.kubernetes.io/aws-load-balancer-type: external

  labels:

    app.kubernetes.io/instance: test-sip

    app.kubernetes.io/name: kamailio-tcp

    app.kubernetes.io/version: 5.5.0-buster

spec:

  type: LoadBalancer

  externalTrafficPolicy: Local

  ports:

    - name: sip-tcp

      nodePort: 30567

      port: 5060

      protocol: TCP

      targetPort: 5060

  selector:

    app.kubernetes.io/instance: test-sip

    app.kubernetes.io/name: kamailio





I have a Netwok Load Balancer create with a listener for TCP 5060 but when I try to connect with Zoiper or Linphone, it does not work :-(



I used tcpdump on my pod and it received well some SIP commands.

Any ideas of what could be wrong here?



Thanks in advance!



Cedric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20211018/74d9395d/attachment.htm>


More information about the sr-users mailing list