Here is sipgrep output:

root@ip-172-31-11-79:/# sipgrep

interface: eth0 (172.31.0.0/255.255.240.0)

filter: (ip or ip6) and ( portrange 5060-5061) or (udp and ip[6:2] & 0x3fff != 0)

T 2021/10/18 08:04:53.496957 194.209.160.181:41733 -> 172.31.11.79:5060

[AP]

Via: SIP/2.0/TCP 194.209.160.181:36703;branch=z9hG4bK-524287-1---0bc0be77eac080a1;rport.

Max-Forwards: 70.

Contact: <sip:cedric@194.209.160.181:36703;rinstance=010fcab4c692c129;transport=tcp>.

To: <sip:cedric@test-sip.sensefly.io:5060;transport=TCP>.

From: <sip:cedric@test-sip.sensefly.io:5060;transport=TCP>;tag=f8166d78.

Call-ID: 9PbwQuHQ1PQQsE0-ZIrNEQ...

CSeq: 1 REGISTER.

Expires: 70.

Allow: INVITE, ACK, CANCEL, BYE, NOTIFY, REFER, MESSAGE, OPTIONS, INFO, SUBSCRIBE.

Supported: replaces, norefersub, extended-refer, timer, outbound, path, X-cisco-serviceuri.

User-Agent: Z 5.5.6 v2.10.15.1-141-g4fc603c55.

Allow-Events: presence, kpml, talk.

Content-Length: 0.

It seems right no?

From: sr-users <sr-users-bounces@lists.kamailio.org> on behalf of Cedric Thiebault <cedric.thiebault@sensefly.com>
Sent: Monday, October 18, 2021 9:28 AM
To: Henning Westerholt <hw@skalatan.de>; Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org>
Subject: Re: [SR-Users] Kamailio on AWS Kubernetes with NLB

Hello,

Thanks for your help :-)

Here are my logs with debug loglevel:

43(48) DEBUG: <core> [core/tcp_read.c:1536]: tcp_read_req(): content-length=0

43(48) INFO: <core> [core/parser/parse_fline.c:147]: parse_first_line(): method not followed by SP

43(48) ERROR: <core> [core/parser/parse_fline.c:271]: parse_first_line(): parse_first_line: bad message (offset: 0)

43(48) DEBUG: <core> [core/parser/msg_parser.c:673]: parse_msg(): invalid message

43(48) ERROR: <core> [core/parser/msg_parser.c:747]: parse_msg(): ERROR: parse_msg: message=<>

43(48) ERROR: <core> [core/receive.c:374]: receive_msg(): core parsing of SIP message failed (194.209.160.181:51603/2)

43(48) DEBUG: <core> [core/usr_avp.c:637]: destroy_avp_list(): destroying list (nil)

43(48) DEBUG: <core> [core/xavp.c:541]: xavp_destroy_list(): destroying xavp list (nil)

43(48) DEBUG: <core> [core/tcp_read.c:1656]: release_tcpconn(): releasing con 0x7f7bcb670558, state -2, fd=7, id=68 ([194.209.160.181]:51603 -> [194.209.160.181]:5060)

43(48) DEBUG: <core> [core/tcp_read.c:1660]: release_tcpconn(): extra_data (nil)

48(53) DEBUG: <core> [core/tcp_main.c:3559]: handle_tcp_child(): reader response= 7f7bcb670558, -2 from 3

48(53) DEBUG: <core> [core/ip_addr.c:571]: print_ip(): tcpconn_new: new tcp connection: 172.31.4.8

48(53) DEBUG: <core> [core/tcp_main.c:1175]: tcpconn_new(): on port 25099, type 2, socket 71

48(53) DEBUG: <core> [core/tcp_main.c:1494]: tcpconn_add(): hashes: 342:3855:3308, 69

48(53) DEBUG: <core> [core/io_wait.h:374]: io_watch_add(): DBG: io_watch_add(0x559718b4f840, 71, 2, 0x7f7bcb670558), fd_no=59

48(53) DEBUG: <core> [core/io_wait.h:599]: io_watch_del(): DBG: io_watch_del (0x559718b4f840, 71, -1, 0x0) fd_no=60 called

48(53) DEBUG: <core> [core/tcp_main.c:4457]: handle_tcpconn_ev(): sending to child, events 2001

48(53) DEBUG: <core> [core/tcp_main.c:4127]: send2child(): selected tcp worker idx:4 proc:44 pid:49 for activity on [tcp:172.31.11.79:5060], 0x7f7bcb670558

44(49) DEBUG: <core> [core/tcp_read.c:1737]: handle_io(): received n=8 con=0x7f7bcb670558, fd=7

44(49) DEBUG: <core> [core/tcp_read.c:300]: tcp_read_data(): EOF on connection 0x7f7bcb670558 (state: 3, flags: 4118) - FD 7, bytes 23, rd-flags 10000 ([172.31.4.8]:25099 -> [172.31.4.8]:5060)
44(49) DEBUG: <core> [core/tcp_read.c:184]: tcp_emit_closed_event(): TCP closed event creation triggered (reason: 0)

44(49) DEBUG: <core> [core/tcp_read.c:192]: tcp_emit_closed_event(): no callback registering for handling TCP closed event

44(49) DEBUG: <core> [core/tcp_read.c:1536]: tcp_read_req(): content-length=0

44(49) DEBUG: <core> [core/tcp_main.c:1590]: _tcpconn_find(): found connection by id: 69

44(49) DEBUG: <core> [core/tcp_main.c:2471]: tcpconn_send_put(): send from reader (49 (44)), reusing fd

44(49) DEBUG: <core> [core/tcp_main.c:2706]: tcpconn_do_send(): sending...

44(49) DEBUG: <core> [core/tcp_main.c:2739]: tcpconn_do_send(): after real write: c= 0x7f7bcb670558 n=2 fd=7

44(49) DEBUG: <core> [core/tcp_main.c:2740]: tcpconn_do_send(): buf=

I'm going to investigate SIP requests with ngrep now...

Cedric

From: Henning Westerholt <hw@skalatan.de>
Sent: Sunday, October 17, 2021 1:20 PM
To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org>
Cc: Cedric Thiebault <cedric.thiebault@sensefly.com>
Subject: RE: Kamailio on AWS Kubernetes with NLB

Hello,

I would try to connect to the kamailio container and increase debug level to see what is happening on a lower level. You can also use tools like ngrep/sngrep to see if the SIP request is routed to the instance.

Cheers,

Henning

Henning Westerholt – https://skalatan.de/blog/

Kamailio services – https://gilawa.com

From: sr-users <sr-users-bounces@lists.kamailio.org> On Behalf Of Cedric Thiebault
Sent: Friday, October 15, 2021 3:37 PM
To: Kamailio (SER) - Users Mailing List <sr-users@lists.kamailio.org>
Subject: Re: [SR-Users] Kamailio on AWS Kubernetes with NLB

Adding a log in kamailio.cfg in request_route method:

xlog("L_INFO", "INFO - request_route - $rm ($fu ($si:$sp) to $tu, $ci)\n");

Using docker-compose I get:

24(30) INFO: {1 1 REGISTER Hotel7dIcm7DCPtmR3jACw..} <script>: INFO - request_route - REGISTER (sip:cedric@localhost:5060;transport=TCP (172.19.0.1:43696) to sip:cedric@localhost:5060;transport=TCP, Hotel7dIcm7DCPtmR3jACw..)

But I don't see any log message on AWS :-(

Cedric

From: sr-users <sr-users-bounces@lists.kamailio.org> on behalf of Cedric Thiebault <cedric.thiebault@sensefly.com>
Sent: Friday, October 15, 2021 11:52 AM
To: sr-users@lists.kamailio.org <sr-users@lists.kamailio.org>
Subject: [SR-Users] Kamailio on AWS Kubernetes with NLB

Hello,

I'm trying to deploy Kamailio on Kubernetes (AWS EKS with Network Load Balancer) but I'm not able to register using TCP.

I'm using official Docker image 5.5.0-buster.

I did some tests with Docker Compose and it worked fine, but on K8S it does not :-(

Here are my logs:

Listening on

udp: 127.0.0.1:5060

udp: 172.31.4.29:5060

udp: 172.31.0.221:5060

tcp: 127.0.0.1:5060

tcp: 172.31.4.29:5060

tcp: 172.31.0.221:5060

Aliases:

tcp: ip-172-31-0-221.eu-central-1.compute.internal:5060

tcp: 172-31-4-29.kube-prometheus-stack-prometheus-node-exporter.infra-prometheus.svc.cluster.local:5060

tcp: localhost4.localdomain4:5060

tcp: localhost4:5060

tcp: localhost.localdomain:5060

tcp: localhost:5060

udp: ip-172-31-0-221.eu-central-1.compute.internal:5060

udp: 172-31-4-29.kube-prometheus-stack-kubelet.kube-system.svc.cluster.local:5060

udp: localhost4.localdomain4:5060

udp: localhost4:5060

udp: localhost.localdomain:5060

udp: localhost:5060

0(1) INFO: <core> [core/tcp_main.c:4983]: init_tcp(): using epoll_lt as the io watch method (auto detected)

0(1) INFO: rr [../outbound/api.h:52]: ob_load_api(): unable to import bind_ob - maybe module is not loaded

0(1) INFO: rr [rr_mod.c:188]: mod_init(): outbound module not available

0(1) INFO: <core> [main.c:3030]: main(): processes (at least): 41 - shm size: 67108864 - pkg size: 8388608

0(1) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992

0(1) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984

0(1) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992

0(1) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984

0(1) INFO: <core> [core/udp_server.c:154]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992

0(1) INFO: <core> [core/udp_server.c:206]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984

29(35) INFO: jsonrpcs [jsonrpcs_sock.c:443]: jsonrpc_dgram_process(): a new child 0/35

31(37) INFO: ctl [io_listener.c:213]: io_listen_loop(): io_listen_loop: using epoll_lt io watch method (config)

35(41) INFO: <core> [core/parser/parse_fline.c:147]: parse_first_line(): method not followed by SP

35(41) ERROR: <core> [core/parser/parse_fline.c:271]: parse_first_line(): parse_first_line: bad message (offset: 0)

35(41) ERROR: <core> [core/parser/msg_parser.c:747]: parse_msg(): ERROR: parse_msg: message=<>

35(41) ERROR: <core> [core/receive.c:374]: receive_msg(): core parsing of SIP message failed (194.230.146.66:9055/2)

And my service:

apiVersion: v1

kind: Service

metadata:

name: kamailio-tcp

annotations:

service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip

service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'

service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing

service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: preserve_client_ip.enabled=true

service.beta.kubernetes.io/aws-load-balancer-type: external

labels:

app.kubernetes.io/instance: test-sip

app.kubernetes.io/name: kamailio-tcp

app.kubernetes.io/version: 5.5.0-buster

spec:

type: LoadBalancer

externalTrafficPolicy: Local

ports:

- name: sip-tcp

nodePort: 30567

port: 5060

protocol: TCP

targetPort: 5060

selector:

app.kubernetes.io/instance: test-sip

app.kubernetes.io/name: kamailio

I have a Netwok Load Balancer create with a listener for TCP 5060 but when I try to connect with Zoiper or Linphone, it does not work :-(

I used tcpdump on my pod and it received well some SIP commands.

Any ideas of what could be wrong here?

Thanks in advance!

Cedric