Hi Team,
Recently we upgraded our kamailo server to 4.2.1 and it was running successfuly with out any issue. We are seeing some issue with the TCP calls which are relayed from the kamilio.
We have set of physical servers and the calls are distributed by checking the dispacther module. Sometimes we are seeing the TCP packets are malformed from kamilio to one server. We can see the UDP calls are working fine with the same server at the same time TCP packets are malformed. That means there is no issue with the network connectivity between the kamailio server and our media server. Once we restart the kamilio we are not seeing this issue and the TCP calls will start working with kamilio and media server.
Can you help us to answer these questions.
1. Any idea about this TCP packet malformed error ?. We were not able to reproduce this TCP packet issue and we are seeing this error in our production environment.
2. Sometimes we have some errors for TCP max conn (ERROR) : 2048 (the default). We are plannig to increase the TCP connection to 4096 (tcp_max_connections=4096). Whether it will create any issue if we are increasing to 4096 or it will affect kamailio performance ?.
Thanks for looking in to this.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51
Question number two is better handled on the sr-users mailing list than in the bug tracker.
Can you give us more details on #1 - a TCP packet can be "malformed" in many ways. How did you notice this - kamailio logs, packet dumps?
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-71367580
Hi oej,
Thanks for looking into. As per our observation we are not seeing any PACKET in our media server from kamailio server and we thought that packets might be malformed.
But after checking the PCAP in kamailio we are not seeing the packet is malformed but packets are not reaching in the server side.
Kamailo Log:
Example one:
Kamailio Log: INFO: <script>: CALL 1183440664-5061-3@BJC.BGI.B.HD new branch at sip:3519709109@ms3-ch-rk.plivo.com:5060 (DIspatching the call to server) INFO: <script>: <<< 1183440664-5061-3@BJC.BGI.B.HD Current Gateway failed, trying next gateway (didn't get any response for 10 seconds)
PCAP: https://dl.dropboxusercontent.com/u/263974527/SIP-1183440664-5061-3%40BJC.BG...
We are not sure what is the problem here ? Thanks again for looking into this.
Once we restarted kamilo we can see the successful TCP calls b/w kamilio server and our media server.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-71414116
It looks like kamailio was not able to connect to your media server. It could be the limit of active connections, try to increase it if you have a lot of tcp connections.
Look in the code at doc/tcp_tunning.txt for hints on heavy use of tcp.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-71913290
Thanks Daniel. We will increase the tcp connections to 4096 and will update you the result.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-71967196
Any updates? Is this still an issue or can we close the bug report?
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-73117283
Hi Oej,
Yes the issue is still occurring. Today we saw the TCP calls to some set of Media servers is failing.
From the PCAP we can see the INVITE packet generated from kamailio is not correct and we are not seeing this error after restarting kamailio.
PCAP: https://www.dropbox.com/s/m3nwcdfk8774al8/KAMILIO-TCP-FRAGEMENTED.pcap?dl=0
Could you please advise what might be the issue here ?
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-73661660
How did you get the pcap file? For example ngrep is usually truncating big packets.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-73663552
We captured using ngrep only. Yeah that can be one reason for the packet truncation.
But when we restarted the kamailio the issue is resolved. Is there anything else we can monitor in our end for debugging this issue in detail ?
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-73672134
Do you get any parsing errors in at the destination? That would be the an indication there is a transmission error.
If you want to see what kamailio is sending out, add onsend_route and print $snd(buf) -- be sure you have the pv buffer big enough (see core cookbook for the global parameter which controls this value).
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-73684293
In the destination i can see the PACKETS reached in the machine and Freeswitch (destination sip ) not able to detect those SIP packets.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-73686848
You can try to use 'tcpdump -s 0 ...' to see what comes there. Beware that tcp is a stream protocol and network sniffing tools can see parts of the same messages coming as separate packet on the wire, which can be a matter of how the kernel flushes the outbuffer.
Also, don't forget about the option with onsend_route to see what kamailio is writting to the socket.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-73687767
Thanks Daniel for the inputs.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-73689400
No followup in a while, closing it as not looking as potential problem in the code. Re-open and add new details if still an issue.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#issuecomment-86726637
Closed #51.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/51#event-266364946
-
Daniel-Constantin Mierla -
Olle E. Johansson
-
Varghese Paul