8 Dec
2015
8 Dec
'15
5:21 p.m.
Hi folk!
Have a strange issue, and cannot understand what is wrong.
Test scheme UA(sip) -> INVITE -> Kamailio
The transport protocol used is TCP.
The issue is reproduced randomly, in case of wrong INVITE, Kamailio does not
parse Method from R-URI and answer "400 CSeq method does not match request
method".
The log with debug level 3 for such INVITE:
https://gist.github.com/vance-od/c4e1c783adba02d80c58
my first confuse is why in line 2 different number of bytes: "read= 1025
bytes, parsed=1037"
the second is in line 12: Method value is empty
after this everithing brokes and kamailio answers 400.
Just another call: the same setup (same device, kamailio instance etc), same
conditions, just another call. Everything is OK, similar part for good call:
https://gist.github.com/vance-od/01b5dff2d81f0878cff4
All my attempts to find the diff between to messages goes to nothing, the 2
INVITEs looks similar, the wireshark analyse of TCP level - also had no
result (all is similar)
The issue happens only with one of our devices and only in case of TCP/TLS
(UDP constantly ok, this is why I am assured the problem is in device, but I
need proofs)
version: kamailio 4.2.5 (i386/linux)
flags: STATS: Off, EXTRA_DEBUG, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS,
DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC,
F_MALLOC, DBG_F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
will appreciate any help or idea ,with such a mess!
Cheers!
--
View this message in context:
http://sip-router.1086192.n5.nabble.com/random-incorrect-method-parsing-tp1…
Sent from the Users mailing list archive at Nabble.com.
9 Dec
9 Dec
9:54 a.m.
Hello,
can you change the sources and replace:
DBG(" method: <%.*s>\n",fl->u.request.method.len,
ZSW(fl->u.request.method.s));
with:
DBG(" method: <%.*s> (%d)\n",fl->u.request.method.len,
ZSW(fl->u.request.method.s), fl->u.request.method.len);
inside parser/msg_parser.c +625
Then recompile, reinstall and wait for same case again.
Maybe there is a '\0', although the logs are not showing it. Or the
length is not properly set.
Cheers,
Daniel
On 08/12/15 16:21, Vasiliy Ganchev wrote:
Hi folk!
Have a strange issue, and cannot understand what is wrong.
Test scheme UA(sip) -> INVITE -> Kamailio
The transport protocol used is TCP.
The issue is reproduced randomly, in case of wrong INVITE, Kamailio does not
parse Method from R-URI and answer "400 CSeq method does not match request
method".
The log with debug level 3 for such INVITE:
https://gist.github.com/vance-od/c4e1c783adba02d80c58
my first confuse is why in line 2 different number of bytes: "read= 1025
bytes, parsed=1037"
the second is in line 12: Method value is empty
after this everithing brokes and kamailio answers 400.
Just another call: the same setup (same device, kamailio instance etc), same
conditions, just another call. Everything is OK, similar part for good call:
https://gist.github.com/vance-od/01b5dff2d81f0878cff4
All my attempts to find the diff between to messages goes to nothing, the 2
INVITEs looks similar, the wireshark analyse of TCP level - also had no
result (all is similar)
The issue happens only with one of our devices and only in case of TCP/TLS
(UDP constantly ok, this is why I am assured the problem is in device, but I
need proofs)
version: kamailio 4.2.5 (i386/linux)
flags: STATS: Off, EXTRA_DEBUG, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS,
DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC,
F_MALLOC, DBG_F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
will appreciate any help or idea ,with such a mess!
Cheers!
--
View this message in context:
http://sip-router.1086192.n5.nabble.com/random-incorrect-method-parsing-tp1…
Sent from the Users mailing list archive at Nabble.com.
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users(a)lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Book: SIP Routing With Kamailio - http://www.asipto.com
http://miconda.eu
1:05 p.m.
Hi Daniel!
Thank you for the suggestion, after the deep investigation, looks like the
root of the problem found:
when device send TCP ACK (before INVITE), in this packet is incapsulated
"VSS-Monitoring ethernet trailer, Source Port: 0", example in file in
attachment.
Looks like kamailio parse it, and add the received /0 to the beginning of
following SIP request (INVITE), as a result - incorrect Method parsing and
400 error.
The way to solve current problem, is to modify the check below:
--- parser/msg_parser.c (revision xxxxx)
+++ parser/msg_parser.c (working copy)
@@ -607,7 +607,7 @@
hdr_flags_t flags;
/* eat crlf from the beginning */
- for (tmp=buf; (*tmp=='\n' || *tmp=='\r')&&
+ for (tmp=buf; (*tmp=='\n' || *tmp=='\r' ||
*tmp=='\0')&&
tmp-buf < len ; tmp++);
Do we need such change in kamailio?
Cheers! issued_packet_with_extra_0.pcap
<http://sip-router.1086192.n5.nabble.com/file/n143771/issued_packet_with_extra_0.pcap>
--
View this message in context:
http://sip-router.1086192.n5.nabble.com/random-incorrect-method-parsing-tp1…
Sent from the Users mailing list archive at Nabble.com.
5:41 p.m.
Hello,
good it was caught. I think it is good/safe to skip the '\0' together
with the white spaces at the beginning for the request. I will analyze
the impact and push the patch for it.
Cheers,
Daniel
On 09/12/15 12:05, Vasiliy Ganchev wrote:
Hi Daniel!
Thank you for the suggestion, after the deep investigation, looks like the
root of the problem found:
when device send TCP ACK (before INVITE), in this packet is incapsulated
"VSS-Monitoring ethernet trailer, Source Port: 0", example in file in
attachment.
Looks like kamailio parse it, and add the received /0 to the beginning of
following SIP request (INVITE), as a result - incorrect Method parsing and
400 error.
The way to solve current problem, is to modify the check below:
--- parser/msg_parser.c (revision xxxxx)
+++ parser/msg_parser.c (working copy)
@@ -607,7 +607,7 @@
hdr_flags_t flags;
/* eat crlf from the beginning */
- for (tmp=buf; (*tmp=='\n' || *tmp=='\r')&&
+ for (tmp=buf; (*tmp=='\n' || *tmp=='\r' ||
*tmp=='\0')&&
tmp-buf < len ; tmp++);
Do we need such change in kamailio?
Cheers! issued_packet_with_extra_0.pcap
<http://sip-router.1086192.n5.nabble.com/file/n143771/issued_packet_with_extra_0.pcap>
--
View this message in context:
http://sip-router.1086192.n5.nabble.com/random-incorrect-method-parsing-tp1…
Sent from the Users mailing list archive at Nabble.com.
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users(a)lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Book: SIP Routing With Kamailio - http://www.asipto.com
http://miconda.eu
10 Dec
10 Dec
10:11 a.m.
Hello,
update on this -- I pushed last evening to master a slightly different
patch, to skip other white space characters before the method is started
to be parsed.
Not having much time, I didn't dig further in the code, but the position
of the patch suggests that these leading whitespaces are still kept in
the message buffer, which may result in sending them out as well, unless
before sending, its another trim of whitespaces.
My opinion is that the tcp read code should trim these whitespaces and
provide msg buffer with the start of the method for sip requests. So I
will analyze more and then see if a different fix would be better,
before considering to backport to stable branches.
Cheers,
Daniel
On 09/12/15 16:41, Daniel-Constantin Mierla wrote:
Hello,
good it was caught. I think it is good/safe to skip the '\0' together
with the white spaces at the beginning for the request. I will analyze
the impact and push the patch for it.
Cheers,
Daniel
On 09/12/15 12:05, Vasiliy Ganchev wrote:
> Hi Daniel!
>
> Thank you for the suggestion, after the deep investigation, looks like the
> root of the problem found:
>
> when device send TCP ACK (before INVITE), in this packet is incapsulated
> "VSS-Monitoring ethernet trailer, Source Port: 0", example in file in
> attachment.
> Looks like kamailio parse it, and add the received /0 to the beginning of
> following SIP request (INVITE), as a result - incorrect Method parsing and
> 400 error.
>
> The way to solve current problem, is to modify the check below:
> --- parser/msg_parser.c (revision xxxxx)
> +++ parser/msg_parser.c (working copy)
> @@ -607,7 +607,7 @@
> hdr_flags_t flags;
>
> /* eat crlf from the beginning */
> - for (tmp=buf; (*tmp=='\n' || *tmp=='\r')&&
> + for (tmp=buf; (*tmp=='\n' || *tmp=='\r' ||
*tmp=='\0')&&
> tmp-buf < len ; tmp++);
>
> Do we need such change in kamailio?
>
> Cheers! issued_packet_with_extra_0.pcap
>
<http://sip-router.1086192.n5.nabble.com/file/n143771/issued_packet_with_extra_0.pcap>
>
>
>
> --
> View this message in context:
http://sip-router.1086192.n5.nabble.com/random-incorrect-method-parsing-tp1…
> Sent from the Users mailing list archive at Nabble.com.
>
> _______________________________________________
> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
> sr-users(a)lists.sip-router.org
> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Book: SIP Routing With Kamailio - http://www.asipto.com
http://miconda.eu
10:35 a.m.
Hi Daniel!
just additional info about the nature of that "extra zero".
Vendor use them as a padding bytes
RFC http://www.ietf.org/rfc/rfc0894.txt says:
"If necessary, the data field should be padded
(with octets of zero) to meet the Ethernet minimum frame size. This
padding is not part of the IP packet and is not included in the total
length field of the IP header."
Looks like these padding bytes are not removed/ignored correctly be NIC,
that leads to the issue in kamailio.
Cheers!
--
View this message in context:
http://sip-router.1086192.n5.nabble.com/random-incorrect-method-parsing-tp1…
Sent from the Users mailing list archive at Nabble.com.
3300
days inactive
3302
days old
5 comments
2 participants
participants (2)
-
Daniel-Constantin Mierla -
Vasiliy Ganchev