[SR-Users] TEXTOPS parser problem with binary data in MIME body

[Klaus Feichtinger]

Hello,

I have an update to the side effects of receiving an INVITE message that
contains a MIME body with binary data:
 - pseudo variables are affected, too
 - the message buffer ($mb) does not include the whole message; it is
ending with the nul character
 - as the message buffer does not include all data, the modification
cannot be done with an external script / program (e.g. perl, bash script)

The behaviour was tested with kamailio 4.0.3, too - no difference!

In general, the whole message is stored in a buffer, but parts of it are
inaccessible for parsing / text functions.

Any idea, what could cause this problem?

regards,
Klaus

> Hello,
>
> I have a problem with Kamailio version 3.2.4 (tested also with 3.3.5)
> regarding "binary data" in message bodies. According RFC3261 it is
> explicitly allowed using binary data within MIME bodies:
>
> RFC 3261 section 7.4.1:   SIP messages MAY contain binary bodies or body
> parts. When no  explicit charset parameter is provided by the sender,
> media subtypes  of the "text" type are defined to have a default charset
> value of  "UTF-8".
>
> However, the Kamailio function "Regular Expression Transformation"
> (re.subst), which is based on the transformation class (exported by the
> textops module) is causing problems in our customer system. In regular
> scenarios, Kamailio is receiving SIP INVITE messages with a MIME body,
> which is containing following parts:
>  - application/sdp (standard conform)
>  - application/x-q931  (Cisco proprietary with BINARY data!)
>  - application/gtd (Cisco proprietary with ASCII strings)
>
> The content of the "x-q931" and "gtd" body parts is depending on (UUS1)
> data that were received on the ISDN line. Kamailio has to forward
> User-User specific information from the ISDN line to SIP user agents (in
> form of the "User-to-User" header field). The content of these UUS1 data
> may contain also byte values "00", which is legitimate. In general,
> Kamailio is in every INVITE message searching specific content in the
> last body (application/gtd) and copying this content to a config
> variable. As soon as the x-q931 body contains nul values (0x00 in binary
> format), the parser stops at this position and cannot parse the rest of
> the message. Therefore, I am missing the information that should be
> copied to the SIP header field, as the parser stops before the end of
> the message body!
>
> As long as the message body does not contain 0x00, it is working fine!
>
> My question is:
>  - is this a bug in Kamailio parsing functions?
>  - is this a design issue of Kamailio text parsers (as binary data are
> allowed acc. RFC3261) - does anybody know a solution for this problem?
> This "bug" is causing big troubles....
>
> Thanks in advance,
> Klaus Feichtinger
>
>
> P.S. trace data (1)...(3) of my problem
>
> (1) exemplary content of the x-q931 body
> ======================Sending: Binary Message Body
> Sep 13 10:50:19: Content-Type: application/x-q931
> 08 01 B4 05 04 03 80 90 A2 18 01 89 1E 02 82 88 6C 05 B1 35 30 30 34 70
> 05 B1 35 30 30 31 7D 02 91 81 7E 18 04 1D 42 75 25 92 43 31 62 94 00 00
> 2C 68 20 64 00 62 F2 10 41 B9 D7 BD 0D 0A
>
> (2) SIP INVITE message (debugger view)
> =============================INVITE sip:115101@<ipv4>:5060 SIP/2.0 Via:
> SIP/2.0/UDP <ipv4>:5060;branch=z9hG4bK171164E
> From: <sip:1101015004@<ipv4>>;tag=29E16410-2541
> To: <sip:115101@<ipv4>>
> Call-ID: 1DBCDACB-1B9911E3-8992FF70-D2BED293@<ipv4>
> Supported: timer,resource-priority,replaces,sdp-anat
> Min-SE:  90
> CSeq: 101 INVITE
> Max-Forwards: 70
> Contact: <sip:1101015004@<ipv4>:5060>
> Expires: 1800
> P-Asserted-Identity: <sip:1101015004@<ipv4>>
> Content-Type: multipart/mixed;boundary=uniqueBoundary
> Mime-Version: 1.0
> Content-Length: 820
>
> --uniqueBoundary
> Content-Type: application/sdp
> Content-Disposition: session;handling=required
>
> v=0
> o=CiscoSystemsSIP-GW-UserAgent 1770 5294 IN IP4 <ipv4>
> s=SIP Call
> c=IN IP4 <ipv4>
> t=0 0
> m=audio 16384 RTP/AVP 8 0
> c=IN IP4 <ipv4>
> a=rtpmap:8 PCMA/8000
> a=rtpmap:0 PCMU/8000
>
> --uniqueBoundary
> Content-Type: application/x-q931
> Content-Disposition: signal;handling=optional
> Content-Length: 62
> 4"      l15004p15001}~Bu%C1b..,h d.brA9W
> --uniqueBoundary
> Content-Type: application/gtd
> Content-Disposition: signal;handling=optional
>
> IAM,
> PRN,isdn*,,,
> USI,rate,c,s,c,1
> USI,lay1,ulaw
> TMR,00
> CPN,08,,1,5001
> CGN,08,,1,,,5004
> UUS,3,1d427525924331629400002c6820640062f21041b9d7bd
> CPC,09
> FCI,,,,,,,y,
> GCI,185ef1291b9911e381d500270dff3fa0
>
> --uniqueBoundary--
>
> (3) config excerpt
> ============if (has_body("multipart/mixed")) {
>    if (filter_body("application/sdp")) {
>       remove_hf("Mime-Version");
>       remove_hf("Content-Type");
>       append_hf("Content-Type: application/sdp\r\n");
>    } else {
>       xlog("L_WARN", " <route> message body part 'application/sdp' not
> found\n");
>    }
>    $var(UUS)=$(rb{re.subst,/^(.*)UUS,.,([a-z0-9,]*)..[A-Z][A-Z][A-Z],(.*)/\2/s});
> append_hf("User-to-User: $var(UUS)\r\n", "CSeq");
>
>    if
> !(subst_body('/Content-Disposition..session.handling=required....//s'))
> {
>       xlog("L_WARN", " <route> substituting Content-Disposition
> FAILED!!! \n");
>    }
>    msg_apply_changes();
>    if (search_body("a=[a-z]+:.+[\r\n]{4}$")) {
>       #!ifdef WITH_XLOGDEBUG
>       xlog("L_INFO", " <route> double CRLF at the end of the message
> body detected - is reduced to one now. \n"); #!endif
>       $var(sdp) = $(rb{s.striptail,2});
>       set_body("$var(sdp)", "application/sdp");
>    }
> }
>
>
>
>
> _______________________________________________
> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
> sr-users at lists.sip-router.org
> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users