I've been tasked to use LD_PRELOAD to log SSL keys for TLS connections using a Diffie-Hellman cipher. The first attempt did not work, so I wanted to sanity check whether Kamailio's TLS support is built in such a way that would defeat LD_PRELOAD.
The instructions from the vendor are to update /etc/init.d/kamailio like this:
env SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" \ start-stop-daemon --start --quiet --pidfile $PIDFILE \ --exec $DAEMON -- $OPTIONS || log_failure_msg " already running"
Is there anything special in Kamailio (5.7.3 on Debian 12) that would prevent this from working? Not necessarily something to defeat a keylogger, but maybe the way tls.so gets loaded?
The only discrepancy I've noticed is the vendor docs refer to libssl.so.3 not libssl.so.1, but the vendor said that should be OK.
I'd love to hear from someone already using VoIPmonitor with Diffie-Hellman ciphers and Kamailio.
You can check this PR https://github.com/kamailio/kamailio/pull/2785
On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
I've been tasked to use LD_PRELOAD to log SSL keys for TLS connections using a Diffie-Hellman cipher. The first attempt did not work, so I wanted to sanity check whether Kamailio's TLS support is built in such a way that would defeat LD_PRELOAD.
The instructions from the vendor are to update /etc/init.d/kamailio like this:
env SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" \ start-stop-daemon --start --quiet --pidfile $PIDFILE \ --exec $DAEMON -- $OPTIONS || log_failure_msg " already running"
Is there anything special in Kamailio (5.7.3 on Debian 12) that would prevent this from working? Not necessarily something to defeat a keylogger, but maybe the way tls.so gets loaded?
The only discrepancy I've noticed is the vendor docs refer to libssl.so.3 not libssl.so.1, but the vendor said that should be OK.
I'd love to hear from someone already using VoIPmonitor with Diffie-Hellman ciphers and Kamailio.
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
It turns out the system I was on really uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio also existing.
I was able to make it work by following the Systemd process:
mkdir /etc/default/kamailio.d/ edit /etc/default/kamailio.d/voipmonitor add lines: SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
The keys are captured by the VoIPmonitor sniffer and everything works as expected from there. I'd be happy to explain further to anyone interested in this setup.
On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com wrote:
You can check this PR https://github.com/kamailio/kamailio/pull/2785
On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
I've been tasked to use LD_PRELOAD to log SSL keys for TLS connections using a Diffie-Hellman cipher. The first attempt did not work, so I wanted to sanity check whether Kamailio's TLS support is built in such a way that would defeat LD_PRELOAD.
The instructions from the vendor are to update /etc/init.d/kamailio like this:
env SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" \ start-stop-daemon --start --quiet --pidfile $PIDFILE \ --exec $DAEMON -- $OPTIONS || log_failure_msg " already running"
Is there anything special in Kamailio (5.7.3 on Debian 12) that would prevent this from working? Not necessarily something to defeat a keylogger, but maybe the way tls.so gets loaded?
The only discrepancy I've noticed is the vendor docs refer to libssl.so.3 not libssl.so.1, but the vendor said that should be OK.
I'd love to hear from someone already using VoIPmonitor with Diffie-Hellman ciphers and Kamailio.
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Hello Calvin,
Thank you for sharing this! This would make for a great tutorial on https://www.kamailio.org/w/documentation/ under the TLS Tutorials section or maybe a new tutorial section for Monitoring. Many would appreciate having this documented there.
-ovidiu
On Mon, Jan 29, 2024 at 8:50 PM Calvin E. via sr-users sr-users@lists.kamailio.org wrote:
It turns out the system I was on really uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio also existing.
I was able to make it work by following the Systemd process:
mkdir /etc/default/kamailio.d/ edit /etc/default/kamailio.d/voipmonitor add lines: SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
The keys are captured by the VoIPmonitor sniffer and everything works as expected from there. I'd be happy to explain further to anyone interested in this setup.
On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com wrote:
You can check this PR https://github.com/kamailio/kamailio/pull/2785
On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users sr-users@lists.kamailio.org wrote:
I've been tasked to use LD_PRELOAD to log SSL keys for TLS connections using a Diffie-Hellman cipher. The first attempt did not work, so I wanted to sanity check whether Kamailio's TLS support is built in such a way that would defeat LD_PRELOAD.
The instructions from the vendor are to update /etc/init.d/kamailio like this:
env SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" \ start-stop-daemon --start --quiet --pidfile $PIDFILE \ --exec $DAEMON -- $OPTIONS || log_failure_msg " already running"
Is there anything special in Kamailio (5.7.3 on Debian 12) that would prevent this from working? Not necessarily something to defeat a keylogger, but maybe the way tls.so gets loaded?
The only discrepancy I've noticed is the vendor docs refer to libssl.so.3 not libssl.so.1, but the vendor said that should be OK.
I'd love to hear from someone already using VoIPmonitor with Diffie-Hellman ciphers and Kamailio.
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Calvin,
Thanks for sharing this, just a question, do you use system-provided OpenSSL or tlsa ?
Le mar. 30 janv. 2024 à 03:00, Calvin E. via sr-users < sr-users@lists.kamailio.org> a écrit :
It turns out the system I was on really uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio also existing.
I was able to make it work by following the Systemd process:
mkdir /etc/default/kamailio.d/ edit /etc/default/kamailio.d/voipmonitor add lines: SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
The keys are captured by the VoIPmonitor sniffer and everything works as expected from there. I'd be happy to explain further to anyone interested in this setup.
On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com wrote:
You can check this PR https://github.com/kamailio/kamailio/pull/2785
On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
I've been tasked to use LD_PRELOAD to log SSL keys for TLS connections using a Diffie-Hellman cipher. The first attempt did not work, so I wanted to sanity check whether Kamailio's TLS support is built in such a way that would defeat LD_PRELOAD.
The instructions from the vendor are to update /etc/init.d/kamailio like this:
env SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" \ start-stop-daemon --start --quiet --pidfile $PIDFILE \ --exec $DAEMON -- $OPTIONS || log_failure_msg " already running"
Is there anything special in Kamailio (5.7.3 on Debian 12) that would prevent this from working? Not necessarily something to defeat a keylogger, but maybe the way tls.so gets loaded?
The only discrepancy I've noticed is the vendor docs refer to libssl.so.3 not libssl.so.1, but the vendor said that should be OK.
I'd love to hear from someone already using VoIPmonitor with Diffie-Hellman ciphers and Kamailio.
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
This was done using the system-provided OpenSSL (Debian 12). It might work for tlsa, but I don't know how Kamilio would respond to LD_PRELOAD affecting one of its own modules.
If your curious how it works, the code is here: https://github.com/voipmonitor/sniffer/blob/master/tools/ssl_keylogger/sslke...
On Fri, Feb 2, 2024 at 1:23 AM Ihor Olkhovskyi via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Thanks for sharing this, just a question, do you use system-provided OpenSSL or tlsa ?
Le mar. 30 janv. 2024 à 03:00, Calvin E. via sr-users < sr-users@lists.kamailio.org> a écrit :
It turns out the system I was on really uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio also existing.
I was able to make it work by following the Systemd process:
mkdir /etc/default/kamailio.d/ edit /etc/default/kamailio.d/voipmonitor add lines: SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
The keys are captured by the VoIPmonitor sniffer and everything works as expected from there. I'd be happy to explain further to anyone interested in this setup.
On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com wrote:
You can check this PR https://github.com/kamailio/kamailio/pull/2785
On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
I've been tasked to use LD_PRELOAD to log SSL keys for TLS connections using a Diffie-Hellman cipher. The first attempt did not work, so I wanted to sanity check whether Kamailio's TLS support is built in such a way that would defeat LD_PRELOAD.
The instructions from the vendor are to update /etc/init.d/kamailio like this:
env SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" \ start-stop-daemon --start --quiet --pidfile $PIDFILE \ --exec $DAEMON -- $OPTIONS || log_failure_msg " already running"
Is there anything special in Kamailio (5.7.3 on Debian 12) that would prevent this from working? Not necessarily something to defeat a keylogger, but maybe the way tls.so gets loaded?
The only discrepancy I've noticed is the vendor docs refer to libssl.so.3 not libssl.so.1, but the vendor said that should be OK.
I'd love to hear from someone already using VoIPmonitor with Diffie-Hellman ciphers and Kamailio.
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Best regards, Ihor (Igor) __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Some weeks ago I learned about [1]. Didn't play with it yet though.
[1] https://medium.com/@yunwei356/ebpf-practical-tutorial-capturing-ssl-tls-plai...
On Tue, Feb 27, 2024, 02:08 Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
This was done using the system-provided OpenSSL (Debian 12). It might work for tlsa, but I don't know how Kamilio would respond to LD_PRELOAD affecting one of its own modules.
If your curious how it works, the code is here: https://github.com/voipmonitor/sniffer/blob/master/tools/ssl_keylogger/sslke...
On Fri, Feb 2, 2024 at 1:23 AM Ihor Olkhovskyi via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Thanks for sharing this, just a question, do you use system-provided OpenSSL or tlsa ?
Le mar. 30 janv. 2024 à 03:00, Calvin E. via sr-users < sr-users@lists.kamailio.org> a écrit :
It turns out the system I was on really uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio also existing.
I was able to make it work by following the Systemd process:
mkdir /etc/default/kamailio.d/ edit /etc/default/kamailio.d/voipmonitor add lines: SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
The keys are captured by the VoIPmonitor sniffer and everything works as expected from there. I'd be happy to explain further to anyone interested in this setup.
On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com wrote:
You can check this PR https://github.com/kamailio/kamailio/pull/2785
On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
I've been tasked to use LD_PRELOAD to log SSL keys for TLS connections using a Diffie-Hellman cipher. The first attempt did not work, so I wanted to sanity check whether Kamailio's TLS support is built in such a way that would defeat LD_PRELOAD.
The instructions from the vendor are to update /etc/init.d/kamailio like this:
env SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" \ start-stop-daemon --start --quiet --pidfile $PIDFILE \ --exec $DAEMON -- $OPTIONS || log_failure_msg " already running"
Is there anything special in Kamailio (5.7.3 on Debian 12) that would prevent this from working? Not necessarily something to defeat a keylogger, but maybe the way tls.so gets loaded?
The only discrepancy I've noticed is the vendor docs refer to libssl.so.3 not libssl.so.1, but the vendor said that should be OK.
I'd love to hear from someone already using VoIPmonitor with Diffie-Hellman ciphers and Kamailio.
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Best regards, Ihor (Igor) __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Calvin,
Voipmonitor-sniffer has support for Kamailio’s ‘siptrace’ module, but this is useful if your goal is to capture SIP over TLS traffic, I’m not sure if that is the reason you have been asked to capture the DH session keys…
If that's the case, any reason you went with LD_PRELOAD method vs kamailio’s siptrace module? Using the later you still get the sip traffic without the need of messing with OpenSSL.
Mind sharing your findings?
Joel.
On Tue, Feb 27, 2024 at 00:18 Bastian Triller via sr-users < sr-users@lists.kamailio.org> wrote:
Some weeks ago I learned about [1]. Didn't play with it yet though.
[1] https://medium.com/@yunwei356/ebpf-practical-tutorial-capturing-ssl-tls-plai...
On Tue, Feb 27, 2024, 02:08 Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
This was done using the system-provided OpenSSL (Debian 12). It might work for tlsa, but I don't know how Kamilio would respond to LD_PRELOAD affecting one of its own modules.
If your curious how it works, the code is here: https://github.com/voipmonitor/sniffer/blob/master/tools/ssl_keylogger/sslke...
On Fri, Feb 2, 2024 at 1:23 AM Ihor Olkhovskyi via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Thanks for sharing this, just a question, do you use system-provided OpenSSL or tlsa ?
Le mar. 30 janv. 2024 à 03:00, Calvin E. via sr-users < sr-users@lists.kamailio.org> a écrit :
It turns out the system I was on really uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio also existing.
I was able to make it work by following the Systemd process:
mkdir /etc/default/kamailio.d/ edit /etc/default/kamailio.d/voipmonitor add lines: SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
The keys are captured by the VoIPmonitor sniffer and everything works as expected from there. I'd be happy to explain further to anyone interested in this setup.
On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com wrote:
You can check this PR https://github.com/kamailio/kamailio/pull/2785
On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
I've been tasked to use LD_PRELOAD to log SSL keys for TLS connections using a Diffie-Hellman cipher. The first attempt did not work, so I wanted to sanity check whether Kamailio's TLS support is built in such a way that would defeat LD_PRELOAD.
The instructions from the vendor are to update /etc/init.d/kamailio like this:
env SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" \ start-stop-daemon --start --quiet --pidfile $PIDFILE \ --exec $DAEMON -- $OPTIONS || log_failure_msg " already running"
Is there anything special in Kamailio (5.7.3 on Debian 12) that would prevent this from working? Not necessarily something to defeat a keylogger, but maybe the way tls.so gets loaded?
The only discrepancy I've noticed is the vendor docs refer to libssl.so.3 not libssl.so.1, but the vendor said that should be OK.
I'd love to hear from someone already using VoIPmonitor with Diffie-Hellman ciphers and Kamailio.
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Best regards, Ihor (Igor) __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
We've been using the siptrace module with Homer to do SIP-only captures, but decided to use a different approach for VoIPmonitor as it affects more than just Kamilio. We're also capturing dozens of FreeSWITCH and rtpengine hosts, which are all using LD_PRELOAD to log their SIP TLS and SRTP DH session keys. We wanted Kamailio and the other components to focus on their real jobs (calling) and let a separate process handle the capturing. This gives us insight/control over any load added by the capturing, and allows us to see things closer to the network perspective rather than the application. It's easy to add the VoIPmonitor sniffer to any host without needing each application to natively support capturing.
I'm sure the siptrace module would have similar results, it's just not part of the "homogenous deployment" approach we're taking with this project.
On Tue, Feb 27, 2024 at 1:29 AM Joel Serrano via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Voipmonitor-sniffer has support for Kamailio’s ‘siptrace’ module, but this is useful if your goal is to capture SIP over TLS traffic, I’m not sure if that is the reason you have been asked to capture the DH session keys…
If that's the case, any reason you went with LD_PRELOAD method vs kamailio’s siptrace module? Using the later you still get the sip traffic without the need of messing with OpenSSL.
Mind sharing your findings?
Joel.
On Tue, Feb 27, 2024 at 00:18 Bastian Triller via sr-users < sr-users@lists.kamailio.org> wrote:
Some weeks ago I learned about [1]. Didn't play with it yet though.
[1] https://medium.com/@yunwei356/ebpf-practical-tutorial-capturing-ssl-tls-plai...
On Tue, Feb 27, 2024, 02:08 Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
This was done using the system-provided OpenSSL (Debian 12). It might work for tlsa, but I don't know how Kamilio would respond to LD_PRELOAD affecting one of its own modules.
If your curious how it works, the code is here: https://github.com/voipmonitor/sniffer/blob/master/tools/ssl_keylogger/sslke...
On Fri, Feb 2, 2024 at 1:23 AM Ihor Olkhovskyi via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Thanks for sharing this, just a question, do you use system-provided OpenSSL or tlsa ?
Le mar. 30 janv. 2024 à 03:00, Calvin E. via sr-users < sr-users@lists.kamailio.org> a écrit :
It turns out the system I was on really uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio also existing.
I was able to make it work by following the Systemd process:
mkdir /etc/default/kamailio.d/ edit /etc/default/kamailio.d/voipmonitor add lines: SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
The keys are captured by the VoIPmonitor sniffer and everything works as expected from there. I'd be happy to explain further to anyone interested in this setup.
On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com wrote:
You can check this PR https://github.com/kamailio/kamailio/pull/2785
On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
> I've been tasked to use LD_PRELOAD to log SSL keys for TLS > connections using a Diffie-Hellman cipher. The first attempt did not work, > so I wanted to sanity check whether Kamailio's TLS support is built in such > a way that would defeat LD_PRELOAD. > > The instructions from the vendor are to update /etc/init.d/kamailio > like this: > > env SSLKEYLOG_UDP='127.0.0.1:1234' > LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so > /usr/lib/x86_64-linux-gnu/libssl.so.3" \ > start-stop-daemon --start --quiet --pidfile $PIDFILE \ > --exec $DAEMON -- $OPTIONS || log_failure_msg " > already running" > > Is there anything special in Kamailio (5.7.3 on Debian 12) that > would prevent this from working? Not necessarily something to defeat a > keylogger, but maybe the way tls.so gets loaded? > > The only discrepancy I've noticed is the vendor docs refer > to libssl.so.3 not libssl.so.1, but the vendor said that should be OK. > > I'd love to hear from someone already using VoIPmonitor > with Diffie-Hellman ciphers and Kamailio. > > __________________________________________________________ > Kamailio - Users Mailing List - Non Commercial Discussions > To unsubscribe send an email to sr-users-leave@lists.kamailio.org > Important: keep the mailing list in the recipients, do not reply > only to the sender! > Edit mailing list options or unsubscribe: > __________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Best regards, Ihor (Igor) __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
I think your plan makes total sense.
Thank you for the insight.
Joel.
On Tue, Feb 27, 2024 at 9:28 AM Calvin E. calvine@gmail.com wrote:
We've been using the siptrace module with Homer to do SIP-only captures, but decided to use a different approach for VoIPmonitor as it affects more than just Kamilio. We're also capturing dozens of FreeSWITCH and rtpengine hosts, which are all using LD_PRELOAD to log their SIP TLS and SRTP DH session keys. We wanted Kamailio and the other components to focus on their real jobs (calling) and let a separate process handle the capturing. This gives us insight/control over any load added by the capturing, and allows us to see things closer to the network perspective rather than the application. It's easy to add the VoIPmonitor sniffer to any host without needing each application to natively support capturing.
I'm sure the siptrace module would have similar results, it's just not part of the "homogenous deployment" approach we're taking with this project.
On Tue, Feb 27, 2024 at 1:29 AM Joel Serrano via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Voipmonitor-sniffer has support for Kamailio’s ‘siptrace’ module, but this is useful if your goal is to capture SIP over TLS traffic, I’m not sure if that is the reason you have been asked to capture the DH session keys…
If that's the case, any reason you went with LD_PRELOAD method vs kamailio’s siptrace module? Using the later you still get the sip traffic without the need of messing with OpenSSL.
Mind sharing your findings?
Joel.
On Tue, Feb 27, 2024 at 00:18 Bastian Triller via sr-users < sr-users@lists.kamailio.org> wrote:
Some weeks ago I learned about [1]. Didn't play with it yet though.
[1] https://medium.com/@yunwei356/ebpf-practical-tutorial-capturing-ssl-tls-plai...
On Tue, Feb 27, 2024, 02:08 Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
This was done using the system-provided OpenSSL (Debian 12). It might work for tlsa, but I don't know how Kamilio would respond to LD_PRELOAD affecting one of its own modules.
If your curious how it works, the code is here: https://github.com/voipmonitor/sniffer/blob/master/tools/ssl_keylogger/sslke...
On Fri, Feb 2, 2024 at 1:23 AM Ihor Olkhovskyi via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Thanks for sharing this, just a question, do you use system-provided OpenSSL or tlsa ?
Le mar. 30 janv. 2024 à 03:00, Calvin E. via sr-users < sr-users@lists.kamailio.org> a écrit :
It turns out the system I was on really uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio also existing.
I was able to make it work by following the Systemd process:
mkdir /etc/default/kamailio.d/ edit /etc/default/kamailio.d/voipmonitor add lines: SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
The keys are captured by the VoIPmonitor sniffer and everything works as expected from there. I'd be happy to explain further to anyone interested in this setup.
On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com wrote:
> You can check this PR > https://github.com/kamailio/kamailio/pull/2785 > > On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < > sr-users@lists.kamailio.org> wrote: > >> I've been tasked to use LD_PRELOAD to log SSL keys for TLS >> connections using a Diffie-Hellman cipher. The first attempt did not work, >> so I wanted to sanity check whether Kamailio's TLS support is built in such >> a way that would defeat LD_PRELOAD. >> >> The instructions from the vendor are to update /etc/init.d/kamailio >> like this: >> >> env SSLKEYLOG_UDP='127.0.0.1:1234' >> LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so >> /usr/lib/x86_64-linux-gnu/libssl.so.3" \ >> start-stop-daemon --start --quiet --pidfile $PIDFILE \ >> --exec $DAEMON -- $OPTIONS || log_failure_msg " >> already running" >> >> Is there anything special in Kamailio (5.7.3 on Debian 12) that >> would prevent this from working? Not necessarily something to defeat a >> keylogger, but maybe the way tls.so gets loaded? >> >> The only discrepancy I've noticed is the vendor docs refer >> to libssl.so.3 not libssl.so.1, but the vendor said that should be OK. >> >> I'd love to hear from someone already using VoIPmonitor >> with Diffie-Hellman ciphers and Kamailio. >> >> __________________________________________________________ >> Kamailio - Users Mailing List - Non Commercial Discussions >> To unsubscribe send an email to sr-users-leave@lists.kamailio.org >> Important: keep the mailing list in the recipients, do not reply >> only to the sender! >> Edit mailing list options or unsubscribe: >> > __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Best regards, Ihor (Igor) __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Hey Calvin,
Did you have to do anything special with OpenSSL and/or Kamailio to get LD_PRELOAD to work and send the keys to voipmonitor?
I can see the env vars are loaded correctly, but I don't see any keys being sent to the sniffer on port 1234 udp.
root@csbc03:~# ps -fe | grep kamailio kamailio 2209068 1 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209069 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209070 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209071 2209068 0 16:33 ? 00:00:02 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209072 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209073 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209074 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209075 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209076 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209077 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209078 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209080 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209082 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209083 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209084 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209086 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209087 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209088 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209089 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209090 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209091 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209092 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209093 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209094 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209095 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209096 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209097 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209098 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209099 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209100 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209101 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209102 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209103 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209104 2209068 0 16:33 ? 00:00:01 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209105 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209106 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209107 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209108 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209109 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209110 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209111 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209112 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209113 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209114 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209115 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209116 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209117 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209118 2209068 4 16:33 ? 00:00:15 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209119 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209120 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209121 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209122 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209123 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209124 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209125 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209126 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209127 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no root 2210501 2210460 0 16:38 pts/0 00:00:00 grep kamailio root@csbc03:~#
root@csbc03:~# cat /proc/2209068/environ LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=2ac0a49bba664c4fbe6c0f5fa7948e4eJOURNAL_STREAM=8:1641955621RUNTIME_DIRECTORY=/run/kamailioCFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yesSSLKEYLOG_UDP=10.2.1.19:1234LD_PRELOAD=/opt/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.1.1 root@csbc03:~#
I tested using the command in voipmonitor docs and that seems to be ok:
root@csbc03:~# env SSLKEYLOG_UDP='10.2.1.19:1234' LD_PRELOAD="/opt/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so" openssl
* SSL KEYLOG : OK detect pointer to function SSL_new : 0x7f10d6adbd30 * SSL KEYLOG : OK detect pointer to function SSL_CTX_set_keylog_callback : 0x7f10d6adcf00 * SSL KEYLOG : log to : 10.2.1.19:1234 OpenSSL> quit root@csbc03:~#
Does anyone have any tips on how to troubleshoot this? I know this might not be directly related to Kamailio...
Thanks, Joel.
On Wed, Feb 28, 2024 at 11:10 AM Joel Serrano joel@textplus.com wrote:
I think your plan makes total sense.
Thank you for the insight.
Joel.
On Tue, Feb 27, 2024 at 9:28 AM Calvin E. calvine@gmail.com wrote:
We've been using the siptrace module with Homer to do SIP-only captures, but decided to use a different approach for VoIPmonitor as it affects more than just Kamilio. We're also capturing dozens of FreeSWITCH and rtpengine hosts, which are all using LD_PRELOAD to log their SIP TLS and SRTP DH session keys. We wanted Kamailio and the other components to focus on their real jobs (calling) and let a separate process handle the capturing. This gives us insight/control over any load added by the capturing, and allows us to see things closer to the network perspective rather than the application. It's easy to add the VoIPmonitor sniffer to any host without needing each application to natively support capturing.
I'm sure the siptrace module would have similar results, it's just not part of the "homogenous deployment" approach we're taking with this project.
On Tue, Feb 27, 2024 at 1:29 AM Joel Serrano via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Voipmonitor-sniffer has support for Kamailio’s ‘siptrace’ module, but this is useful if your goal is to capture SIP over TLS traffic, I’m not sure if that is the reason you have been asked to capture the DH session keys…
If that's the case, any reason you went with LD_PRELOAD method vs kamailio’s siptrace module? Using the later you still get the sip traffic without the need of messing with OpenSSL.
Mind sharing your findings?
Joel.
On Tue, Feb 27, 2024 at 00:18 Bastian Triller via sr-users < sr-users@lists.kamailio.org> wrote:
Some weeks ago I learned about [1]. Didn't play with it yet though.
[1] https://medium.com/@yunwei356/ebpf-practical-tutorial-capturing-ssl-tls-plai...
On Tue, Feb 27, 2024, 02:08 Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
This was done using the system-provided OpenSSL (Debian 12). It might work for tlsa, but I don't know how Kamilio would respond to LD_PRELOAD affecting one of its own modules.
If your curious how it works, the code is here: https://github.com/voipmonitor/sniffer/blob/master/tools/ssl_keylogger/sslke...
On Fri, Feb 2, 2024 at 1:23 AM Ihor Olkhovskyi via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Thanks for sharing this, just a question, do you use system-provided OpenSSL or tlsa ?
Le mar. 30 janv. 2024 à 03:00, Calvin E. via sr-users < sr-users@lists.kamailio.org> a écrit :
> It turns out the system I was on really > uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio > also existing. > > I was able to make it work by following the Systemd process: > > mkdir /etc/default/kamailio.d/ > edit /etc/default/kamailio.d/voipmonitor > add lines: > SSLKEYLOG_UDP='127.0.0.1:1234' > LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so > /usr/lib/x86_64-linux-gnu/libssl.so.3" > > The keys are captured by the VoIPmonitor sniffer and everything > works as expected from there. I'd be happy to explain further to anyone > interested in this setup. > > On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com > wrote: > >> You can check this PR >> https://github.com/kamailio/kamailio/pull/2785 >> >> On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < >> sr-users@lists.kamailio.org> wrote: >> >>> I've been tasked to use LD_PRELOAD to log SSL keys for TLS >>> connections using a Diffie-Hellman cipher. The first attempt did not work, >>> so I wanted to sanity check whether Kamailio's TLS support is built in such >>> a way that would defeat LD_PRELOAD. >>> >>> The instructions from the vendor are to update >>> /etc/init.d/kamailio like this: >>> >>> env SSLKEYLOG_UDP='127.0.0.1:1234' >>> LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so >>> /usr/lib/x86_64-linux-gnu/libssl.so.3" \ >>> start-stop-daemon --start --quiet --pidfile $PIDFILE \ >>> --exec $DAEMON -- $OPTIONS || log_failure_msg " >>> already running" >>> >>> Is there anything special in Kamailio (5.7.3 on Debian 12) that >>> would prevent this from working? Not necessarily something to defeat a >>> keylogger, but maybe the way tls.so gets loaded? >>> >>> The only discrepancy I've noticed is the vendor docs refer >>> to libssl.so.3 not libssl.so.1, but the vendor said that should be OK. >>> >>> I'd love to hear from someone already using VoIPmonitor >>> with Diffie-Hellman ciphers and Kamailio. >>> >>> __________________________________________________________ >>> Kamailio - Users Mailing List - Non Commercial Discussions >>> To unsubscribe send an email to sr-users-leave@lists.kamailio.org >>> Important: keep the mailing list in the recipients, do not reply >>> only to the sender! >>> Edit mailing list options or unsubscribe: >>> >> __________________________________________________________ > Kamailio - Users Mailing List - Non Commercial Discussions > To unsubscribe send an email to sr-users-leave@lists.kamailio.org > Important: keep the mailing list in the recipients, do not reply > only to the sender! > Edit mailing list options or unsubscribe: >
-- Best regards, Ihor (Igor) __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Ok so for whatever reason Kamailio is not sending keys, but if I try on the same machine with openssl binary directly I do see keys being sent to the voipmonitor-sniffer.
I will report here what I find for anyone else that runs into this.
On Tue, Mar 5, 2024 at 1:43 PM Joel Serrano joel@textplus.com wrote:
Hey Calvin,
Did you have to do anything special with OpenSSL and/or Kamailio to get LD_PRELOAD to work and send the keys to voipmonitor?
I can see the env vars are loaded correctly, but I don't see any keys being sent to the sniffer on port 1234 udp.
root@csbc03:~# ps -fe | grep kamailio kamailio 2209068 1 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209069 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209070 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209071 2209068 0 16:33 ? 00:00:02 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209072 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209073 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209074 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209075 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209076 2209068 0 16:33 ? 00:00:03 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209077 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209078 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209080 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209082 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209083 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209084 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209086 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209087 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209088 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209089 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209090 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209091 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209092 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209093 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209094 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209095 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209096 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209097 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209098 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209099 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209100 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209101 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209102 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209103 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209104 2209068 0 16:33 ? 00:00:01 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209105 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209106 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209107 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209108 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209109 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209110 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209111 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209112 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209113 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209114 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209115 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209116 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209117 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209118 2209068 4 16:33 ? 00:00:15 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209119 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209120 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209121 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209122 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209123 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209124 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209125 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209126 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no kamailio 2209127 2209068 0 16:33 ? 00:00:00 /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -f /etc/kamailio/csbc.cfg -m 512 -M 32 --atexit=no root 2210501 2210460 0 16:38 pts/0 00:00:00 grep kamailio root@csbc03:~#
root@csbc03:~# cat /proc/2209068/environ
LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=2ac0a49bba664c4fbe6c0f5fa7948e4eJOURNAL_STREAM=8:1641955621RUNTIME_DIRECTORY=/run/kamailioCFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yesSSLKEYLOG_UDP=10.2.1.19:1234LD_PRELOAD=/opt/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.1.1 root@csbc03:~#
I tested using the command in voipmonitor docs and that seems to be ok:
root@csbc03:~# env SSLKEYLOG_UDP='10.2.1.19:1234' LD_PRELOAD="/opt/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so" openssl
- SSL KEYLOG : OK detect pointer to function SSL_new : 0x7f10d6adbd30
- SSL KEYLOG : OK detect pointer to function SSL_CTX_set_keylog_callback
: 0x7f10d6adcf00
- SSL KEYLOG : log to : 10.2.1.19:1234
OpenSSL> quit root@csbc03:~#
Does anyone have any tips on how to troubleshoot this? I know this might not be directly related to Kamailio...
Thanks, Joel.
On Wed, Feb 28, 2024 at 11:10 AM Joel Serrano joel@textplus.com wrote:
I think your plan makes total sense.
Thank you for the insight.
Joel.
On Tue, Feb 27, 2024 at 9:28 AM Calvin E. calvine@gmail.com wrote:
We've been using the siptrace module with Homer to do SIP-only captures, but decided to use a different approach for VoIPmonitor as it affects more than just Kamilio. We're also capturing dozens of FreeSWITCH and rtpengine hosts, which are all using LD_PRELOAD to log their SIP TLS and SRTP DH session keys. We wanted Kamailio and the other components to focus on their real jobs (calling) and let a separate process handle the capturing. This gives us insight/control over any load added by the capturing, and allows us to see things closer to the network perspective rather than the application. It's easy to add the VoIPmonitor sniffer to any host without needing each application to natively support capturing.
I'm sure the siptrace module would have similar results, it's just not part of the "homogenous deployment" approach we're taking with this project.
On Tue, Feb 27, 2024 at 1:29 AM Joel Serrano via sr-users < sr-users@lists.kamailio.org> wrote:
Calvin,
Voipmonitor-sniffer has support for Kamailio’s ‘siptrace’ module, but this is useful if your goal is to capture SIP over TLS traffic, I’m not sure if that is the reason you have been asked to capture the DH session keys…
If that's the case, any reason you went with LD_PRELOAD method vs kamailio’s siptrace module? Using the later you still get the sip traffic without the need of messing with OpenSSL.
Mind sharing your findings?
Joel.
On Tue, Feb 27, 2024 at 00:18 Bastian Triller via sr-users < sr-users@lists.kamailio.org> wrote:
Some weeks ago I learned about [1]. Didn't play with it yet though.
[1] https://medium.com/@yunwei356/ebpf-practical-tutorial-capturing-ssl-tls-plai...
On Tue, Feb 27, 2024, 02:08 Calvin E. via sr-users < sr-users@lists.kamailio.org> wrote:
This was done using the system-provided OpenSSL (Debian 12). It might work for tlsa, but I don't know how Kamilio would respond to LD_PRELOAD affecting one of its own modules.
If your curious how it works, the code is here: https://github.com/voipmonitor/sniffer/blob/master/tools/ssl_keylogger/sslke...
On Fri, Feb 2, 2024 at 1:23 AM Ihor Olkhovskyi via sr-users < sr-users@lists.kamailio.org> wrote:
> Calvin, > > Thanks for sharing this, just a question, do you use system-provided > OpenSSL or tlsa ? > > Le mar. 30 janv. 2024 à 03:00, Calvin E. via sr-users < > sr-users@lists.kamailio.org> a écrit : > >> It turns out the system I was on really >> uses /lib/systemd/system/kamailio.service, despite /etc/init.d/kamailio >> also existing. >> >> I was able to make it work by following the Systemd process: >> >> mkdir /etc/default/kamailio.d/ >> edit /etc/default/kamailio.d/voipmonitor >> add lines: >> SSLKEYLOG_UDP='127.0.0.1:1234' >> LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so >> /usr/lib/x86_64-linux-gnu/libssl.so.3" >> >> The keys are captured by the VoIPmonitor sniffer and everything >> works as expected from there. I'd be happy to explain further to anyone >> interested in this setup. >> >> On Sun, Jan 28, 2024 at 3:20 AM Sergey Safarov s.safarov@gmail.com >> wrote: >> >>> You can check this PR >>> https://github.com/kamailio/kamailio/pull/2785 >>> >>> On Fri, Jan 26, 2024 at 8:58 PM Calvin E. via sr-users < >>> sr-users@lists.kamailio.org> wrote: >>> >>>> I've been tasked to use LD_PRELOAD to log SSL keys for TLS >>>> connections using a Diffie-Hellman cipher. The first attempt did not work, >>>> so I wanted to sanity check whether Kamailio's TLS support is built in such >>>> a way that would defeat LD_PRELOAD. >>>> >>>> The instructions from the vendor are to update >>>> /etc/init.d/kamailio like this: >>>> >>>> env SSLKEYLOG_UDP='127.0.0.1:1234' >>>> LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so >>>> /usr/lib/x86_64-linux-gnu/libssl.so.3" \ >>>> start-stop-daemon --start --quiet --pidfile $PIDFILE \ >>>> --exec $DAEMON -- $OPTIONS || log_failure_msg " >>>> already running" >>>> >>>> Is there anything special in Kamailio (5.7.3 on Debian 12) that >>>> would prevent this from working? Not necessarily something to defeat a >>>> keylogger, but maybe the way tls.so gets loaded? >>>> >>>> The only discrepancy I've noticed is the vendor docs refer >>>> to libssl.so.3 not libssl.so.1, but the vendor said that should be OK. >>>> >>>> I'd love to hear from someone already using VoIPmonitor >>>> with Diffie-Hellman ciphers and Kamailio. >>>> >>>> __________________________________________________________ >>>> Kamailio - Users Mailing List - Non Commercial Discussions >>>> To unsubscribe send an email to sr-users-leave@lists.kamailio.org >>>> Important: keep the mailing list in the recipients, do not reply >>>> only to the sender! >>>> Edit mailing list options or unsubscribe: >>>> >>> __________________________________________________________ >> Kamailio - Users Mailing List - Non Commercial Discussions >> To unsubscribe send an email to sr-users-leave@lists.kamailio.org >> Important: keep the mailing list in the recipients, do not reply >> only to the sender! >> Edit mailing list options or unsubscribe: >> > > > -- > Best regards, > Ihor (Igor) > __________________________________________________________ > Kamailio - Users Mailing List - Non Commercial Discussions > To unsubscribe send an email to sr-users-leave@lists.kamailio.org > Important: keep the mailing list in the recipients, do not reply > only to the sender! > Edit mailing list options or unsubscribe: > __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe send an email to sr-users-leave@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Make sure you are preloading the correct OpenSSL library. On my Debian 12 box it is libssl.so.3 not libssl.so.1.1. You can confirm which is loaded by checking the "maps" of a running proc:
$ sudo fgrep ssl /proc/2951676/maps 7f26647a4000-7f26647c3000 r--p 00000000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f26647c3000-7f2664821000 r-xp 0001f000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664821000-7f2664840000 r--p 0007d000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664840000-7f266484a000 r--p 0009c000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484a000-7f266484e000 rw-p 000a6000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484e000-7f266484f000 r--p 00000000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f266484f000-7f2664850000 r-xp 00001000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664850000-7f2664851000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664851000-7f2664852000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664852000-7f2664853000 rw-p 00003000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so
My systemd /lib/systemd/system/kamailio.service has a line "EnvironmentFile=-/etc/default/kamailio.d/*" so I dropped a file there:
$ cat /etc/default/kamailio.d/voipmonitor SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
In my environment we're using "packetbuffer_sender = yes" to copy all packets to a central processor. I'm sending the keys to localhost so they can get picked up by the sniffer instead of sending them separately to the central processor. For this to work, the sniffer also must capture the "lo" interface.
Hi Calvin,
Thanks for the tip on capturing on LO interface, I'm sure you just saved me some headaches ;)
Interestingly when I check the environ I do see the env vars being set, but in the maps I don't see the keylogger:
root@csbc03:~# cat /proc/2216899/environ LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=fb5d2818a5434319ab2381262737dcffJOURNAL_STREAM=8:1642042024RUNTIME_DIRECTORY=/run/kamailioCFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32SSLKEYLOG_UDP=10.2.1.19:1234LD_PRELOAD=/opt/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.1.1RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yes root@csbc03:~#
root@csbc03:~# fgrep ssl /proc/2216899/maps 7f1ceef99000-7f1ceefb6000 r--p 00000000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1ceefb6000-7f1cef004000 r-xp 0001d000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef004000-7f1cef01e000 r--p 0006b000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef01e000-7f1cef01f000 ---p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef01f000-7f1cef028000 r--p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef028000-7f1cef02c000 rw-p 0008e000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 root@csbc03:~#
This is on a debian 10 box. I have another box for testing on debian12, I set the exact same config as you and I still don't see the keylogger being loaded:
root@csbc01:~# lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 12 (bookworm) Release: 12 Codename: bookworm root@csbc01:~#
root@csbc01:~# cat /etc/default/kamailio.d/voipmonitor # ANSIBLE_MANAGED_FILE - Do NOT edit this file as it is auto-generated by Ansible. SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" root@csbc01:~#
root@csbc01:~# file /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=f1a884cad7648cc38a579b1d00a9ad523297b78c, with debug_info, not stripped root@csbc01:~#
root@csbc01:~# file /usr/lib/x86_64-linux-gnu/libssl.so.3 /usr/lib/x86_64-linux-gnu/libssl.so.3: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=dd6b0615fc5d03f9c698d6d0c9d2da1b1e8f2d24, stripped root@csbc01:~#
root@csbc01:~# cat /proc/181454/environ LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=059a5e15f1bb4e2bae17c0b5ec9c731eJOURNAL_STREAM=8:2661302RUNTIME_DIRECTORY=/run/kamailioSYSTEMD_EXEC_PID=181394CFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yesSSLKEYLOG_UDP=127.0.0.1:1234LD_PRELOAD=/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3 root@csbc01:~#
root@csbc01:~# fgrep ssl /proc/181454/maps 7f0c537b6000-7f0c537d5000 r--p 00000000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c537d5000-7f0c53833000 r-xp 0001f000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c53833000-7f0c53852000 r--p 0007d000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c53852000-7f0c5385c000 r--p 0009c000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c5385c000-7f0c53860000 rw-p 000a6000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 root@csbc01:~#
Any other ideas of what I can be missing?
On Tue, Mar 5, 2024 at 2:30 PM Calvin E. calvine@gmail.com wrote:
Make sure you are preloading the correct OpenSSL library. On my Debian 12 box it is libssl.so.3 not libssl.so.1.1. You can confirm which is loaded by checking the "maps" of a running proc:
$ sudo fgrep ssl /proc/2951676/maps 7f26647a4000-7f26647c3000 r--p 00000000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f26647c3000-7f2664821000 r-xp 0001f000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664821000-7f2664840000 r--p 0007d000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664840000-7f266484a000 r--p 0009c000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484a000-7f266484e000 rw-p 000a6000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484e000-7f266484f000 r--p 00000000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f266484f000-7f2664850000 r-xp 00001000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664850000-7f2664851000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664851000-7f2664852000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664852000-7f2664853000 rw-p 00003000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so
My systemd /lib/systemd/system/kamailio.service has a line "EnvironmentFile=-/etc/default/kamailio.d/*" so I dropped a file there:
$ cat /etc/default/kamailio.d/voipmonitor SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
In my environment we're using "packetbuffer_sender = yes" to copy all packets to a central processor. I'm sending the keys to localhost so they can get picked up by the sniffer instead of sending them separately to the central processor. For this to work, the sniffer also must capture the "lo" interface.
Does your sslkeylog.so work on that same host with the openssl test? I noticed you're using ansible, so I'm curious if you're compiling on some other host that could have different versions of the openssl-dev stuff. Other things could be file or path permissions, or maybe a security tool blocking it (would auditd do that?).
At this point I'd reach out to their support.
On Tue, Mar 5, 2024 at 10:24 PM Joel Serrano joel@textplus.com wrote:
Hi Calvin,
Thanks for the tip on capturing on LO interface, I'm sure you just saved me some headaches ;)
Interestingly when I check the environ I do see the env vars being set, but in the maps I don't see the keylogger:
root@csbc03:~# cat /proc/2216899/environ LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=fb5d2818a5434319ab2381262737dcffJOURNAL_STREAM=8:1642042024RUNTIME_DIRECTORY=/run/kamailioCFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32SSLKEYLOG_UDP=10.2.1.19:1234LD_PRELOAD=/opt/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.1.1RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yes root@csbc03:~#
root@csbc03:~# fgrep ssl /proc/2216899/maps 7f1ceef99000-7f1ceefb6000 r--p 00000000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1ceefb6000-7f1cef004000 r-xp 0001d000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef004000-7f1cef01e000 r--p 0006b000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef01e000-7f1cef01f000 ---p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef01f000-7f1cef028000 r--p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef028000-7f1cef02c000 rw-p 0008e000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 root@csbc03:~#
This is on a debian 10 box. I have another box for testing on debian12, I set the exact same config as you and I still don't see the keylogger being loaded:
root@csbc01:~# lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 12 (bookworm) Release: 12 Codename: bookworm root@csbc01:~#
root@csbc01:~# cat /etc/default/kamailio.d/voipmonitor # ANSIBLE_MANAGED_FILE - Do NOT edit this file as it is auto-generated by Ansible. SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" root@csbc01:~#
root@csbc01:~# file /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=f1a884cad7648cc38a579b1d00a9ad523297b78c, with debug_info, not stripped root@csbc01:~#
root@csbc01:~# file /usr/lib/x86_64-linux-gnu/libssl.so.3 /usr/lib/x86_64-linux-gnu/libssl.so.3: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=dd6b0615fc5d03f9c698d6d0c9d2da1b1e8f2d24, stripped root@csbc01:~#
root@csbc01:~# cat /proc/181454/environ LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=059a5e15f1bb4e2bae17c0b5ec9c731eJOURNAL_STREAM=8:2661302RUNTIME_DIRECTORY=/run/kamailioSYSTEMD_EXEC_PID=181394CFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yesSSLKEYLOG_UDP=127.0.0.1:1234LD_PRELOAD=/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3 root@csbc01:~#
root@csbc01:~# fgrep ssl /proc/181454/maps 7f0c537b6000-7f0c537d5000 r--p 00000000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c537d5000-7f0c53833000 r-xp 0001f000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c53833000-7f0c53852000 r--p 0007d000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c53852000-7f0c5385c000 r--p 0009c000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c5385c000-7f0c53860000 rw-p 000a6000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 root@csbc01:~#
Any other ideas of what I can be missing?
On Tue, Mar 5, 2024 at 2:30 PM Calvin E. calvine@gmail.com wrote:
Make sure you are preloading the correct OpenSSL library. On my Debian 12 box it is libssl.so.3 not libssl.so.1.1. You can confirm which is loaded by checking the "maps" of a running proc:
$ sudo fgrep ssl /proc/2951676/maps 7f26647a4000-7f26647c3000 r--p 00000000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f26647c3000-7f2664821000 r-xp 0001f000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664821000-7f2664840000 r--p 0007d000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664840000-7f266484a000 r--p 0009c000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484a000-7f266484e000 rw-p 000a6000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484e000-7f266484f000 r--p 00000000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f266484f000-7f2664850000 r-xp 00001000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664850000-7f2664851000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664851000-7f2664852000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664852000-7f2664853000 rw-p 00003000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so
My systemd /lib/systemd/system/kamailio.service has a line "EnvironmentFile=-/etc/default/kamailio.d/*" so I dropped a file there:
$ cat /etc/default/kamailio.d/voipmonitor SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
In my environment we're using "packetbuffer_sender = yes" to copy all packets to a central processor. I'm sending the keys to localhost so they can get picked up by the sniffer instead of sending them separately to the central processor. For this to work, the sniffer also must capture the "lo" interface.
Damm that was a rabbit hole...
So the key pointers were found thanks to reading two very helpful links [1] [2].
The TL;DR is that I use setcap to add capabilities to Kamailio to allow to listen on ports <1024 without root. Once you add capabilities, any LD_* env var gets stripped out and is not accessible to the process for security reasons. The solution is to have the sslkeylogger.so lib in a system LD path with setuid bit added (chmod +s), and load it without any "/" in the name.
So basically doing this (pseudo commands):
mv keylogger.so /system/ld/path/keylogger.so chmod u+s /system/ld/path/keylogger.so
And then have the /etc/default/kamailio.d/voipmonitor file as:
SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="sslkeylog.so libssl.so.1.1"
Restart and boom, sslkeylogger is loaded:
root@csbc03:~# fgrep ssl /proc/2633948/maps 7f97ffb92000-7f97ffbaf000 r--p 00000000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffbaf000-7f97ffbfd000 r-xp 0001d000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffbfd000-7f97ffc17000 r--p 0006b000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffc17000-7f97ffc18000 ---p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffc18000-7f97ffc21000 r--p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffc21000-7f97ffc25000 rw-p 0008e000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f9800173000-7f9800174000 r--p 00000000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so 7f9800174000-7f9800175000 r-xp 00001000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so 7f9800175000-7f9800176000 r--p 00002000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so 7f9800176000-7f9800177000 r--p 00002000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so 7f9800177000-7f9800178000 rw-p 00003000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so root@csbc03:~#
I have one last question for you Calvin,
Can you share the settings you have on your local and remote voipmonitor-sniffers to allow Kamailio to send keys to 127.0.0.1:1234, and then have the voipmonitor-client forward that to voipmonitor-server for processing?
These are mine:
-CLIENT-
[general] id_sensor = 23 query_cache = yes server_destination = XXX server_destination_port = XXX server_password = XXX packetbuffer_sender = yes packetbuffer_enable = yes packetbuffer_total_maxheap = 512 #in MB packetbuffer_compress = yes packetbuffer_file_totalmaxsize = 2000 #MB. Default is disabled. packetbuffer_file_path = /var/spool/voipmonitor/packetbuffer interface = eno1,lo sipport = 5060 sipport = 5061 sipport = 5062
NOTES: 5060 is regular UDP, 5061 and 5062 are both TLS ports.
-SERVER- (only the ssl_* settings)
ssl = yes ssl_ipport = A.B.C.D : 5061 ssl_ipport = A.B.C.E : 5061 ssl_ipport = A.B.C.F : 5061 ssl_ipport = A.B.C.G : 5061 ssl_ipport = A.B.C.D : 5062 ssl_ipport = A.B.C.E : 5062 ssl_ipport = A.B.C.F : 5062 ssl_ipport = A.B.C.G : 5062 ssl_store_sessions_expiration_hours = 48 ssl_sessionkey_udp = yes ssl_sessionkey_udp_port = 1234 ssl_sessionkey_udp_ip = 192.168.1.0/24 ssl_sessionkey_udp_maxwait_ms = 10000 ssl_store_sessions = persistent ssl_ignore_error_invalid_mac = yes
NOTES: All the A.B.C.X are Kamailio instances Public IPs.
I know I'm missing something to get the combo Kamailio->Local-Sniffer->Remote-Sniffer to work, any hints there?
Thanks, Joel.
[1] https://stackoverflow.com/questions/18058426/does-using-linux-capabilities-d... [2] https://unix.stackexchange.com/questions/757484/ld-preload-does-not-work-and...
On Thu, Mar 7, 2024 at 4:33 PM Calvin E. calvine@gmail.com wrote:
Does your sslkeylog.so work on that same host with the openssl test? I noticed you're using ansible, so I'm curious if you're compiling on some other host that could have different versions of the openssl-dev stuff. Other things could be file or path permissions, or maybe a security tool blocking it (would auditd do that?).
At this point I'd reach out to their support.
On Tue, Mar 5, 2024 at 10:24 PM Joel Serrano joel@textplus.com wrote:
Hi Calvin,
Thanks for the tip on capturing on LO interface, I'm sure you just saved
me some headaches ;)
Interestingly when I check the environ I do see the env vars being set,
but in the maps I don't see the keylogger:
root@csbc03:~# cat /proc/2216899/environ
LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=fb5d2818a5434319ab2381262737dcffJOURNAL_STREAM=8:1642042024RUNTIME_DIRECTORY=/run/kamailioCFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32SSLKEYLOG_UDP=10.2.1.19:1234LD_PRELOAD=/opt/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.1.1RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yes
root@csbc03:~#
root@csbc03:~# fgrep ssl /proc/2216899/maps 7f1ceef99000-7f1ceefb6000 r--p 00000000 08:06 266231
/usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f1ceefb6000-7f1cef004000 r-xp 0001d000 08:06 266231
/usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f1cef004000-7f1cef01e000 r--p 0006b000 08:06 266231
/usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f1cef01e000-7f1cef01f000 ---p 00085000 08:06 266231
/usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f1cef01f000-7f1cef028000 r--p 00085000 08:06 266231
/usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f1cef028000-7f1cef02c000 rw-p 0008e000 08:06 266231
/usr/lib/x86_64-linux-gnu/libssl.so.1.1
root@csbc03:~#
This is on a debian 10 box. I have another box for testing on debian12,
I set the exact same config as you and I still don't see the keylogger being loaded:
root@csbc01:~# lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 12 (bookworm) Release: 12 Codename: bookworm root@csbc01:~#
root@csbc01:~# cat /etc/default/kamailio.d/voipmonitor # ANSIBLE_MANAGED_FILE - Do NOT edit this file as it is auto-generated
by Ansible.
SSLKEYLOG_UDP='127.0.0.1:1234'
LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
root@csbc01:~#
root@csbc01:~# file
/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so
/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so: ELF
64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=f1a884cad7648cc38a579b1d00a9ad523297b78c, with debug_info, not stripped
root@csbc01:~#
root@csbc01:~# file /usr/lib/x86_64-linux-gnu/libssl.so.3 /usr/lib/x86_64-linux-gnu/libssl.so.3: ELF 64-bit LSB shared object,
x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=dd6b0615fc5d03f9c698d6d0c9d2da1b1e8f2d24, stripped
root@csbc01:~#
root@csbc01:~# cat /proc/181454/environ
LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=059a5e15f1bb4e2bae17c0b5ec9c731eJOURNAL_STREAM=8:2661302RUNTIME_DIRECTORY=/run/kamailioSYSTEMD_EXEC_PID=181394CFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yesSSLKEYLOG_UDP= 127.0.0.1:1234LD_PRELOAD=/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3
root@csbc01:~#
root@csbc01:~# fgrep ssl /proc/181454/maps 7f0c537b6000-7f0c537d5000 r--p 00000000 08:01 3674382
/usr/lib/x86_64-linux-gnu/libssl.so.3
7f0c537d5000-7f0c53833000 r-xp 0001f000 08:01 3674382
/usr/lib/x86_64-linux-gnu/libssl.so.3
7f0c53833000-7f0c53852000 r--p 0007d000 08:01 3674382
/usr/lib/x86_64-linux-gnu/libssl.so.3
7f0c53852000-7f0c5385c000 r--p 0009c000 08:01 3674382
/usr/lib/x86_64-linux-gnu/libssl.so.3
7f0c5385c000-7f0c53860000 rw-p 000a6000 08:01 3674382
/usr/lib/x86_64-linux-gnu/libssl.so.3
root@csbc01:~#
Any other ideas of what I can be missing?
On Tue, Mar 5, 2024 at 2:30 PM Calvin E. calvine@gmail.com wrote:
Make sure you are preloading the correct OpenSSL library. On my Debian 12 box it is libssl.so.3 not libssl.so.1.1. You can confirm which is loaded by checking the "maps" of a running proc:
$ sudo fgrep ssl /proc/2951676/maps 7f26647a4000-7f26647c3000 r--p 00000000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f26647c3000-7f2664821000 r-xp 0001f000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664821000-7f2664840000 r--p 0007d000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664840000-7f266484a000 r--p 0009c000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484a000-7f266484e000 rw-p 000a6000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484e000-7f266484f000 r--p 00000000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f266484f000-7f2664850000 r-xp 00001000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664850000-7f2664851000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664851000-7f2664852000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664852000-7f2664853000 rw-p 00003000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so
My systemd /lib/systemd/system/kamailio.service has a line "EnvironmentFile=-/etc/default/kamailio.d/*" so I dropped a file there:
$ cat /etc/default/kamailio.d/voipmonitor SSLKEYLOG_UDP='127.0.0.1:1234'
LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so
/usr/lib/x86_64-linux-gnu/libssl.so.3"
In my environment we're using "packetbuffer_sender = yes" to copy all packets to a central processor. I'm sending the keys to localhost so they can get picked up by the sniffer instead of sending them separately to the central processor. For this to work, the sniffer also must capture the "lo" interface.
Try removing `ssl_sessionkey_udp_ip`, I don't have that set on mine. Changing it to 127.0.0.1 might also work, but that's only a guess.
Keep in mind this part is open source, if you want to check it out: https://github.com/voipmonitor/sniffer
Only the GUI is licensed, but isn't required if you want to use the database and pcaps directly.
On Thu, Mar 7, 2024 at 7:56 PM Joel Serrano joel@textplus.com wrote:
Damm that was a rabbit hole...
So the key pointers were found thanks to reading two very helpful links [1] [2].
The TL;DR is that I use setcap to add capabilities to Kamailio to allow to listen on ports <1024 without root. Once you add capabilities, any LD_* env var gets stripped out and is not accessible to the process for security reasons. The solution is to have the sslkeylogger.so lib in a system LD path with setuid bit added (chmod +s), and load it without any "/" in the name.
So basically doing this (pseudo commands):
mv keylogger.so /system/ld/path/keylogger.so chmod u+s /system/ld/path/keylogger.so
And then have the /etc/default/kamailio.d/voipmonitor file as:
SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="sslkeylog.so libssl.so.1.1"
Restart and boom, sslkeylogger is loaded:
root@csbc03:~# fgrep ssl /proc/2633948/maps 7f97ffb92000-7f97ffbaf000 r--p 00000000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffbaf000-7f97ffbfd000 r-xp 0001d000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffbfd000-7f97ffc17000 r--p 0006b000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffc17000-7f97ffc18000 ---p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffc18000-7f97ffc21000 r--p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f97ffc21000-7f97ffc25000 rw-p 0008e000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f9800173000-7f9800174000 r--p 00000000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so 7f9800174000-7f9800175000 r-xp 00001000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so 7f9800175000-7f9800176000 r--p 00002000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so 7f9800176000-7f9800177000 r--p 00002000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so 7f9800177000-7f9800178000 rw-p 00003000 08:06 262170 /usr/lib/x86_64-linux-gnu/sslkeylog.so root@csbc03:~#
I have one last question for you Calvin,
Can you share the settings you have on your local and remote voipmonitor-sniffers to allow Kamailio to send keys to 127.0.0.1:1234, and then have the voipmonitor-client forward that to voipmonitor-server for processing?
These are mine:
-CLIENT-
[general] id_sensor = 23 query_cache = yes server_destination = XXX server_destination_port = XXX server_password = XXX packetbuffer_sender = yes packetbuffer_enable = yes packetbuffer_total_maxheap = 512 #in MB packetbuffer_compress = yes packetbuffer_file_totalmaxsize = 2000 #MB. Default is disabled. packetbuffer_file_path = /var/spool/voipmonitor/packetbuffer interface = eno1,lo sipport = 5060 sipport = 5061 sipport = 5062
NOTES: 5060 is regular UDP, 5061 and 5062 are both TLS ports.
-SERVER- (only the ssl_* settings)
ssl = yes ssl_ipport = A.B.C.D : 5061 ssl_ipport = A.B.C.E : 5061 ssl_ipport = A.B.C.F : 5061 ssl_ipport = A.B.C.G : 5061 ssl_ipport = A.B.C.D : 5062 ssl_ipport = A.B.C.E : 5062 ssl_ipport = A.B.C.F : 5062 ssl_ipport = A.B.C.G : 5062 ssl_store_sessions_expiration_hours = 48 ssl_sessionkey_udp = yes ssl_sessionkey_udp_port = 1234 ssl_sessionkey_udp_ip = 192.168.1.0/24 ssl_sessionkey_udp_maxwait_ms = 10000 ssl_store_sessions = persistent ssl_ignore_error_invalid_mac = yes
NOTES: All the A.B.C.X are Kamailio instances Public IPs.
I know I'm missing something to get the combo Kamailio->Local-Sniffer->Remote-Sniffer to work, any hints there?
Thanks, Joel.
[1] https://stackoverflow.com/questions/18058426/does-using-linux-capabilities-d... [2] https://unix.stackexchange.com/questions/757484/ld-preload-does-not-work-and...
On Thu, Mar 7, 2024 at 4:33 PM Calvin E. calvine@gmail.com wrote:
Does your sslkeylog.so work on that same host with the openssl test? I noticed you're using ansible, so I'm curious if you're compiling on some other host that could have different versions of the openssl-dev stuff. Other things could be file or path permissions, or maybe a security tool blocking it (would auditd do that?).
At this point I'd reach out to their support.
On Tue, Mar 5, 2024 at 10:24 PM Joel Serrano joel@textplus.com wrote:
Hi Calvin,
Thanks for the tip on capturing on LO interface, I'm sure you just saved me some headaches ;)
Interestingly when I check the environ I do see the env vars being set, but in the maps I don't see the keylogger:
root@csbc03:~# cat /proc/2216899/environ LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=fb5d2818a5434319ab2381262737dcffJOURNAL_STREAM=8:1642042024RUNTIME_DIRECTORY=/run/kamailioCFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32SSLKEYLOG_UDP=10.2.1.19:1234LD_PRELOAD=/opt/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.1.1RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yes root@csbc03:~#
root@csbc03:~# fgrep ssl /proc/2216899/maps 7f1ceef99000-7f1ceefb6000 r--p 00000000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1ceefb6000-7f1cef004000 r-xp 0001d000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef004000-7f1cef01e000 r--p 0006b000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef01e000-7f1cef01f000 ---p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef01f000-7f1cef028000 r--p 00085000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 7f1cef028000-7f1cef02c000 rw-p 0008e000 08:06 266231 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 root@csbc03:~#
This is on a debian 10 box. I have another box for testing on debian12, I set the exact same config as you and I still don't see the keylogger being loaded:
root@csbc01:~# lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 12 (bookworm) Release: 12 Codename: bookworm root@csbc01:~#
root@csbc01:~# cat /etc/default/kamailio.d/voipmonitor # ANSIBLE_MANAGED_FILE - Do NOT edit this file as it is auto-generated by Ansible. SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3" root@csbc01:~#
root@csbc01:~# file /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=f1a884cad7648cc38a579b1d00a9ad523297b78c, with debug_info, not stripped root@csbc01:~#
root@csbc01:~# file /usr/lib/x86_64-linux-gnu/libssl.so.3 /usr/lib/x86_64-linux-gnu/libssl.so.3: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=dd6b0615fc5d03f9c698d6d0c9d2da1b1e8f2d24, stripped root@csbc01:~#
root@csbc01:~# cat /proc/181454/environ LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPIDFILE=/run/kamailio/kamailio.pidHOME=/run/kamailioLOGNAME=kamailioUSER=kamailioINVOCATION_ID=059a5e15f1bb4e2bae17c0b5ec9c731eJOURNAL_STREAM=8:2661302RUNTIME_DIRECTORY=/run/kamailioSYSTEMD_EXEC_PID=181394CFGFILE=/etc/kamailio/csbc.cfgSHM_MEMORY=512PKG_MEMORY=32RUN_KAMAILIO=yesGROUP=kamailioDUMP_CORE=yesSSLKEYLOG_UDP=127.0.0.1:1234LD_PRELOAD=/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3 root@csbc01:~#
root@csbc01:~# fgrep ssl /proc/181454/maps 7f0c537b6000-7f0c537d5000 r--p 00000000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c537d5000-7f0c53833000 r-xp 0001f000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c53833000-7f0c53852000 r--p 0007d000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c53852000-7f0c5385c000 r--p 0009c000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f0c5385c000-7f0c53860000 rw-p 000a6000 08:01 3674382 /usr/lib/x86_64-linux-gnu/libssl.so.3 root@csbc01:~#
Any other ideas of what I can be missing?
On Tue, Mar 5, 2024 at 2:30 PM Calvin E. calvine@gmail.com wrote:
Make sure you are preloading the correct OpenSSL library. On my Debian 12 box it is libssl.so.3 not libssl.so.1.1. You can confirm which is loaded by checking the "maps" of a running proc:
$ sudo fgrep ssl /proc/2951676/maps 7f26647a4000-7f26647c3000 r--p 00000000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f26647c3000-7f2664821000 r-xp 0001f000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664821000-7f2664840000 r--p 0007d000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f2664840000-7f266484a000 r--p 0009c000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484a000-7f266484e000 rw-p 000a6000 08:01 131274 /usr/lib/x86_64-linux-gnu/libssl.so.3 7f266484e000-7f266484f000 r--p 00000000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f266484f000-7f2664850000 r-xp 00001000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664850000-7f2664851000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664851000-7f2664852000 r--p 00002000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so 7f2664852000-7f2664853000 rw-p 00003000 08:01 154916 /usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so
My systemd /lib/systemd/system/kamailio.service has a line "EnvironmentFile=-/etc/default/kamailio.d/*" so I dropped a file there:
$ cat /etc/default/kamailio.d/voipmonitor SSLKEYLOG_UDP='127.0.0.1:1234' LD_PRELOAD="/usr/local/src/voipmonitor-git/tools/ssl_keylogger/sslkeylog.so /usr/lib/x86_64-linux-gnu/libssl.so.3"
In my environment we're using "packetbuffer_sender = yes" to copy all packets to a central processor. I'm sending the keys to localhost so they can get picked up by the sniffer instead of sending them separately to the central processor. For this to work, the sniffer also must capture the "lo" interface.
-
Bastian Triller -
Calvin E. -
Ihor Olkhovskyi -
Joel Serrano -
Ovidiu Sas -
Sergey Safarov